Log In Sign Up

directg.jp
robots.txt

Robots Exclusion Standard data for directg.jp

Resource Scan

Scan Details

Site Domain directg.jp
Base Domain directg.jp
Scan Status Ok
Last Scan2024-06-19T21:16:14+00:00
Next Scan 2024-07-19T21:16:14+00:00

Last Scan

Scanned2024-06-19T21:16:14+00:00
URL https://directg.jp/robots.txt
Domain IPs 1.234.83.241
Response IP 1.234.83.241
Found Yes
Hash ec1b4e58edb496e6503a3c4d5c11c2ce68845031a3bbcf8b7fc0913b09bd1908
SimHash 621d17038c01

Groups

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

gtwek

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot@gmail.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/1.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/2.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/3.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/4.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/5.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

askpeterbot

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

mozilla/5.0 (compatible; null; +null; null)

Rule Path
Disallow /

js-kit url resolver

Rule Path
Disallow /

js-kit url resolver, http://js-kit.com/

Rule Path
Disallow /

http://siteexplorer.info/

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mozilla/5.0 (compatible; butterfly/1.0; +http://labs.topsy.com/butterfly/) gecko/2009032608 firefox/3.0.8

Rule Path
Disallow /

tweetmeme

Rule Path
Disallow /

mozilla/5.0 (macintosh; u; intel mac os x 10.6; en-us; rv:1.9.2) gecko/20100115 firefox/3.6 (flipboardproxy/0.0.5; +http://flipboard.com/browserproxy)

Rule Path
Disallow /

mozilla/5.0 (macintosh; u; intel mac os x 10.6; en-us; rv:1.9.2) gecko/20100115 firefox/3.6 (flipboardproxy/1.1; +http://flipboard.com/browserproxy)

Rule Path
Disallow /

badbot

Rule Path
Disallow /

mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; msie 6.0b; windows nt 5.0) gecko/2009011913 firefox/3.0.6 tweetmemebot

Rule Path
Disallow /

tweetedtimes bot/1.0

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

mozilla/5.0 (compatible; exabot/3.0 (biggerbetter); +http://www.exabot.com/go/robot)

Rule Path
Disallow /

mozilla/5.0 (compatible; exabot/3.0; +http:/www.exabot.com/go/robot)

Rule Path
Disallow /

mozilla/5.0 (compatible; tweetmemebot/2.11; +http://tweetmeme.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; tweetmemebot/3.0; +http://tweetmeme.com/)

Rule Path
Disallow /

tweetmemebot/3.0; +http://tweetmeme.com/

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 8.0; windows nt 6.1; trident/4.0; apcpms=^n201309130831584094993260671919b0e33f_4^; slcc2; .net clr 2.0.50727; .net clr 3.5.30729; .net clr 3.0.30729; media center pc 6.0

Rule Path
Disallow /

bingbot
daumoa
googlebot
yahoo! slurp
naverbot
msnbot
yeti

Rule Path
Disallow /order/
Disallow /mypage/
Disallow /member/
Disallow /_Admin2/
Disallow /ads.txt
Back to top