netoff.co.jp
robots.txt

Robots Exclusion Standard data for netoff.co.jp

Resource Scan

Scan Details

Site Domain netoff.co.jp
Base Domain netoff.co.jp
Scan Status Ok
Last Scan2024-05-21T04:02:25+00:00
Next Scan 2024-05-28T04:02:25+00:00

Last Scan

Scanned2024-05-21T04:02:25+00:00
URL https://netoff.co.jp/robots.txt
Redirect https://www.netoff.co.jp/robots.txt
Redirect Domain www.netoff.co.jp
Redirect Base netoff.co.jp
Domain IPs 210.141.21.25
Redirect IPs 23.32.29.16, 96.17.180.42
Response IP 96.17.180.42
Found Yes
Hash 34e917b33cfec5bf469fe979698b4a51a0796765deb96aaca775cace14cfeba9
SimHash 01167a28a313

Groups

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

reap-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /