Log In Sign Up

mid.syosetu.com
robots.txt

Robots Exclusion Standard data for mid.syosetu.com

Resource Scan

Scan Details

Site Domain mid.syosetu.com
Base Domain syosetu.com
Scan Status Ok
Last Scan2025-09-03T09:39:51+00:00
Next Scan 2025-09-10T09:39:51+00:00

Last Scan

Scanned2025-09-03T09:39:51+00:00
URL https://mid.syosetu.com/robots.txt
Domain IPs 203.137.161.27, 203.137.162.157, 203.137.162.199, 203.137.192.156, 203.137.192.158, 203.137.192.168, 203.137.193.203, 203.137.195.68, 210.152.14.33
Response IP 203.137.162.157
Found Yes
Hash 9318c06ce3d7d56ee7ba35092bce6bb78887175dafa56019445c5e18f78e5dfa
SimHash 2f06c020e332

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mediapartners-google

Rule Path
Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)

Rule Path
Disallow /

meta-externalagent/1.1

Rule Path
Disallow /
Back to top