bungeishunju.com
robots.txt

Robots Exclusion Standard data for bungeishunju.com

Resource Scan

Scan Details

Site Domain bungeishunju.com
Base Domain bungeishunju.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-23T20:35:27+00:00
Next Scan 2025-06-21T20:35:27+00:00

Last Successful Scan

Scanned2024-05-21T19:13:18+00:00
URL https://bungeishunju.com/robots.txt
Domain IPs 52.199.77.78
Response IP 52.199.77.78
Found Yes
Hash 994493830807b8708b16a5f72e4105d0982d59b1eb4d6b1bd0d8ff93cd7d5c83
SimHash 611dd866c751

Groups

*

Rule Path
Allow /n/*
Allow /m/*
Allow /archives/*
Allow /follows
Allow /followers
Allow /likes
Allow /membership/*
Disallow /*/
Disallow /embed/*
Disallow /intent/*

bingbot

Rule Path
Allow /n/*
Allow /m/*
Allow /archives/*
Allow /follows
Allow /followers
Allow /likes
Disallow /*/
Disallow /embed/*
Disallow /intent/*

Other Records

Field Value
crawl-delay 30

megalodon

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
sitemap https://bungeishunju.com/sitemap.xml.gz