chirashi.kurashiru.com
robots.txt

Robots Exclusion Standard data for chirashi.kurashiru.com

Resource Scan

Scan Details

Site Domain chirashi.kurashiru.com
Base Domain kurashiru.com
Scan Status Ok
Last Scan2024-09-19T21:27:06+00:00
Next Scan 2024-09-26T21:27:06+00:00

Last Scan

Scanned2024-09-19T21:27:06+00:00
URL https://chirashi.kurashiru.com/robots.txt
Domain IPs 13.33.88.10, 13.33.88.17, 13.33.88.41, 13.33.88.9
Response IP 13.33.88.41
Found Yes
Hash 68a6513b9592444bc7d6e9fefdb82a387560ccb5515f43425615f0b968d93264
SimHash 4d0edc70c213

Groups

*

Rule Path
Disallow /admin_api
Disallow /wapi
Disallow /retail_contents_api
Disallow /widgets

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

baiduspider+

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://chirashi.kurashiru.com/sitemap.xml.gz