chronostorysearch.com
robots.txt

Robots Exclusion Standard data for chronostorysearch.com

Resource Scan

Scan Details

Site Domain chronostorysearch.com
Base Domain chronostorysearch.com
Scan Status Ok
Last Scan2026-03-20T11:33:28+00:00
Next Scan 2026-03-27T11:33:28+00:00

Last Scan

Scanned2026-03-20T11:33:28+00:00
URL https://chronostorysearch.com/robots.txt
Redirect https://www.chronostorysearch.com/robots.txt
Redirect Domain www.chronostorysearch.com
Redirect Base chronostorysearch.com
Domain IPs 104.21.64.49, 172.67.176.13, 2606:4700:3032::ac43:b00d, 2606:4700:3037::6815:4031
Redirect IPs 104.21.64.49, 172.67.176.13, 2606:4700:3032::ac43:b00d, 2606:4700:3037::6815:4031
Response IP 104.21.64.49
Found Yes
Hash fc3dbf727eb16f8b3db36caf5d207cbafbd0e8c1316b6d17748d15896b84325b
SimHash 400a1403f653

Groups

*

Rule Path
Allow /
Disallow /api/

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bytespider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://chrono-story-search.vercel.app/sitemap.xml

Comments

  • Robots.txt for chrono-story-search
  • 限制爬蟲行為以減少 Edge Requests
  • 限制爬取頻率(秒)
  • 針對特定爬蟲的規則
  • Sitemap