blog.claustrophobia-game.com
robots.txt

Robots Exclusion Standard data for blog.claustrophobia-game.com

Resource Scan

Scan Details

Site Domain blog.claustrophobia-game.com
Base Domain claustrophobia-game.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-10-06T18:52:37+00:00
Next Scan 2026-01-04T18:52:37+00:00

Last Successful Scan

Scanned2023-09-11T00:33:31+00:00
URL https://blog.claustrophobia-game.com/robots.txt
Domain IPs 74.114.154.18, 74.114.154.22
Response IP 74.114.154.22
Found Yes
Hash e0f6c86bd20b2e690eab1bbd9999348850d10de5843fd49cf79ff4022f2ed19c
SimHash 6b34d845cd33

Groups

*

Rule Path
Disallow /random
Disallow /day
Disallow /sticky-ad-iframe.html
Disallow /privacy/consent

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://blog.claustrophobia-game.com/sitemap.xml

Comments

  • OpenAI's crawler
  • Common Crawl's crawler