ghostrobot.com
robots.txt

Robots Exclusion Standard data for ghostrobot.com

Resource Scan

Scan Details

Site Domain ghostrobot.com
Base Domain ghostrobot.com
Scan Status Ok
Last Scan2025-12-31T01:25:53+00:00
Next Scan 2026-01-07T01:25:53+00:00

Last Scan

Scanned2025-12-31T01:25:53+00:00
URL https://ghostrobot.com/robots.txt
Domain IPs 104.21.45.91, 172.67.212.170, 2606:4700:3035::6815:2d5b, 2606:4700:3036::ac43:d4aa
Response IP 104.21.45.91
Found Yes
Hash 23b2541b118a6500d4bbf5cbc0dbc6d2581ec2435a6a00939ed2aaacd453502a
SimHash c130191225b5

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://ghostrobot.com/sitemaps-2-sitemap.xml
sitemap https://ghostrobot.tv/sitemaps-2-sitemap.xml

Comments

  • robots.txt for https://ghostrobot.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/