careergoalie.com
robots.txt

Robots Exclusion Standard data for careergoalie.com

Resource Scan

Scan Details

Site Domain careergoalie.com
Base Domain careergoalie.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-15T15:46:28+00:00
Next Scan 2025-04-14T15:46:28+00:00

Last Successful Scan

Scanned2025-02-14T15:45:37+00:00
URL https://careergoalie.com/robots.txt
Domain IPs 104.21.3.185, 172.67.131.23, 2606:4700:3032::6815:3b9, 2606:4700:3037::ac43:8317
Response IP 172.67.131.23
Found Yes
Hash 92b707bcfa8f58adf6a2529f368c981c002a52b2afef1646c7556db7feebb663
SimHash a2848d877770

Groups

*

Rule Path
Allow /

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines: