sturbridgeyankee.com
robots.txt

Robots Exclusion Standard data for sturbridgeyankee.com

Resource Scan

Scan Details

Site Domain sturbridgeyankee.com
Base Domain sturbridgeyankee.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-04-06T04:29:03+00:00
Next Scan 2024-07-05T04:29:03+00:00

Last Successful Scan

Scanned2023-05-13T20:52:20+00:00
URL https://sturbridgeyankee.com/robots.txt
Redirect https://www.sturbridgeyankee.com/robots.txt
Redirect Domain www.sturbridgeyankee.com
Redirect Base sturbridgeyankee.com
Domain IPs 104.21.48.15, 172.67.175.184, 2606:4700:3031::ac43:afb8, 2606:4700:3033::6815:300f
Redirect IPs 23.32.29.106, 23.32.29.96
Response IP 23.215.7.23
Found Yes
Hash 233d7838e3dd87c2f44d4fb614efbb73fc59019c6e53ff3d1ca847395dac6c52
SimHash 25f0cf52cda5

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 20

Comments

  • Allow all robots to spider everything by disallowing nothing