crayhats.com
robots.txt

Robots Exclusion Standard data for crayhats.com

Resource Scan

Scan Details

Site Domain crayhats.com
Base Domain crayhats.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-06-08T19:51:02+00:00
Next Scan 2024-07-08T19:51:02+00:00

Last Successful Scan

Scanned2024-05-10T19:23:24+00:00
URL https://crayhats.com/robots.txt
Domain IPs 104.21.67.163, 172.67.178.26, 2606:4700:3036::6815:43a3, 2606:4700:3036::ac43:b21a
Response IP 104.21.67.163
Found Yes
Hash ce7e9b78db67021fda12bf9c463f85eacb4b1ca66476b8fdc2de755451820488
SimHash 67089041c3b1

Groups

*

Rule Path
Disallow

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://crayhats.com/sitemap.xml

Warnings

  • `host` is not a known field.