crosswordbuzz.com
robots.txt

Robots Exclusion Standard data for crosswordbuzz.com

Resource Scan

Scan Details

Site Domain crosswordbuzz.com
Base Domain crosswordbuzz.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-31T00:17:22+00:00
Next Scan 2025-01-29T00:17:22+00:00

Last Successful Scan

Scanned2024-01-05T21:20:55+00:00
URL https://crosswordbuzz.com/robots.txt
Domain IPs 104.21.25.189, 172.67.134.123, 2606:4700:3033::ac43:867b, 2606:4700:3036::6815:19bd
Response IP 104.21.25.189
Found Yes
Hash 04dab62c9b6e0910c6214d71e0d43fd4987e3ecbe2c1473e85de16928e1850b6
SimHash 65181c67cdb5

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 9

ezoic

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

ezoicbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://crosswordbuzz.com/daily/crosswordbuzz-daily.xml