czech-stuff.com
robots.txt

Robots Exclusion Standard data for czech-stuff.com

Resource Scan

Scan Details

Site Domain czech-stuff.com
Base Domain czech-stuff.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-30T07:50:55+00:00
Next Scan 2025-12-30T07:50:55+00:00

Last Successful Scan

Scanned2025-10-09T11:07:38+00:00
URL https://czech-stuff.com/robots.txt
Domain IPs 185.151.30.159, 2a07:7800::159
Response IP 185.151.30.159
Found Yes
Hash 3511533656fb032fa8154ab9f56d54f46d76bba1dcecbf7f37160a24f6a831b7
SimHash 6410da326731

Groups

scrapy

Rule Path
Allow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2