nomadnotmad.com
robots.txt

Robots Exclusion Standard data for nomadnotmad.com

Resource Scan

Scan Details

Site Domain nomadnotmad.com
Base Domain nomadnotmad.com
Scan Status Ok
Last Scan2024-11-15T07:30:13+00:00
Next Scan 2024-11-22T07:30:13+00:00

Last Scan

Scanned2024-11-15T07:30:13+00:00
URL https://nomadnotmad.com/robots.txt
Domain IPs 104.21.58.120, 172.67.159.109, 2606:4700:3033::ac43:9f6d, 2606:4700:3037::6815:3a78
Response IP 172.67.159.109
Found Yes
Hash 390c7787114afade2fc0485f15e626f460165ae7bc7f3135c735929c2679c08f
SimHash 650898f243a1

Groups

scrapy

Rule Path
Allow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20