thebigtodolist.com
robots.txt

Robots Exclusion Standard data for thebigtodolist.com

Resource Scan

Scan Details

Site Domain thebigtodolist.com
Base Domain thebigtodolist.com
Scan Status Ok
Last Scan2024-05-19T03:18:31+00:00
Next Scan 2024-05-26T03:18:31+00:00

Last Scan

Scanned2024-05-19T03:18:31+00:00
URL https://thebigtodolist.com/robots.txt
Domain IPs 104.21.32.97, 172.67.150.174, 2606:4700:3031::ac43:96ae, 2606:4700:3033::6815:2061
Response IP 104.21.32.97
Found Yes
Hash 9ba8fee1c28378b8e83d11fefc0b82490ad5bac94ccd07a34279506b08a1423b
SimHash 6510cd5061b5

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500