jimwilliamson.info
robots.txt

Robots Exclusion Standard data for jimwilliamson.info

Resource Scan

Scan Details

Site Domain jimwilliamson.info
Base Domain jimwilliamson.info
Scan Status Ok
Last Scan6/15/2025, 8:37:52 AM
Next Scan 6/16/2025, 8:37:52 AM

Last Scan

Scanned6/15/2025, 8:37:52 AM
URL https://jimwilliamson.info/robots.txt
Domain IPs 104.21.94.195, 172.67.168.162, 2606:4700:3031::6815:5ec3, 2606:4700:3035::ac43:a8a2
Response IP 104.21.94.195
Found Yes
Hash 39b0832ae6b5ac78748e7aed417b8a019bdd28e0fa12e0dbc6eab214ada4bdcc
SimHash 0858c8a0ea72

Groups

*

Rule Path
Disallow /

Comments

  • Example of how to block all crawlers