gavinhollinger.com
robots.txt

Robots Exclusion Standard data for gavinhollinger.com

Resource Scan

Scan Details

Site Domain gavinhollinger.com
Base Domain gavinhollinger.com
Scan Status Ok
Last Scan2025-10-26T10:57:18+00:00
Next Scan 2025-11-02T10:57:18+00:00

Last Scan

Scanned2025-10-26T10:57:18+00:00
URL https://www.gavinhollinger.com/robots.txt
Domain IPs 104.21.1.87, 172.67.128.241, 2606:4700:3030::6815:157, 2606:4700:3036::ac43:80f1
Response IP 172.67.128.241
Found Yes
Hash 7d5f8e2daf409299b279197b38dac8c88691e39b4dae6c2d9828c007892ee374
SimHash a8145950d5de

Groups

*

Rule Path Comment
Disallow /tmp/ these will soon disappear, don't index anything in this directory
Disallow /robotstxtorg.html don't index the file robotstxtorg.html

ia_archiver

Rule Path
Disallow /

Comments

  • EXAMPLE robots.txt file - see http://www.robotstxt.org/wc/norobots.html
  • block alexa http://pages.alexa.com/help/webmasters/#prevent
  • http://web.archive.org/collections/web.html