de9straatjes.nl
robots.txt

Robots Exclusion Standard data for de9straatjes.nl

Resource Scan

Scan Details

Site Domain de9straatjes.nl
Base Domain de9straatjes.nl
Scan Status Ok
Last Scan5/19/2025, 5:08:12 PM
Next Scan 6/18/2025, 5:08:12 PM

Last Scan

Scanned5/19/2025, 5:08:12 PM
URL https://de9straatjes.nl/robots.txt
Domain IPs 141.138.168.120, 2a03:3c00:a002:175::1006
Response IP 141.138.168.120
Found Yes
Hash 92660a43463162577e4632491c6a911474b8e03ae58f67c075c881abce98329c
SimHash e90909548ed3

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /webstat/

Other Records

Field Value
sitemap https://de9straatjes.nl/sitemap.xml

Comments

  • www.robotstxt.org
  • code.google.com/web/controlcrawlindex