insaonline.org
robots.txt

Robots Exclusion Standard data for insaonline.org

Resource Scan

Scan Details

Site Domain insaonline.org
Base Domain insaonline.org
Scan Status Ok
Last Scan2025-10-20T20:13:50+00:00
Next Scan 2025-11-19T20:13:50+00:00

Last Scan

Scanned2025-10-20T20:13:50+00:00
URL https://insaonline.org/robots.txt
Redirect https://www.insaonline.org:443/robots.txt
Redirect Domain www.insaonline.org
Redirect Base insaonline.org
Domain IPs 52.54.112.189
Redirect IPs 3.213.52.137, 3.228.199.49
Response IP 3.228.199.49
Found Yes
Hash 19ecd75c1640d90895e2a85801f26a37eb63510a837060fc76472049434f95f4
SimHash 6915d9754ff0

Groups

*

Rule Path
Disallow /Sitefinity
Disallow /sandbox
Disallow /search-results

Other Records

Field Value
crawl-delay 120

Comments

  • Do not delete /Sitefinity. Never any reason to allow indexing here
  • The same goes for sandbox
  • Also disallow search. We already have it set to "noindex", but keep getting googlebot hits