Log In Sign Up

ncra.org
robots.txt

Robots Exclusion Standard data for ncra.org

Resource Scan

Scan Details

Site Domain ncra.org
Base Domain ncra.org
Scan Status Ok
Last Scan2025-10-18T08:22:42+00:00
Next Scan 2025-11-17T08:22:42+00:00

Last Scan

Scanned2025-10-18T08:22:42+00:00
URL https://ncra.org/robots.txt
Redirect https://www.ncra.org:443/robots.txt
Redirect Domain www.ncra.org
Redirect Base ncra.org
Domain IPs 52.54.112.189
Redirect IPs 44.217.76.202, 54.164.69.38
Response IP 54.164.69.38
Found Yes
Hash b03e5e3cc6ed6f5a7060821380a49dc61f59a67db7ed65c4baa7c5f82f6997c1
SimHash 6817c975cff0

Groups

*

Rule Path
Disallow /Sitefinity
Disallow /sandbox
Disallow /search-results
Disallow /advanced-search

Other Records

Field Value
crawl-delay 120
Back to top

Comments

  • Do not delete /Sitefinity. Never any reason to allow indexing here
  • The same goes for sandbox
  • Also disallow search. We already have it set to "noindex", but keep getting googlebot hits
  • Also disallow advanced-search as an amazonbot is hitting it hard as of 04/08/2025. Wayne Floyd
Back to top