Log In Sign Up

ctbar.org
robots.txt

Robots Exclusion Standard data for ctbar.org

Resource Scan

Scan Details

Site Domain ctbar.org
Base Domain ctbar.org
Scan Status Ok
Last Scan2025-10-12T12:06:14+00:00
Next Scan 2025-11-11T12:06:14+00:00

Last Scan

Scanned2025-10-12T12:06:14+00:00
URL https://www.ctbar.org/robots.txt
Domain IPs 18.208.242.254, 35.170.22.248
Response IP 35.170.22.248
Found Yes
Hash 5e06bce84516666ac4e9a62ad56452e2c35c4100c896cf7cdf11ac9a71c62592
SimHash 689e993367f0

Groups

*

Rule Path
Disallow /Sitefinity
Disallow /sandbox
Disallow /search-results

Other Records

Field Value
crawl-delay 120
Back to top

Comments

  • Do not delete /Sitefinity. Never any reason to allow indexing here
  • The same goes for sandbox
  • Also disallow search. We already have it set to "noindex", but keep getting googlebot hits
  • At launch, remove this Disallow. Add any other folders that should not be indexed
Back to top