thecatterycc.org
robots.txt

Robots Exclusion Standard data for thecatterycc.org

Resource Scan

Scan Details

Site Domain thecatterycc.org
Base Domain thecatterycc.org
Scan Status Ok
Last Scan2025-10-25T18:08:10+00:00
Next Scan 2025-11-01T18:08:10+00:00

Last Scan

Scanned2025-10-25T18:08:10+00:00
URL https://thecatterycc.org/robots.txt
Redirect https://www.thecatterycc.org/robots.txt
Redirect Domain www.thecatterycc.org
Redirect Base thecatterycc.org
Domain IPs 199.34.228.42
Redirect IPs 199.34.228.42
Response IP 199.34.228.42
Found Yes
Hash a2dc0c50738c22c12f5a764499bdc28c750d870fff6de347bd369ec6b4d36011
SimHash 8954dc6427d3

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /about.html
Disallow /adopt.html
Disallow /clinic.html
Disallow /get-involved.html

Other Records

Field Value
sitemap https://www.thecatterycc.org/sitemap.xml