capcvet.org
robots.txt

Robots Exclusion Standard data for capcvet.org

Resource Scan

Scan Details

Site Domain capcvet.org
Base Domain capcvet.org
Scan Status Ok
Last Scan2025-12-09T03:30:59+00:00
Next Scan 2025-12-16T03:30:59+00:00

Last Scan

Scanned2025-12-09T03:30:59+00:00
URL https://capcvet.org/robots.txt
Domain IPs 104.21.76.252, 172.67.202.133, 2606:4700:3031::ac43:ca85, 2606:4700:3035::6815:4cfc
Response IP 104.21.76.252
Found Yes
Hash cc8a1c1d6daade82fa4223aa26d9177ce5c39c2a9d5b28b9b7d9199a16b6a82a
SimHash 637a19126ab2

Groups

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env

Other Records

Field Value
sitemap https://capcvet.org/sitemaps-1-sitemap.xml
sitemap https://petdiseasealerts.org/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://capcvet.org/
  • live - don't allow web crawlers to index cpresources/ or vendor/
  • Algolia-Crawler-Verif: 83EDDFD3F8E8AF1C