petfolk.com
robots.txt

Robots Exclusion Standard data for petfolk.com

Resource Scan

Scan Details

Site Domain petfolk.com
Base Domain petfolk.com
Scan Status Ok
Last Scan2025-10-10T21:44:12+00:00
Next Scan 2025-10-24T21:44:12+00:00

Last Scan

Scanned2025-10-10T21:44:12+00:00
URL https://petfolk.com/robots.txt
Domain IPs 104.26.14.100, 104.26.15.100, 172.67.70.156, 2606:4700:20::681a:e64, 2606:4700:20::681a:f64, 2606:4700:20::ac43:469c
Response IP 172.67.70.156
Found Yes
Hash f9fe42fc8f194155d4db73294c114d911f540f2ab5150c639c034049bbe0463c
SimHash a05039474d81

Groups

*

Rule Path
Disallow /book/*
Disallow /book?*
Allow /assets/
Allow /static/
Allow /services/
Allow /blog/
Allow /locations/

Other Records

Field Value
sitemap https://petfolk.com/sitemap.xml

Comments

  • See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • Disallow acquisition booking funnel
  • Allow bots to access essential resources
  • Sitemap location