petworks.com
robots.txt
Robots Exclusion Standard data for petworks.com
Resource Scan
Scan Details
| Site Domain | petworks.com |
| Base Domain | petworks.com |
| Scan Status | Ok |
| Last Scan | 2025-06-20T11:45:25+00:00 |
| Next Scan | 2025-06-27T11:45:25+00:00 |
Last Scan
| Scanned | 2025-06-20T11:45:25+00:00 |
| URL | https://petworks.com/robots.txt |
| Redirect | https://www.petworks.com/robots.txt |
| Redirect Domain | www.petworks.com |
| Redirect Base | petworks.com |
| Domain IPs | 104.26.6.58, 104.26.7.58, 172.67.69.91, 2606:4700:20::681a:63a, 2606:4700:20::681a:73a, 2606:4700:20::ac43:455b |
| Redirect IPs | 104.26.6.58, 104.26.7.58, 172.67.69.91, 2606:4700:20::681a:63a, 2606:4700:20::681a:73a, 2606:4700:20::ac43:455b |
| Response IP | 104.26.7.58 |
| Found | Yes |
| Hash | 944c33bc813e8f715518ef39622c3f980a8c728da4479a16eeeacbd8aa6425f1 |
| SimHash | 68108808adb2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*auth$ |
| Disallow | /search/autocomplete |
| Disallow | /login |
| Disallow | /en/login |
| Disallow | /en/request_quote |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.petworks.com/sitemap-listing.xml.gz |
| sitemap | https://www.petworks.com/sitemap-hire.xml.gz |
| sitemap | https://www.petworks.com/sitemap-user.xml.gz |
| sitemap | https://www.petworks.com/sitemap.xml.gz |