caoago.com
robots.txt
Robots Exclusion Standard data for caoago.com
Resource Scan
Scan Details
| Site Domain | caoago.com |
| Base Domain | caoago.com |
| Scan Status | Ok |
| Last Scan | 2024-09-29T21:03:09+00:00 |
| Next Scan | 2024-10-06T21:03:09+00:00 |
Last Scan
| Scanned | 2024-09-29T21:03:09+00:00 |
| URL | https://www.caoago.com/robots.txt |
| Domain IPs | 2404:6800:4003:c03::79, 74.125.24.121 |
| Response IP | 74.125.24.121 |
| Found | Yes |
| Hash | deb217112ecac61b407129897e6d36f3d451fe6b3c8655e6a5255c947fb6baff |
| SimHash | 4b5cba40c691 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search?updated-min= |
| Disallow | /search?updated-max= |
| Disallow | /search/label/*?updated-min= |
| Disallow | /search/label/*?updated-max= |
| Disallow | /search |
| Disallow | /*?updated-max=* |
| Disallow | *archive.html |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.caoago.com/atom.xml?redirect=false&start-index=1&max-results=500 |