cheapbooks.com
robots.txt
Robots Exclusion Standard data for cheapbooks.com
Resource Scan
Scan Details
| Site Domain | cheapbooks.com |
| Base Domain | cheapbooks.com |
| Scan Status | Ok |
| Last Scan | 2024-10-12T02:07:10+00:00 |
| Next Scan | 2024-10-19T02:07:10+00:00 |
Last Scan
| Scanned | 2024-10-12T02:07:10+00:00 |
| URL | http://cheapbooks.com/robots.txt |
| Domain IPs | 143.198.141.174 |
| Response IP | 143.198.141.174 |
| Found | Yes |
| Hash | 7199d2707d91e7fd05a80e5a36664739854999bdd7d9b1bc93cffbf62210b290 |
| SimHash | ca5d5ce24610 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /click*.cgi |
| Disallow | /price*.cgi |
| Disallow | /pics/* |
| Disallow | /thumbnails/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://cdn.cheapbooks.com/sub/sitemaps/index.xml |
Comments