comics.ha.com
robots.txt
Robots Exclusion Standard data for comics.ha.com
Resource Scan
Scan Details
| Site Domain | comics.ha.com |
| Base Domain | ha.com |
| Scan Status | Ok |
| Last Scan | 2024-11-11T08:23:25+00:00 |
| Next Scan | 2024-11-25T08:23:25+00:00 |
Last Scan
| Scanned | 2024-11-11T08:23:25+00:00 |
| URL | https://comics.ha.com/robots.txt |
| Domain IPs | 104.18.38.129, 172.64.149.127, 2606:4700:4400::6812:2681, 2606:4700:4400::ac40:957f |
| Response IP | 104.18.38.129 |
| Found | Yes |
| Hash | c65595832642217e95e12e20d4cb16497cd5020306e7756c61847c549a19b68e |
| SimHash | 1324dec1c6ea |
Groups
mozilla/5.0 (compatible; gluten free crawler/1.0; +http://glutenfreepleasure.com/)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /c/greetings.zx |
| Disallow | /c/ha-rebuttal.zx |
| Disallow | /c/cart/ |
| Disallow | /c/error/invalid-lot.zx |
| Disallow | /c/my/collection/ |
| Disallow | /c/bid-review.zx |
| Disallow | /c/bid.zx |
| Disallow | /c/prlink.zx |
| Disallow | /c/error/maintenance.zx |
| Disallow | /c/print-prices-realized.zx |
| Disallow | /c/catalog-print.zx |
| Disallow | /c/s/d/frontmatter/ |
| Disallow | /c/ecatalog.zx |
| Disallow | /c/auction-home.zx |
| Disallow | /c/invoice/ |
| Disallow | /c/rank.zx |
| Disallow | /c/moto/ |
| Disallow | /c/my/wantlist.zx |
| Disallow | /c/webservices/ebay-banner.zx |
| Disallow | /c/ebay-results.zx |
| Disallow | /c/phone-bid.zx |
| Disallow | /live.zx |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://comics.ha.com/sitemap.zx |