classifieds.theadvocate.com
robots.txt
Robots Exclusion Standard data for classifieds.theadvocate.com
Resource Scan
Scan Details
| Site Domain | classifieds.theadvocate.com |
| Base Domain | theadvocate.com |
| Scan Status | Ok |
| Last Scan | 2024-11-04T01:33:55+00:00 |
| Next Scan | 2024-12-04T01:33:55+00:00 |
Last Scan
| Scanned | 2024-11-04T01:33:55+00:00 |
| URL | https://classifieds.theadvocate.com/robots.txt |
| Domain IPs | 34.234.152.169, 54.165.237.149 |
| Response IP | 54.165.237.149 |
| Found | Yes |
| Hash | bbeb794b7209442cb3f2d39995f38d03aab9953d0365c057694a2064cf931c63 |
| SimHash | 931ecd074a81 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /marketplace/login/ |
| Disallow | /marketplace/registration/ |
| Disallow | /marketplace/search/ |
| Disallow | /marketplace/account/ |
| Disallow | /marketplace/basket/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://classifieds.theadvocate.com/marketplace/sitemap.xml |
Comments