cycletrader.com
robots.txt
Robots Exclusion Standard data for cycletrader.com
Resource Scan
Scan Details
| Site Domain | cycletrader.com |
| Base Domain | cycletrader.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-11-08T04:45:47+00:00 |
| Next Scan | 2024-11-15T04:45:47+00:00 |
Last Successful Scan
| Scanned | 2024-08-01T01:16:38+00:00 |
| URL | https://cycletrader.com/robots.txt |
| Redirect | https://www.cycletrader.com/robots.txt |
| Redirect Domain | www.cycletrader.com |
| Redirect Base | cycletrader.com |
| Domain IPs | 44.205.37.65, 44.209.176.239, 54.165.164.24 |
| Redirect IPs | 2600:9000:2190:1600:15:5353:5fc0:93a1, 2600:9000:2190:4e00:15:5353:5fc0:93a1, 2600:9000:2190:600:15:5353:5fc0:93a1, 2600:9000:2190:9200:15:5353:5fc0:93a1, 2600:9000:2190:a600:15:5353:5fc0:93a1, 2600:9000:2190:a800:15:5353:5fc0:93a1, 2600:9000:2190:e000:15:5353:5fc0:93a1, 2600:9000:2190:f000:15:5353:5fc0:93a1, 52.222.214.114, 52.222.214.80, 52.222.214.81, 52.222.214.83 |
| Response IP | 65.9.112.83 |
| Found | Yes |
| Hash | cefd57452ca3969ff500acf541cff9f642e3341f8530439dd2782db7751774f4 |
| SimHash | 186878edce82 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*listing/http |
| Disallow | /find/listing/print/ |
| Disallow | /find/listing/photos/print/ |
| Disallow | /*email/seller/ |
| Disallow | /find/listing-emaillisting/ |
| Disallow | /find/website/ |
| Disallow | /find/website/listing/ |
| Disallow | */myt/ |
| Disallow | /find/dealers/print |
| Disallow | /motorcycle-dealers/print |
| Disallow | /advertiser-agreement |
| Disallow | /community-guidelines |
| Disallow | /mediainfo.html |
| Disallow | /security |
| Disallow | /dealer-search-results/ |
| Disallow | /*datalink$ |
| Disallow | /*MOBILE%3Dfalse |
| Disallow | /*vrsn%3Drss* |
| Disallow | /*lat%3D* |
| Disallow | /*long%3D* |
| Disallow | /*affilid%3D* |
| Disallow | /*media%3D* |
| Disallow | /*vrsn%3Diframe* |
| Disallow | */email/* |
| Disallow | /*make%3DTRIUMPH*2321194 |
| Disallow | /mileage%3D |
| Disallow | /fuelType%3D |
| Disallow | /grossVehicleWeight%3D |
| Disallow | /sleepingCapacity%3D |
| Disallow | /slideouts%3D |
| Disallow | /propellerHours%3D |
| Disallow | /airframeTime%3D |
| Disallow | /engineHours%3D |
| Disallow | /*Buy-Here |
| Disallow | /research/resources/motocross-resources |
| Disallow | */Gettiledata/* |
| Disallow | /gettiledata/topic/ |
| Disallow | /*model%3DRuckus%20ONLY%20%242499.00%20OTD%21%21%21%7C764857008* |
| Disallow | /*sort%3Dfeatured |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.cycletrader.com/sitemaps/cycle-sitemap-root.xml |