machinerypete.com
robots.txt
Robots Exclusion Standard data for machinerypete.com
Resource Scan
Scan Details
| Site Domain | machinerypete.com |
| Base Domain | machinerypete.com |
| Scan Status | Ok |
| Last Scan | 2024-06-20T18:53:10+00:00 |
| Next Scan | 2024-06-27T18:53:10+00:00 |
Last Scan
| Scanned | 2024-06-20T18:53:10+00:00 |
| URL | https://machinerypete.com/robots.txt |
| Redirect | https://assets.machinerypete.com/uploads/sitemaps/robots.txt |
| Redirect Domain | assets.machinerypete.com |
| Redirect Base | machinerypete.com |
| Domain IPs | 34.149.14.124 |
| Redirect IPs | 13.33.30.104, 13.33.30.122, 13.33.30.41, 13.33.30.77 |
| Response IP | 13.33.30.122 |
| Found | Yes |
| Hash | af8fea7259b2e7dc3018f5855dbcacfc1079217778ef0540e7983b1c00f00191 |
| SimHash | fb1e9960cff7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cms |
| Disallow | /assets/21 |
| Disallow | /ajax |
| Disallow | /auction_results |
| Disallow | /auction_events |
| Disallow | /mobile |
| Disallow | /specs |
| Disallow | /users |
| Disallow | /conversations |
| Disallow | *custom_search%3D |
| Disallow | *dynamic_site_fields%5B |
| Disallow | *hours%5B |
| Disallow | *limit%3D |
| Disallow | *manual_sort%3D |
| Disallow | *old_location_str%3D |
| Disallow | *price%5B |
| Disallow | *sort_term%3D |
| Disallow | *year%5B |
| Disallow | *zip_miles%3D |
| Disallow | *zip_code%3D |
| Disallow | */jserrors/aggregate |
| Disallow | */metrics/aggregate |
| Disallow | */spa/aggregate |
| Disallow | */page_action/aggregate |
| Disallow | */page_view_event/aggregate |
| Disallow | */session_trace/aggregate |
| Disallow | */ajax/aggregate |
| Disallow | */page_view_timing/aggregate |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.machinerypete.com/generic_sitemap_index.xml.gz |
| sitemap | https://www.machinerypete.com/blogs_index.xml.gz |
| sitemap | https://www.machinerypete.com/tractors_index.xml.gz |
| sitemap | https://www.machinerypete.com/harvesting_index.xml.gz |
| sitemap | https://www.machinerypete.com/planting_index.xml.gz |
| sitemap | https://www.machinerypete.com/applicators_index.xml.gz |
| sitemap | https://www.machinerypete.com/hay-and-forage_index.xml.gz |
| sitemap | https://www.machinerypete.com/tillage_index.xml.gz |
| sitemap | https://www.machinerypete.com/grain-handling_index.xml.gz |
| sitemap | https://www.machinerypete.com/loaders-and-lifts_index.xml.gz |
| sitemap | https://www.machinerypete.com/trucks-and-trailers_index.xml.gz |
| sitemap | https://www.machinerypete.com/livestock-equipment_index.xml.gz |
| sitemap | https://www.machinerypete.com/specialty-crops_index.xml.gz |
| sitemap | https://www.machinerypete.com/other_index.xml.gz |
| sitemap | https://www.machinerypete.com/construction_index.xml.gz |
| sitemap | https://www.machinerypete.com/lawn-and-garden_index.xml.gz |
| sitemap | https://www.machinerypete.com/used_tractors_index.xml.gz |
| sitemap | https://www.machinerypete.com/listings_index.xml.gz |