agrisupply.com
robots.txt
Robots Exclusion Standard data for agrisupply.com
Resource Scan
Scan Details
| Site Domain | agrisupply.com |
| Base Domain | agrisupply.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-09T01:41:54+00:00 |
| Next Scan | 2024-10-09T01:41:54+00:00 |
Last Successful Scan
| Scanned | 2024-07-18T21:21:51+00:00 |
| URL | https://agrisupply.com/robots.txt |
| Redirect | https://www.agrisupply.com:443/robots.txt |
| Redirect Domain | www.agrisupply.com |
| Redirect Base | agrisupply.com |
| Domain IPs | 75.2.73.217 |
| Redirect IPs | 13.224.14.39, 13.224.14.47, 13.224.14.8, 13.224.14.91 |
| Response IP | 3.160.246.72 |
| Found | Yes |
| Hash | abf1343ed753c83e829eb11c22df5fd469df13fbcf34c6882a0ba2eb481516ca |
| SimHash | 09009f63c171 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /controls/ |
| Disallow | /logs/ |
| Disallow | /bin/ |
| Disallow | /cart.aspx |
| Disallow | /checkout.aspx |
| Disallow | /summary.aspx |
| Disallow | /store/* |
| Disallow | /tellafriendpopup.aspx?* |
| Disallow | /pop_tell_friend.asp?* |
| Disallow | /pop_product_inventory.asp?* |
| Disallow | /equipment-supplies/* |
| Disallow | /heavy-duty-leather-gloves/p/G87471/* |
| Disallow | *brand%3D* |
| Disallow | *keyword%3D* |
| Disallow | *category_1%3D* |
| Disallow | *searchcenter.aspx* |
| Disallow | *hawkcustom%3D* |
| Disallow | *sort%3D* |
| Disallow | *tire_rim_diameter%3D* |
| Disallow | *tire_overall_diameter%3D* |
| Disallow | *tire_load_range%3D* |
| Disallow | *tire_section_width%3D* |
| Disallow | */filter/* |
| Disallow | /pop_product_inventory.asp* |
| Disallow | /search.asp* |
| Disallow | /pop_tell_friend.asp* |
| Disallow | /pop_product_inventory.asp* |
| Disallow | *MBCH_1001_* |
| Disallow | *%26sid%3D%26eid%3D* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.agrisupply.com/sitemap.xml |