shop.deere.com
robots.txt
Robots Exclusion Standard data for shop.deere.com
Resource Scan
Scan Details
Site Domain | shop.deere.com |
Base Domain | deere.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-08-31T14:16:48+00:00 |
Next Scan | 2024-11-29T14:16:48+00:00 |
Last Successful Scan
Scanned | 2024-01-12T13:57:46+00:00 |
URL | https://shop.deere.com/robots.txt |
Redirect | https://shop.deere.com/us/robots.txt |
Domain IPs | 65.8.11.102, 65.8.11.24, 65.8.11.4, 65.8.11.97 |
Response IP | 18.66.218.81 |
Found | Yes |
Hash | 079147abf1438f7457f4e522986bbc21309acbdc05d2a039a96909f4c911e46a |
SimHash | e852d79fedf8 |
Groups
*
Rule | Path |
---|---|
Disallow | /us/cart |
Disallow | /us/checkout |
Disallow | /us/my-account |
Disallow | /us/jdb2cstorefront |
Other Records
Field | Value | Comment |
---|---|---|
crawl-delay | 10 | 10 seconds between page requests |
Other Records
Field | Value |
---|---|
sitemap | /us/sitemap.xml |
Warnings
- `request-rate` is not a known field.
- `visit-time` is not a known field.
Comments