starkbros.com
robots.txt
Robots Exclusion Standard data for starkbros.com
Resource Scan
Scan Details
| Site Domain | starkbros.com |
| Base Domain | starkbros.com |
| Scan Status | Ok |
| Last Scan | 2024-09-20T12:44:36+00:00 |
| Next Scan | 2024-09-27T12:44:36+00:00 |
Last Scan
| Scanned | 2024-09-20T12:44:36+00:00 |
| URL | https://starkbros.com/robots.txt |
| Redirect | https://www.starkbros.com/robots.txt |
| Redirect Domain | www.starkbros.com |
| Redirect Base | starkbros.com |
| Domain IPs | 20.9.25.138 |
| Redirect IPs | 20.9.25.138 |
| Response IP | 20.9.25.138 |
| Found | Yes |
| Hash | 96674cf28a598d935e8f7942fff250b8b9c71ef258f69e67fa4280b445343428 |
| SimHash | dc44307a8e0b |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /META-INF/ |
| Disallow | /WEB-INF/ |
| Disallow | /favicon.ico |
| Disallow | /error/ |
| Disallow | /checkout/ |
| Disallow | /about/request-a-catalog |
| Disallow | /shopping-cart/ |
| Disallow | /account/ |
| Disallow | /resources/ |
| Disallow | /api/ |
| Disallow | /growing-guide/printed-guides/ |
| Disallow | /*wicket%3Ainterface*$ |
| Disallow | /*IBehavior*$ |
| Disallow | /*ILinkListener*$ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.starkbros.com/datafeeds/sitemap.xml |