bensherman.co.uk
robots.txt
Robots Exclusion Standard data for bensherman.co.uk
Resource Scan
Scan Details
| Site Domain | bensherman.co.uk |
| Base Domain | bensherman.co.uk |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-11-15T21:45:44+00:00 |
| Next Scan | 2026-02-13T21:45:44+00:00 |
Last Successful Scan
| Scanned | 2025-06-25T19:16:39+00:00 |
| URL | https://bensherman.co.uk/robots.txt |
| Redirect | https://www.bensherman.co.uk/robots.txt |
| Redirect Domain | www.bensherman.co.uk |
| Redirect Base | bensherman.co.uk |
| Domain IPs | 104.18.18.239, 104.18.19.239, 2606:4700::6812:12ef, 2606:4700::6812:13ef |
| Redirect IPs | 104.18.18.239, 104.18.19.239, 2606:4700::6812:12ef, 2606:4700::6812:13ef |
| Response IP | 104.18.18.239 |
| Found | Yes |
| Hash | 523521dd0a63702e59db47466acf2ebb702b372d0f77cc26207ececa1833211c |
| SimHash | 6930c8609d43 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /popup* |
| Disallow | /zoom |
| Disallow | /Checkout/ |
| Disallow | /MembersArea/ |
| Disallow | /Product/Reminder |
| Disallow | /member* |
| Disallow | /search-results* |
| Disallow | /viewall?* |
| Disallow | /?f=t&c* |
| Disallow | /the-blog/* |
| Disallow | /blog/* |
| Disallow | /static/ |
| Disallow | /*?menu= |
| Disallow | /se-* |
| Disallow | /product/api/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.bensherman.co.uk/sitemap.xml |
Comments