charleskeith.co.uk
robots.txt
Robots Exclusion Standard data for charleskeith.co.uk
Resource Scan
Scan Details
| Site Domain | charleskeith.co.uk |
| Base Domain | charleskeith.co.uk |
| Scan Status | Ok |
| Last Scan | 2025-12-01T22:21:09+00:00 |
| Next Scan | 2025-12-15T22:21:09+00:00 |
Last Scan
| Scanned | 2025-12-01T22:21:09+00:00 |
| URL | https://charleskeith.co.uk/robots.txt |
| Redirect | https://www.charleskeith.co.uk/robots.txt |
| Redirect Domain | www.charleskeith.co.uk |
| Redirect Base | charleskeith.co.uk |
| Domain IPs | 13.248.160.137, 76.223.34.124 |
| Redirect IPs | 104.18.26.112, 104.18.27.112 |
| Response IP | 104.18.26.112 |
| Found | Yes |
| Hash | c6912dccb740241f3573cca478c6cf3e788c337c98f8adfdd720f136c03aa9cb |
| SimHash | 67155f5676fa |
Groups
claude-web
| Rule | Path |
|---|---|
| Disallow | *contentsearch* |
| Disallow | /*pmin* |
| Disallow | /*pmax* |
| Disallow | /*prefn1* |
| Disallow | /*prefn2* |
| Disallow | /*prefn3* |
| Disallow | /*prefn4* |
| Disallow | /*prefv1* |
| Disallow | /*prefv2* |
| Disallow | /*prefv3* |
| Disallow | /*prefv4* |
| Disallow | /*srule* |
| Disallow | *search?q=* |
| Disallow | /cart |
| Disallow | /*/cart |
| Disallow | /*/orderconfirmation |
| Disallow | /*/revieworder |
| Disallow | /*/thank-you |
| Disallow | /*/check-out |
| Disallow | /login |
| Disallow | *?q=* |
| Disallow | /CSRF-Fail |
| Disallow | /on/demandware.store/* |
| Disallow | */Search-Show |
| Disallow | /*/profile |
| Disallow | /*/account |
| Disallow | /*/orders |
| Disallow | /*/order-details* |
| Disallow | /*/my-preferences |
| Disallow | /*/addressbook |
| Disallow | /*/wallet |
| Allow | /*/wallets |
| Disallow | /*/membership |
| Allow | /*/thank-you-mothers-day |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.charleskeith.co.uk/gb/sitemap_index.xml |