nest.co.uk
robots.txt
Robots Exclusion Standard data for nest.co.uk
Resource Scan
Scan Details
| Site Domain | nest.co.uk |
| Base Domain | nest.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-10-19T14:47:58+00:00 |
| Next Scan | 2024-11-18T14:47:58+00:00 |
Last Scan
| Scanned | 2024-10-19T14:47:58+00:00 |
| URL | https://nest.co.uk/robots.txt |
| Redirect | https://www.nest.co.uk/robots.txt |
| Redirect Domain | www.nest.co.uk |
| Redirect Base | nest.co.uk |
| Domain IPs | 35.233.64.221 |
| Redirect IPs | 35.233.64.221 |
| Response IP | 35.233.64.221 |
| Found | Yes |
| Hash | 0b55c5dc65b33b7a9f1f3b5b9cbf014829d0ed1542c09239df4a7effa20565b0 |
| SimHash | 6c84dbb2ea12 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /basket |
| Disallow | /login |
| Disallow | /logout |
| Disallow | /search |
| Disallow | /json_account |
| Disallow | /json_basket |
| Disallow | /json_listing |
| Disallow | /json_order |
| Disallow | /json_subscribe |
| Disallow | /json_wishlist |
| Disallow | /product/*?currency* |
| Disallow | /contact?product* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.nest.co.uk/sitemap.xml |