putneytravel.co.uk
robots.txt
Robots Exclusion Standard data for putneytravel.co.uk
Resource Scan
Scan Details
| Site Domain | putneytravel.co.uk |
| Base Domain | putneytravel.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-09-21T07:33:55+00:00 |
| Next Scan | 2024-10-21T07:33:55+00:00 |
Last Scan
| Scanned | 2024-09-21T07:33:55+00:00 |
| URL | https://putneytravel.co.uk/robots.txt |
| Domain IPs | 104.21.20.122, 172.67.192.217, 2606:4700:3034::6815:147a, 2606:4700:3037::ac43:c0d9 |
| Response IP | 104.21.20.122 |
| Found | Yes |
| Hash | cedb1420f4c1f2ba2993f3b572cc77c46cb4f063aa0a8700e126174491d4e2b0 |
| SimHash | 60648bc192ff |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /get-holiday-type-destination |
| Disallow | /get-holiday-type-combo |
| Disallow | /get-ship-month-combo |
| Disallow | /holiday-offer |
| Disallow | /compared-property |
| Disallow | /get-featured-record |
| Disallow | /get-price-list |
| Disallow | /holiday-list |
| Disallow | /holiday-type |
| Disallow | /holiday-type-featured |
| Disallow | /make-inquiry |
| Disallow | /make-inquiry-compared-property |
| Disallow | /offer-detail |
| Disallow | /offers |
| Disallow | /property-detail |
| Disallow | /browse-by-activity/activity-child-one/ |
| Disallow | /browse-by-activity/activity-child-two |
| Disallow | /browse-by-destination |
| Disallow | /browse-by-destination/destination-child-one |
| Disallow | /browse-by-destination/destination-child-two |
| Disallow | /browse-by-theme |
| Disallow | /make-inquiry-compared-property |
| Disallow | /get-grid-price |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1000 |
googlebot
googlebot-image
mediapartners-google
msnbot
msnbot-media
slurp
yahoo-blogs
yahoo-mmcrawler
| Rule | Path |
|---|---|
| Disallow | /get-holiday-type-destination |
| Disallow | /get-holiday-type-combo |
| Disallow | /get-ship-month-combo |
| Disallow | /holiday-offer |
| Disallow | /compared-property |
| Disallow | /get-featured-record |
| Disallow | /get-price-list |
| Disallow | /holiday-list |
| Disallow | /holiday-type |
| Disallow | /holiday-type-featured |
| Disallow | /make-inquiry |
| Disallow | /make-inquiry-compared-property |
| Disallow | /offer-detail |
| Disallow | /offers |
| Disallow | /property-detail |
| Disallow | /browse-by-activity/activity-child-one/ |
| Disallow | /browse-by-activity/activity-child-two |
| Disallow | /browse-by-destination |
| Disallow | /browse-by-destination/destination-child-one |
| Disallow | /browse-by-destination/destination-child-two |
| Disallow | /browse-by-theme |
| Disallow | /make-inquiry-compared-property |
| Disallow | /get-grid-price |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
Comments