airfarewatchdog.co.uk
robots.txt
Robots Exclusion Standard data for airfarewatchdog.co.uk
Resource Scan
Scan Details
| Site Domain | airfarewatchdog.co.uk |
| Base Domain | airfarewatchdog.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-09-27T19:01:07+00:00 |
| Next Scan | 2024-10-04T19:01:07+00:00 |
Last Scan
| Scanned | 2024-09-27T19:01:07+00:00 |
| URL | https://airfarewatchdog.co.uk/robots.txt |
| Redirect | https://www.airfarewatchdog.com/robots.txt |
| Redirect Domain | www.airfarewatchdog.com |
| Redirect Base | airfarewatchdog.com |
| Domain IPs | 34.224.54.119 |
| Redirect IPs | 104.18.18.87, 104.18.19.87, 2606:4700::6812:1257, 2606:4700::6812:1357 |
| Response IP | 104.18.19.87 |
| Found | Yes |
| Hash | 4f3fac9a4742a4563a3a4a23bfbc042e1cde6b600b58d0bda51d7b83683bb700 |
| SimHash | a80dbe94cf82 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ad-wrapper/ |
| Disallow | /fare-alerts/* |
| Disallow | /hotelPricing/* |
| Disallow | /hotelContent/* |
| Disallow | /track |
| Disallow | /location |
| Disallow | /airport-suggest |
| Disallow | /edgecast-route-object.html |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.airfarewatchdog.com/sitemap.xml |