airfarewatchdog.com
robots.txt
Robots Exclusion Standard data for airfarewatchdog.com
Resource Scan
Scan Details
| Site Domain | airfarewatchdog.com |
| Base Domain | airfarewatchdog.com |
| Scan Status | Ok |
| Last Scan | 2024-06-15T02:40:28+00:00 |
| Next Scan | 2024-06-22T02:40:28+00:00 |
Last Scan
| Scanned | 2024-06-15T02:40:28+00:00 |
| URL | https://airfarewatchdog.com/robots.txt |
| Redirect | https://www.airfarewatchdog.com/robots.txt |
| Redirect Domain | www.airfarewatchdog.com |
| Redirect Base | airfarewatchdog.com |
| Domain IPs | 34.224.54.119 |
| Redirect IPs | 23.54.118.34, 23.54.118.46 |
| Response IP | 23.52.171.202 |
| Found | Yes |
| Hash | 4f3fac9a4742a4563a3a4a23bfbc042e1cde6b600b58d0bda51d7b83683bb700 |
| SimHash | a80dbe94cf82 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ad-wrapper/ |
| Disallow | /fare-alerts/* |
| Disallow | /hotelPricing/* |
| Disallow | /hotelContent/* |
| Disallow | /track |
| Disallow | /location |
| Disallow | /airport-suggest |
| Disallow | /edgecast-route-object.html |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.airfarewatchdog.com/sitemap.xml |