airfarewatchdog.com
robots.txt
Robots Exclusion Standard data for airfarewatchdog.com
Resource Scan
Scan Details
Site Domain | airfarewatchdog.com |
Base Domain | airfarewatchdog.com |
Scan Status | Ok |
Last Scan | 2024-09-21T14:32:09+00:00 |
Next Scan | 2024-09-28T14:32:09+00:00 |
Last Scan
Scanned | 2024-09-21T14:32:09+00:00 |
URL | https://airfarewatchdog.com/robots.txt |
Redirect | https://www.airfarewatchdog.com/robots.txt |
Redirect Domain | www.airfarewatchdog.com |
Redirect Base | airfarewatchdog.com |
Domain IPs | 34.224.54.119 |
Redirect IPs | 34.193.117.31, 35.170.222.220, 52.23.25.136 |
Response IP | 52.23.25.136 |
Found | Yes |
Hash | 4f3fac9a4742a4563a3a4a23bfbc042e1cde6b600b58d0bda51d7b83683bb700 |
SimHash | a80dbe94cf82 |
Groups
*
Rule | Path |
---|---|
Disallow | /ad-wrapper/ |
Disallow | /fare-alerts/* |
Disallow | /hotelPricing/* |
Disallow | /hotelContent/* |
Disallow | /track |
Disallow | /location |
Disallow | /airport-suggest |
Disallow | /edgecast-route-object.html |
Other Records
Field | Value |
---|---|
sitemap | https://www.airfarewatchdog.com/sitemap.xml |