flightgift.com
robots.txt

Robots Exclusion Standard data for flightgift.com

Resource Scan

Scan Details

Site Domain flightgift.com
Base Domain flightgift.com
Scan Status Ok
Last Scan2024-05-30T02:51:17+00:00
Next Scan 2024-06-29T02:51:17+00:00

Last Scan

Scanned2024-05-30T02:51:17+00:00
URL https://flightgift.com/robots.txt
Redirect https://www.flightgift.com/robots.txt
Redirect Domain www.flightgift.com
Redirect Base flightgift.com
Domain IPs 104.26.8.4, 104.26.9.4, 172.67.68.9, 2606:4700:20::681a:804, 2606:4700:20::681a:904, 2606:4700:20::ac43:4409
Redirect IPs 104.26.8.4, 104.26.9.4, 172.67.68.9, 2606:4700:20::681a:804, 2606:4700:20::681a:904, 2606:4700:20::ac43:4409
Response IP 172.67.68.9
Found Yes
Hash df9225563f16dbd5d19d18a6e6aa0d137a6edf38d533bbc1739415c1a2d05e46
SimHash 9aab6220d7d2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /de/kurzurlaub-mallorca/es-pontas-2/
Disallow /nl/faq/can-cancel-change-booking/
Disallow /weekend-in-barcelona-a-complete-guide/tibidabo/
Disallow /nl/things-you-must-pack-for-your-next-flight/annie-spratt-256172-unsplash-2/
Disallow /nl/blog/vier-oud-en-nieuw-in-frankrijk%20%EE%85%8D%20%EE%A2%9E%20%EE%A2%B6
Disallow /nl/blog/jouw-bier-vakantie-5-bestemmingen-voor-de-echte-kenners/
Disallow /nl/de-10-mooiste-plekken-in-italie-voor-de-meest-instagrammable-fotos/%20%EE%85%8D
Disallow /nl/comedy-bar_resized-2/
Disallow /es/amsterdam-dos-from-the-hotelgift-team/
Disallow /fr/?wordfence_lh=1&hid=5E7600B7C6ACD971E584108BC458B643%2F
Disallow /nl/la-rambla-2/
Disallow /fgc_airline_logos_desktop_latest-no-delta-2/
Disallow /es/blog/5-best-destinations-for-your-october-trip/
Disallow /faq/image-printable-pdf-upside-2/
Disallow /delayed-flight-make/jay-wen-361388-unsplash/
Disallow /es/top-5-christmas-activities-in-london-2018/
Disallow /es/blog/5-instagram-accounts-who-will-inspire-you-to-travel-more/
Disallow /de/wochenende-in-lille/carlton_hotel-a-lille-2/
Disallow /nl/blog/5-leuke-winterbestemmingen-in-europa-voor-koppels/winter-destinations-couple-2/
Disallow /nl/tibidabo-2/
Disallow /nl/about/fg_quote-icon-2/
Disallow /nl/transavia-1680x700/
Disallow /nl/?attachment_id=27516
Disallow /nl/fgc_airline_logos_desktop_new/
Disallow /es/3-6-2/
Disallow /blog/restaurants-in-bristol-you-cannot-miss/screenshot-2019-04-05-at-12-33-51/
Disallow /home/fg-home-header-2/
Disallow /fr/f3/
Disallow /nl/fgc_physical_example_no_text/
Disallow /es/img_0462/
Disallow /nl/english_fgc_physical_example_white_background/
Disallow /de/fg_demo_printout_de/
Disallow /es/fgc_airline_logos_mobile3/
Disallow /en/b2b2/
Disallow /en/fgc_airline_logos_desktop2/
Disallow /es/airlinelogos/
Disallow /es/home1318/
Disallow /nl/vogue/
Disallow /es/b2b4-2/
Disallow /es/?attachment_id=28011
Disallow /es/?attachment_id=27891
Disallow /en/enjoy/made-blue/
Disallow /fr/sab/
Disallow /de/blog/das-perfekte-wochenende-in-edinburgh/scottish-national-gallery-2/
Disallow /de/blog/weihnachten-in-frankreich-so-erlebst-du-das-beste-weihnachten-aller-zeiten/france_xmas_traditions-2/
Disallow /en/blog/christmas-activities-in-london-2018/
Disallow /fr/sab/
Disallow /en/home-2/
Disallow /us/home-2/
Disallow /nl/home-2/
Disallow /de/home-2/
Disallow /es/home-2/
Disallow /fr/home-2/
Disallow /blog/preview/