fairfx.com
robots.txt

Robots Exclusion Standard data for fairfx.com

Resource Scan

Scan Details

Site Domain fairfx.com
Base Domain fairfx.com
Scan Status Ok
Last Scan2024-05-26T07:37:30+00:00
Next Scan 2024-06-25T07:37:30+00:00

Last Scan

Scanned2024-05-26T07:37:30+00:00
URL https://fairfx.com/robots.txt
Redirect https://www.fairfx.com:443/robots.txt
Redirect Domain www.fairfx.com
Redirect Base fairfx.com
Domain IPs 18.133.35.115, 18.134.171.171, 3.10.252.196
Redirect IPs 108.156.133.127, 108.156.133.20, 108.156.133.73, 108.156.133.92
Response IP 108.156.133.127
Found Yes
Hash f8db3c95eb929b25ab1d6250a061a71b2a9165e1b5c88c1641897bb9358c88ca
SimHash 9d64ddd20757

Groups

*

Rule Path
Disallow /opodo/
Disallow /matalan-offer/
Disallow /travelmail/
Disallow /refer/
Disallow /l/
Disallow /aberdeen-airport-guide-card/
Disallow /aberdeen-airport-guide-cash/
Disallow /adwords-ispy/
Disallow /adwords-ispy-dollar/
Disallow /adwords-ispy-euro/
Disallow /affiliatefuture/
Disallow /affiliatefuture-dollar/
Disallow /affiliatefuture-euro/
Disallow /affinity/
Disallow /affinity-dollar/
Disallow /affinity-euro/
Disallow /airportmag/
Disallow /alpharooms/
Disallow /amber/
Disallow /beatthatquote/
Disallow /belfast-airport-guide-card/
Disallow /belfast-airport-guide-cash/
Disallow /birmingham-airport-guide-card/
Disallow /birmingham-airport-guide-cash/
Disallow /blackpool-airport-guide-card/
Disallow /blackpool-airport-guide-cash/
Disallow /bluelight/
Disallow /bristol-airport-guide-card/
Disallow /bristol-airport-guide-cash/
Disallow /campamerica/
Disallow /cardiff-airport-guide-card/
Disallow /cardiff-airport-guide-cash/
Disallow /carterphillips/
Disallow /carterphillips-dollar/
Disallow /carterphillips-euro/
Disallow /comparecurrencycards/
Disallow /comparemoneytransfer-card/
Disallow /comparemoneytransfer-cash/
Disallow /compareprepaid/
Disallow /compareswitchandsave/
Disallow /comparetravelmoney/
Disallow /confused/
Disallow /dibb/
Disallow /digitallook/
Disallow /doncaster-sheffield-airport-guide-card/
Disallow /doncaster-sheffield-airport-guide-cash/
Disallow /dtz/
Disallow /east-midlands-airport-guide-card/
Disallow /east-midlands-airport-guide-cash/
Disallow /easyBus/
Disallow /easybus/
Disallow /edinburgh-airport-guide-card/
Disallow /edinburgh-airport-guide-cash/
Disallow /fool/
Disallow /fpa/
Disallow /freedominsurance/
Disallow /gatwick-airport-guide-card/
Disallow /gatwick-airport-guide-cash/
Disallow /gitecompare/
Disallow /glasgow-airport-guide-card/
Disallow /glasgow-airport-guide-cash/
Disallow /gocompare/
Disallow /guardian/
Disallow /haringeyfamilies/
Disallow /heathrow-airport-guide-card/
Disallow /heathrow-airport-guide-cash/
Disallow /hiltonmiami/
Disallow /holidayextrascard/
Disallow /holidayextrascard-confirmation/
Disallow /holidayextrascard-domain/
Disallow /holidayextrascard-email/
Disallow /holidayextrascash/
Disallow /hostelworld/
Disallow /humberside-airport-guide-card/
Disallow /humberside-airport-guide-cash/
Disallow /italianproperties/
Disallow /italymag/
Disallow /laband/
Disallow /leadingrates/
Disallow /leeds-airport-guide-card/
Disallow /leeds-airport-guide-cash/
Disallow /liverpool-airport-guide-card/
Disallow /liverpool-airport-guide-cash/
Disallow /london-city-airport-guide-card/
Disallow /london-city-airport-guide-cash/
Disallow /lovemoney/
Disallow /luton-airport-guide-card/
Disallow /luton-airport-guide-cash/
Disallow /mailonline/
Disallow /manchester-airport-guide-card/
Disallow /manchester-airport-guide-cash/
Disallow /matalan/
Disallow /moneyhighstreet/
Disallow /moneymagpie/
Disallow /moneynet/
Disallow /moneysupermarket-cash/
Disallow /moneytip/
Disallow /mrandmrssmith/
Disallow /msn-ispy/
Disallow /msn-ispy-dollar/
Disallow /msn-ispy-euro/
Disallow /ncp/
Disallow /neverblue/
Disallow /neverblue-dollar/
Disallow /neverblue-euro/
Disallow /newcastle-airport-guide-card/
Disallow /newcastle-airport-guide-cash/
Disallow /opodo/
Disallow /prestwick-airport-guide-card/
Disallow /prestwick-airport-guide-cash/
Disallow /profitistic/
Disallow /quantum/
Disallow /raileurope/
Disallow /servisair/
Disallow /shankly/
Disallow /skymag/
Disallow /socialnetworking/
Disallow /southampton-airport-guide-card/
Disallow /southampton-airport-guide-cash/
Disallow /mirror/
Disallow /stanstead-airport-guide-card/
Disallow /stanstead-airport-guide-cash/
Disallow /studentdiscounts/
Disallow /studentdiscounts-dollar/
Disallow /studentdiscounts-euro/
Disallow /students/
Disallow /teesside-airport-guide-card/
Disallow /teesside-airport-guide-cash/
Disallow /thisismoney/
Disallow /tradedoubler/
Disallow /travelguru/
Disallow /tripadvisor/
Disallow /tripadvisor-dollar/
Disallow /tripadvisor-euro/
Disallow /uswitch/
Disallow /uswitch-email/
Disallow /wanderlust/
Disallow /whichwaytopay/
Disallow /whichwaytopay-dollar/
Disallow /whichwaytopay-euro/
Disallow /womansweekly/
Disallow /worldtravelguide/
Disallow /yotel/
Disallow /ysm-ispy/
Disallow /ysm-ispy-dollar/
Disallow /ysm-ispy-euro/
Disallow /myfinances/
Disallow /apcoa/
Disallow /ebookers/
Disallow /whichwaytopaycash/
Disallow /fairinvestment/
Disallow /compareandsave/
Disallow /uswitch1/
Disallow /uswitch2/
Disallow /uswitch3/
Disallow /uswitch4/
Disallow /uswitch5/
Disallow /clcmembers/
Disallow /ukcreditcards/
Disallow /resorthoppa-offer/
Disallow /resorthoppa-offa/
Disallow /lowcosttaxis-offer/
Disallow /lowcostholidays-offer/
Disallow /countrysidealliance/
Disallow /twitter/
Disallow /thetraveleditor/
Disallow /gmtv/
Disallow /express/
Disallow /moneyexpert/
Disallow /loanrunner/
Disallow /wwtp-card/
Disallow /wwtp-cash/
Disallow /survivingthecreditcrunch/
Disallow /price-watchdog/
Disallow /prepaidcreditdebitcards/
Disallow /prepaycreditdebitcards/
Disallow /freeprepaidcards/
Disallow /gcen/
Disallow /beepmoney/
Disallow /carrentals/
Disallow /self-catering-breaks/
Disallow /countrylife/
Disallow /bestattravel/
Disallow /beepmoney-cash/
Disallow /gp/
Disallow /travelmag/
Disallow /prepaidvisacard/
Disallow /corporateclients/
Disallow /cruisecompany/
Disallow /cp-moneyexpert/
Disallow /cp-loans/
Disallow /cp-tiscali/
Disallow /cp-aolmoney/
Disallow /cp-thesun/
Disallow /cp-autotrader/
Disallow /cp-notw/
Disallow /cp-moneynet/
Disallow /cp-aolmoney-cash/
Disallow /cp-autotrader-cash/
Disallow /cp-loans-cash/
Disallow /cp-moneyexpert-cash/
Disallow /cp-moneynet-cash/
Disallow /cp-notw-cash/
Disallow /cp-thesun-cash/
Disallow /cp-tiscali-cash/
Disallow /beachballpass/
Disallow /snowballpass/
Disallow /vacationworld/
Disallow /moneymona/
Disallow /uswitch-anywhere/
Disallow /uswitch-anywhere-email/
Disallow /businesses-for-sale/
Disallow /knowyourmoney/
Disallow /freecard/
Disallow /matalan-offer/
Disallow /balpa/
Disallow /truphone/
Disallow /trekamerica/
Disallow /bargaintravelhouse/
Disallow /pinkelephantparking/
Disallow /whatagreatbuy/
Disallow /moneywise/
Disallow /travel-money-org/
Disallow /moneywiseplc/
Disallow /motorhomefacts/
Disallow /crewtravelcard/
Disallow /leisureworld/
Disallow /stafftravel-eu/
Disallow /livingwiththelions/
Disallow /questoverseas/
Disallow /clingantravel/
Disallow /traveladvicetv/
Disallow /dailyasia/
Disallow /propaganda/
Disallow /moneycouk-cash/
Disallow /wwtp-fairpay/
Disallow /fxcompared/
Disallow /thisismoney-fairpay/
Disallow /cp-fairpay/
Disallow /comparecurrency-fairpay/
Disallow /corporateoffer/
Disallow /special-offer/
Disallow /helpmetosave/
Disallow /mytravelmoney/
Disallow /diy/
Disallow /rewardgateway/
Disallow /lovemoney2/
Disallow /creditcardexpert/
Disallow /concur/
Disallow /portmanstaff/
Disallow /moneysupermarket-offer/
Disallow /knowyourmoney-cash/
Disallow /cashcustomers/
Disallow /compareholidaymoney/
Disallow /travelweekly/
Disallow /hostelbookers/
Disallow /lonelyplanet/
Disallow /facebook/
Disallow /georgeadams/
Disallow /comparemoneytransfer/
Disallow /damatt/
Disallow /mti/
Disallow /touristupdates/
Disallow /mytravelmoney-euro/
Disallow /mytravelmoney-dollar/
Disallow /mytravelmoney-sterling/
Disallow /fairpayemail/
Disallow /debbiesvillas/
Disallow /debbiesvillas-fairpay/
Disallow /bmi/
Disallow /siddalls/
Disallow /carrentals2/
Disallow /suitcase/
Disallow /cashoffer/
Disallow /grandamericanadventures/
Disallow /lifeandtravel/
Disallow /lisa/
Disallow /dpltravel/
Disallow /gerson/
Disallow /gersoncards/
Disallow /gersoncash/
Disallow /gts/
Disallow /moneycouk1/
Disallow /mirrorfx/
Disallow /telegraphfx/
Disallow /postfx/
Disallow /examfx/
Disallow /echofx/
Disallow /chroniclefx/
Disallow /journalfx/
Disallow /menfx/
Disallow /westfx/
Disallow /gazettefx/
Disallow /bmailfx/
Disallow /hertz/
Disallow /holiday/
Disallow /raf/
Disallow /cash/
Disallow /tube/
Disallow /abuyeashah/
Disallow /alexheath/
Disallow /jackmiles/
Disallow /jakeclarke/
Disallow /jenniferwilson/
Disallow /louiseevans/
Disallow /marykelly/
Disallow /rosa/
Disallow /shermaine/
Disallow /stephenwilson/
Disallow /williamleverett/
Disallow /ampem/
Disallow /andrewtaylor/
Disallow /alessandro/
Disallow /bentubby/
Disallow /claudio/
Disallow /curtlangston/
Disallow /dorward/
Disallow /durley/
Disallow /errietta/
Disallow /garymason/
Disallow /grahamrichards/
Disallow /jatin/
Disallow /jules/
Disallow /mrmg/
Disallow /orla/
Disallow /pavel/
Disallow /bobby/
Disallow /darren/
Disallow /julian/
Disallow /mudassar/
Disallow /thanim/
Disallow /binita/
Disallow /cliveatkinson/
Disallow /emmataylor/
Disallow /johnzablocki/
Disallow /kelliesmith/
Disallow /nerea/
Disallow /tony/
Disallow /tyson/
Disallow /bencauley/
Disallow /freya/
Disallow /garybelton/
Disallow /hal/
Disallow /johnmurphy/
Disallow /niall/
Disallow /robfisher/
Disallow /robertsmith/
Disallow /fernando/
Disallow /jamesw/
Disallow /johnh/
Disallow /lincoln/
Disallow /mariette/
Disallow /ian/
Disallow /simon/
Disallow /matthewgoacher/
Disallow /josephnorton/
Disallow /jraw/
Disallow /edpearce/
Disallow /richardo/
Disallow /kirk/
Disallow /neo/
Disallow /michaelrimmer/
Disallow /andrewshanahan/
Disallow /reuben/
Disallow /manoj/
Disallow /johnpearson/
Disallow /jamie/
Disallow /miles/
Disallow /ramona/
Disallow /kelvin/
Disallow /prefit/
Disallow /ukis/
Disallow /romania/
Disallow /bulgaria/
Disallow /lcfccash/
Disallow /lithuania/
Disallow /skystaff/
Disallow /city/
Disallow /cityforex/
Disallow /freetrial/
Disallow /indulgefootball/
Disallow /mos/
Disallow /3d/
Disallow /login/forgotpassword/
Disallow /wizard/
Disallow /bf/wizard/
Disallow /gatwick/
Disallow /contact/quote/
Disallow /contact/dealingdeskcurrencyconverter/
Disallow /cash-converter-widget/
Disallow /cash-converter-widget-responsive/

Other Records

Field Value
crawl-delay 10

webreaper

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

Comments

  • Robots.txt for www.fairfx.com
  • 13/07/2017
  • Affiliate landing pages
  • Any referer
  • Any landing referrer
  • Referers that generate a redirect
  • Error generating pages
  • Site areas where indexing makes no sense
  • Widgets
  • States that crawlers must give a 10 second delay between requests to the same server
  • ----------------------------------------------
  • This section covers bots that we dont like, for whatever reason.
  • If you have an objection to the inclusion of your bot, email _enquiries@fairfx.com_