Log In Sign Up

mastershoe.co.uk
robots.txt

Robots Exclusion Standard data for mastershoe.co.uk

Resource Scan

Scan Details

Site Domain mastershoe.co.uk
Base Domain mastershoe.co.uk
Scan Status Ok
Last Scan2024-09-09T04:57:57+00:00
Next Scan 2024-10-09T04:57:57+00:00

Last Scan

Scanned2024-09-09T04:57:57+00:00
URL https://mastershoe.co.uk/robots.txt
Domain IPs 104.26.0.40, 104.26.1.40, 172.67.68.208, 2606:4700:20::681a:128, 2606:4700:20::681a:28, 2606:4700:20::ac43:44d0
Response IP 104.26.1.40
Found Yes
Hash e4f0dcba8d23df516f95459911888d7660af5327b7d3657c3c75d7ece41928b2
SimHash 2a1ed2cacafb

Groups

ninjabot

Rule Path
Disallow /

amazon-kendra-web-crawler-*

Rule Path
Disallow /

ninja-crawler81

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

*

Rule Path
Disallow */shopby/*
Disallow *--*
Allow */shopby/ikon$
Allow */shopby/aigle$
Allow */shopby/asics$
Allow */shopby/base_london$
Allow */shopby/bates$
Allow */shopby/berghaus$
Allow */shopby/birkenstock$
Allow */shopby/blundstone$
Allow */shopby/bogs$
Allow */shopby/camper$
Allow */shopby/caterpillar$
Allow */shopby/chatham$
Allow */shopby/converse$
Allow */shopby/crocs$
Allow */shopby/dc$
Allow */shopby/dr_martens$
Allow */shopby/dude$
Allow */shopby/dvs$
Allow */shopby/el_natura_lista$
Allow */shopby/earth_spirit$
Allow */shopby/evercreatures$
Allow */shopby/eastpak$
Allow */shopby/etnies$
Allow */shopby/fitflop$
Allow */shopby/fly_london$
Allow */shopby/geox$
Allow */shopby/globe$
Allow */shopby/grafters$
Allow */shopby/grinders$
Allow */shopby/gringos$
Allow */shopby/harley_davidson$
Allow */shopby/heavenly_feet$
Allow */shopby/helly_hansen$
Allow */shopby/hoka_one_one$
Allow */shopby/holees$
Allow */shopby/helly_hansen$
Allow */shopby/hummel$
Allow */shopby/hunter$
Allow */shopby/hush_puppies$
Allow */shopby/hype$
Allow */shopby/irregular_choice$
Allow */shopby/jane_klain$
Allow */shopby/joules$
Allow */shopby/joya$
Allow */shopby/kanyon$
Allow */shopby/keen$
Allow */shopby/kickers$
Allow */shopby/k_swiss$
Allow */shopby/lacoste$
Allow */shopby/le_chameau$
Allow */shopby/london_brogues$
Allow */shopby/mephisto$
Allow */shopby/marco_tozzi$
Allow */shopby/merrell$
Allow */shopby/moheda$
Allow */shopby/muck_boots$
Allow */shopby/mustang$
Allow */shopby/natural_world$
Allow */shopby/nikwax$
Allow */shopby/oxygen$
Allow */shopby/oak_hyde$
Allow */shopby/padders$
Allow */shopby/palladium$
Allow */shopby/panama_jack$
Allow */shopby/puma$
Allow */shopby/ralph_lauren$
Allow */shopby/reef$
Allow */shopby/rocket_dog$
Allow */shopby/rockport$
Allow */shopby/romika$
Allow */shopby/ruby_shoo$
Allow */shopby/salomon$
Allow */shopby/scholl$
Allow */shopby/skechers$
Allow */shopby/sneaky_steve$
Allow */shopby/softinos$
Allow */shopby/solovair$
Allow */shopby/sorel$
Allow */shopby/sperry$
Allow */shopby/strive$
Allow */shopby/superga$
Allow */shopby/teva$
Allow */shopby/the_north_face$
Allow */shopby/timberland$
Allow */shopby/toms$
Allow */shopby/toni_pons$
Allow */shopby/t_u_k$
Allow */shopby/vagabond$
Allow */shopby/vans$
Allow */shopby/vibram$
Allow */shopby/wrangler$
Allow */shopby/new-balance$
Allow */shopby/nordikas$
Allow */shopby/on-foot$
Allow */shopby/plakton$
Allow */shopby/roamers$
Allow */shopby/sebago$
Allow */shopby/woodland$
Allow */shopby/art$
Allow */shopby/blowfish$
Allow */shopby/buffalo$
Allow */shopby/butterfly-twists$
Allow */shopby/cipriata$
Allow */shopby/emu-australia$
Allow */shopby/ecco$
Allow */shopby/gola$
Allow */shopby/grisport$
Allow */shopby/ikon$
Allow */shopby/barefoot$
Allow */shopby/biker_boots$
Allow */shopby/boat_shoes$
Allow */shopby/brogues$
Allow */shopby/casual_shoes$
Allow */shopby/chelsea_boots$
Allow */shopby/chukka_boots$
Allow */shopby/classic_sandals$
Allow */shopby/clogs_mules$
Allow */shopby/cowboy_boots$
Allow */shopby/espadrille$
Allow */shopby/fisherman_sandals$
Allow */shopby/loafers$
Allow */shopby/military_boots$
Allow */shopby/monkey_boots$
Allow */shopby/neoprene_wellingtons$
Allow */shopby/running$
Allow */shopby/safety_boots$
Allow */shopby/skate_trainers$
Allow */shopby/slides_sandals$
Allow */shopby/snow_boots$
Allow */shopby/tennis_shoes$
Allow */shopby/toe_post_sandals$
Allow */shopby/trainers$
Allow */shopby/walking_boots$
Allow */shopby/walking_sandals$
Allow */shopby/walking_shoes$
Allow */shopby/wellington_boots$
Allow */shopby/gladiator_sandals$
Allow */shopby/heeled_sandals$
Allow */shopby/heels$
Allow */shopby/limited_edition$
Allow */shopby/mary_janes$
Allow */shopby/peep_toe$
Allow */shopby/pumps$
Allow */shopby/slingbacks$
Allow */shopby/slippers$
Allow */shopby/strappy_sandals$
Allow */shopby/leather$
Allow */shopby/vegan$
Allow */shopby/canvas$
Allow /aigle-history
Allow /back-to-school
Allow /what-makes-a-birkenstock
Allow /birkenstock-arizona
Allow /birkenstock-history
Allow /birkenstock-gizeh
Allow /camper-slippers
Allow /camper-peu-cami
Allow /camper_pelotas
Allow /camper-history
Allow /caterpillar-history
Allow /caterpillar-colorado
Allow /crocs-specialist
Allow /crocs-bistro
Allow /crocs-work
Allow /crocs-history
Allow /fitflop-history
Allow /fitflop-lulu
Allow /fitflop-sole-units
Allow /flylondon-history
Allow /fly_mes_boots
Allow /fly_salv_boots
Allow /fly_tram_sandals
Allow /heavenly_feet_history
Allow /hunter-history
Allow /hoka-one-one-arahi
Allow /hoka-one-one-arahi
Allow /hoka-one-one-clifton
Allow /hoka-one-one-history
Allow /hunter_argyll
Allow /hunter_balmoral
Allow /hunter_chelsea_boot
Allow /irregular-choice-festive
Allow /irregular-choice-history
Allow /joya-history
Allow /joya_emotion
Allow /joya_faq
Allow /joya_motion
Allow /joya_size_info
Allow /keen-austin
Allow /keen-newport
Allow /keen-history
Allow /keen-better-takes-action
Allow /kickers-kick-hi
Allow /kickers-kick-lo
Allow /kickers-history
Allow /kickers-back-to-school
Allow /le-chameau-history
Allow /lechameau-vierzonord
Allow /lechameau-vierzon
Allow /merrell-barefoot
Allow /merrell-history
Allow /merrell-ice-grip
Allow /merrell-moab
Allow /muck-boots-history
Allow /muck-boots-arctic-range
Allow /oak-and-hyde-coast-mid
Allow /oak-and-hyde-crest
Allow /oak-and-hyde-history
Allow /salomon-history
Allow /salomon-quest
Allow /skechers-flex-appeal
Allow /skechers-dlites
Allow /skechers-history
Allow /skechers-memory-foam
Allow /the-making-of-le-chameau
Allow /timberland-6inchclassic
Allow /timberland-boatshoes
Allow /timberland-history
Allow /vans-history
Allow /vans_authentic
Allow /vans_old_skool
Allow /vans_sk8_hi
Allow /vegan
Allow /vibram-fivefingers
Allow /vibram-furoshiki
Allow /what-makes-a-birkenstock
Allow /strive-technologies
Allow /strive-history
Allow /sorel-caribou
Allow /sorel-out-n-about
Allow /sorel-history
Allow /superga-2750
Allow /superga-2790
Allow /superga-metallics
Allow /superga-history
Allow /lacoste-ampthill
Allow /lacoste-chaymon
Allow /lacoste-history
Allow /etnies-callicut
Allow /etnies-icons
Allow /etnies-history
Allow /etnies-buy-a-shoe-plant-a-tree
Allow /toms-alpargata
Allow /toms-history
Allow /k-swiss-tennis
Allow /k-swiss-history
Allow /the_art_company_history
Allow /teva-history
Allow /buffalo-classics
Allow /buffalo-history
Allow /harley-davidson-history
Allow /ikon-history
Allow /asics-history
Allow /chatham-boat-shoes
Allow /chatham-history
Allow /dc-history
Allow /berghaus-supalite
Allow /berghaus-history
Allow /bogs-history
Allow /blundstone-history
Allow /el-natura-lista-history
Allow /base-london-history
Allow /the_north_face_hedgehog
Allow /the_north_face_history
Allow /vagabond-dioon
Allow /vagabond-history
Allow /skinners-history
Allow /hey-dude-farty
Allow /hey-dude-history
Allow /geox-history
Allow /globe-tilt
Allow /globe-sabre
Allow /globe-history
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /magento/
Disallow /pkginfo/
Disallow /report/
Disallow /scripts/
Disallow /shell/
Disallow /stats/
Disallow /var/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /*refine*
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /catalog/product/gallery/
Disallow /catalog/seo_sitemap/category/
Disallow /checkout/cart/
Disallow /catalogsearch
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /*.php$
Disallow /*?SID=
Disallow /*?cat=
Back to top

Other Records

Field Value
sitemap https://www.mastershoe.co.uk/sitemap.xml
sitemap https://www.mastershoe.co.uk/Shopby_sitemap.xml
Back to top

Comments

  • Released Pages
  • Landing Page Releases
  • Small Landing Page Releases
  • Directories
  • Disallow: /js/
  • Disallow: /lib/
  • Disallow: /media/
  • Disallow: /skin/
  • Paths (clean URLs)
  • Files
  • Paths (no clean URLs)
  • Disallow: /*.js$
  • Disallow: /*.css$
Back to top