madeindesign.co.uk
robots.txt

Robots Exclusion Standard data for madeindesign.co.uk

Resource Scan

Scan Details

Site Domain madeindesign.co.uk
Base Domain madeindesign.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-04T14:17:51+00:00
Next Scan 2024-12-03T14:17:51+00:00

Last Successful Scan

Scanned2024-04-15T14:15:53+00:00
URL https://madeindesign.co.uk/robots.txt
Redirect https://www.madeindesign.co.uk/robots.txt
Redirect Domain www.madeindesign.co.uk
Redirect Base madeindesign.co.uk
Domain IPs 104.18.26.57, 104.18.27.57, 2606:4700::6812:1a39, 2606:4700::6812:1b39
Redirect IPs 104.18.26.57, 104.18.27.57, 2606:4700::6812:1a39, 2606:4700::6812:1b39
Response IP 104.18.26.57
Found Yes
Hash 3f421faa615c9446227598180d98f43df9a194d9096087d93790bff423dc763f
SimHash 2af65c79a35b

Groups

searchmetricsbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

onpagebot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pi-monster

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /*gclid%3D
Disallow /*esl-k%3D
Disallow /*msclkid%3D
Disallow /*fbclid%3D
Disallow /*utm%3D
Disallow /use_ajax
Disallow /turpentineCookies
Disallow /sav/
Disallow /review/ajax/
Disallow /shipping/
Disallow /*category_level*
Disallow /*collection_b2c*
Disallow /*category%3D
Disallow /*id%3D
Disallow /*part%3D
Disallow /*category%3D
Disallow /*id%3D
Disallow /*part%3D
Disallow /*mid_product_room%3D
Disallow /*mid_designer%3D
Disallow /*mid_color_family%3D
Disallow /*mid_material_family%3D
Disallow /*mid_type%3D
Disallow /*mid_price_range%3D
Disallow /*final_price_EUR%3D
Disallow /*cat%3D
Disallow /*mid_magazine_brand%3D
Disallow /*mid_magazine_designer%3D
Disallow /*mid_magazine_event%3D
Disallow /*mid_brand%3D
Disallow /*mid_designer%3D
Disallow /*designer_id%3D
Disallow /*selectionOrder%3D
Disallow /b2b/index/devis/*
Disallow /directory/currency/*
Disallow /be2bill/tpe/saveAjax*
Disallow /b2b/index/devisCart*
Disallow /*mode%3D
Disallow /*dir%3D
Disallow /*limit%3D
Disallow /*q%3D
Disallow /*vp_id%3D
Disallow /*order%3D
Disallow /*ectrans%3D
Disallow /*___SID%3D
Disallow /*Page%3D
Disallow /*keywords%3D
Disallow /catalog/
Disallow /*/catalog/
Disallow /prod*-vp.html
Disallow /prod*-savp.html
Disallow /prod*-fpc.html
Disallow /catalogsearch/
Disallow /midseo/
Disallow /midsearch/
Disallow /s-*.html
Disallow /poll/
Disallow /midgui/
Disallow /pdf
Disallow /skin/adminhtml/
Disallow /skin/install/
Disallow /directory/currency/
Disallow /giftcard/
Disallow /wishlist/
Disallow /customer/
Disallow /account/
Disallow /smilenewsletter/
Disallow /checkout/
Disallow /midcustomer/
Disallow /criteo/
Disallow /sales/order/history/
Disallow /recommended-products.html
Disallow /midcatalog/product/print/
Disallow /midcatalog/product/
Disallow /midcatalog/category/
Disallow /midfredhopper/
Disallow /b2b/index/proxy/url
Disallow /product-selections/*/d-*
Disallow /product-selections/*/b-*
Disallow /sale/*/b-*
Disallow /sale/*/d-*
Disallow /*checkout%3D
Disallow /*email_newsletter%3D
Disallow /*kk%3D
Disallow /*kt%3D
Disallow /*affc%3D
Disallow /*cto_pld%3D
Disallow /*from%3D
Disallow /*yitest%3D
Disallow /showcase/
Disallow /private-sales/
Disallow /private-sales.html*
Disallow /sale-preview/
Disallow /mag-private-sale-*
Disallow /midwidget/
Disallow /?cpkey*
Disallow /?awc=*
Disallow *?eaf-publisher=*
Disallow *?etf-media=*

Other Records

Field Value
sitemap https://www.madeindesign.co.uk/sitemap/sitemap_uk_index.xml

Comments

  • Robots# 02.2019
  • disallow bots
  • SearchmetricsBot
  • Linguee
  • SEOkicks
  • YandexBot
  • OnPageBot
  • BLEXBot
  • Pricepi
  • MegaIndex
  • MemoryBot
  • RIDDLER
  • twenga
  • domaincrawler
  • Rogerbot
  • Seekport
  • delay bots
  • Pinterest
  • Apply rules to all user agents
  • Filtered pages exclusion
  • Parameters pages exclusion
  • Product pages
  • Private sales product
  • Search result pages exclusion
  • poll exclusion
  • Skin images
  • minified js and css are in /skin/m/
  • Customer pages exclusion
  • Print pages exclusion
  • 2945 - 20160921 désactive urls FH
  • Exclusion d'url #3066
  • 5074 - Update 08.2019
  • website custom rules
  • Update 02.2019
  • Presales 08/06/2018
  • Sitemap
  • END of rules