designfliesen.de
robots.txt

Robots Exclusion Standard data for designfliesen.de

Resource Scan

Scan Details

Site Domain designfliesen.de
Base Domain designfliesen.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-17T19:18:24+00:00
Next Scan 2024-12-01T19:18:24+00:00

Last Successful Scan

Scanned2024-10-10T18:42:27+00:00
URL https://designfliesen.de/robots.txt
Redirect https://www.designfliesen.de/robots.txt
Redirect Domain www.designfliesen.de
Redirect Base designfliesen.de
Domain IPs 5.172.42.203
Redirect IPs 5.172.42.203
Response IP 5.172.42.203
Found Yes
Hash cfd85c7b8ea5db71cb74c3cb5d8b50f6eb9d442908b7619f6aa20d8408982718
SimHash ddd67050883d

Groups

*

Rule Path
Disallow /bestellungen.html
Disallow /de/bestellungen.html
Disallow /login.html
Disallow /de/login.html
Disallow /bestellen/login.html
Disallow /de/bestellen/login.html
Disallow /bestellen/informationen.html
Disallow /de/bestellen/informationen.html
Disallow /warenkorb.html
Disallow /de/warenkorb.html
Disallow /wunschliste.html
Disallow /de/wunschliste.html
Disallow /registrieren.html
Disallow /de/registrieren.html
Disallow /profil.html
Disallow /de/profil.html
Disallow /passwort-vergessen.html
Disallow /de/passwort-vergessen.html
Disallow /passwort-resetten.html
Disallow /de/passwort-resetten.html
Disallow /passwort-andern.html
Disallow /de/passwort-andern.html
Disallow /zahlen/$
Disallow /de/zahlen/$
Disallow /suche.html?*
Disallow /de/suche.html?*
Disallow /preisanfrage.html
Disallow /de/preisanfrage.html
Disallow /management-gebruikers-duits.html
Disallow /de/management-gebruikers-duits.html
Disallow /management-lieferadressen.html
Disallow /de/management-lieferadressen.html
Disallow /management-berechnung-adressen.html
Disallow /de/management-berechnung-adressen.html
Disallow /abmelden-uberprufungsantrag.html
Disallow /de/abmelden-uberprufungsantrag.html
Disallow /zuruck-bestellen.html
Disallow /de/zuruck-bestellen.html
Disallow /zuruck-bestellen-bestellung-auswahlen.html
Disallow /de/zuruck-bestellen-bestellung-auswahlen.html
Disallow /waehle-externen-verkaufer.html
Disallow /de/waehle-externen-verkaufer.html
Disallow /Ruckgabeanforderung.html
Disallow /de/Ruckgabeanforderung.html
Disallow /meine-digitalen-Produkte.html
Disallow /de/meine-digitalen-Produkte.html
Disallow /x19y46/dashboard.html
Disallow /de/x19y46/dashboard.html
Disallow /x19y46/dashboard/login.html
Disallow /de/x19y46/dashboard/login.html
Disallow /benotigte-produkte/$
Disallow /de/benotigte-produkte/$
Disallow /angebots.html
Disallow /de/angebots.html
Disallow /angebots-detail.html
Disallow /de/angebots-detail.html
Disallow /angebots-detail/$
Disallow /de/angebots-detail/$
Disallow /uberprufen-sie-die-e-mail-adresse.html
Disallow /de/uberprufen-sie-die-e-mail-adresse.html
Disallow /verpackungsartikel.html
Disallow /de/verpackungsartikel.html
Disallow /Angebotsliste.html
Disallow /de/Angebotsliste.html
Disallow /webfunctions/utypia$
Disallow /zoek/$
Disallow /setcrossdomaintoken/$
Disallow /m/93/OfferteAanvraag.aspx$
Disallow /m/93/MonsterAanvraag.aspx$
Disallow /afterpay/$
Disallow /riverty/$
Disallow /cardgate/$
Disallow /pin/$
Disallow /multisafe-pay/$
Disallow /cfg/$
Disallow /debtorlogo/$
Disallow /orderdetail/$
Disallow /orderdetail/$
Disallow /bestelldetails/$
Disallow /details-de-la-commande/$
Disallow /detalle-del-pedido/$
Disallow /dettaglio-dell-ordine/$
Disallow /bedankt/$
Disallow /thanks/$
Disallow /danke/$
Disallow /merci/$
Disallow /gracias/$
Disallow /grazie/$
Disallow /hulu-egg-zip-jack-bestbuy-l4.html
Disallow /TradeTracker/$
Disallow /tradetracker/$
Disallow /rp/$
Disallow *.axd$
Disallow *.axd
Disallow *.ashx$
Disallow *.ashx

Other Records

Field Value
crawl-delay 3

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

twengabot-2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ucrawler/1.0

Rule Path
Disallow /

ucrawler

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

yandeximages

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

vagabondo

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

pricebot

Rule Path
Disallow /

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://www.designfliesen.de/sitemap_index.xml

Comments

  • https://megaindex.com/crawler
  • Version:
  • IIS24\182.2.7.3208
  • Directory:
  • Next-2024-01-10-S182.2.7
  • Style:
  • s198