übergrössen-kaufhaus.de
robots.txt

Robots Exclusion Standard data for übergrössen-kaufhaus.de

Resource Scan

Scan Details

Site Domain übergrössen-kaufhaus.de
Base Domain übergrössen-kaufhaus.de
Scan Status Ok
Last Scan2024-10-03T14:31:36+00:00
Next Scan 2024-10-10T14:31:36+00:00

Last Scan

Scanned2024-10-03T14:31:36+00:00
URL https://übergrössen-kaufhaus.de/robots.txt
Redirect https://xn--bergrssen-kaufhaus-h3b8i.de/robots.txt
Domain IPs 2a01:488:42:1000:57e6:2d90:a3:c813, 87.230.45.144
Response IP 87.230.45.144
Found Yes
Hash 8e640114cccbb0dff4ecc3340f87f0cff2a9c3d288919d102641c1d6de3cde70
SimHash 329e7e42c838

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

blexbot

Rule Path
Disallow /

mozilla/5.0 (compatible; dataforseobot/1.0; +https://dataforseo.com/dataforseo-bot)

Rule Path
Disallow /

semrushbot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
splitsignalbot
yandex
baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

jobs.de-robot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yeti

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

yak

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

wonderbot

Rule Path
Disallow /

yandexantivirus

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

changedetection

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

buck

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

grobbot

Rule Path
Disallow /

baiduspider-render

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mega-index

Rule Path
Disallow /

dataforseo-bot

Rule Path
Disallow /

Comments

  • bots anweisen nur alle 60 Sekunden die Seite zu crawlen.
  • den Bot von webmeup ausschliessen
  • den Bot von DataForSeoBot ausschliessen
  • den Bot von Semrush ausschliessen