grosse-herrenschuhe.com
robots.txt

Robots Exclusion Standard data for grosse-herrenschuhe.com

Resource Scan

Scan Details

Site Domain grosse-herrenschuhe.com
Base Domain grosse-herrenschuhe.com
Scan Status Ok
Last Scan2024-10-01T07:42:34+00:00
Next Scan 2024-10-08T07:42:34+00:00

Last Scan

Scanned2024-10-01T07:42:34+00:00
URL https://grosse-herrenschuhe.com/robots.txt
Redirect https://www.grosse-herrenschuhe.com/robots.txt
Redirect Domain www.grosse-herrenschuhe.com
Redirect Base grosse-herrenschuhe.com
Domain IPs 2a01:488:42:1000:57e6:2d90:a3:c813, 87.230.45.144
Redirect IPs 2a01:488:42:1000:57e6:2d90:a3:c813, 87.230.45.144
Response IP 87.230.45.144
Found Yes
Hash ddedf071cd5b3ede81f2e856de58471d3c30f93d3c4bba441371a59caee4462a
SimHash 329e7e42c838

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

blexbot

Rule Path
Disallow /

mozilla/5.0 (compatible; dataforseobot/1.0; +https://dataforseo.com/dataforseo-bot)

Rule Path
Disallow /

semrushbot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
splitsignalbot
yandex
baiduspider

Rule Path
Disallow /
Disallow

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

jobs.de-robot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yeti

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

yak

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

wonderbot

Rule Path
Disallow /

yandexantivirus

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

changedetection

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

buck

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

grobbot

Rule Path
Disallow /

baiduspider-render

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mega-index

Rule Path
Disallow /

Comments

  • bots anweisen nur alle 60 Sekunden die Seite zu crawlen.
  • den Bot von webmeup ausschliessen
  • den Bot von DataForSeoBot ausschliessen
  • den Bot von Semrush ausschliessen