petshop.de
robots.txt

Robots Exclusion Standard data for petshop.de

Resource Scan

Scan Details

Site Domain petshop.de
Base Domain petshop.de
Scan Status Ok
Last Scan2024-06-26T04:42:54+00:00
Next Scan 2024-07-26T04:42:54+00:00

Last Scan

Scanned2024-06-26T04:42:54+00:00
URL https://www.petshop.de/robots.txt
Domain IPs 185.104.74.229
Response IP 185.104.74.229
Found Yes
Hash 9b096731db943c0a2196a9d8c8a056880f7aecf00713a990d6d8fc4663349f64
SimHash 5ad74642ee90

Groups

*

Rule Path
Disallow /login
Disallow /admin
Disallow /seite/impressum
Disallow /seite/datenschutz
Disallow /seite/agb
Disallow /addProduct
Disallow /image_ajax
Disallow /search_ajax

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

auskunftbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seekport

Rule Path
Disallow /

applebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow
Disallow /
Disallow /

blexbot/1.0

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mqqbrowser

Rule Path
Disallow /

nimbostratus-bot/v1.3.2

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

ucbrowser

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gabanzabot

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

Warnings

  • `user agent` is not a known field.