shoptin.de
robots.txt

Robots Exclusion Standard data for shoptin.de

Resource Scan

Scan Details

Site Domain shoptin.de
Base Domain shoptin.de
Scan Status Ok
Last Scan2024-10-13T19:14:00+00:00
Next Scan 2024-10-20T19:14:00+00:00

Last Scan

Scanned2024-10-13T19:14:00+00:00
URL https://shoptin.de/robots.txt
Redirect https://www.shoptin.de/robots.txt
Redirect Domain www.shoptin.de
Redirect Base shoptin.de
Domain IPs 104.21.81.50, 172.67.156.249, 2606:4700:3031::6815:5132, 2606:4700:3031::ac43:9cf9
Redirect IPs 104.21.81.50, 172.67.156.249, 2606:4700:3031::6815:5132, 2606:4700:3031::ac43:9cf9
Response IP 104.21.81.50
Found Yes
Hash 0f46399876b99aa6ee1a81b9c648981548eb11a90ec0d2c64af4696a3317a1f6
SimHash 4a1cc962a733

Groups

*

Rule Path
Allow /*.css
Allow /*.js
Disallow /product/click-eshop/
Disallow /eshop-click/

spbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

kingkevinbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Disallow /admin/
Disallow /auth/

Other Records

Field Value
sitemap https://www.shoptin.de/sitemap_index.xml
sitemap https://www.shoptin.de/sitemap_category.xml
sitemap https://www.shoptin.de/sitemap_product_1.xml
sitemap https://www.shoptin.de/blog/sitemap_index.xml