Log In Sign Up

designtagebuch.de
robots.txt

Robots Exclusion Standard data for designtagebuch.de

Resource Scan

Scan Details

Site Domain designtagebuch.de
Base Domain designtagebuch.de
Scan Status Ok
Last Scan2024-11-10T23:08:11+00:00
Next Scan 2024-11-17T23:08:11+00:00

Last Scan

Scanned2024-11-10T23:08:11+00:00
URL https://designtagebuch.de/robots.txt
Domain IPs 185.215.156.20
Response IP 185.215.156.20
Found Yes
Hash deee66eb0b5e658bb19a45e99621c7e1136c7ae0cdcc9d2868cbb4907bb34e2a
SimHash 4469c2a0472b

Groups

ia_archiver

Rule Path
Disallow /

bot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

bingbot

Rule Path
Disallow

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

feed a fever

Rule Path
Disallow /

web core

Rule Path
Disallow /

simplepie

Rule Path
Disallow /

superfeedr

Rule Path
Disallow /

crawler

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /
Allow /
Disallow /wp-admin/
Disallow /mysqldumper/
Disallow /produkt/
Disallow /shop/
Disallow /login/
Disallow /register/
Disallow /account/

*

Rule Path
Disallow /wp-content/uploads/wp-import-export-lite/
Back to top

Other Records

Field Value
sitemap https://www.designtagebuch.de/sitemap.xml
Back to top

Comments

  • Bots ausschliessen
  • grundsaetzlich erlaubt
  • vom Spidern ausschliessen
  • WP Import Export Rule
Back to top

Warnings

  • 2 invalid lines.
Back to top