Log In Sign Up

berlios.de
robots.txt

Robots Exclusion Standard data for berlios.de

Resource Scan

Scan Details

Site Domain berlios.de
Base Domain berlios.de
Scan Status Ok
Last Scan2024-09-03T06:51:21+00:00
Next Scan 2024-10-03T06:51:21+00:00

Last Scan

Scanned2024-09-03T06:51:21+00:00
URL https://berlios.de/robots.txt
Domain IPs 89.107.187.162
Response IP 89.107.187.162
Found Yes
Hash fa088cc394bd9714d450c8227d614932a0268c7860c2f369d33dde587bc79e84
SimHash 311c973a5d4a

Groups

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

spyfu

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

cognitiveseo

Rule Path
Disallow /

findthatlead

Rule Path
Disallow /
Back to top

Comments

  • AhrefsBot
  • MajesticSEO Bot
  • SEMrush Bot
  • Moz's Dotbot
  • Sistrix Bot
  • Searchmetrics Bot
  • SpyFu
  • Alexa (Amazon) Bot
  • Cognitive SEO
  • FindThatLead Bot
Back to top