Log In Sign Up

german-syslinux-blog.de
robots.txt

Robots Exclusion Standard data for german-syslinux-blog.de

Resource Scan

Scan Details

Site Domain german-syslinux-blog.de
Base Domain german-syslinux-blog.de
Scan Status Ok
Last Scan2025-06-11T19:14:40+00:00
Next Scan 2025-07-11T19:14:40+00:00

Last Scan

Scanned2025-06-11T19:14:40+00:00
URL https://german-syslinux-blog.de/robots.txt
Redirect https://www.german-syslinux-blog.de/robots.txt
Redirect Domain www.german-syslinux-blog.de
Redirect Base german-syslinux-blog.de
Domain IPs 85.13.130.122
Redirect IPs 85.13.130.122
Response IP 85.13.130.122
Found Yes
Hash 82f1a7032cf97c263f9c954183761299df964759a8d4cd32d0d7bb7021940ad6
SimHash 507280c28e1b

Groups

*

Rule Path
Disallow /download/
Disallow /998705a1c6b08c5e3f3c698dd155799a/

webreaper
webcopier
offline explorer
httrack
microsoft.url.control
emailcollector
penthesilea
ahrefsbot
dotbot
mj12bot
metajobbot
seoscanners.net
seokicks-robot
seznambot
magpie-crawler
icjobs
grapeshotcrawler
aihitbot
linkdexbot
wbsearchbot
qwantify
baiduspider
archive.org_bot
ia_archiver

Rule Path
Disallow /
Back to top

Comments

  • http://de.selfhtml.org/diverses/robots.htm
  • ======================================
  • Crawling-Geschwindingkeit 30sec:
  • ======================================
  • ======================================
  • Gültig für alle Bots:
  • ======================================
  • ======================================
  • Sorry man, no secrets here. But
  • eventually the next time. Who
  • knows... *hustle* By the way, u
  • believe all what I'm saying ? XD
  • Maybe u should decode the HASH!
  • ======================================
  • ======================================
  • Schließe folgende Spider komplett aus:
  • ======================================
Back to top

Warnings

  • 1 invalid line.
Back to top