Log In Sign Up

suche.t-online.de
robots.txt

Robots Exclusion Standard data for suche.t-online.de

Resource Scan

Scan Details

Site Domain suche.t-online.de
Base Domain t-online.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-08T14:15:59+00:00
Next Scan 2024-11-22T14:15:59+00:00

Last Successful Scan

Scanned2024-10-01T09:09:13+00:00
URL https://suche.t-online.de/robots.txt
Domain IPs 2a05:d014:c20:c700:932f:1d6e:212b:185a, 2a05:d014:c20:c701:fab8:566a:a6f4:f022, 2a05:d014:c20:c702:5d03:551e:c37c:1c01, 3.124.28.51, 3.75.59.204, 52.58.175.82
Response IP 3.75.59.204
Found Yes
Hash 7d4e70182b4a013ff2d9be384270b77334b56adfe3928567ca055c4211fb0a8b
SimHash 38195342ce33

Groups

*

Rule Path
Disallow /fast-cgi
Disallow /statics
Disallow /geo-service

slurp

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

proximic

Rule Path
Disallow /

exabot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

yacy

Rule Path
Disallow /

daum

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

nutch

Rule Path
Disallow /

custo

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

magpierss

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

feed a fever

Rule Path
Disallow /

psbot

Rule Path
Disallow /

stackrambler

Rule Path
Disallow /

linkdex

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

protopage

Rule Path
Disallow /

bspider

Rule Path
Disallow /

phantom

Rule Path
Disallow /

arks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

yet-another-spider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ssearch crawler

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top