addefend.motor-talk.de
robots.txt

Robots Exclusion Standard data for addefend.motor-talk.de

Resource Scan

Scan Details

Site Domain addefend.motor-talk.de
Base Domain motor-talk.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-07T22:00:36+00:00
Next Scan 2024-10-05T22:00:36+00:00

Last Successful Scan

Scanned2023-12-11T21:56:55+00:00
URL http://addefend.motor-talk.de/robots.txt
Redirect https://www.motor-talk.de/robots.txt
Redirect Domain www.motor-talk.de
Redirect Base motor-talk.de
Domain IPs 194.50.69.71, 91.211.75.211
Redirect IPs 2.22.248.31, 2.22.248.5
Response IP 2.22.248.31
Found Yes
Hash ba9c7029f02ab33077a4af0110b651b78a05f52a0cf2ae402f0b93df9da0c0b7
SimHash 99a8500981b2

Groups

*

Rule Path
Disallow /out
Disallow /forum/erfahrungen-mit-www-reifenonlineversand-de-oder-www-stahlkomplettrad-de-t1263122.html
Disallow /forum/markenautoteile-de-da-bestelle-ich-nichts-mehr-t1048828.html
Disallow /forum/security-test-track-b157.html
Disallow /forum/fahrzeuge-b769.html
Disallow /2521ef5d58fc027d3121662b7d8f9ac2/images.html
Disallow /suche.html
Disallow /blogs/rogertennis2*
Disallow /blogs/technik-und-ot-blog*
Disallow /blogs/blogs/berlin-paul*

sogou

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

Warnings

  • 2 invalid lines.