vebidoo.de
robots.txt

Robots Exclusion Standard data for vebidoo.de

Resource Scan

Scan Details

Site Domain vebidoo.de
Base Domain vebidoo.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-04-08T19:12:09+00:00
Next Scan 2024-07-07T19:12:09+00:00

Last Successful Scan

Scanned2022-02-21T00:58:53+00:00
URL https://vebidoo.de/robots.txt
Redirect https://www.vebidoo.de/robots.txt
Redirect Domain www.vebidoo.de
Redirect Base vebidoo.de
Response IP 136.243.17.97
Found Yes
Hash 6c42dcc54a6a9f6b072ddc9ab90054f5e8a0635b185831d8f74d5cadeca72bdd
SimHash 135ad0b34ff0

Groups

googlebot

Rule Path
Allow /

googlebot-mobile

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

*

Rule Path
Disallow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

bingbot

Rule Path
Allow /
Allow /

libwww

Rule Path
Disallow /

wget

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mama casper

Rule Path
Disallow /

mama xirio

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

goscraper

Rule Path
Disallow /

java

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

keybot translation-search-machine

Rule Path
Disallow /

bender

Rule Path
Disallow /

Warnings

  • 2 invalid lines.
  • `user agent` is not a known field.