antonianum.de
robots.txt

Robots Exclusion Standard data for antonianum.de

Archived Snapshots

Resource Scan

Scan Details

Site Domain	antonianum.de
Base Domain	antonianum.de
Scan Status	Ok
Last Scan	2024-10-26T12:16:12+00:00
Next Scan	2024-11-25T12:16:12+00:00

Last Scan

Scanned	2024-10-26T12:16:12+00:00
URL	https://antonianum.de/robots.txt
Domain IPs	2001:8d8:100f:f000::2de, 217.160.0.121
Response IP	217.160.0.121
Found	Yes
Hash	2b828c5b93d647326d031dd46e7f4dd6499523abaaba444728924de16dd03498
SimHash	f23fbd4a49f0

Groups

*

Rule	Path
Disallow	/administrator/
Disallow	/bin/
Disallow	/cache/
Disallow	/cli/
Disallow	/components/
Disallow	/includes/
Disallow	/installation/
Disallow	/language/
Disallow	/layouts/
Disallow	/libraries/
Disallow	/logs/
Disallow	/modules/
Disallow	/plugins/
Disallow	/tmp/

Rule

Path

Disallow

/administrator/

Disallow

/bin/

Disallow

/cache/

Disallow

/cli/

Disallow

/components/

Disallow

/includes/

Disallow

/installation/

Disallow

/language/

Disallow

/layouts/

Disallow

/libraries/

Disallow

/logs/

Disallow

/modules/

Disallow

/plugins/

Disallow

/tmp/

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

giftghostbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seznam

Rule	Path
Disallow	/

Rule

Path

Disallow

paperlibot

Rule	Path
Disallow	/

Rule

Path

Disallow

genieo

Rule	Path
Disallow	/

Rule

Path

Disallow

dataprovider/6.101

Rule	Path
Disallow	/

Rule

Path

Disallow

dataprovidersiteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

dazoobot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot

Rule	Path
Disallow	/

Rule

Path

Disallow

domainstatsbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot/1.1

Rule	Path
Disallow	/

Rule

Path

Disallow

dubaiindex

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

ecommercebot

Rule	Path
Disallow	/

Rule

Path

Disallow

expertsearchspider

Rule	Path
Disallow	/

Rule

Path

Disallow

feedbin

Rule	Path
Disallow	/

Rule

Path

Disallow

fetch/2.0a

Rule	Path
Disallow	/

Rule

Path

Disallow

ffbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

focusbot/1.1

Rule	Path
Disallow	/

Rule

Path

Disallow

huaweisymantecspider

Rule	Path
Disallow	/

Rule

Path

Disallow

huaweisymantecspider/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

jobdiggerspider

Rule	Path
Disallow	/

Rule

Path

Disallow

lemurwebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

lipperheylinkexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

lssrocketcrawler/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

lyt.srv1.5

Rule	Path
Disallow	/

Rule

Path

Disallow

miadev/0.0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

najdi.si/3.1

Rule	Path
Disallow	/

Rule

Path

Disallow

bountiibot

Rule	Path
Disallow	/

Rule

Path

Disallow

experibot_v1

Rule	Path
Disallow	/

Rule

Path

Disallow

bixocrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

bixocrawler testcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

crawler4j

Rule	Path
Disallow	/

Rule

Path

Disallow

crowsnest/0.5

Rule	Path
Disallow	/

Rule

Path

Disallow

cukbot

Rule	Path
Disallow	/

Rule

Path

Disallow

dataprovider/6.92

Rule	Path
Disallow	/

Rule

Path

Disallow

dblbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot/0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

digg deeper/v1

Rule	Path
Disallow	/

Rule

Path

Disallow

discobot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

discobot/1.1

Rule	Path
Disallow	/

Rule

Path

Disallow

discobot/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

discoverybot/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

dlvr.it/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

domainstatsbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

drupact/0.7

Rule	Path
Disallow	/

Rule

Path

Disallow

ezooms/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

fastbot crawler beta 2.0

Rule

Path

Disallow

fastbot crawler beta 4.0

Rule

Path

Disallow

feedly social

Rule

Path

Disallow

feedly/1.0

Rule

Path

Disallow

feedlybot/1.0

Rule

Path

Disallow

feedspot

Rule

Path

Disallow

feedspotbot/1.0

Rule

Path

Disallow

clickagy intelligence bot v2

Rule

Path

Disallow

classbot

Rule

Path

Disallow

cispa vulnerability notification

Rule

Path

Disallow

cirrusexplorer/1.1

Rule

Path

Disallow

checksem/nutch-1.10

Rule

Path

Disallow

catchbot/5.0

Rule

Path

Disallow

catchbot/3.0

Rule

Path

Disallow

catchbot/2.0

Rule

Path

Disallow

catchbot/1.0

Rule

Path

Disallow

camontspider/1.0

Rule

Path

Disallow

buzzbot/1.0

Rule

Path

Disallow

buzzbot

Rule

Path

Disallow

businessseek.biz_spider

Rule

Path

Disallow

bubing

Rule

Path

Disallow

blexbot

Rule

Path

Disallow

fyberspider/1.3

Rule

Path

Disallow

findlinks/1.1.6-beta5

Rule

Path

Disallow

g2reader-bot/1.0

Rule

Path

Disallow

findlinks/1.1.6-beta6

Rule

Path

Disallow

findlinks/2.0

Rule

Path

Disallow

findlinks/2.0.1

Rule

Path

Disallow

findlinks/2.0.2

Rule

Path

Disallow

findlinks/2.0.4

Rule

Path

Disallow

findlinks/2.0.5

Rule

Path

Disallow

findlinks/2.0.9

Rule

Path

Disallow

findlinks/2.1

Rule

Path

Disallow

findlinks/2.1.5

Rule

Path

Disallow

findlinks/2.1.3

Rule

Path

Disallow

findlinks/2.2

Rule

Path

Disallow

findlinks/2.5

Rule

Path

Disallow

findlinks/2.6

Rule

Path

Disallow

ffbot/1.0

Rule

Path

Disallow

findlinks/1.0

Rule

Path

Disallow

findlinks/1.1.3-beta8

Rule

Path

Disallow

findlinks/1.1.3-beta9

Rule

Path

Disallow

findlinks/1.1.4-beta7

Rule

Path

Disallow

findlinks/1.1.6-beta1

Rule

Path

Disallow

findlinks/1.1.6-beta1 yacy

Rule

Path

Disallow

findlinks/1.1.6-beta2

Rule

Path

Disallow

findlinks/1.1.6-beta3

Rule

Path

Disallow

findlinks/1.1.6-beta4

Rule

Path

Disallow

bixo

Rule

Path

Disallow

bixolabs/1.0

Rule

Path

Disallow

crawlera/1.10.2

Rule

Path

Disallow

dataprovider site explorer

Rule

Path

Disallow

vagabondo

Rule

Path

Disallow

seokicks-robot

Rule

Path

Disallow

sistrix

Rule

Path

Disallow

wbsearchbot

Rule

Path

Disallow

proximic

Rule

Path

Disallow

queryseekerspider

Rule

Path

Disallow

ia_archiver

Rule

Path

Disallow

archive.org_bot

Rule

Path

Disallow

special_archiver

Rule

Path

Disallow

heritrix

Rule

Path

Disallow

netestate ne crawler

Rule

Path

Disallow

ahrefsbot

Rule

Path

Disallow

alexibot

Rule

Path

Disallow

mj12bot

Rule

Path

Disallow

surveybot

Rule

Path

Disallow

xenu's

Rule

Path

Disallow

xenu's link sleuth 1.1c

Rule

Path

Disallow

rogerbot

Rule

Path

Disallow

truliabot

Rule

Path

Disallow

feedjira

Rule

Path

Disallow

semrushbot

Rule

Path

Disallow

semrushbot-sa

Rule

Path

Disallow

semrushbot-ba

Rule

Path

Disallow

semrushbot-si

Rule

Path

Disallow

semrushbot-swa

Rule

Path

Disallow

semrushbot-ct

Rule

Path

Disallow

semrushbot-bm

Rule

Path

Disallow

teleport

Rule

Path

Disallow

teleportpro

Rule

Path

Disallow

emailcollector

Rule

Path

Disallow

emailsiphon

Rule

Path

Disallow

webbandit

Rule

Path

Disallow

webzip

Rule

Path

Disallow

webreaper

Rule

Path

Disallow

webstripper

Rule

Path

Disallow

web downloader

Rule

Path

Disallow

webcopier

Rule

Path

Disallow

offline explorer pro

Rule

Path

Disallow

offline explorer

Rule

Path

Disallow

httrack website copier

Rule

Path

Disallow

offline commander

Rule

Path

Disallow

leech

Rule

Path

Disallow

websnake

Rule

Path

Disallow

blackwidow

Rule

Path

Disallow

http weazel

Rule

Path

Disallow

Comments

If the Joomla site is installed within a folder such as at
e.g. www.example.com/joomla/ the robots.txt file MUST be
moved to the site root at e.g. www.example.com/robots.txt
AND the joomla folder name MUST be prefixed to the disallowed
path, e.g. the Disallow rule for the /administrator/ folder
MUST be changed to read Disallow: /joomla/administrator/
For more information about the robots.txt standard, see:
http://www.robotstxt.org/orig.html
For syntax checking, see:
http://www.sxw.org.uk/computing/robots/check.html

Warnings

8 invalid lines.

antonianum.derobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

dotbot

giftghostbot

seznam

paperlibot

genieo

dataprovider/6.101

dataprovidersiteexplorer

dazoobot/1.0

diffbot

domainstatsbot/1.0

dotbot/1.1

dubaiindex

siteexplorer

ecommercebot

expertsearchspider

feedbin

fetch/2.0a

ffbot/1.0

focusbot/1.1

huaweisymantecspider

huaweisymantecspider/1.0

jobdiggerspider

lemurwebcrawler

lipperheylinkexplorer

lssrocketcrawler/1.0

lyt.srv1.5

miadev/0.0.1

najdi.si/3.1

bountiibot

experibot_v1

bixocrawler

bixocrawler testcrawler

crawler4j

crowsnest/0.5

cukbot

dataprovider/6.92

dblbot/1.0

diffbot/0.1

digg deeper/v1

discobot/1.0

discobot/1.1

discobot/2.0

discoverybot/2.0

dlvr.it/1.0

domainstatsbot/1.0

drupact/0.7

ezooms/1.0

fastbot crawler beta 2.0

fastbot crawler beta 4.0

feedly social

feedly/1.0

feedlybot/1.0

feedspot

feedspotbot/1.0

clickagy intelligence bot v2

classbot

cispa vulnerability notification

cirrusexplorer/1.1

checksem/nutch-1.10

catchbot/5.0

catchbot/3.0

catchbot/2.0

catchbot/1.0

camontspider/1.0

buzzbot/1.0

buzzbot

businessseek.biz_spider

bubing

blexbot

fyberspider/1.3

findlinks/1.1.6-beta5

g2reader-bot/1.0

findlinks/1.1.6-beta6

findlinks/2.0

antonianum.de
robots.txt