notizie24.eu
robots.txt

Robots Exclusion Standard data for notizie24.eu

Archived Snapshots

Resource Scan

Scan Details

Site Domain	notizie24.eu
Base Domain	notizie24.eu
Scan Status	Ok
Last Scan	2024-11-10T23:31:17+00:00
Next Scan	2024-11-17T23:31:17+00:00

Last Scan

Scanned	2024-11-10T23:31:17+00:00
URL	http://notizie24.eu/robots.txt
Redirect	https://www.okmedicina.it/robots.txt
Redirect Domain	www.okmedicina.it
Redirect Base	okmedicina.it
Domain IPs	2.236.15.120
Redirect IPs	2.236.15.120
Response IP	2.236.15.120
Found	Yes
Hash	9af368ed55db9c45ab20a5768eb126c2f49b148bf71d4cf2760e0dcecf09d22f
SimHash	72991d70e6f8

Groups

*

Rule	Path
Disallow	/administrator/
Disallow	/bin/
Disallow	/cache/
Disallow	/cli/
Disallow	/includes/
Disallow	/language/
Disallow	/layouts/
Disallow	/libraries/
Disallow	/logs/
Disallow	/log/
Disallow	/plugins/
Disallow	/tmp/
Disallow	/medici/mail/

Rule

Path

Disallow

/administrator/

Disallow

/bin/

Disallow

/cache/

Disallow

/cli/

Disallow

/includes/

Disallow

/language/

Disallow

/layouts/

Disallow

/libraries/

Disallow

/logs/

Disallow

/log/

Disallow

/plugins/

Disallow

/tmp/

Disallow

/medici/mail/

seekportbot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Product	Comment
amazonbot	Amazon's user agent

Product

Comment

amazonbot

Amazon's user agent

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/
Disallow	/

Rule

Path

Disallow

abonti

Rule	Path
Disallow	/

Rule

Path

Disallow

abonti/0.92

Rule	Path
Disallow	/

Rule

Path

Disallow

abot v1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

aboutthedomain

Rule	Path
Disallow	/

Rule

Path

Disallow

add catalog

Rule	Path
Disallow	/

Rule

Path

Disallow

add catalog/2.1

Rule	Path
Disallow	/

Rule

Path

Disallow

advbot

Rule	Path
Disallow	/

Rule

Path

Disallow

advbot/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs-bot/6.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs-bot/7.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs-bot/3.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs-bot/4.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs-bot/5.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

No rules defined. All paths allowed.

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aihitbot/2.9

Rule	Path
Disallow	/

Rule

Path

Disallow

anonymous/0.0

Rule	Path
Disallow	/

Rule

Path

Disallow

arachnida

Rule	Path
Disallow	/

Rule

Path

Disallow

associative spider

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

baidu spider

Rule	Path
Disallow	/

Rule

Path

Disallow

battleztar bazinga

Rule	Path
Disallow	/

Rule

Path

Disallow

battleztar bazinga/0.01

Rule	Path
Disallow	/

Rule

Path

Disallow

bdfetch

Rule	Path
Disallow	/

Rule

Path

Disallow

betabot

Rule	Path
Disallow	/

Rule

Path

Disallow

bieshu

Rule	Path
Disallow	/

Rule

Path

Disallow

bigli seo

Rule	Path
Disallow	/

Rule

Path

Disallow

blackboard safeassign

Rule	Path
Disallow	/

Rule

Path

Disallow

blazer 1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

blp_bbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blp_bbot/0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

boia-accessibility-agent/pr 1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

bot for jce

Rule	Path
Disallow	/

Rule

Path

Disallow

bot/0.1 (bot for jce)

Rule	Path
Disallow	/

Rule

Path

Disallow

bpimagewalker

Rule	Path
Disallow	/

Rule

Path

Disallow

bpimagewalker/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

buibui-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

buibui-bot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

ca-crawler

Rule

Path

Disallow

ca-crawler/1.0

Rule

Path

Disallow

cakephp

Rule

Path

Disallow

calypso v/0.01

Rule

Path

Disallow

calypso

Rule

Path

Disallow

cb/nutch-1.7

Rule

Path

Disallow

ccbot

Rule

Path

Disallow

ccbot/2.0

Rule

Path

Disallow

checkbot

Rule

Path

Disallow

checkgzipcompression.com

Rule

Path

Disallow

chushou

Rule

Path

Disallow

cloudservermarketspider

Rule

Path

Disallow

cloudservermarketspider/1.0

Rule

Path

Disallow

clushbot/3.x-binaryfury

Rule

Path

Disallow

cms crawler

Rule

Path

Disallow

cms crawler: http://www.cmscrawler.com

Rule

Path

Disallow

coccoc

Rule

Path

Disallow

coincornerbot

Rule

Path

Disallow

coincornerbot/1.1

Rule

Path

Disallow

copyscape

Rule

Path

Disallow

crawler4j

Rule

Path

Disallow

crazywebcrawler 0.9.0

Rule

Path

Disallow

crazywebcrawler 0.9.1

Rule

Path

Disallow

crazywebcrawler 0.9.7

Rule

Path

Disallow

crazywebcrawler

Rule

Path

Disallow

crazywebcrawler-spider

Rule

Path

Disallow

crowsnest

Rule

Path

Disallow

crowsnest/0.5

Rule

Path

Disallow

curious george - www.analyticsseo.com/crawler

Rule

Path

Disallow

curious george

Rule

Path

Disallow

cuwhois

Rule

Path

Disallow

cuwhois/1.0

Rule

Path

Disallow

dahoms

Rule

Path

Disallow

datagnionbot

Rule

Path

Disallow

deusu/5.0.2

Rule

Path

Disallow

digincore

Rule

Path

Disallow

digincore bot

Rule

Path

Disallow

dispatch/0.11.0

Rule

Path

Disallow

domain re-animator bot

Rule

Path

Disallow

domainappender /1.0

Rule

Path

Disallow

domainappender

Rule

Path

Disallow

domaincrawler/3.0

Rule

Path

Disallow

domainsigmacrawler

Rule

Path

Disallow

domainsigmacrawler/0.1

Rule

Path

Disallow

domnutch

Rule

Path

Disallow

domnutch-bot

Rule

Path

Disallow

domnutch-bot/nutch

Rule

Path

Disallow

domnutch-bot/nutch-1.0

Rule

Path

Disallow

dotbot

Rule

Path

Disallow

eccp/1.2.1

Rule

Path

Disallow

ecommercebot

Rule

Path

Disallow

enlle punto com/nutch-1.9

Rule

Path

Disallow

episerver link checker

Rule

Path

Disallow

euripbot

Rule

Path

Disallow

euripbot/2.0

Rule

Path

Disallow

evc/2.0

Rule

Path

Disallow

evc-batch

Rule

Path

Disallow

evc-batch/2.0

Rule

Path

Disallow

express webpictures

Rule

Path

Disallow

faraday v0.8.8

Rule

Path

Disallow

faraday

Rule

Path

Disallow

findxbot

Rule

Path

Disallow

findxbot/1.0

Rule

Path

Disallow

flamingo_searchengine

Rule

Path

Disallow

flipboard robot

Rule

Path

Disallow

getproxi.es-bot

Rule

Path

Disallow

getproxi.es-bot/1.1

Rule

Path

Disallow

gigablastopensource

Rule

Path

Disallow

gigablastopensource/1.0

Rule

Path

Disallow

girafabot

Rule

Path

Disallow

gluten free crawler

Rule

Path

Disallow

gluten free crawler/1.0

Rule

Path

Disallow

griffinbot

Rule

Path

Disallow

grifinbot/0.01

Rule

Path

Disallow

gwpimages

Rule

Path

Disallow

gwpimages/1.0

Rule

Path

Disallow

haiula

Rule

Path

Disallow

haiula/1.4

Rule

Path

Disallow

haosouspider

Rule

Path

Disallow

hivemind

Rule

Path

Disallow

hostharvest

Rule

Path

Disallow

hostharvest/0.4.28

Rule

Path

Disallow

hrcrawler

Rule

Path

Disallow

hrcrawler/2.0

Rule

Path

Disallow

http://git.io/tl_s2w

Rule

Path

Disallow

http://www.checkprivacy.or.kr:6600/rs/privacy_enfaq.jsp

Rule

Path

Disallow

hubspot links crawler 1.0

Rule

Path

Disallow

hubspot webcrawler

Rule

Path

Disallow

hubspot

Rule

Path

Disallow

hunchan

Rule

Path

Disallow

hypercrawl

Rule

Path

Disallow

hypercrawl/0.2

Rule

Path

Disallow

icap-iod

Rule

Path

Disallow

icc-crawler

Rule

Path

Disallow

icc-crawler/2.0

Rule

Path

Disallow

ichiro robot

Rule

Path

Disallow

image.coccoc/1.0

Rule

Path

Disallow

image2play

Rule

Path

Disallow

image2play/0.1

Rule

Path

Disallow

indy library

Rule

Path

Disallow

insightscollector

Rule

Path

Disallow

insightscollector/0.1

Rule

Path

Disallow

insightscollector/0.1beta

Rule

Path

Disallow

integrity/5

Rule

Path

Disallow

internaetboten

Rule

Path

Disallow

internaetboten/0.99

Rule

Path

Disallow

irl crawler

Rule

Path

Disallow

james bot - webcrawler

Rule

Path

Disallow

james bot

Rule

Path

Disallow

jamesbot

Rule

Path

Disallow

jetbrains 5.0

Rule

Path

Disallow

jetbrains

Rule

Path

Disallow

kraken

Rule

Path

Disallow

kraken/0.1

Rule

Path

Disallow

kyoto-tohoku-crawler/v1

Rule

Path

Disallow

larbin

Rule

Path

Disallow

lechenie

Rule

Path

Disallow

libwww-perl

Rule

Path

Disallow

link checker

Rule

Path

Disallow

link/1.0

Rule

Path

Disallow

linkcheck

Rule

Path

Disallow

linkcheckv3.0

Rule

Path

Disallow

linkdex

Rule

Path

Disallow

linkdex.com/v2.0

Rule

Path

Disallow

linkdex.com/v2.1

Rule

Path

Disallow

linkdexbot

Rule

Path

Disallow

linkdexbot/2.0

Rule

Path

Disallow

linkdexbot/2.1

Rule

Path

Disallow

linkdexbot-mobile/2.1

Rule

Path

Disallow

linkpadbot

Rule

Path

Disallow

linkpadbot/1.06

Rule

Path

Disallow

linqiascrapebot

Rule

Path

Disallow

linqiascrapebot/1.0

Rule

Path

Disallow

lipperhey seo service

Rule

Path

Disallow

lipperhey

Rule

Path

Disallow

lipperhey-kaus-australis

Rule

Path

Disallow

lipperhey-kaus-australis/5.0

Rule

Path

Disallow

listicka

Rule

Path

Disallow

lssrocketcrawler

Rule

Path

Disallow

lssrocketcrawler/1.0 lightspeedsystems

Rule

Path

Disallow

lssrocketcrawler/1.0

Rule

Path

Disallow

ltx71

Rule

Path

Disallow

lwnutch/nutch-1.4

Rule

Path

Disallow

mail.ru

Rule

Path

Disallow

mail.ru_bot

Rule

Path

Disallow

mail.ru_bot/2.0

Rule

Path

Disallow

mail.ru_bot/fast/2.0

Rule

Path

Disallow

meanpathbot

Rule

Path

Disallow

megaindex.ru

Rule

Path

Disallow

megaindex.ru/2.0

Rule

Path

Disallow

mezhpozvonochnoi

Rule

Path

Disallow

mike-crawler

Rule

Path

Disallow

mixbot

Rule

Path

Disallow

mixrankbot

Rule

Path

Disallow

mj12bot

Rule

Path

Disallow

monkeybot/0.1

Rule

Path

Disallow

my crawler

Rule

Path

Disallow

my nutch spider/nutch-1.9

Rule

Path

Disallow

mycrowl/nutch-1.9

Rule

Path

Disallow

mygreatua/2.0

Rule

Path

Disallow

myiptest

Rule

Path

Disallow

nameprotect robot

Rule

Path

Disallow

nerdybot

Rule

Path

Disallow

netcraft spider

Rule

Path

Disallow

netestate ne crawler

Rule

Path

Disallow

netlyzer fastprobe

Rule

Path

Disallow

netresearchserver

Rule

Path

Disallow

netresearchserver/4.0

Rule

Path

Disallow

nmap scripting engine

Rule

Path

Disallow

node.io

Rule

Path

Disallow

node.js

Rule

Path

Disallow

node/simplecrawler 0.5.2

Rule

Path

Disallow

node/simplecrawler

Rule

Path

Disallow

obot/2.3.1

Rule

Path

Disallow

omgilibot

Rule

Path

Disallow

omgilibot/0.4

Rule

Path

Disallow

online domain tools - online website link checker

Rule

Path

Disallow

online domain tools - online website link checker/1.2

Rule

Path

Disallow

openfind robot

Rule

Path

Disallow

openhosebot

Rule

Path

Disallow

openhosebot/2.1

Rule

Path

Disallow

openstat

Rule

Path

Disallow

openstat/0.1

Rule

Path

Disallow

optimizationcrawler

Rule

Path

Disallow

optimizationcrawler/0.2

Rule

Path

Disallow

page analyzer v4.0

Rule

Path

Disallow

page analyzer

Rule

Path

Disallow

pageanalyzer

Rule

Path

Disallow

pageanalyzer/1.1

Rule

Path

Disallow

pageanalyzer/1.5

Rule

Path

Disallow

pagesinventory

Rule

Path

Disallow

pagespeed/1.1 fetcher

Rule

Path

Disallow

pagespeed/1.1

Rule

Path

Disallow

pagespeedbot

Rule

Path

Disallow

perl lwp

Rule

Path

Disallow

phpcrawl

Rule

Path

Disallow

phpsitecheck 1.0

Rule

Path

Disallow

phpsitecheck

Rule

Path

Disallow

plukkie

Rule

Path

Disallow

pogs/2.0

Rule

Path

Disallow

powermarks

Rule

Path

Disallow

powerpivot

Rule

Path

Disallow

privacy_enfaq.jsp

Rule

Path

Disallow

prlog

Rule

Path

Disallow

prlog/1.0

Rule

Path

Disallow

publiclibraryarchive.org

Rule

Path

Disallow

publiclibraryarchive.org/1.0

Rule

Path

Disallow

pu_in crawler

Rule

Path

Disallow

putin

Rule

Path

Disallow

putin spider

Rule

Path

Disallow

qingdao

Rule

Path

Disallow

qlikview

Rule

Path

Disallow

quipu

Rule

Path

Disallow

quipu/1.0

Rule

Path

Disallow

quipu/2.0

Rule

Path

Disallow

r6_commentreader

Rule

Path

Disallow

r6_feedfetcher

Rule

Path

Disallow

riddler

Rule

Path

Disallow

rivalseek.com-bot

Rule

Path

Disallow

rogerbot

Rule

Path

Disallow

rogerbot/1.0

Rule

Path

Disallow

rootlink

Rule

Path

Disallow

ru_bot/2.0

Rule

Path

Disallow

Comments

If the Joomla site is installed within a folder such as at
e.g. www.example.com/joomla/ the robots.txt file MUST be
moved to the site root at e.g. www.example.com/robots.txt
AND the joomla folder name MUST be prefixed to the disallowed
path, e.g. the Disallow rule for the /administrator/ folder
MUST be changed to read Disallow: /joomla/administrator/
For syntax checking, see:
http://tool.motoricerca.info/robots-checker.phtml

Warnings

7 invalid lines.
`ser-agent` is not a known field.

notizie24.eurobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

seekportbot

amazonbot

mj12bot

bytespider

abonti

abonti/0.92

abot v1.0

aboutthedomain

add catalog

add catalog/2.1

advbot

advbot/2.0

ahrefs-bot

ahrefsbot/1.0

ahrefs-bot/6.0

ahrefs-bot/7.0

ahrefs-bot/3.0

ahrefs-bot/4.0

ahrefs-bot/5.0

ahrefsbot

aihitbot

aihitbot/2.9

anonymous/0.0

arachnida

associative spider

baiduspider

baidu spider

battleztar bazinga

battleztar bazinga/0.01

bdfetch

betabot

bieshu

bigli seo

blackboard safeassign

blazer 1.0

blexbot

blexbot/1.0

blp_bbot

blp_bbot/0.1

boia-accessibility-agent/pr 1.0

bot for jce

bot/0.1 (bot for jce)

bpimagewalker

bpimagewalker/2.0

bubing

buibui-bot

buibui-bot/1.0

ca-crawler

ca-crawler/1.0

cakephp

calypso v/0.01

calypso

cb/nutch-1.7

ccbot

ccbot/2.0

checkbot

checkgzipcompression.com

chushou

cloudservermarketspider

cloudservermarketspider/1.0

clushbot/3.x-binaryfury

cms crawler

cms crawler: http://www.cmscrawler.com

coccoc

coincornerbot

coincornerbot/1.1

copyscape

crawler4j

crazywebcrawler 0.9.0

crazywebcrawler 0.9.1

crazywebcrawler 0.9.7

crazywebcrawler

crazywebcrawler-spider

crowsnest

notizie24.eu
robots.txt