Log In Sign Up

bari.repubblica.it
robots.txt

Robots Exclusion Standard data for bari.repubblica.it

Resource Scan

Scan Details

Site Domain bari.repubblica.it
Base Domain repubblica.it
Scan Status Ok
Last Scan2025-03-10T03:12:24+00:00
Next Scan 2025-03-17T03:12:24+00:00

Last Scan

Scanned2025-03-10T03:12:24+00:00
URL https://bari.repubblica.it/robots.txt
Domain IPs 3.165.75.113, 3.165.75.25, 3.165.75.28, 3.165.75.55
Response IP 3.165.75.113
Found Yes
Hash dc744bbc3d706c27314a57cfef960e58e58da939ba7cd24568e6f5249b0e974e
SimHash 324451109987

Groups

*

Rule Path
Disallow /ristoranti/
Disallow /multimedia/
Disallow /images/2013/11/12/161242653-dc034546-e1c0-426d-905f-14cc2020cabc.jpg
Disallow /dettaglio/
Disallow /dettaglio-news/palermo-13%3A28/2756419
Disallow /dettaglio-news/
Disallow /cronaca/2022/12/17/news/giustizia_capristo_e_nardi_a_processo_hanno_fatto_mercimonio_della_funzione_di_procuratore_della_repubblica-379383874/
Disallow /cronaca/2021/11/05/news/bari_falso_investimento_in_pozzi_di_petrolio_in_mozambico_e_oklahoma_ingegnere_a_processo_per_truffa-325232865/
Disallow /cronaca/2021/10/30/news/primo_novembre_a_bari_apertura_straordinaria_del_mercato_del_lunedi_-324359553/
Disallow /cronaca/2018/04/13/news/lecce_scomparsa_da_giorni_una_44enne_ricerche_anche_sui_traghetti_in_partenza_da_brindisi-193755159/
Disallow /cronaca/2018/02/17/news/barletta_due_quintali_di_hashish_a_bordo_di_auto_rubate_arrestati_in_tre-189073699/
Disallow /cronaca/2015/03/16/foto/escort-109637844/1/
Disallow /cronaca/2014/05/17/news/appalti_pilotati-86423324
Disallow /blaize/datalayer

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

linkarchiver

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gumgum bot

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

youbot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

web-archive-net.com.bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

nicecrawler

Rule Path
Disallow /

url_spider_pro

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

nabot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

lexxebot/1.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

archivebot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

wotbot

Rule Path
Disallow /

fetch

Rule Path
Disallow /

nutch

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

europarchive.org

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

umbot-ln

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

true_robot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

slurp

Rule Path
Disallow /

linkextractorpro

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

searchpreview

Rule Path
Disallow /

bixocrawler

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

quora-bot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

primalbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

openbot

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

npbot

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

nnetseer crawler

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

dloader(naverrobot)

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

moreoverbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

sitebot/0.1

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://bari.repubblica.it/sitemap-n.xml
Back to top