libero.it
robots.txt

Robots Exclusion Standard data for libero.it

Resource Scan

Scan Details

Site Domain libero.it
Base Domain libero.it
Scan Status Ok
Last Scan2024-11-13T17:21:13+00:00
Next Scan 2024-11-20T17:21:13+00:00

Last Scan

Scanned2024-11-13T17:21:13+00:00
URL https://libero.it/robots.txt
Redirect https://www.libero.it/robots.txt
Redirect Domain www.libero.it
Redirect Base libero.it
Domain IPs 213.209.17.209
Redirect IPs 52.222.144.18, 52.222.144.25, 52.222.144.67, 52.222.144.81
Response IP 52.85.49.29
Found Yes
Hash 674e95078f314a775f2bff9edb378de9a1f398eb9fdbe3669964a878ad2de744
SimHash f0dd31c8ce21

Groups

*

Rule Path
Disallow /temicaldi.php?*
Disallow /magazine/guida-tv/*?
Disallow /magazine/*ricerca/*

ahrefsbot

Rule Path
Disallow /magazine/
Disallow /tv/

blexbot

Rule Path
Disallow /magazine/
Disallow /tv/

bloglines/3.1

Rule Path
Disallow /magazine/
Disallow /tv/

cityreview

Rule Path
Disallow /magazine/
Disallow /tv/

doc

Rule Path
Disallow /magazine/
Disallow /tv/

dotbot

Rule Path
Disallow /magazine/
Disallow /tv/

download ninja

Rule Path
Disallow /magazine/
Disallow /tv/

exabot

Rule Path
Disallow /magazine/
Disallow /tv/

fetch

Rule Path
Disallow /magazine/
Disallow /tv/

grapeshot

Rule Path
Disallow /magazine/
Disallow /tv/

grub-client

Rule Path
Disallow /magazine/
Disallow /tv/

httrack

Rule Path
Disallow /magazine/
Disallow /tv/

jyxobot/1

Rule Path
Disallow /magazine/
Disallow /tv/

k2spider

Rule Path
Disallow /magazine/
Disallow /tv/

larbin

Rule Path
Disallow /magazine/
Disallow /tv/

libwww

Rule Path
Disallow /magazine/
Disallow /tv/

linko

Rule Path
Disallow /magazine/
Disallow /tv/

microsoft.url.control

Rule Path
Disallow /magazine/
Disallow /tv/

msiecrawler

Rule Path
Disallow /magazine/
Disallow /tv/

npbot

Rule Path
Disallow /magazine/
Disallow /tv/

offline explorer

Rule Path
Disallow /magazine/
Disallow /tv/

petalbot

Rule Path
Disallow /magazine/
Disallow /tv/

proximic

Rule Path
Disallow /magazine/
Disallow /tv/

psbot

Rule Path
Disallow /magazine/
Disallow /tv/

semrushbot

Rule Path
Disallow /magazine/
Disallow /tv/

sitecheck.internetseer.com

Rule Path
Disallow /magazine/
Disallow /tv/

sitesnagger

Rule Path
Disallow /magazine/
Disallow /tv/

speedy

Rule Path
Disallow /magazine/
Disallow /tv/

teleport

Rule Path
Disallow /magazine/
Disallow /tv/

teleportpro

Rule Path
Disallow /magazine/
Disallow /tv/

ubicrawler

Rule Path
Disallow /magazine/
Disallow /tv/

webcopier

Rule Path
Disallow /magazine/
Disallow /tv/

webreaper

Rule Path
Disallow /magazine/
Disallow /tv/

webstripper

Rule Path
Disallow /magazine/
Disallow /tv/

webzip

Rule Path
Disallow /magazine/
Disallow /tv/

wget

Rule Path
Disallow /magazine/
Disallow /tv/

xenu

Rule Path
Disallow /magazine/
Disallow /tv/

zao

Rule Path
Disallow /magazine/
Disallow /tv/

zealbot

Rule Path
Disallow /magazine/
Disallow /tv/

zyborg

Rule Path
Disallow /magazine/
Disallow /tv/

Other Records

Field Value
sitemap https://www.libero.it/magazine/sitemap/sitemap_magazine.xml
sitemap https://www.libero.it/tv/sitemap/sitemap.xml