diariodecastillayleon.elmundo.es
robots.txt

Robots Exclusion Standard data for diariodecastillayleon.elmundo.es

Archived Snapshots

Resource Scan

Scan Details

Site Domain	diariodecastillayleon.elmundo.es
Base Domain	elmundo.es
Scan Status	Ok
Last Scan	2024-05-01T04:50:38+00:00
Next Scan	2024-05-08T04:50:38+00:00

Last Scan

Scanned	2024-05-01T04:50:38+00:00
URL	https://diariodecastillayleon.elmundo.es/robots.txt
Redirect	https://www.diariodecastillayleon.es/robots.txt
Redirect Domain	www.diariodecastillayleon.es
Redirect Base	diariodecastillayleon.es
Domain IPs	13.33.21.120, 13.33.21.124, 13.33.21.51, 13.33.21.83, 2600:9000:200f:3000:1c:370b:3c40:93a1, 2600:9000:200f:3200:1c:370b:3c40:93a1, 2600:9000:200f:3600:1c:370b:3c40:93a1, 2600:9000:200f:5c00:1c:370b:3c40:93a1, 2600:9000:200f:7c00:1c:370b:3c40:93a1, 2600:9000:200f:9000:1c:370b:3c40:93a1, 2600:9000:200f:b000:1c:370b:3c40:93a1, 2600:9000:200f:fc00:1c:370b:3c40:93a1
Redirect IPs	13.33.21.120, 13.33.21.124, 13.33.21.51, 13.33.21.83, 2600:9000:2363:1200:1c:370b:3c40:93a1, 2600:9000:2363:3400:1c:370b:3c40:93a1, 2600:9000:2363:5c00:1c:370b:3c40:93a1, 2600:9000:2363:8e00:1c:370b:3c40:93a1, 2600:9000:2363:a400:1c:370b:3c40:93a1, 2600:9000:2363:c400:1c:370b:3c40:93a1, 2600:9000:2363:ce00:1c:370b:3c40:93a1, 2600:9000:2363:f600:1c:370b:3c40:93a1
Response IP	18.165.171.99
Found	Yes
Hash	b706b208b837bc0681ee8a5fdd0ab14720a0b0b0a8bfb69b3a069acaf2f34c1e
SimHash	a0d358c8ced3

Groups

*

Rule	Path
Disallow	/*.swf$
Disallow	/*.tif$
Disallow	/*.mp3$
Disallow	/*.flv$
Disallow	/*.mp4$
Disallow	/*.avi$
Disallow	/*.ics$

Rule

Path

Disallow

/*.swf$

Disallow

/*.tif$

Disallow

/*.mp3$

Disallow

/*.flv$

Disallow

/*.mp4$

Disallow

/*.avi$

Disallow

/*.ics$

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

ubicrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

doc

Rule	Path
Disallow	/

Rule

Path

Disallow

zao

Rule	Path
Disallow	/

Rule

Path

Disallow

sitecheck.internetseer.com

Rule	Path
Disallow	/

Rule

Path

Disallow

zealbot

Rule	Path
Disallow	/

Rule

Path

Disallow

msiecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

sitesnagger

Rule	Path
Disallow	/

Rule

Path

Disallow

webstripper

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

fetch

Rule	Path
Disallow	/

Rule

Path

Disallow

offline explorer

Rule	Path
Disallow	/

Rule

Path

Disallow

teleport

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

linko

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

microsoft.url.control

Rule	Path
Disallow	/

Rule

Path

Disallow

xenu

Rule	Path
Disallow	/

Rule

Path

Disallow

larbin

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww

Rule	Path
Disallow	/

Rule

Path

Disallow

zyborg

Rule	Path
Disallow	/

Rule

Path

Disallow

download ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

maxthon

Rule	Path
Disallow	/

Rule

Path

Disallow

cncdialer

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mailto: techinfo@ubermetrics-technologies.com

Rule	Path
Disallow	/

Rule

Path

Disallow

Comments

Agentes nocivos conocidos

diariodecastillayleon.elmundo.esrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

ia_archiver

ubicrawler

doc

zao

sitecheck.internetseer.com

zealbot

msiecrawler

sitesnagger

webstripper

webcopier

fetch

offline explorer

teleport

teleportpro

webzip

linko

httrack

microsoft.url.control

xenu

larbin

libwww

zyborg

download ninja

slurp

maxthon

cncdialer

ahrefsbot

mailto: techinfo@ubermetrics-technologies.com

Comments

diariodecastillayleon.elmundo.es
robots.txt