diariodevalladolid.elmundo.es
robots.txt

Robots Exclusion Standard data for diariodevalladolid.elmundo.es

Archived Snapshots

Resource Scan

Scan Details

Site Domain	diariodevalladolid.elmundo.es
Base Domain	elmundo.es
Scan Status	Ok
Last Scan	2024-05-04T01:39:48+00:00
Next Scan	2024-05-11T01:39:48+00:00

Last Scan

Scanned	2024-05-04T01:39:48+00:00
URL	https://diariodevalladolid.elmundo.es/robots.txt
Redirect	https://www.diariodevalladolid.es/robots.txt
Redirect Domain	www.diariodevalladolid.es
Redirect Base	diariodevalladolid.es
Domain IPs	13.226.228.101, 13.226.228.112, 13.226.228.4, 13.226.228.86, 2600:9000:2201:2600:11:a435:9340:93a1, 2600:9000:2201:2800:11:a435:9340:93a1, 2600:9000:2201:3400:11:a435:9340:93a1, 2600:9000:2201:600:11:a435:9340:93a1, 2600:9000:2201:6200:11:a435:9340:93a1, 2600:9000:2201:8000:11:a435:9340:93a1, 2600:9000:2201:8600:11:a435:9340:93a1, 2600:9000:2201:d200:11:a435:9340:93a1
Redirect IPs	13.226.228.101, 13.226.228.112, 13.226.228.4, 13.226.228.86, 2600:9000:2201:1600:11:a435:9340:93a1, 2600:9000:2201:2e00:11:a435:9340:93a1, 2600:9000:2201:5c00:11:a435:9340:93a1, 2600:9000:2201:5e00:11:a435:9340:93a1, 2600:9000:2201:9600:11:a435:9340:93a1, 2600:9000:2201:a400:11:a435:9340:93a1, 2600:9000:2201:b400:11:a435:9340:93a1, 2600:9000:2201:e600:11:a435:9340:93a1
Response IP	18.165.171.102
Found	Yes
Hash	c59bbeeb291d4a4e7b8756b59c15c08f00616ee06e87a316a82a912ae70dc922
SimHash	a0d358c8ced3

Groups

*

Rule	Path
Disallow	/*.swf$
Disallow	/*.tif$
Disallow	/*.mp3$
Disallow	/*.flv$
Disallow	/*.mp4$
Disallow	/*.avi$
Disallow	/*.ics$

Rule

Path

Disallow

/*.swf$

Disallow

/*.tif$

Disallow

/*.mp3$

Disallow

/*.flv$

Disallow

/*.mp4$

Disallow

/*.avi$

Disallow

/*.ics$

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

ubicrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

doc

Rule	Path
Disallow	/

Rule

Path

Disallow

zao

Rule	Path
Disallow	/

Rule

Path

Disallow

sitecheck.internetseer.com

Rule	Path
Disallow	/

Rule

Path

Disallow

zealbot

Rule	Path
Disallow	/

Rule

Path

Disallow

msiecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

sitesnagger

Rule	Path
Disallow	/

Rule

Path

Disallow

webstripper

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

fetch

Rule	Path
Disallow	/

Rule

Path

Disallow

offline explorer

Rule	Path
Disallow	/

Rule

Path

Disallow

teleport

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

linko

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

microsoft.url.control

Rule	Path
Disallow	/

Rule

Path

Disallow

xenu

Rule	Path
Disallow	/

Rule

Path

Disallow

larbin

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww

Rule	Path
Disallow	/

Rule

Path

Disallow

zyborg

Rule	Path
Disallow	/

Rule

Path

Disallow

download ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

maxthon

Rule	Path
Disallow	/

Rule

Path

Disallow

cncdialer

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mailto: techinfo@ubermetrics-technologies.com

Rule	Path
Disallow	/

Rule

Path

Disallow

Comments

Agentes nocivos conocidos

diariodevalladolid.elmundo.esrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

ia_archiver

ubicrawler

doc

zao

sitecheck.internetseer.com

zealbot

msiecrawler

sitesnagger

webstripper

webcopier

fetch

offline explorer

teleport

teleportpro

webzip

linko

httrack

microsoft.url.control

xenu

larbin

libwww

zyborg

download ninja

slurp

maxthon

cncdialer

ahrefsbot

mailto: techinfo@ubermetrics-technologies.com

Comments

diariodevalladolid.elmundo.es
robots.txt