elcorreodeburgos.elmundo.es
robots.txt

Robots Exclusion Standard data for elcorreodeburgos.elmundo.es

Archived Snapshots

Resource Scan

Scan Details

Site Domain	elcorreodeburgos.elmundo.es
Base Domain	elmundo.es
Scan Status	Ok
Last Scan	2024-04-26T23:44:39+00:00
Next Scan	2024-05-03T23:44:39+00:00

Last Scan

Scanned	2024-04-26T23:44:39+00:00
URL	https://elcorreodeburgos.elmundo.es/robots.txt
Redirect	https://www.elcorreodeburgos.com/robots.txt
Redirect Domain	www.elcorreodeburgos.com
Redirect Base	elcorreodeburgos.com
Domain IPs	18.161.6.125, 18.161.6.3, 18.161.6.60, 18.161.6.93, 2600:9000:24db:1800:1b:4c26:a80:93a1, 2600:9000:24db:4600:1b:4c26:a80:93a1, 2600:9000:24db:4c00:1b:4c26:a80:93a1, 2600:9000:24db:6600:1b:4c26:a80:93a1, 2600:9000:24db:9400:1b:4c26:a80:93a1, 2600:9000:24db:a000:1b:4c26:a80:93a1, 2600:9000:24db:d400:1b:4c26:a80:93a1, 2600:9000:24db:dc00:1b:4c26:a80:93a1
Redirect IPs	18.154.132.123, 18.154.132.23, 18.154.132.39, 18.154.132.46, 2600:9000:24db:1400:1b:4c26:a80:93a1, 2600:9000:24db:5000:1b:4c26:a80:93a1, 2600:9000:24db:7c00:1b:4c26:a80:93a1, 2600:9000:24db:aa00:1b:4c26:a80:93a1, 2600:9000:24db:b600:1b:4c26:a80:93a1, 2600:9000:24db:d600:1b:4c26:a80:93a1, 2600:9000:24db:f200:1b:4c26:a80:93a1, 2600:9000:24db:f600:1b:4c26:a80:93a1
Response IP	18.165.171.69
Found	Yes
Hash	8b1e593f65ffe7e857446743139a7a7ba2529d0eda9ad58f0a76b270d9365141
SimHash	a0d259d3cef7

Groups

*

Rule	Path
Allow	/ads.txt
Disallow	/*.swf$
Disallow	/*.tif$
Disallow	/*.mp3$
Disallow	/*.flv$
Disallow	/*.mp4$
Disallow	/*.avi$
Disallow	/*.txt$
Disallow	/*.ics$

Rule

Path

Allow

/ads.txt

Disallow

/*.swf$

Disallow

/*.tif$

Disallow

/*.mp3$

Disallow

/*.flv$

Disallow

/*.mp4$

Disallow

/*.avi$

Disallow

/*.txt$

Disallow

/*.ics$

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

ubicrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

doc

Rule	Path
Disallow	/

Rule

Path

Disallow

zao

Rule	Path
Disallow	/

Rule

Path

Disallow

sitecheck.internetseer.com

Rule	Path
Disallow	/

Rule

Path

Disallow

zealbot

Rule	Path
Disallow	/

Rule

Path

Disallow

msiecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

sitesnagger

Rule	Path
Disallow	/

Rule

Path

Disallow

webstripper

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

fetch

Rule	Path
Disallow	/

Rule

Path

Disallow

offline explorer

Rule	Path
Disallow	/

Rule

Path

Disallow

teleport

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

linko

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

microsoft.url.control

Rule	Path
Disallow	/

Rule

Path

Disallow

xenu

Rule	Path
Disallow	/

Rule

Path

Disallow

larbin

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww

Rule	Path
Disallow	/

Rule

Path

Disallow

zyborg

Rule	Path
Disallow	/

Rule

Path

Disallow

download ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

maxthon

Rule	Path
Disallow	/

Rule

Path

Disallow

cncdialer

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mailto: techinfo@ubermetrics-technologies.com

Rule	Path
Disallow	/

Rule

Path

Disallow

Other Records

Field	Value
sitemap	https://www.elcorreodeburgos.com/sitemap-google-news.xml

Field

Value

sitemap

https://www.elcorreodeburgos.com/sitemap-google-news.xml

Comments

Extensiones de contenidos no indexables
Agentes nocivos conocidos

elcorreodeburgos.elmundo.esrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

ia_archiver

ubicrawler

doc

zao

sitecheck.internetseer.com

zealbot

msiecrawler

sitesnagger

webstripper

webcopier

fetch

offline explorer

teleport

teleportpro

webzip

linko

httrack

microsoft.url.control

xenu

larbin

libwww

zyborg

download ninja

slurp

maxthon

cncdialer

ahrefsbot

mailto: techinfo@ubermetrics-technologies.com

Other Records

Comments

elcorreodeburgos.elmundo.es
robots.txt