diariovasco.com
robots.txt

Robots Exclusion Standard data for diariovasco.com

Resource Scan

Scan Details

Site Domain diariovasco.com
Base Domain diariovasco.com
Scan Status Ok
Last Scan2024-05-09T06:08:31+00:00
Next Scan 2024-05-16T06:08:31+00:00

Last Scan

Scanned2024-05-09T06:08:31+00:00
URL https://diariovasco.com/robots.txt
Redirect https://www.diariovasco.com/robots.txt
Redirect Domain www.diariovasco.com
Redirect Base diariovasco.com
Domain IPs 23.215.7.14, 23.215.7.21
Redirect IPs 23.32.29.11, 96.17.180.50
Response IP 23.52.171.218
Found Yes
Hash ebb5d2a1f6e6a3953c021b0d6d676b31af428412878f26d188f5078d6e08edf3
SimHash 2a249c640e71

Groups

mediapartners-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

twitterbot

Rule Path
Disallow *

grapeshot

Rule Path
Allow *2020*

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /acd/
Disallow /fcgi-bin/
Disallow /*-preview.html
Disallow /registro/
Disallow /popups/
Disallow /preview/
Disallow /_catalogo/
Disallow /includes/manuales/_catalogo/
Disallow /portada/modulo/blockadblocker/
Disallow */suscr/
Disallow /diario_vasco_2009/
Disallow /_cabeceraExterna/
Disallow /_config/
Disallow /externo/
Disallow /includes/
Disallow /metas.html
Disallow /_minicabecera/
Disallow /MM/
Disallow /modulos/
Disallow /nullmetas.html
Disallow /SysConfig/
Disallow /4900/vocento.diariovasco/
Disallow /zHomePrueba/index.html
Disallow /pruebaspubli.html
Disallow */guia-tv/
Disallow /4900/webm.DIARIOVASCO/
Disallow /RC/MM/
Disallow /diario_vasco/MM/
Disallow /diario_vasco/noticias/
Disallow /apoyos/documentos/
Disallow /gl-d/
Disallow */gl-d$
Disallow /ht-d/
Disallow */ht-d$
Disallow /bb-d/
Disallow */bb-d$
Disallow /*?ns_
Disallow /module-amp/
Disallow */aggregate*
Disallow */multimedia/fotos/
Disallow */pg*/prensa/
Disallow */planes/2014
Disallow */planes/2016
Disallow */v/
Disallow */fotos/*.html
Disallow */prensa/
Disallow /backend/
Disallow /hemeroteca/*.html
Disallow *-mm.html

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.diariovasco.com/sitemap.xml
sitemap https://www.diariovasco.com/sitemap.incremental.xml
sitemap https://www.diariovasco.com/sitemap-video.xml
sitemap https://www.diariovasco.com/sitemap-temas.xml

Comments

  • Robots www.diariovasco.com
  • User Agents
  • redi14
  • mob
  • Sitemaps