okdiario.com
robots.txt

Robots Exclusion Standard data for okdiario.com

Resource Scan

Scan Details

Site Domain okdiario.com
Base Domain okdiario.com
Scan Status Ok
Last Scan2024-05-03T20:40:31+00:00
Next Scan 2024-05-10T20:40:31+00:00

Last Scan

Scanned2024-05-03T20:40:31+00:00
URL https://okdiario.com/robots.txt
Domain IPs 104.18.18.5, 104.18.19.5, 2606:4700::6812:1205, 2606:4700::6812:1305
Response IP 104.18.18.5
Found Yes
Hash 44ba22b3c644777b240f5b7020fa40dd0fe46851802761c8348c528f06afffe9
SimHash 22d659dc8a93

Groups

*

Rule Path
Disallow /buscador/
Disallow /?*pid
Disallow /newsletter/
Disallow /euro-dolar/*
Disallow /bonos/*
Disallow /criptodivisas/*
Disallow /materias-primas/*
Disallow */en-la-red
Disallow /okclub/*
Disallow /okclub
Disallow /ok-vs-ko/*
Disallow /ok-vs-ko
Disallow /?attachment_id*
Disallow /?s=
Disallow /resultados/*
Disallow /guiltybit/*
Disallow /*.swf$
Disallow /*.tif$
Disallow /*.mp3$
Disallow /*.flv$
Disallow /*.mp4$
Disallow /*.avi$
Disallow /*.ics$
Disallow /*.pdf$
Disallow /*.ppt$
Disallow /*.pptx$
Disallow /*.doc$
Disallow /*.docx$

ia_archiver

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

slurp

Rule Path
Disallow /

maxthon

Rule Path
Disallow /

cncdialer

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://okdiario.com/sitemaps/okd-sitemap-loteria-navidad-2021.xml.gz
sitemap https://okdiario.com/sitemaps/okd-sitemap-google-news.xml
sitemap https://okdiario.com/sitemaps/okd-sitemap-index.xml
sitemap https://okdiario.com/sitemaps/okd-sitemap-happyfm-2021.xml.gz
sitemap https://okdiario.com/sitemaps/okd-sitemap-baleares-2021.xml.gz
sitemap https://okdiario.com/look/sitemap_index.xml
sitemap https://okdiario.com/look/news-sitemap.xml
sitemap https://okdiario.com/diariomadridista/sitemap_index.xml
sitemap https://okdiario.com/diariomadridista/news-sitemap.xml
sitemap https://okdiario.com/sitemaps/okd-sitemap-author-1.xml.gz

Comments

  • Directorios bloqueados
  • Extensiones de contenidos no indexables
  • Bloqueo de agentes