olimpicos.clarosports.com
robots.txt

Robots Exclusion Standard data for olimpicos.clarosports.com

Resource Scan

Scan Details

Site Domain olimpicos.clarosports.com
Base Domain clarosports.com
Scan Status Ok
Last Scan2024-09-07T21:08:35+00:00
Next Scan 2024-10-07T21:08:35+00:00

Last Scan

Scanned2024-09-07T21:08:35+00:00
URL https://olimpicos.clarosports.com/robots.txt
Domain IPs 2600:1413:b000:6::17d5:2bcc, 2600:1413:b000:6::17d5:2bdf, 96.17.96.4, 96.17.96.9
Response IP 23.59.168.105
Found Yes
Hash 7a25642a5666783afbc0602db08082e0da159a05c7ff5dc5cd74987755cb0f55
SimHash a198515a6ca5

Groups

*

Rule Path
Allow /*.css
Allow /*.js
Disallow /akamai/*
Disallow /121173452/*
Disallow /pyeongchang-2018/akamai/*
Disallow /pyeongchang-2018/assets/*
Disallow /pyeongchang-2018/utils/*
Disallow /pyeongchang-2018/componentes_estaticos/*
Disallow /pyeongchang-2018/app/*
Disallow /pyeongchang-2018/widgets/*
Disallow /pyeongchang-2018/ooyala/*
Disallow /buenos-aires-2018/json/*
Disallow /buenos-aires-2018/magazine/*
Disallow /buenos-aires-2018/prerender/*
Disallow /buenos-aires-2018/assets/*
Disallow /*/feed/
Disallow /*/trackback/
Disallow /*/attachment/
Disallow /tag/*/
Disallow /xmlrpc.php
Disallow /b*?s=
Disallow /*/*/*/feed.xml
Disallow /?attachment_id*
Disallow /*?view=
Disallow /*?slide=
Disallow /*?s=
Disallow /*?share=

orthogaffe

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

Other Records

Field Value
sitemap https://olimpicos.marcaclaro.com/sitemap.xml

Comments

  • Rule OG
  • Rule PC2018
  • Rule BAYOG2018
  • Impedir URLs de poco valor y feeds que se podrían generar
  • Lista de bots y scrapers bloqueados
  • sitemap