estaticos.expansion.com
robots.txt
Robots Exclusion Standard data for estaticos.expansion.com
Resource Scan
Scan Details
Site Domain | estaticos.expansion.com |
Base Domain | expansion.com |
Scan Status | Ok |
Last Scan | 2024-05-10T21:48:07+00:00 |
Next Scan | 2024-05-17T21:48:07+00:00 |
Last Scan
Scanned | 2024-05-10T21:48:07+00:00 |
URL | https://estaticos.expansion.com/robots.txt |
Domain IPs | 18.238.192.119, 18.238.192.3, 18.238.192.39, 18.238.192.96 |
Response IP | 18.165.171.104 |
Found | Yes |
Hash | dd0ac7234c3f9fd91a3ea8627efdf0d6ebf42aeb7836d3965941adaac7ce552e |
SimHash | 8888535ed8b8 |
Groups
*
Rule | Path |
---|---|
Disallow | /s/ |
Disallow | /registro/* |
Disallow | /buscador/* |
Disallow | /avisolegal/index.html |
Disallow | /usuario/panelcontrol/AtencionCliente |
Disallow | /ed/* |
Disallow | /especiales/philips* |
Disallow | /especiales/philips* |
Disallow | /especiales/cepsa/2015/11/27/5656f5efca4741354a8b456c.html |
Disallow | /economia-digital/2015/12/02/565ee35de2704e08348b4579.html |
Disallow | /2013/05/24/valencia/1369421699.html |
Disallow | /accesible/2013/04/24/catalunya/1366824807.html |
Disallow | /2013/04/24/catalunya/1366824807.html |
Disallow | /2013/04/22/juridico/1366657780.html |
Disallow | /juridico/sentencias/2015/12/28/56816af522601dd7178b459c.html |
Disallow | /2015/02/03/juridico/1422988139.html |
Disallow | /accesible/2011/11/18/catalunya/1321648436.html |
Disallow | /2011/12/09/catalunya/1323418073.html |
Disallow | /ejecutivo-administrador/fernandez-jambrina-alicia_2265029_G50.html |
Disallow | /blogs/peon-de-dama/2012/11/15/bosques-naturales-la-ultima-estafa.html |
Disallow | /agencia/efe/2012/07/30/17492097.html |
Disallow | /2009/04/28/empresas/1240930114.html |
Disallow | /accesible/2011/12/09/catalunya/1323418073.html |
Disallow | /2014/10/31/juridico/1414757650.html |
Disallow | /2011/11/18/catalunya/1321648436.html |
Disallow | /2008/04/16/empresas/energia/1112875.html |
Disallow | /2012/08/08/empresas/1344440327.html |
Disallow | *zonadescargas/obtenerDocumento.html?codigo=* |
Disallow | /ultima_hora/index.html?year=* |
Disallow | /edicion_impresa/calendario.html?month=* |
Disallow | /includes/calendarios/calendarioRadar.html?year* |
Disallow | /opinion/documentos/hemeroteca.html?* |
addthis.com disallow: /
admantx disallow: /
ahrefsbot disallow: /
bdcbot disallow: /
bender disallow: /
bixocrawler disallow: /
bl.uk_lddc_bot disallow: /
blexbot disallow: /
bubing disallow: /
cliqzbot disallow: /
cncdialer disallow: /
crawler4j disallow: /
crystalsemanticsbot disallow: /
cyberalert disallow: /
digext disallow: /
discobot disallow: /
discoverybot disallow: /
dloader disallow: /
dloader(naverrobot) disallow: /
doc disallow: /
dotbot disallow: /
download ninja disallow: /
dts agent disallow: /
exabot disallow: /
ezooms disallow: /
fairshare disallow: /
fetch disallow: /
flamingo_searchengine disallow: /
genieo disallow: /
gigabot disallow: /
grub-client disallow: /
heritrix disallow: /
heritrix/3.3.0 disallow: /
httrack disallow: /
ia_archiver disallow: /
integromedb disallow: /
istellabot disallow: /
jikespider disallow: /
jyxobot disallow: /
k2spider disallow: /
kimengi disallow: /
kimengi/nineconnections.com disallow: /
larbin disallow: /
lexxebot/1.0 disallow: /
libwww disallow: /
linko disallow: /
livelapbot disallow: /
magpie-crawler disallow: /
maxthon disallow: /
metauri disallow: /
microsoft.url.control disallow: /
mj12bot disallow: /
moreover disallow: /
moreoverbot disallow: /
msiecrawler disallow: /
nabot disallow: /
naverbot disallow: /
nerdbynature.bot disallow: /
netestate ne crawler disallow: /
netseer crawler disallow: /
newscan disallow: /
nextgensearchbot disallow: /
npbot disallow: /
nutch disallow: /
offline explorer disallow: /
omgilibot disallow: /
orthogaffe disallow: /
piplbot disallow: /
pixray-seeker disallow: /
proximic disallow: /
psbot disallow: /
queryseekerspider disallow: /
rogerbot disallow: /
seokicks disallow: /
seokicks-robot disallow: /
sitebot disallow: /
sitebot/0.1 disallow: /
sitecheck.internetseer.com disallow: /
sitesnagger disallow: /
slurp disallow: /
sogou disallow: /
sosospider disallow: /
spbot disallow: /
spinn3r disallow: /
teleport disallow: /
teleportpro disallow: /
trendictionbot disallow: /
trovitbot disallow: /
turnitinbot disallow: /
ubicrawler disallow: /
umbot-ln disallow: /
unisterbot disallow: /
universalfeedparser disallow: /
wbsearchbot disallow: /
webcopier disallow: /
webreaper disallow: /
webstripper disallow: /
webzip disallow: /
wesee:search disallow: /
wget disallow: /
wotbot disallow: /
wotbox disallow: /
xenu disallow: /
yasni disallow: /
zao disallow: /
zealbot disallow: /
zyborg disallow: /
gptbot disallow: /
ccbot disallow: /
anthropic-ai disallow: /
chatgpt-user disallow: /
No rules defined. All paths allowed.
Warnings
- 1 invalid line.
Comments