vanitatis.com
robots.txt

Robots Exclusion Standard data for vanitatis.com

Resource Scan

Scan Details

Site Domain vanitatis.com
Base Domain vanitatis.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-18T11:01:50+00:00
Next Scan 2024-10-17T11:01:50+00:00

Last Successful Scan

Scanned2024-05-28T11:00:45+00:00
URL https://vanitatis.com/robots.txt
Redirect https://www.vanitatis.elconfidencial.com/robots.txt
Redirect Domain www.vanitatis.elconfidencial.com
Redirect Base elconfidencial.com
Domain IPs 108.139.10.120, 108.139.10.19, 108.139.10.43, 108.139.10.88
Redirect IPs 154.47.23.177, 212.102.42.89, 2a02:6ea0:d342::4, 2a02:6ea0:d638::4
Response IP 212.102.42.89
Found Yes
Hash 54e0ffef0839853baad9950eeda8c90835051246fa7edc8e4161acc3882a989c
SimHash 7960c66afd30

Groups

*

Rule Path
Disallow /buscar/
Disallow /fonts/
Disallow /contacto/
Disallow /ClassoraProxy.php
Disallow /access.php
Disallow /css.php
Disallow /img.php
Disallow /js.php
Disallow /response/
Disallow /comunidad/
Disallow /comment/
Disallow /service/
Disallow /access_bigdata.php
Disallow /access/
Disallow /ofensivo/
Disallow /enviar/
Disallow /valorar/
Disallow /repositorio/
Disallow /comentar/
Disallow /*.asp$
Allow /_Incapsula_Resource
Disallow /*/%7B%7Bampurl%7D%7D$
Disallow /*/%7B%7Bampurl%7D%7D$
Disallow /*/%7B%7Bshots*%7D%7D$
Disallow /*/%7B%7Bshots*%7D%7D$
Disallow /35003347/*
Disallow /*.asp?*
Disallow /*/pass_*
Disallow /*/%7B%7Burl%7D%7D$
Disallow /hemeroteca/
Disallow /stats/
Disallow /multimedia/fotos/*
Disallow /suscribete/?*
Disallow */21682112617/*
Disallow /try/home*
Disallow /try/home*

fairshare

Rule Path
Disallow /

metauri

Rule Path
Disallow /

nabot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

kimengi/nineconnections.com

Rule Path
Disallow /

kimengi

Rule Path
Disallow /

digext

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

dloader

Rule Path
Disallow /

dloader(naverrobot)

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

wotbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

bixocrawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

genieo

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

umbot-ln

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

heritrix/3.3.0

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

moreover

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /