vlex.es
robots.txt

Robots Exclusion Standard data for vlex.es

Resource Scan

Scan Details

Site Domain vlex.es
Base Domain vlex.es
Scan Status Ok
Last Scan2024-11-12T19:44:03+00:00
Next Scan 2024-11-19T19:44:03+00:00

Last Scan

Scanned2024-11-12T19:44:03+00:00
URL https://vlex.es/robots.txt
Domain IPs 108.157.254.32, 108.157.254.5, 108.157.254.69, 108.157.254.91
Response IP 108.157.254.69
Found Yes
Hash 6a0919b80ddd36ad89e8e55fe33e7ede9af182f21c2c705d974712ddabec1fa4
SimHash b2de79094767

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /tour
Disallow /redirect
Disallow /stats
Disallow /search
Disallow /account/login_ip
Disallow /account/check_if_logged?callback=jsonp*
Disallow /pdf/
Disallow /*/search
Disallow /*/auto_complete_for_norma_citada
Disallow /*/auto_complete_for_voces_partes
Disallow /vid/1234$
Disallow /vid/*ix_resultado*
Disallow /async
Disallow /*st_dominio
Disallow /*textolibre
Disallow /voces
Disallow /vid/*/translate?mt=
Disallow /checkout
Disallow /vid/*.json
Disallow /vid/*/content
Disallow /tags/*?format=pdf
Disallow /vid?*ix_resultado*
Disallow /source/*/*/thesaurus/starting_with
Disallow /corporate
Disallow /help_center
Disallow /for_publishers
Disallow /librarian_center
Disallow /offline_trials
Disallow /promos
Disallow /countries
Disallow /customers
Disallow /publisher
Disallow /languages
Disallow /freetrial_30
Disallow /center-search*
Disallow /session_ip*
Disallow /session.json*

bingbot

Rule Path
Disallow /tags/*/page/*

msnbot

Rule Path
Disallow /tags/*/page/*

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

psbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bloglines/3.1

Rule Path
Disallow /

jyxobot/1

Rule Path
Disallow /

cityreview

Rule Path
Disallow /

proximic

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file