lexington.es
robots.txt

Robots Exclusion Standard data for lexington.es

Resource Scan

Scan Details

Site Domain lexington.es
Base Domain lexington.es
Scan Status Ok
Last Scan2024-09-07T13:06:02+00:00
Next Scan 2024-10-07T13:06:02+00:00

Last Scan

Scanned2024-09-07T13:06:02+00:00
URL https://lexington.es/robots.txt
Redirect https://www.lexington.es/robots.txt
Redirect Domain www.lexington.es
Redirect Base lexington.es
Domain IPs 178.32.106.65
Redirect IPs 178.32.106.65
Response IP 178.32.106.65
Found Yes
Hash 5b3e77a23b47db701df7f3c3e30d61df0428ffaddbb94af74e68cf0fcc6a2cbe
SimHash ab540a504830

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-login
Disallow /wp-admin
Disallow /*/feed/
Disallow /*/trackback/
Disallow /*/attachment/
Disallow /author/
Disallow *?replytocom
Disallow /tag/*/page/
Disallow /tag/*/feed/
Disallow /comments/
Disallow /xmlrpc.php
Disallow /*?s=
Disallow /*/*/*/feed.xml
Disallow /?attachment_id*
Disallow /search

googlebot

Rule Path
Allow /*.css$
Allow /*.js$

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

curious george

Rule Path
Disallow /

cybeye.com

Rule Path
Disallow /

docomo

Rule Path
Disallow /

exb language crawler

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

genieo

Rule Path
Disallow /

genio

Rule Path
Disallow /

lwnutch

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

openwebindex

Rule Path
Disallow /

rediffnewsbot

Rule Path
Disallow /

seoengworldbot

Rule Path
Disallow /

scanmine

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

wocbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

libcrawl

Rule Path
Disallow /

linkdex

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

repparser

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sindice-site-manager

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

woriobot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yolinkbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.lexington.es/sitemap_index.xml