libreriadelau.com
robots.txt

Robots Exclusion Standard data for libreriadelau.com

Resource Scan

Scan Details

Site Domain libreriadelau.com
Base Domain libreriadelau.com
Scan Status Ok
Last Scan2024-05-24T13:33:15+00:00
Next Scan 2024-06-23T13:33:15+00:00

Last Scan

Scanned2024-05-24T13:33:15+00:00
URL https://www.libreriadelau.com/robots.txt
Domain IPs 18.154.7.22, 18.154.7.34, 18.154.7.51, 18.154.7.73, 2600:9000:2666:1a00:e:e621:dd80:93a1, 2600:9000:2666:7a00:e:e621:dd80:93a1, 2600:9000:2666:8600:e:e621:dd80:93a1, 2600:9000:2666:ae00:e:e621:dd80:93a1, 2600:9000:2666:d200:e:e621:dd80:93a1, 2600:9000:2666:dc00:e:e621:dd80:93a1, 2600:9000:2666:e00:e:e621:dd80:93a1, 2600:9000:2666:ec00:e:e621:dd80:93a1
Response IP 108.157.254.72
Found Yes
Hash 4b525085285b9fed803edb46f9176c60b487a2e90d656cf2cecb604f49c8743a
SimHash b49c5982cc73

Groups

*

Rule Path
Disallow /img/*
Disallow /account/*
Disallow /login/*
Disallow /busca?*
Disallow /checkout/*
Disallow /quick-view/*
Disallow /espiar/*
Disallow /directory/currency/switch/currency/*
Disallow /productotherpaymentsystems/*
Disallow /ViewPagedUserComments*

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baidu

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

altavista

Rule Path
Disallow /

slurp

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.libreriadelau.com/sitemap.xml

Comments

  • Disallow all crawlers access to certain pages.