lensonline.be
robots.txt
Robots Exclusion Standard data for lensonline.be
Resource Scan
Scan Details
| Site Domain | lensonline.be |
| Base Domain | lensonline.be |
| Scan Status | Ok |
| Last Scan | 2024-09-26T07:36:43+00:00 |
| Next Scan | 2024-10-26T07:36:43+00:00 |
Last Scan
| Scanned | 2024-09-26T07:36:43+00:00 |
| URL | https://lensonline.be/robots.txt |
| Redirect | https://www.lensonline.be/nl/robots.txt |
| Redirect Domain | www.lensonline.be |
| Redirect Base | lensonline.be |
| Domain IPs | 185.3.217.195, 2a02:5b40:4:228::a2 |
| Redirect IPs | 185.3.217.195, 2a02:5b40:4:228::a2 |
| Response IP | 185.3.217.195 |
| Found | Yes |
| Hash | 4902f56f37db623eb54b5aa82d8167b123e57c279f3467cd3e14394d188725e2 |
| SimHash | 2233190160e8 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | */faq/* |
| Disallow | */account/ |
| Disallow | */checkout/ |
| Disallow | */widgets/ |
| Disallow | */navigation/ |
| Disallow | */bundles/ |
| Disallow | */imprint$ |
| Disallow | */privacy$ |
| Disallow | */gtc$ |
| Disallow | /*?*p= |
| Disallow | /*?*manufacturer= |
| Disallow | /*?*properties= |
| Disallow | */search |
ahrefsbot
cazoodlebot
fasterfox
jyxobot
mj12bot
shopwiki
etaospider
baiduspider
baiduspider-image
baiduspider-news
baiduspider-video
bublupbot
ccbot
cliqzbot
coccoc
coccocbot-image
coccocbot-web
daumoa
euripbot
exploratodo
feedly
findxbot
gooblog
haosouspider
ichiro
istellabot
jikespider
lycos
mail.ru
mojeekbot
orangebot
plukkie
qwantify
rambler
seznambot
sosospider
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
sputnikbot
teoma
wotbox
yacybot
yandex
yandexmobilebot
yeti
yioopbot
yoozbot
youdaobot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.lensonline.be/nl/sitemap.xml |
| sitemap | https://www.lensonline.be/fr/sitemap.xml |
| sitemap | https://www.lensonline.be/en/sitemap.xml |
Warnings
- 3 invalid lines.
Comments