luminaire.fr
robots.txt

Robots Exclusion Standard data for luminaire.fr

Resource Scan

Scan Details

Site Domain luminaire.fr
Base Domain luminaire.fr
Scan Status Ok
Last Scan2024-09-15T02:35:05+00:00
Next Scan 2024-10-15T02:35:05+00:00

Last Scan

Scanned2024-09-15T02:35:05+00:00
URL https://luminaire.fr/robots.txt
Redirect https://www.luminaire.fr/robots.txt
Redirect Domain www.luminaire.fr
Redirect Base luminaire.fr
Domain IPs 34.111.122.25
Redirect IPs 34.111.122.25
Response IP 34.111.122.25
Found Yes
Hash 6a1ea35f5ac9f8330e509e6bd72bcf98a698e70165a2e8fe8ef575e00f07e9ce
SimHash 4106f068cbf3

Groups

*
adidxbot

Product Comment
adidxbot explicit mentioning of bing ads bot. It does not adhere to the agent wildcard.
Rule Path Comment
Disallow /checkout/ -
Disallow /customer/ -
Disallow /wishlist/ -
Disallow /catalogsearch/ -
Disallow /app/ -
Disallow /lib/ -
Disallow /*.php$ -
Disallow /*SID%3D -
Disallow /index.php/ -
Disallow /banner/ajax -
Disallow /lw_related/ajax -
Disallow /documents/ -
Disallow */article/*nocache%3D -
Disallow /bloom/widget/* -
Disallow /*?*&* block irrelevant filter combinations
Disallow /*?*~* block irrelevant filter value combinations
Disallow /*?*~* block tilde
Disallow /*?*&*&*p=* block irrelevant filter combinations with pagination
Disallow /*?*~*&p=* block irrelevant filter value combinations with pagination
Disallow /*?*~*&p=* block tilde
Allow /*?*&p=* allow relevant filter with pagination
Disallow /*?*___* -
Disallow /*?*category=* -
Disallow /*?*light_bulb=* -
Disallow /*?SESSIONNAME=* -
Allow /*utm_* -
Allow /*lw_view%3Dnocontent* -
Allow /*display%3Dproducts* -
Allow /*lw_om_view%3Drecotop* -
Allow /*block_sku%3D* -

barkrowler
buck
velenpublicwebcrawler
imagesiftbot

Rule Path
Disallow *

gptbot
applebot

Rule Path
Disallow /*?*p=*

adsbot-google-mobile

Rule Path
Disallow /bloom/widget/*

adsbot-google

Rule Path
Disallow /bloom/widget/*

mediapartners-google

Rule Path
Disallow /bloom/widget/*

googlebot-image

Rule Path
Disallow /bloom/widget/*

googlebot-video

Rule Path
Disallow /bloom/widget/*

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.luminaire.fr/media/sitemap/sitemap_fr_fr.xml

Comments

  • Disallow not needed crawlers
  • Disallow pagination crawls where not needed
  • Disallow widget crawls