lampade.it
robots.txt

Robots Exclusion Standard data for lampade.it

Resource Scan

Scan Details

Site Domain lampade.it
Base Domain lampade.it
Scan Status Ok
Last Scan2024-10-11T08:45:47+00:00
Next Scan 2024-11-10T08:45:47+00:00

Last Scan

Scanned2024-10-11T08:45:47+00:00
URL https://lampade.it/robots.txt
Redirect https://www.lampade.it/robots.txt
Redirect Domain www.lampade.it
Redirect Base lampade.it
Domain IPs 34.111.122.25
Redirect IPs 34.111.122.25
Response IP 34.111.122.25
Found Yes
Hash 6145bec261d2269c5243cf6b2c7e1823dbd2a5d72b63319b42ac1f3a2d02fb59
SimHash 4106f060cbf3

Groups

*
adidxbot

Product Comment
adidxbot explicit mentioning of bing ads bot. It does not adhere to the agent wildcard.
Rule Path Comment
Disallow /checkout/ -
Disallow /customer/ -
Disallow /wishlist/ -
Disallow /catalogsearch/ -
Disallow /app/ -
Disallow /lib/ -
Disallow /*.php$ -
Disallow /*SID%3D -
Disallow /index.php/ -
Disallow /banner/ajax -
Disallow /lw_related/ajax -
Disallow /documents/ -
Disallow */article/*nocache%3D -
Disallow /bloom/widget/* -
Disallow /*?*&* block irrelevant filter combinations
Disallow /*?*~* block irrelevant filter value combinations
Disallow /*?*~* block tilde
Disallow /*?*&*&*p=* block irrelevant filter combinations with pagination
Disallow /*?*~*&p=* block irrelevant filter value combinations with pagination
Disallow /*?*~*&p=* block tilde
Allow /*?*&p=* allow relevant filter with pagination
Disallow /*?*___* -
Disallow /*?*category=* -
Disallow /*?*light_bulb=* -
Disallow /*?SESSIONNAME=* -
Allow /*utm_* -
Allow /*lw_view%3Dnocontent* -
Allow /*display%3Dproducts* -
Allow /*lw_om_view%3Drecotop* -
Allow /*block_sku%3D* -

barkrowler
buck
velenpublicwebcrawler
imagesiftbot

Rule Path
Disallow *

gptbot
applebot

Rule Path
Disallow /*?*p=*

adsbot-google-mobile

Rule Path
Disallow /bloom/widget/*

adsbot-google

Rule Path
Disallow /bloom/widget/*

mediapartners-google

Rule Path
Disallow /bloom/widget/*

googlebot-image

Rule Path
Disallow /bloom/widget/*

googlebot-video

Rule Path
Disallow /bloom/widget/*

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.lampade.it/media/sitemap/sitemap_it_it.xml

Comments

  • Disallow not needed crawlers
  • Disallow pagination crawls where not needed
  • Disallow widget crawls