lampy.pl
robots.txt

Robots Exclusion Standard data for lampy.pl

Resource Scan

Scan Details

Site Domain lampy.pl
Base Domain lampy.pl
Scan Status Ok
Last Scan2024-09-15T03:53:05+00:00
Next Scan 2024-10-15T03:53:05+00:00

Last Scan

Scanned2024-09-15T03:53:05+00:00
URL https://lampy.pl/robots.txt
Redirect https://www.lampy.pl/robots.txt
Redirect Domain www.lampy.pl
Redirect Base lampy.pl
Domain IPs 34.111.187.42
Redirect IPs 34.111.187.42
Response IP 34.111.187.42
Found Yes
Hash 75f748ffd186fd51fc150ad1595a78da22f0dc9a1f60a20b75a2c445db350db5
SimHash 4106f060cbf3

Groups

*
adidxbot

Product Comment
adidxbot explicit mentioning of bing ads bot. It does not adhere to the agent wildcard.
Rule Path Comment
Disallow /checkout/ -
Disallow /customer/ -
Disallow /wishlist/ -
Disallow /catalogsearch/ -
Disallow /app/ -
Disallow /lib/ -
Disallow /*.php$ -
Disallow /*SID%3D -
Disallow /index.php/ -
Disallow /banner/ajax -
Disallow /lw_related/ajax -
Disallow /documents/ -
Disallow */article/*nocache%3D -
Disallow /bloom/widget/* -
Disallow /*?*&* block irrelevant filter combinations
Disallow /*?*~* block irrelevant filter value combinations
Disallow /*?*~* block tilde
Disallow /*?*&*&*p=* block irrelevant filter combinations with pagination
Disallow /*?*~*&p=* block irrelevant filter value combinations with pagination
Disallow /*?*~*&p=* block tilde
Allow /*?*&p=* allow relevant filter with pagination
Disallow /*?*___* -
Disallow /*?*category=* -
Disallow /*?*light_bulb=* -
Disallow /*?SESSIONNAME=* -
Allow /*utm_* -
Allow /*lw_view%3Dnocontent* -
Allow /*display%3Dproducts* -
Allow /*lw_om_view%3Drecotop* -
Allow /*block_sku%3D* -

barkrowler
buck
velenpublicwebcrawler
imagesiftbot

Rule Path
Disallow *

gptbot
applebot

Rule Path
Disallow /*?*p=*

adsbot-google-mobile

Rule Path
Disallow /bloom/widget/*

adsbot-google

Rule Path
Disallow /bloom/widget/*

mediapartners-google

Rule Path
Disallow /bloom/widget/*

googlebot-image

Rule Path
Disallow /bloom/widget/*

googlebot-video

Rule Path
Disallow /bloom/widget/*

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.lampy.pl/media/sitemap/sitemap_pl_pl.xml

Comments

  • Disallow not needed crawlers
  • Disallow pagination crawls where not needed
  • Disallow widget crawls