Log In Sign Up

lipscosme.com
robots.txt

Robots Exclusion Standard data for lipscosme.com

Resource Scan

Scan Details

Site Domain lipscosme.com
Base Domain lipscosme.com
Scan Status Ok
Last Scan2024-11-08T23:41:42+00:00
Next Scan 2024-11-15T23:41:42+00:00

Last Scan

Scanned2024-11-08T23:41:42+00:00
URL https://lipscosme.com/robots.txt
Domain IPs 108.157.254.101, 108.157.254.16, 108.157.254.74, 108.157.254.96
Response IP 108.157.254.74
Found Yes
Hash 531f28db547282fed3f8d77869d10aa66dd48eca3e622be85f760aacc958f3f0
SimHash c3060f86ab42

Groups

gptbot
chatgpt-user
ccbot
google-extended
claudebot
anthropic-ai
facebookbot
applebot-extended

Rule Path
Disallow /

*

Rule Path
Disallow /products.*
Disallow /products/search
Disallow /products/*/commerce_button
Disallow /posts/search
Disallow /brands/search
Disallow /users/search
Disallow /articles/search
Disallow /search
Disallow /categories/call_children
Disallow /chats/
Disallow /clip_folders/
Disallow /coupons/
Disallow /lives/
Disallow /logs/
Disallow /webview/
Disallow /browsing_histories
Disallow /shops?brand_name=
Disallow /brands/*/stations/*
Disallow /products/*/media
Disallow /*?*&*&*&*
Disallow /*?age_type=*&*
Disallow /*%26age_type%3D
Disallow /*%26brand_id%3D
Disallow /*%26category_id%3D
Disallow /*?only_this_product=
Disallow /*%26only_this_product%3D
Disallow /*?filter_gifting_posts=
Disallow /*%26filter_gifting_posts%3D
Disallow /*?rate=
Disallow /*%26rate%3D
Disallow /*?skin_type=*&*
Disallow /*%26skin_type%3D
Disallow /*?sort=
Disallow /*%26sort%3D
Allow /products/*/review?sort=latest
Disallow /products/*/review?sort=latest&*
Allow /products/*/review?page=*&sort=latest
Disallow /products/*/review?page=*&*&sort=latest
Disallow /products/*/review?page=*&sort=latest&*
Disallow /*?product_pattern_id=*&
Disallow /*%26product_pattern_id%3D
Allow /products/*/review?page=*&product_pattern_id=
Disallow /products/*/review?page=*&product_pattern_id=*&
Disallow /products/*/review?page=*&*&product_pattern_id=
Disallow /*?text=
Disallow /*%26text%3D
Disallow /*?title=
Disallow /*%26title%3D
Disallow /*?layout=
Disallow /*%26layout%3D
Disallow /*%26type%3D
Disallow /*?category_feature_name=*&*
Disallow /rankings/*?period=*
Disallow /rankings/*%26period%3D*
Disallow /cosmetic_calendar?*
Disallow /cosmetic_calendar/*?*
Disallow /users/n/
Disallow /users/sign_up?
Disallow /users/sign_in?
Disallow /secondary_use/
Disallow /commerce/mail_magazines/
Back to top

Other Records

Field Value
sitemap https://lipscosme.com/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next lines:
Back to top