Log In Sign Up

six-scents.com
robots.txt

Robots Exclusion Standard data for six-scents.com

Resource Scan

Scan Details

Site Domain six-scents.com
Base Domain six-scents.com
Scan Status Ok
Last Scan2026-02-17T13:05:40+00:00
Next Scan 2026-03-03T13:05:40+00:00

Last Scan

Scanned2026-02-17T13:05:40+00:00
URL https://six-scents.com/robots.txt
Domain IPs 104.21.94.169, 172.67.138.70, 2606:4700:3031::6815:5ea9, 2606:4700:3033::ac43:8a46
Response IP 172.67.138.70
Found Yes
Hash 5285f7e079246473ea48132691861c829723d08c069a3af6730fb0bb2a6ab844
SimHash 6d89dd712ec7

Groups

*
googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow
Disallow /admin
Disallow /cart
Disallow /checkout
Disallow /checkouts/
Disallow /orders
Disallow /account
Disallow /policies/
Disallow /search
Disallow /*?*sort_by=*
Disallow /*?*variant=*
Disallow /*?*filter*
Disallow /*?page=*
Disallow /*?*q=*
Disallow /*/cart
Allow /products/
Allow /collections/
Allow /blogs/
Allow /pages/
Allow /*.css
Allow /*.js

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://six-scents.com/sitemap.xml
Back to top

Comments

  • Parfums Quartana - Optimized Robots.txt
  • Last updated: 2025-11-08
  • Allow google bots
  • Block administrative and transactional pages
  • Block search and filter pages (prevent duplicate content)
  • Block tracking and cart parameters
  • Allow important content (explicit)
  • Allow CSS/JS for rendering (critical for mobile-first indexing)
  • Sitemap location (update with your actual domain)
  • Block aggressive bots (optional but recommended)
Back to top