Log In Sign Up

vperfumes.com
robots.txt

Robots Exclusion Standard data for vperfumes.com

Resource Scan

Scan Details

Site Domain vperfumes.com
Base Domain vperfumes.com
Scan Status Ok
Last Scan2025-12-05T01:13:45+00:00
Next Scan 2025-12-19T01:13:45+00:00

Last Scan

Scanned2025-12-05T01:13:45+00:00
URL https://vperfumes.com/robots.txt
Redirect https://www.vperfumes.com/robots.txt
Redirect Domain www.vperfumes.com
Redirect Base vperfumes.com
Domain IPs 104.26.4.21, 104.26.5.21, 172.67.68.46, 2606:4700:20::681a:415, 2606:4700:20::681a:515, 2606:4700:20::ac43:442e
Redirect IPs 104.26.4.21, 104.26.5.21, 172.67.68.46, 2606:4700:20::681a:415, 2606:4700:20::681a:515, 2606:4700:20::ac43:442e
Response IP 172.67.68.46
Found Yes
Hash 4c109effce97cd8d689169e2bbd36addebb411bb16938500885d2093670a54d7
SimHash 28344611e8b8

Groups

*

Rule Path
Disallow /
Disallow /*?search=
Allow /$
Allow /ae-en$
Allow /ae-ar$
Allow /sa-ar$
Allow /sa-en$
Allow /ae-en/$
Allow /ae-ar/$
Allow /sa-ar/$
Allow /sa-en/$
Allow /ae-en/store-locator
Allow /ae-ar/store-locator
Allow /sa-en/store-locator
Allow /sa-ar/store-locator
Allow /ae-en/page
Allow /ae-ar/page
Allow /sa-en/page
Allow /sa-ar/page
Allow /ae-en/men
Allow /ae-ar/men
Allow /sa-en/men
Allow /sa-ar/men
Allow /ae-en/women
Allow /ae-ar/women
Allow /sa-en/women
Allow /sa-ar/women
Allow /ae-en/online-exclusive
Allow /ae-ar/online-exclusive
Allow /sa-en/online-exclusive
Allow /sa-ar/online-exclusive
Allow /ae-en/faq
Allow /ae-ar/faq
Allow /sa-en/faq
Allow /sa-ar/faq
Allow /ae-en/collection
Allow /ae-ar/collection
Allow /sa-en/collection
Allow /sa-ar/collection
Allow /ae-en/category
Allow /ae-ar/category
Allow /sa-en/category
Allow /sa-ar/category
Allow /ae-en/brands
Allow /ae-ar/brands
Allow /sa-en/brands
Allow /sa-ar/brands
Allow /ae-en/product
Allow /ae-ar/product
Allow /sa-en/product
Allow /sa-ar/product
Allow /api/v1/w/admin/auth/export-feed?type=google&lang=en
Allow /sitemap
Allow /ae-en/sitemap
Allow /ae-ar/sitemap
Allow /sa-en/sitemap
Allow /sa-ar/sitemap
Allow /favicon.ico

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

anthropic-user-agent

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.vperfumes.com/sitemap.xml
Back to top

Comments

  • All Denied Rule
  • This section explicitly blocks all crawlers by default
  • Block search query URLs
  • Allow Specific Pages and Paths
  • This section lists specific paths that are allowed for crawlers
  • These will override the previous "Disallow: /"
  • Allow Homepages
  • Allow Store Locator Pages
  • Allow Additional Pages
  • Allow Main Pages
  • Allow Collection Pages
  • Allow Categories Pages
  • Allow Brands Pages
  • Allow Product Pages
  • Allow Feed URL
  • Allow Sitemap URL
  • Allow Favicon
  • Allow Specific Bots
  • Crawl Delay Control
  • This section specifies crawl delays for specific bots
  • Sitemap Location
Back to top