miladbazar.com
robots.txt

Robots Exclusion Standard data for miladbazar.com

Resource Scan

Scan Details

Site Domain miladbazar.com
Base Domain miladbazar.com
Scan Status Ok
Last Scan2026-03-02T16:39:34+00:00
Next Scan 2026-03-16T16:39:34+00:00

Last Scan

Scanned2026-03-02T16:39:34+00:00
URL https://miladbazar.com/robots.txt
Domain IPs 104.21.23.104, 172.67.210.126, 2606:4700:3031::ac43:d27e, 2606:4700:3032::6815:1768
Response IP 172.67.210.126
Found Yes
Hash 3a7d015447bef1da989ad5eb024809ecf32b25db666e045cfda2bab2ad487881
SimHash f526a31bec22

Groups

*

Rule Path
Disallow /cart/
Disallow /account/
Disallow /profile/
Disallow /checkout/
Disallow /search?*
Disallow /search/*
Disallow /*?sort_by*
Disallow /*?filter*
Disallow /*?page=*
Disallow /*?limit=*
Disallow /api/
Disallow /cdn-cgi/
Allow /_app/
Allow /app/
Disallow /staging/
Disallow /test/
Disallow /_next/
Disallow /en/account/
Disallow /fa/account/
Disallow /ar/account/
Disallow /tr/account/
Disallow /en/cart/
Disallow /fa/cart/
Disallow /ar/cart/
Disallow /tr/cart/
Disallow /en/profile/
Disallow /fa/profile/
Disallow /ar/profile/
Disallow /tr/profile/
Allow /
Allow /en/
Allow /collections/
Allow /collections/*
Allow /brands/
Allow /brands/*
Allow /fa/
Allow /fa/collections/
Allow /fa/collections/*
Allow /fa/brands/
Allow /fa/brands/*
Allow /ar/
Allow /ar/collections/
Allow /ar/collections/*
Allow /ar/brands/
Allow /ar/brands/*
Allow /tr/
Allow /tr/collections/
Allow /tr/collections/*
Allow /tr/brands/
Allow /tr/brands/*
Allow /assets/
Allow *.css
Allow *.js
Allow *.jpg
Allow *.jpeg
Allow *.png
Allow *.gif
Allow *.webp
Allow *.svg
Allow *.woff
Allow *.woff2
Allow *.ttf

Other Records

Field Value
sitemap https://miladbazar.com/sitemap.xml
sitemap https://miladbazar.com/sitemap-en.xml
sitemap https://miladbazar.com/sitemap-fa.xml

Comments

  • Robots.txt for MiladBazar E-commerce Site
  • E-commerce specific pages - no SEO value
  • Search and filter pages - prevent duplicate content
  • API endpoints and dynamic content
  • Cloudflare internal paths (email protection, etc.)
  • Explicitly allow SvelteKit assets (critical for page rendering)
  • Development and staging content
  • Language redirects (prevent duplicate indexing)
  • Allow important pages explicitly
  • FA
  • AR
  • TR
  • Allow critical resources for proper page rendering
  • Sitemap location