Log In Sign Up

arcollectionbd.com
robots.txt

Robots Exclusion Standard data for arcollectionbd.com

Resource Scan

Scan Details

Site Domain arcollectionbd.com
Base Domain arcollectionbd.com
Scan Status Ok
Last Scan2026-02-25T19:29:10+00:00
Next Scan 2026-03-11T19:29:10+00:00

Last Scan

Scanned2026-02-25T19:29:10+00:00
URL https://arcollectionbd.com/robots.txt
Domain IPs 2.57.91.165, 2a02:4780:84:af98:e5f1:be25:2881:2e8e, 2a02:4780:85:5c5d:6561:de4a:2460:b61c, 84.32.84.255
Response IP 77.37.66.45
Found Yes
Hash f99f2c8692ea11b512607314aaf80ac374b87ce3817d9eacc329de76e99e1dbd
SimHash ad801610ae21

Groups

*

Rule Path
Allow /
Allow /category/
Allow /brand/
Allow /product/
Allow /page/
Allow /about
Allow /contact
Allow /shop/
Disallow /admin/
Disallow /backend/
Disallow /api/
Disallow /reseller/
Disallow /merchant/
Disallow /storage/
Disallow /vendor/
Disallow /node_modules/
Disallow /bootstrap/
Disallow /config/
Disallow /database/
Disallow /resources/
Disallow /chunks/
Disallow /*.json$
Disallow /*.xml$
Disallow /*.env
Disallow /artisan
Disallow /server.php
Disallow /webpack.mix.js
Disallow /*?*sort=
Disallow /*?*filter=
Disallow /*?*page=
Disallow /cart
Disallow /checkout
Disallow /order/
Disallow /login
Disallow /register
Disallow /password/

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://arcollectionbd.com/sitemap.xml
Back to top

Comments

  • Robots.txt for AR Collection
  • Allow crawling of main content
  • Disallow admin and backend areas
  • Disallow system directories
  • Disallow sensitive files
  • Disallow search and filter pages with parameters (prevent duplicate content)
  • Disallow cart and checkout (user-specific pages)
  • Disallow login/register pages
  • Crawl delay (optional - be gentle on server)
  • Sitemap location
Back to top