Log In Sign Up

vinosyspirits.com
robots.txt

Robots Exclusion Standard data for vinosyspirits.com

Resource Scan

Scan Details

Site Domain vinosyspirits.com
Base Domain vinosyspirits.com
Scan Status Ok
Last Scan2025-10-20T00:33:07+00:00
Next Scan 2025-11-19T00:33:07+00:00

Last Scan

Scanned2025-10-20T00:33:07+00:00
URL https://vinosyspirits.com/robots.txt
Redirect https://www.vinosyspirits.com/robots.txt
Redirect Domain www.vinosyspirits.com
Redirect Base vinosyspirits.com
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 146.75.45.124
Found Yes
Hash 3a13fba4092aa3dff08cf01ce557abb9cdbeb0c5d56ee23f0b75d2fa8849deea
SimHash 2002a753e6b1

Groups

*

Rule Path
Allow /*?p=
Disallow /media/captcha/
Disallow /media/customer/
Disallow /media/dhl/
Disallow /media/downloadable/
Disallow /media/import/
Disallow /media/pdf/
Disallow /media/sales/
Disallow /media/tmp/
Disallow /media/xmlconnect/
Disallow */index.php/
Disallow */catalog/product_compare/
Disallow */catalog/category/view/
Disallow */catalog/product/view/
Disallow */catalog/product/gallery/
Disallow */catalogsearch/
Disallow */control/
Disallow */customer/
Disallow */customize/
Disallow */newsletter/
Disallow */poll/
Disallow */review/
Disallow */sendfriend/
Disallow */tag/
Disallow */wishlist/
Disallow */checkout/
Disallow */onestepcheckout/
Disallow /*?dir*
Disallow /*?limit*
Disallow /*?mode*
Disallow /*?___from_store=*
Disallow /*?___store=*
Disallow /*?cat=*
Disallow /*?q=*
Disallow /*?price=*
Disallow /*?availability=*
Disallow /*?brand=*
Disallow /*?p=*&
Disallow /*.php$
Disallow /*?SID=
Back to top

Other Records

Field Value
sitemap https://www.vinosyspirits.com/media/sitemap.xml
Back to top

Comments

  • Crawlers Setup
  • Allow paging (unless paging inside a listing with more params, as disallowed below)
  • Directories (technical images only)
  • Paths (leading * to make work for single- and multilocale versions)
  • Do not crawl sub category pages that are sorted or filtered.
  • NB: Avoid wider rules like /*?* as these would block also assets with timestamp/version as parameter!
  • These are more specific, pick what you need - and do not forget to add your custom filters!
  • Paths that can be safely ignored (no clean URLs)
Back to top