Log In Sign Up

bztactical.co.uk
robots.txt

Robots Exclusion Standard data for bztactical.co.uk

Resource Scan

Scan Details

Site Domain bztactical.co.uk
Base Domain bztactical.co.uk
Scan Status Ok
Last Scan2024-05-26T10:51:58+00:00
Next Scan 2024-06-25T10:51:58+00:00

Last Scan

Scanned2024-05-26T10:51:58+00:00
URL https://bztactical.co.uk/robots.txt
Redirect https://www.bztactical.co.uk/robots.txt
Redirect Domain www.bztactical.co.uk
Redirect Base bztactical.co.uk
Domain IPs 104.26.10.26, 104.26.11.26, 172.67.72.21, 2606:4700:20::681a:a1a, 2606:4700:20::681a:b1a, 2606:4700:20::ac43:4815
Redirect IPs 104.26.10.26, 104.26.11.26, 172.67.72.21, 2606:4700:20::681a:a1a, 2606:4700:20::681a:b1a, 2606:4700:20::ac43:4815
Response IP 172.67.72.21
Found Yes
Hash 548123d8058c75c858041ff4d8428361c2d06f252e4ec2afd39fac904b590beb
SimHash cb04bb736fe2

Groups

*

Rule Path
Allow /
Disallow /wishlist/index/add/
Disallow /auction/
Disallow /directory/
Disallow /catalog/product_compare/add/

*

Rule Path
Disallow /*?dir*
Disallow /*?dir=desc
Disallow /*?dir=asc
Disallow /*?limit=all
Disallow /*?mode*
Disallow /*?cat*
Disallow /*?manufacturer*
Disallow /*?price*
Disallow /*?C*
Disallow /*?O*
Disallow /*?p*
Disallow /*?item*
Disallow /*?size*
Disallow /*?colour*
Disallow /*?thread*
Disallow /*?caliber*
Disallow /*?gas*
Disallow /*?SID=
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Back to top

Other Records

Field Value
sitemap https://www.bzpaintball.co.uk/sitemap.xml
sitemap https://www.bztactical.co.uk/sitemaps/tactical_sitemap.xml
Back to top

Comments

  • robots.txt for Magento Community and Enterprise
  • GENERAL SETTINGS
  • Enable robots.txt rules for all crawlers
  • Do not crawl sub category pages that are sorted or filtered.
  • Do not crawl links with session IDs
  • Do not crawl checkout and user account pages
  • Do not crawl seach pages and not-SEO optimized catalog links
Back to top