Log In Sign Up

grandtourcollection.com
robots.txt

Robots Exclusion Standard data for grandtourcollection.com

Resource Scan

Scan Details

Site Domain grandtourcollection.com
Base Domain grandtourcollection.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-09T14:59:49+00:00
Next Scan 2026-01-08T14:59:49+00:00

Last Successful Scan

Scanned2025-10-17T01:20:04+00:00
URL https://grandtourcollection.com/robots.txt
Redirect https://www.grandtourcollection.com/robots.txt
Redirect Domain www.grandtourcollection.com
Redirect Base grandtourcollection.com
Domain IPs 104.26.4.35, 104.26.5.35, 172.67.70.114, 2606:4700:20::681a:423, 2606:4700:20::681a:523, 2606:4700:20::ac43:4672
Redirect IPs 104.26.4.35, 104.26.5.35, 172.67.70.114, 2606:4700:20::681a:423, 2606:4700:20::681a:523, 2606:4700:20::ac43:4672
Response IP 104.26.4.35
Found Yes
Hash 6945693ec79b05e6cbdc812f4020932484fee0af7fe7c4005f607e034541511a
SimHash 2444ad32eeb2

Groups

*

Rule Path
Allow /*?p=
Disallow */index.php/
Disallow */catalog/product_compare/
Disallow */catalog/category/view/
Disallow */catalog/product/gallery/
Disallow */catalogsearch/
Disallow */checkout/
Disallow */control/
Disallow */contacts/
Disallow */customer/
Disallow */customize/
Disallow */newsletter/
Disallow */poll/
Disallow */review/
Disallow */sales/
Disallow */sendfriend/
Disallow */tag/
Disallow */wishlist/
Disallow /*?dir*
Disallow /*?limit*
Disallow /*?mode*
Disallow /*?price=*
Disallow /*?___from_store=*
Disallow /*?___store=*
Disallow /*?q=*
Disallow /*?p=*&
Disallow /*.php$
Disallow /*?SID=

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.grandtourcollection.com/sitemap.xml
Back to top

Comments

  • For more information about the robots.txt standard visit
  • http://www.robotstxt.org/orig.html
  • For syntax checking visit
  • http://tool.motoricerca.info/robots-checker.phtml
  • Crawl the Sitemap. Set the correct URL before uncomment
  • Crawlers Setup
  • How many seconds a crawler should wait before loading and crawling page content
  • Set a custom crawl rate if you are experiencing traffic issues with your server
  • https://www.contentkingapp.com/academy/robotstxt/faq/crawl-delay-10/
  • Allow to crawl paging (paging inside a listing with more params are disallowed below)
  • Do not crawl non-SEF paths and generated content (if you use a store id in URL you must prefix with * or copy for each store)
  • Disallow: */catalog/product/view/
  • Allow: */catalogsearch/seo_sitemap
  • Allow: */catalogsearch/term/popular
  • Do not crawl dynamic filters. Uncomment what you need or add custom filters
  • Disallow: /*?cat=*
  • Disallow: /*?availability=*
  • Disallow: /*?brand=*
  • Do not crawl paths that can be safely ignored by search engines (no clean URLs)
  • Do not allow media indexing for the following bots
  • Disallow all or add custom paths. For example */media/ or */skin/
  • User-agent: baiduspider-image
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: baiduspider-video
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: msnbot-media
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: Googlebot-Image
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: Googlebot-Video
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
Back to top