Log In Sign Up

stayhealthy.ch
robots.txt

Robots Exclusion Standard data for stayhealthy.ch

Resource Scan

Scan Details

Site Domain stayhealthy.ch
Base Domain stayhealthy.ch
Scan Status Ok
Last Scan2024-09-06T10:52:38+00:00
Next Scan 2024-10-06T10:52:38+00:00

Last Scan

Scanned2024-09-06T10:52:38+00:00
URL https://stayhealthy.ch/robots.txt
Domain IPs 172.66.40.55, 172.66.43.201, 2606:4700:3108::ac42:2837, 2606:4700:3108::ac42:2bc9
Response IP 172.66.43.201
Found Yes
Hash 34a583306d7d87632ecc13a2572b6c1bf493bc740e3cd3640b1001081ffd75dd
SimHash 45347f154186

Groups

*

Rule Path
Disallow
Disallow /wishlist/
Disallow /admin/
Disallow /sendfriend/
Disallow /enable-cookies/
Disallow /skin/
Disallow /js/
Disallow /directory/
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /cartquickpro/
Disallow /comparing/
Disallow /tag/
Disallow /review/
Disallow /review/product/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Disallow /*?dir*
Disallow /*?dir=desc
Disallow /*?dir=asc
Disallow /*?limit=all
Disallow /*?mode*
Disallow /*?SID=
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /*.git
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$
Allow /pub/media/
Allow /media/

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.stayhealthy.ch/sitemaps/stayhealthy.ch/sitemap.xml
Back to top

Comments

  • robots.txt for Magento 2.x Community and Enterprise
  • GENERAL SETTINGS
  • Website Sitemap
  • Crawl-delay parameter: number of seconds to wait between successive requests to the same server.
  • Set a custom crawl rate if you're experiencing traffic problems with your server.
  • Crawl-delay: 30
  • Enable robots.txt rules for all crawlers
  • Disallow: /index.php/
  • Stop crawling user account and checkout pages by search engine robot:
  • Blocking native catalog and search pages:
  • Disable Tag & Review (Avoid duplicate content)
  • Disable Common files
  • Disable sorting (Avoid duplicate content)
  • Disallow: /*?
  • Blocking CMS directories.
  • Disable version control folders and others
  • Allow media
  • CRAWLERS SETTINGS
  • Bad Bots Disallow
  • User-agent: Baiduspider
  • Disallow: /
Back to top