Log In Sign Up

scfreiburg.com
robots.txt

Robots Exclusion Standard data for scfreiburg.com

Resource Scan

Scan Details

Site Domain scfreiburg.com
Base Domain scfreiburg.com
Scan Status Ok
Last Scan2025-12-11T02:59:05+00:00
Next Scan 2026-01-10T02:59:05+00:00

Last Scan

Scanned2025-12-11T02:59:05+00:00
URL https://scfreiburg.com/robots.txt
Redirect https://www.scfreiburg.com/robots.txt
Redirect Domain www.scfreiburg.com
Redirect Base scfreiburg.com
Domain IPs 185.48.220.111
Redirect IPs 185.48.220.111
Response IP 185.48.220.111
Found Yes
Hash 2117989ab0884f97c9ab0130c824d518d61c3455917d865f61e0128e435a3048
SimHash 30715354aeb3

Groups

*

Rule Path
Disallow /forum/?*
Disallow /forum/members/*
Disallow /*.git$
Disallow /*.idea$
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*.tar.gz$
Disallow /*.zip$
Disallow /session.php
Disallow /shop/admin/
Disallow /shop/app/
Disallow /shop/downloader/
Disallow /shop/errors/
Disallow /shop/includes/
Disallow /shop/lib/
Disallow /shop/pkginfo/
Disallow /shop/shell/
Disallow /shop/skin/
Disallow /shop/var/
Disallow /shop/api.php
Disallow /shop/cron.php
Disallow /shop/cron.sh
Disallow /shop/error_log
Disallow /shop/get.php
Disallow /shop/install.php
Disallow /shop/LICENSE.html
Disallow /shop/LICENSE.txt
Disallow /shop/LICENSE_AFL.txt
Disallow /shop/README.txt
Disallow /shop/RELEASE_NOTES.txt
Disallow /shop/STATUS.txt
Disallow /shop/*?Dir*
Disallow /shop/*?Dir=desc
Disallow /shop/*?Dir=asc
Disallow /shop/*?Limit=all
Disallow /shop/*?Mode*
Disallow /shop/index.php/
Disallow /shop/*?SID=
Disallow /shop/checkout/
Disallow /shop/catalog/product_compare/
Disallow /shop/onestepcheckout/
Disallow /shop/customer/
Disallow /shop/customer/account/
Disallow /shop/customer/account/login/
Disallow /shop/newsletter/
Disallow /shop/poll/
Disallow /shop/review/
Disallow /shop/sendfriend/
Disallow /shop/wishlist/
Disallow /shop/catalogsearch/
Disallow /shop/catalog/category/view/
Disallow /shop/catalog/product/view/
Disallow /shop/catalog/product/gallery/

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

trendictionbot0.5.0

Rule Path
Disallow /

owlin

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

stashbot

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.scfreiburg.com/typo3temp/sitemapindex.xml
Back to top

Comments

  • Enables robots.txt rules for all crawlers
  • Do not allow indexing files and folders that are required during development: CVS, SVN directory and dump files
  • GENERAL SETTINGS For MAGENTO
  • Do not index the general technical Magento directory
  • Do not index the shared files Magento
  • MAGENTO SEA IMPROVEMENT
  • Do not index the page subcategories that are sorted or filtered.
  • Do not index the second copy of the home page (example.com / index.php /). Un-comment only if you have activated Magento SEO URLs.
  • Do not index the link from the session ID
  • Do not index the page checkout and user account
  • Do not index the search page and CEO, non-optimized link categories
  • Exclude Various Robots
Back to top