Log In Sign Up

traumsofas.de
robots.txt

Robots Exclusion Standard data for traumsofas.de

Resource Scan

Scan Details

Site Domain traumsofas.de
Base Domain traumsofas.de
Scan Status Ok
Last Scan2025-09-22T08:14:41+00:00
Next Scan 2025-10-22T08:14:41+00:00

Last Scan

Scanned2025-09-22T08:14:41+00:00
URL https://traumsofas.de/robots.txt
Redirect https://www.traumsofas.de/robots.txt
Redirect Domain www.traumsofas.de
Redirect Base traumsofas.de
Domain IPs 104.26.10.35, 104.26.11.35, 172.67.74.111, 2606:4700:20::681a:a23, 2606:4700:20::681a:b23, 2606:4700:20::ac43:4a6f
Redirect IPs 104.26.10.35, 104.26.11.35, 172.67.74.111, 2606:4700:20::681a:a23, 2606:4700:20::681a:b23, 2606:4700:20::ac43:4a6f
Response IP 104.26.10.35
Found Yes
Hash c59bcafc20f1212393c8b0b4a8ebbfa71b43623646bf302c49b558a6d32cc64a
SimHash 2444edb2eef2

Groups

*

Rule Path
Allow /
Allow /*?p=
Disallow */index.php/
Disallow */catalog/product_compare/
Disallow */catalog/category/view/
Disallow */catalog/product/view/
Disallow */catalog/product/gallery/
Disallow */catalogsearch/
Disallow */checkout/
Disallow */control/
Disallow */contacts/
Disallow */customer/
Disallow */customize/
Disallow */newsletter/
Disallow */poll/
Disallow */review/
Disallow */sales/
Disallow */sendfriend/
Disallow */tag/
Disallow */wishlist/
Disallow /lightboxcms/ajax/loadCmsPage/
Disallow /lightboxcms/ajax/loadCmsBlock/
Disallow /fabricsamples
Disallow /*?dir*
Disallow /*?limit*
Disallow /*?mode*
Disallow /*?price=*
Disallow /*?___from_store=*
Disallow /*?___store=*
Disallow /*?q=*
Disallow /*?p=*&
Disallow /*.php$
Disallow /*?SID=

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.traumsofas.de/sitemap/sitemap.xml
Back to top

Comments

  • For more information about the robots.txt standard visit
  • http://www.robotstxt.org/orig.html
  • For syntax checking visit
  • http://tool.motoricerca.info/robots-checker.phtml
  • Crawl the Sitemap. Set the correct URL before uncomment
  • Crawlers Setup
  • How many seconds a crawler should wait before loading and crawling page content
  • Set a custom crawl rate if you are experiencing traffic issues with your server
  • https://www.contentkingapp.com/academy/robotstxt/faq/crawl-delay-10/
  • Allow to crawl paging (paging inside a listing with more params are disallowed below)
  • Do not crawl non-SEF paths and generated content (if you use a store id in URL you must prefix with * or copy for each store)
  • Allow: */catalogsearch/seo_sitemap
  • Allow: */catalogsearch/term/popular
  • Do not crawl custom urls
  • Do not crawl dynamic filters. Uncomment what you need or add custom filters
  • Disallow: /*?cat=*
  • Disallow: /*?availability=*
  • Disallow: /*?brand=*
  • Do not crawl paths that can be safely ignored by search engines (no clean URLs)
  • Do not allow media indexing for the following bots
  • Disallow all or add custom paths. For example */media/ or */skin/
  • User-agent: baiduspider-image
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: baiduspider-video
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: msnbot-media
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: Googlebot-Image
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
  • User-agent: Googlebot-Video
  • Disallow: /
  • Disallow: */media/
  • Disallow: */skin/
Back to top