aromahuen.com
robots.txt

Robots Exclusion Standard data for aromahuen.com

Resource Scan

Scan Details

Site Domain aromahuen.com
Base Domain aromahuen.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-03-29T02:45:33+00:00
Next Scan 2025-05-28T02:45:33+00:00

Last Successful Scan

Scanned2025-01-06T01:55:38+00:00
URL https://aromahuen.com/robots.txt
Domain IPs 104.21.42.106, 172.67.161.62, 2606:4700:3030::6815:2a6a, 2606:4700:3030::ac43:a13e
Response IP 104.21.42.106
Found Yes
Hash 819e9cbaa0f7ebc74c72b14f58c37ebf5e245100e0a50bf43bbc87978dd9f40f
SimHash 85208843cd73

Groups

*

Rule Path
Disallow /404/
Disallow /app/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /js/
Disallow /lib/
Disallow /magento/
Disallow /pkginfo/
Disallow /shell/
Disallow /skin/
Disallow /var/
Disallow /tmp/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /catalog/product/gallery/
Disallow /admin
Disallow /index.php/catalog/product_compare/
Disallow /index.php/catalog/category/view/
Disallow /index.php/catalog/product/view/
Disallow /index.php/catalogsearch/
Disallow /index.php/checkout/
Disallow /index.php/control/
Disallow /index.php/contacts/
Disallow /index.php/customer/
Disallow /index.php/customize/
Disallow /index.php/newsletter/
Disallow /index.php/poll/
Disallow /index.php/review/
Disallow /index.php/sendfriend/
Disallow /index.php/tag/
Disallow /index.php/wishlist/
Disallow /index.php/catalog/product/gallery/
Disallow /index.php/admin
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /get.php
Disallow /robots.txt
Disallow /*.php$
Disallow /*?p=*&
Disallow /*?color=*
Disallow /*?producent=*
Disallow /*?price=*
Disallow /*?limit=all&*
Disallow /*?taste_profile=*
Disallow /*?*

Comments

  • Default settings for bots:
  • Dirs that should not be indexed:
  • Disallow: /media/
  • Minimize duplicate content, very important!:
  • And for users without SEO-rewrites:
  • Files with no interest:
  • No reason to index these: