Log In Sign Up

sleepsolutions.com.au
robots.txt

Robots Exclusion Standard data for sleepsolutions.com.au

Resource Scan

Scan Details

Site Domain sleepsolutions.com.au
Base Domain sleepsolutions.com.au
Scan Status Ok
Last Scan2024-06-10T16:40:51+00:00
Next Scan 2024-07-10T16:40:51+00:00

Last Scan

Scanned2024-06-10T16:40:51+00:00
URL https://sleepsolutions.com.au/robots.txt
Redirect https://www.sleepsolutions.com.au/robots.txt
Redirect Domain www.sleepsolutions.com.au
Redirect Base sleepsolutions.com.au
Domain IPs 172.66.40.74, 172.66.43.182, 2606:4700:3108::ac42:284a, 2606:4700:3108::ac42:2bb6
Redirect IPs 172.66.40.74, 172.66.43.182, 2606:4700:3108::ac42:284a, 2606:4700:3108::ac42:2bb6
Response IP 172.66.40.74
Found Yes
Hash fc64d0dc5100eee5b9c74671f74880d9f0d3045795efd8019fa9b80f8adef851
SimHash 2f01731587b2

Groups

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /CVS
Disallow /*.svn$
Disallow /*.idea$
Disallow /*.sql$
Disallow /*.tgz$
Disallow /admin/
Disallow /errors/
Disallow /opt/
Disallow /api.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /get.php
Disallow /*?gclid=
Disallow /*?limit=
Disallow /*?mode*
Disallow /*?playerInstance=
Disallow /*?datetime=
Disallow /*?price=
Disallow /*?product_id=
Disallow /*?technologydisplay=
Disallow /*?quiltweight=
Disallow /*?sort=
Disallow /*?seatecsize=
Disallow /*?m=
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /control/
Disallow /contacts/
Disallow /customize/
Disallow /newsletter/
Disallow /review/
Disallow /sendfriend/
Disallow /wishlist/
Disallow /*?SID=
Disallow /catalog/product/gallery/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalog/seo_sitemap/category/
Disallow /sales/guest/form/
Disallow /blog/test_Category/
Disallow /cleanup.php
Disallow /apc.php
Disallow /memcache.php
Disallow /phpinfo.php
Allow /*.js$
Allow /*.css$
Allow /*.png$
Allow /*.jpg$
Allow /*.gif$
Back to top

Other Records

Field Value
sitemap https://www.sleepsolutions.com.au/sitemap.xml
Back to top

Comments

  • GENERAL SETTINGS For MAGENTO
  • Do not index the page Magento admin
  • Do not index the general technical Magento directory
  • Do not index the shared files Magento
  • MAGENTO SEO IMPROVEMENT
  • Do not index the page subcategories that are sorted or filtered..
  • Disallow: */shopby/*
  • Disallow: /amfilerating/
  • Do not index the second copy of the home page (example.com / index.php /). Un-comment only if you have activated Magento SEO URLs.
  • Disallow: /index.php/
  • Paths (clean URLs)
  • Do not index the link from the session ID
  • Do not index the search page and CEO, non-optimized link categories
  • Server Settings
  • Do not index the general technical directories and files on a server
Back to top

Warnings

  • `host` is not a known field.
Back to top