Log In Sign Up

doolz.co.nz
robots.txt

Robots Exclusion Standard data for doolz.co.nz

Resource Scan

Scan Details

Site Domain doolz.co.nz
Base Domain doolz.co.nz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-02T11:40:22+00:00
Next Scan 2024-10-16T11:40:22+00:00

Last Successful Scan

Scanned2024-08-25T11:34:09+00:00
URL https://doolz.co.nz/robots.txt
Redirect https://www.doolz.co.nz/robots.txt
Redirect Domain www.doolz.co.nz
Redirect Base doolz.co.nz
Domain IPs 149.28.175.70, 2001:19f0:5801:1764:5400:1ff:fef1:d1c2
Redirect IPs 149.28.175.70, 2001:19f0:5801:1764:5400:1ff:fef1:d1c2
Response IP 149.28.175.70
Found Yes
Hash 62c611a8839a7059fc4d8b544e541f584bcaf84f80431654d2959531831c2bdd
SimHash 131f15590ba8

Groups

lcc

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

daum

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

seekport

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

*

Rule Path
Disallow /_api/
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Disallow /store/catalogue/cart.html
Disallow /store/catalogue/checkout.html
Disallow /store/catalogue/checkout/
Disallow /store/search/
Disallow /store/price-drops.html
Disallow /store/price-drops/
Back to top

Other Records

Field Value
sitemap https://www.doolz.co.nz/sitemap.txt
sitemap https://www.doolz.co.nz/sitemap1.txt
sitemap https://www.doolz.co.nz/sitemap2.txt
sitemap https://www.doolz.co.nz/sitemap3.txt
sitemap https://www.doolz.co.nz/sitemap4.txt
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
Back to top