Log In Sign Up

allaboutwine.co.uk
robots.txt

Robots Exclusion Standard data for allaboutwine.co.uk

Resource Scan

Scan Details

Site Domain allaboutwine.co.uk
Base Domain allaboutwine.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-24T03:55:49+00:00
Next Scan 2025-01-22T03:55:49+00:00

Last Successful Scan

Scanned2023-09-23T10:31:57+00:00
URL https://allaboutwine.co.uk/robots.txt
Domain IPs 104.26.0.174, 104.26.1.174, 172.67.72.109, 2606:4700:20::681a:1ae, 2606:4700:20::681a:ae, 2606:4700:20::ac43:486d
Response IP 172.67.72.109
Found Yes
Hash 07f95ee7132d17882778640bf1ec6f6e44f176623c16984c7d3e9fc9993d5982
SimHash a31e155843e5

Groups

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
  • Disallow: /layouts/
  • Disallow: /media/
  • Disallow: /templates/
Back to top