Log In Sign Up

cdld.nl
robots.txt

Robots Exclusion Standard data for cdld.nl

Resource Scan

Scan Details

Site Domain cdld.nl
Base Domain cdld.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-12-15T19:41:46+00:00
Next Scan 2025-03-15T19:41:46+00:00

Last Successful Scan

Scanned2023-10-30T18:44:59+00:00
URL https://cdld.nl/robots.txt
Domain IPs 185.104.29.152, 2a06:2ec0:1::158
Response IP 185.104.29.152
Found Yes
Hash ae815efcfb5fdf45b1cc3afc2cd4f3a4ae15183e0ba075bfd2ae60737012c44e
SimHash e31d1d5bc3f4

Groups

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/
Back to top

Other Records

Field Value
sitemap https://cdld.nl/index.php?option=com_jmap&view=sitemap&format=xml
sitemap https://cdld.nl/index.php?option=com_jmap&view=sitemap&format=videos
sitemap https://cdld.nl/index.php?option=com_jmap&view=sitemap&format=images
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
  • JSitemap entries
Back to top