Log In Sign Up

dost.gov.ph
robots.txt

Robots Exclusion Standard data for dost.gov.ph

Resource Scan

Scan Details

Site Domain dost.gov.ph
Base Domain dost.gov.ph
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-10T02:31:27+00:00
Next Scan 2026-02-09T02:31:27+00:00

Last Successful Scan

Scanned2025-10-25T12:08:46+00:00
URL https://dost.gov.ph/robots.txt
Redirect https://www.dost.gov.ph/robots.txt
Redirect Domain www.dost.gov.ph
Redirect Base dost.gov.ph
Domain IPs 20.44.210.117
Redirect IPs 20.44.210.117
Response IP 20.44.210.117
Found Yes
Hash 200459daa9512b4c94418e170ba85835a34b8b09ff32b9c50d2060587ddd61b9
SimHash e21f1d5949fc

Groups

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • https://www.robotstxt.org/orig.html
Back to top