dgt.org.au
robots.txt

Robots Exclusion Standard data for dgt.org.au

Resource Scan

Scan Details

Site Domain dgt.org.au
Base Domain dgt.org.au
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-11T07:33:54+00:00
Next Scan 2026-03-11T07:33:54+00:00

Last Successful Scan

Scanned2025-08-04T21:41:45+00:00
URL https://dgt.org.au/robots.txt
Domain IPs 104.26.12.122, 104.26.13.122, 172.67.72.62, 2606:4700:20::681a:c7a, 2606:4700:20::681a:d7a, 2606:4700:20::ac43:483e
Response IP 104.26.12.122
Found Yes
Hash 06d91edd7a917b0eda24b27fedb06f836fb362fbfff3afcc8443d0cb8d22b5c3
SimHash 43381d522f93

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env

Other Records

Field Value
sitemap https://www.dgt.org.au/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.dgt.org.au/
  • live - don't allow web crawlers to index cpresources/ or vendor/