nccred.org.au
robots.txt

Robots Exclusion Standard data for nccred.org.au

Resource Scan

Scan Details

Site Domain nccred.org.au
Base Domain nccred.org.au
Scan Status Ok
Last Scan2026-03-23T18:40:18+00:00
Next Scan 2026-03-30T18:40:18+00:00

Last Scan

Scanned2026-03-23T18:40:18+00:00
URL https://nccred.org.au/robots.txt
Domain IPs 170.64.235.18
Response IP 170.64.235.18
Found Yes
Hash 2f5be6dc4c5ed55e7e2f670af9fd8b3018c3ed93ac8d0864a1fbbd288c202bd1
SimHash a8284f264fd8

Groups

*

Rule Path
Disallow /uploads/main/cms-manual/
Disallow /uploads/main/Demo/
Disallow /uploads/main/For-Developers/
Disallow /imager/main/cms-manual/
Disallow /imager/main/Demo/
Disallow /imager/main/For-Developers/
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /dynamic
Disallow /docs

Comments

  • Allow robots access to everything
  • It's also highly recommended that any URLs blocked in robots.txt also have an X-Robots-Tag
  • header, otherwise they could still potentially make it into search results
  • See http://webmasters.stackexchange.com/questions/65184/will-the-already-indexed-urls-be-removed-if-i-use-a-robots-txt