Log In Sign Up

aspect.org.au
robots.txt

Robots Exclusion Standard data for aspect.org.au

Resource Scan

Scan Details

Site Domain aspect.org.au
Base Domain aspect.org.au
Scan Status Ok
Last Scan2024-06-15T01:03:59+00:00
Next Scan 2024-06-22T01:03:59+00:00

Last Scan

Scanned2024-06-15T01:03:59+00:00
URL https://aspect.org.au/robots.txt
Redirect https://www.aspect.org.au/robots.txt
Redirect Domain www.aspect.org.au
Redirect Base aspect.org.au
Domain IPs 103.42.111.74
Redirect IPs 23.44.4.144, 23.44.4.178
Response IP 23.50.232.235
Found Yes
Hash 89cde1647e86f4b3ccb5ce0b806be6961a8cda591d7c3be271c0b4373069d017
SimHash 080c87044ff1

Groups

*

Rule Path
Disallow /uploads/main/cms-manual/
Disallow /dynamic
Back to top

Comments

  • Allow robots access to everything
  • It's also highly recommended that any URLs blocked in robots.txt also have an X-Robots-Tag
  • header, otherwise they could still potentially make it into search results
  • See http://webmasters.stackexchange.com/questions/65184/will-the-already-indexed-urls-be-removed-if-i-use-a-robots-txt
Back to top