haileybury.com.au
robots.txt

Robots Exclusion Standard data for haileybury.com.au

Resource Scan

Scan Details

Site Domain haileybury.com.au
Base Domain haileybury.com.au
Scan Status Ok
Last Scan2024-05-21T22:36:04+00:00
Next Scan 2024-06-20T22:36:04+00:00

Last Scan

Scanned2024-05-21T22:36:04+00:00
URL https://haileybury.com.au/robots.txt
Redirect https://www.haileybury.com.au/robots.txt
Redirect Domain www.haileybury.com.au
Redirect Base haileybury.com.au
Domain IPs 54.153.229.39
Redirect IPs 13.33.30.36, 13.33.30.37, 13.33.30.53, 13.33.30.91, 2600:9000:229f:1000:17:e72d:74c0:93a1, 2600:9000:229f:1600:17:e72d:74c0:93a1, 2600:9000:229f:1800:17:e72d:74c0:93a1, 2600:9000:229f:1c00:17:e72d:74c0:93a1, 2600:9000:229f:2a00:17:e72d:74c0:93a1, 2600:9000:229f:5400:17:e72d:74c0:93a1, 2600:9000:229f:8200:17:e72d:74c0:93a1, 2600:9000:229f:a800:17:e72d:74c0:93a1
Response IP 13.33.30.36
Found Yes
Hash 021a6d877c686ca5cdb7e8d849faba8beff79e6478955f6c8cc9600bf4357ff7
SimHash 41601f523d91

Groups

*

Rule Path
Disallow /cpresources/
Disallow /pangea-fullsite/
Disallow /actions/queue/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://www.haileybury.com.au/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.haileybury.com.au/
  • live - don't allow web crawlers to index cpresources/ or vendor/