acorn.org.au
robots.txt

Robots Exclusion Standard data for acorn.org.au

Resource Scan

Scan Details

Site Domain acorn.org.au
Base Domain acorn.org.au
Scan Status Ok
Last Scan2025-10-21T04:24:39+00:00
Next Scan 2025-11-20T04:24:39+00:00

Last Scan

Scanned2025-10-21T04:24:39+00:00
URL https://acorn.org.au/robots.txt
Redirect https://www.acorn.org.au/robots.txt
Redirect Domain www.acorn.org.au
Redirect Base acorn.org.au
Domain IPs 104.21.56.188, 172.67.155.138, 2606:4700:3030::ac43:9b8a, 2606:4700:3035::6815:38bc
Redirect IPs 104.21.56.188, 172.67.155.138, 2606:4700:3030::ac43:9b8a, 2606:4700:3035::6815:38bc
Response IP 172.67.155.138
Found Yes
Hash 9e7bf317499d1fd941d0672d4171272210da8e5ce0dcd73ab27a782777950a8d
SimHash a113d8306303

Groups

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /ajax.cfm
Disallow /api.cfm
Disallow /content.cfm
Disallow /site/ajax.cfm
Disallow /site/api.cfm
Disallow /site/content.cfm
Disallow
Allow /client_images/*.jpg$
Allow /client_images/*.jpeg$
Allow /client_images/*.png$
Allow /client_images/*.gif$
Allow /client_images/*.JPG$
Allow /client_images/*.JPEG$
Allow /client_images/*.PNG$
Allow /client_images/*.GIF$
Disallow /client_images/

Comments

  • robots.txt
  • Crawl-delay: 20