my.cqu.edu.au
robots.txt

Robots Exclusion Standard data for my.cqu.edu.au

Resource Scan

Scan Details

Site Domain my.cqu.edu.au
Base Domain cqu.edu.au
Scan Status Ok
Last Scan2024-11-03T09:04:49+00:00
Next Scan 2024-12-03T09:04:49+00:00

Last Scan

Scanned2024-11-03T09:04:49+00:00
URL https://my.cqu.edu.au/robots.txt
Domain IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 6e4d773c657026706fe2fd10640fc43c3415a82a2b5f34d7cbefcf104ee82624
SimHash 6852cd5b0374

Groups

*

Rule Path
Allow /$
Allow /login
Disallow /

funnelback

Rule Path
Allow *

Comments

  • Set all agent to disallow crawling of the whole site except root and /login.