study.unimelb.edu.au
robots.txt

Robots Exclusion Standard data for study.unimelb.edu.au

Resource Scan

Scan Details

Site Domain study.unimelb.edu.au
Base Domain unimelb.edu.au
Scan Status Ok
Last Scan2025-03-03T07:01:31+00:00
Next Scan 2025-04-02T07:01:31+00:00

Last Scan

Scanned2025-03-03T07:01:31+00:00
URL https://study.unimelb.edu.au/robots.txt
Domain IPs 185.64.255.1
Response IP 202.9.95.188
Found Yes
Hash a53d9573e09d47b27fae0275ece31efdbf2085756b332539195ba65af4e43ece
SimHash 518c50f3a316

Groups

funnelback
*

Rule Path
Disallow /assets/
Disallow /web_services/
Disallow /dev/
Disallow /dev-nonfac/
Disallow /404*
Disallow /search
Disallow /opendays/_sections/
Disallow /opendays/assets/
Disallow /opendays/dev/
Disallow /opendays/images/
Disallow /opendays/thank-you*
Disallow /__data/
Allow /__data/assets/image/0025/393271/favicon.png
Disallow /how-to-apply/graduate-coursework-study/graduate-coursework-options/conditional-offers/
Disallow /connect-with-us/revision-series-unit-3
Disallow /connect-with-us/events-for-future-students/events-for-review
Disallow /study-tags/
Disallow /study-with-us/programs-for-high-achievers/kwong-lee-dow-young-scholars-program/kld-portal
Disallow /lp/templates/
Disallow /lp/assets/

twitterbot

Rule Path
Allow /__data/assets/image*

powermapper

Rule Path
Allow /

Other Records

Field Value
sitemap https://study.unimelb.edu.au/sitemap.xml

Comments

  • Specific crawler advisories