cat.lib.unimelb.edu.au
robots.txt
Robots Exclusion Standard data for cat.lib.unimelb.edu.au
Resource Scan
Scan Details
| Site Domain | cat.lib.unimelb.edu.au |
| Base Domain | unimelb.edu.au |
| Scan Status | Ok |
| Last Scan | 2025-10-12T01:04:50+00:00 |
| Next Scan | 2025-11-11T01:04:50+00:00 |
Last Scan
| Scanned | 2025-10-12T01:04:50+00:00 |
| URL | https://cat.lib.unimelb.edu.au/robots.txt |
| Domain IPs | 54.153.201.132 |
| Response IP | 54.153.201.132 |
| Found | Yes |
| Hash | 2016df8af895d47c63b41025f150983cfb8d2148590f373cf7538e9d18e61a48 |
| SimHash | 3774dde26050 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /acquire |
| Disallow | /airpac |
| Disallow | /airwkst |
| Disallow | /articles |
| Disallow | /availlim |
| Disallow | /bookill |
| Disallow | /bookit |
| Disallow | /circhistlim |
| Disallow | /circpix |
| Disallow | /cisti_order |
| Disallow | /clearhist |
| Disallow | /documents |
| Disallow | /donate |
| Disallow | /extlang |
| Disallow | /feeds |
| Disallow | /ftlist |
| Disallow | /goto |
| Disallow | /iii |
| Disallow | /ill |
| Disallow | /illframe |
| Disallow | /indexsort |
| Disallow | /journill |
| Disallow | /kids |
| Disallow | /launch |
| Disallow | /logout |
| Disallow | /manage |
| Disallow | /manual |
| Disallow | /metafind |
| Disallow | /mfgo |
| Disallow | /netli |
| Disallow | /nonret |
| Disallow | /patroninfo |
| Disallow | /programs |
| Disallow | /review |
| Disallow | /search~S1 |
| Disallow | /search~S2 |
| Disallow | /search~S3 |
| Disallow | /search~S4 |
| Disallow | /search~S5 |
| Disallow | /search~S6 |
| Disallow | /search~S7 |
| Disallow | /search~S8 |
| Disallow | /search~S9 |
| Disallow | /search~S10 |
| Disallow | /search~S11 |
| Disallow | /search~S12 |
| Disallow | /search~S13 |
| Disallow | /search~S14 |
| Disallow | /search~S15 |
| Disallow | /search~S16 |
| Disallow | /search~S17 |
| Disallow | /search~S18 |
| Disallow | /search~S19 |
| Disallow | /search~S20 |
| Disallow | /search~S21 |
| Disallow | /search~S22 |
| Disallow | /search~S23 |
| Disallow | /search~S24 |
| Disallow | /search~S25 |
| Disallow | /search~S26 |
| Disallow | /search~S27 |
| Disallow | /search~S28 |
| Disallow | /search~S29 |
| Disallow | /search~S32 |
| Disallow | /search~S33 |
| Disallow | /search~S34 |
| Disallow | /search~S35 |
| Disallow | /search~S36 |
| Disallow | /search~S37 |
| Disallow | /search~S38 |
| Disallow | /search~S39 |
| Disallow | /search~S40 |
| Disallow | /search~S41 |
| Disallow | /search~S42 |
| Disallow | /search~S43 |
| Disallow | /search~S44 |
| Disallow | /search~S45 |
| Disallow | /search~S46 |
| Disallow | /search~S47 |
| Disallow | /search~S48 |
| Disallow | /search~S49 |
| Disallow | /search~S50 |
| Disallow | /selfreg |
| Disallow | /setlang |
| Disallow | /setscope |
| Disallow | /suggest |
| Disallow | /tmp |
| Disallow | /validate |
| Disallow | /VERIFYPATRON |
| Disallow | /VERSION |
| Disallow | /weblang |
| Disallow | /wm |
| Disallow | /xrecord%3D |
| Disallow | /z39 |
| Disallow | /z39m |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot-ia
| Rule | Path |
|---|---|
| Disallow | /acquire |
| Disallow | /airpac |
| Disallow | /airwkst |
| Disallow | /articles |
| Disallow | /availlim |
| Disallow | /bookill |
| Disallow | /bookit |
| Disallow | /circhistlim |
| Disallow | /circpix |
| Disallow | /cisti_order |
| Disallow | /clearhist |
| Disallow | /documents |
| Disallow | /donate |
| Disallow | /extlang |
| Disallow | /feeds |
| Disallow | /ftlist |
| Disallow | /goto |
| Disallow | /iii |
| Disallow | /ill |
| Disallow | /illframe |
| Disallow | /indexsort |
| Disallow | /journill |
| Disallow | /kids |
| Disallow | /launch |
| Disallow | /logout |
| Disallow | /manage |
| Disallow | /manual |
| Disallow | /metafind |
| Disallow | /mfgo |
| Disallow | /netli |
| Disallow | /nonret |
| Disallow | /patroninfo |
| Disallow | /programs |
| Disallow | /review |
| Disallow | /selfreg |
| Disallow | /setlang |
| Disallow | /setscope |
| Disallow | /suggest |
| Disallow | /tmp |
| Disallow | /validate |
| Disallow | /VERIFYPATRON |
| Disallow | /VERSION |
| Disallow | /weblang |
| Disallow | /wm |
| Disallow | /xrecord%3D |
| Disallow | /z39 |
| Disallow | /z39m |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Comments