Log In Sign Up

monash.edu
robots.txt

Robots Exclusion Standard data for monash.edu

Resource Scan

Scan Details

Site Domain monash.edu
Base Domain monash.edu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-13T06:47:10+00:00
Next Scan 2025-01-11T06:47:10+00:00

Last Successful Scan

Scanned2024-03-11T06:45:27+00:00
URL https://monash.edu/robots.txt
Redirect https://www.monash.edu/robots.txt
Redirect Domain www.monash.edu
Redirect Base monash.edu
Domain IPs 43.245.41.7
Redirect IPs 104.18.41.238, 172.64.146.18
Response IP 172.64.146.18
Found Yes
Hash 7ab17863d1b43e0506249e408253c2fc55bfd7f3677c801bd9bddf90247fc07f
SimHash 35db3b688cd7

Groups

*
*

Rule Path Comment
Disallow /*?sq_content_src= -
Disallow /*_recache -
Disallow /*_nocache -
Disallow /*_edit -
Disallow /*_admin -
Disallow /*_login -
Disallow /*_performance -
Disallow /_content -
Disallow /_designs -
Disallow /__lib -
Disallow /__fudge -
Disallow /_dev -
Disallow /global-search -
Disallow /search -
Disallow /_webservices -
Disallow /monash-mango/ -
Disallow /staff/web/templates/ -
Disallow /about/itsp/monash_only/ -
Disallow /about/support_services/ -
Disallow /about/support_servicesold/ -
Disallow /journals/analytica-moscow/ -
Disallow /journals/old-moscow/ -
Disallow /magpie/restrict/ -
Disallow /monashplan/draft/ -
Disallow /monashplan/global99/ -
Disallow /monashplan/monashonly/ -
Disallow /mrs/support/news2/ -
Disallow /news/media/ -
Disallow /pso-old/staff/ -
Disallow /staff/announce/ -
Disallow /staff/identity/ -
Disallow /wwwdev/support/webauthors-l/ -
Disallow /colours/ -
Disallow /copyright/MonStaff/ -
Disallow /pub/ The FTP tree should not be indexed as HTTP
Disallow /pub/ftp -
Disallow /ftp -
Disallow /old/ Where we keep things we think we might need again
Disallow /wwwdev_src/ seems logical
Disallow /images/ Not a lot of point
Disallow /cgi-bin/ -
Disallow /ausapec/ -
Disallow /cclcs/ -
Disallow /korean/ -
Disallow /phil/ -
Disallow /nksc/ -
Disallow /visarts/ -
Disallow /ling/ -
Disallow /mai/ -
Disallow /ncas/ -
Disallow /psych/ -
Disallow /deeb/ -
Disallow /genetics/ -
Disallow /appsci/ -
Disallow /newwww/ -
Disallow /informatics/ -
Disallow /study/study-archive -
Disallow /pubs/1999handbooks/ -
Disallow /pubs/98handbooks/ -
Disallow /pubs/97handbooks/ -
Disallow /pubs/96handbooks/ -
Disallow /pubs/1995handbooks/ -
Disallow /pubs/2001handbooks/ -
Disallow /pubs/2000handbooks/ -
Disallow /pubs/2002handbooks/ -
Disallow /pubs/2003handbooks/ -
Disallow /pubs/2004handbooks/ -
Disallow /pubs/2005handbooks/ -
Disallow /pubs/2006handbooks/ -
Disallow /pubs/handbooks/offcampus/ -
Disallow /pubs/2007handbooks/ -
Disallow /pubs/2008handbooks/ -
Disallow /pubs/2009handbooks/ -
Disallow /pubs/2010handbooks/ -
Disallow /pubs/2011handbooks/ -
Disallow /pubs/2012handbooks/ -
Disallow /pubs/2013handbooks/ -
Disallow /pubs/2014handbooks/ -
Disallow /pubs/2015handbooks/ -
Disallow /pubs/2016handbooks/ -
Disallow /pubs/2017handbooks/ -
Disallow /pubs/2018handbooks/ -
Disallow /pubs/2019handbooks/ -
Disallow /esolutions/accounts-passwords/access/ -
Disallow /wwwdev/stats/ -
Disallow /servers/stats/ -
Disallow /cc/ccst1 -
Disallow /cc/ccst2 -
Disallow /cc/ccst3 -
Disallow /cc/ccst4 -
Disallow /cc/ccst5 -
Disallow /cc/ccst6 -
Disallow /cc/ccst7 -
Disallow /cc/ccst8 -
Disallow /cc/ccst9 -
Disallow /cc/ccstaff1 -
Disallow /cc/ccstaff2 -
Disallow /cc/ccstaff3 -
Disallow /cc/ccstaff4 -
Disallow /cc/ccstaff5 -
Disallow /cc/ccstaff6 -
Disallow /cc/student -
Disallow /cc/staff -
Disallow /cc/staff2 -
Disallow /cc/alstaff -
Disallow /cc/alst -
Disallow /eestaff -
Disallow /eestaff2 -
Disallow /eest1 -
Disallow /eest2 -
Disallow /mobstaff1 -
Disallow /mobstaff2 -
Disallow /cc/ -
Disallow /journals/analytica-moscow/ -
Disallow /journals/old-moscow/ -
Disallow /wwwdev/yale/ -
Disallow /pso/psoarchive2000/ -
Disallow /pso/psoarchive_pre2000/ -
Disallow /resgrant/digest/ -
Disallow /resgrant/gresults/ -
Disallow /resgrant/expertise/ -
Disallow /resgrant/rscpa95/ -
Disallow /resgrant/1996resdata/ -
Disallow /resgrant/1996resdata/software/ -
Disallow /resgrant/CADRES/minutes/ -
Disallow /resgrant/CADRES/agenda/ -
Disallow /phdschol/cmmittee/agendas/ -
Disallow /Anatomy/students/ -
Disallow /Anatomy/Notices/ -
Disallow /unisec/pdc/ -
Disallow /unisec/taci/ -
Disallow /unisec/council/ -
Disallow /unisec/protect/ -
Disallow /magpie/ -
Disallow /personnel/handbook/ -
Disallow /personnel/empcond/entbarg/ -
Disallow /personnel/sdu/training/fptemplates/ -
Disallow /.new/ -
Disallow /moved/ -
Disallow /servers/ -
Disallow /vo/ -
Disallow /wwwdev/ -
Disallow /phdschol/apr2004/ -
Disallow /alumni/assets/images/ -
Disallow /research/directory/ -
Disallow /international/muelc/ -
Disallow /international/studyabroad/ -
Disallow /students/studyabroad/ -
Disallow /research/profiles/ -
Disallow /research/people/profiles/profile.html?sid=2519&pid=3226 -
Disallow /research/people/profiles/profile.html?sid=3933097&pid=9467 -
Disallow /study/assets/includes/pdfs/destination-monash.pdf -
Disallow /mcpl/ -
Disallow /timetables-staging -
Disallow /library/sandpit/ -
Disallow /library/sandbox/ -
Disallow /news/internal/ -
Disallow /study/_designs -
Disallow /study/examples -
Disallow /study/_courses -
Disallow /study/search -
Disallow /*/study-archive -
Disallow /study/media/course-feature-blocks -
Disallow /study/courses/majors-minors-specialisations/2017/ -
Disallow /get-started-new/ -
Disallow /news/internal/ -
Disallow /commission-seminar/designing-for-the-future -
Disallow /fees/statement/about/csp -
Disallow /fees/statement/about/full-fee -
Disallow /fees/statement/about/ssaf -
Disallow /_search_web_services/news-listing -
Disallow /education/mentor-report/ -
Disallow /muma/_designs/ -
Disallow *?*query= -
Disallow *?*f.*%7C*= -
Disallow /artdes/ -
Disallow /study/courses/complete-my-profile -
Disallow /confirm-your-details -
Disallow /media/images -
Disallow /monash-scholars/preparing-for-year-11-and-12/maf-form/master-registration-form-preparing-for-year-11-and-12 -
Disallow /monash-scholars/preparing-for-year-11-and-12/maf-form/privacy-statement-preparing-for-year-11-and-12 -
Disallow /monash-scholars/preparing-for-year-11-and-12/maf-form/_thankyoumessage-preparing-for-year-11-and-12 -
Disallow /about/our-locations/peninsula-campus/configuration/_thankyoumessage-peninsula-experience-day -
Disallow /about/our-locations/peninsula-campus/configuration/privacy-statement-peninsula-experience-day -
Disallow /about/our-locations/peninsula-campus/configuration/register-for-updates-peninsula-experience-day -
Disallow /international/our-locations/peninsula-campus/configuration/_thankyoumessage-peninsula-experience-day -
Disallow /international/our-locations/peninsula-campus/configuration/privacy-statement-peninsula-experience-day -
Disallow /international/our-locations/peninsula-campus/configuration/register-for-updates-peninsula-experience-day -
Disallow /lens-publishing-guide -

Other Records

Field Value
crawl-delay 30
crawl-delay 30
Back to top

Other Records

Field Value
sitemap http://www.monash.edu/sitemap.xml
Back to top

Comments

  • www.monash.edu
  • Added for Dey Alexander. Templates no be indexed. RK dec 2003
  • Added for migration access issue 10/9/03 sms
  • Disallow: /library/ # Removed 21/11/2012 by DMa - Google needs to index library's site
  • Disallow: /arts/ # removed on 20/11/19 - after domain change for arts
  • INS555156 HJiang, will be on 1st of Oct
  • INC000001675021
  • Updated 24/09
  • INC000001890005
  • Disallow: /library/search
  • INC000001918907
  • Don't index web server statistics
  • Don't index user disks - they should be accessed as ~username
  • lls site moved to celts server
  • Disallow: /lls/
  • Disallow: /orientation/
  • fixing issue with mrgs progress reports indexing
  • HEAT 450520 - removing /alumni./assets/images/ from Google image search
  • User-agent: Googlebot-Image
  • for Gary Gopinathan REMEDY INC200916 by HM 21 Feb 2012
  • for Rachel Zelada REMEDY INC378928 by DMa 14 Dec 2012
  • added for Derek Brown REMEDY INC400190 - 30 Jan 2013 by HM
  • REMEDY INC513114 - 1 Aug 2013 by DMa
  • REMEDY INC542349 - 11 Nov by DMa
  • REMEDY INC693434 - 15 May 2014 by DMa
  • REMEDY INC842290 - 12/01/2015 by MathewR
  • Squiz Zd 38741
  • INC000002051888
  • Don't index Internal news folder as requested by Internal communications team - 2 June 2017 - done by Shefali Joshi
  • MS-81 - Move Study to monash.edu domain - 19 Aug 2017 by dcook
  • SDVIC-607 - Prevent crawl of old majors
  • added for Fiona McQueen by SMC digital team 09/18
  • INC000001972784 added for Greg McKeown by HDo 14 Sep 2018
  • INC000002074170
  • Disallow MUMA's Design Files
  • SDVIC-4380 Prevent crawling of Funnelback search queries or facets.
  • OTHER SETTINGS
  • INC000002334371 - Disallow MADA's /artdes/ paths
  • added by Simiao Luo from SMC, requested by Harriosn Gist, 11/12/2019
  • added by Wilson for Jenny Legg INC000002497957
  • added by Angelene Wong, 22/10/2020
  • added by Angelene Wong, 27/10/2020
  • added Dennis Gellert, 26/03/2021 INC000002859750
Back to top