york.ac.uk
robots.txt

Robots Exclusion Standard data for york.ac.uk

Resource Scan

Scan Details

Site Domain york.ac.uk
Base Domain york.ac.uk
Scan Status Ok
Last Scan2024-06-07T12:44:12+00:00
Next Scan 2024-07-07T12:44:12+00:00

Last Scan

Scanned2024-06-07T12:44:12+00:00
URL https://york.ac.uk/robots.txt
Redirect https://www.york.ac.uk/robots.txt
Redirect Domain www.york.ac.uk
Redirect Base york.ac.uk
Domain IPs 144.32.128.115, 2001:630:61:180::1:73
Redirect IPs 104.18.24.130, 104.18.25.130, 2606:4700::6812:1882, 2606:4700::6812:1982
Response IP 104.18.25.130
Found Yes
Hash 0baba73d8ced1cabbdeeb8358cf7640d4d2722cb57d172ee1918dc5d88005643
SimHash ad7c4b6544e8

Groups

linecker/*

Rule Path
Disallow /cgi-bin-auth/
Disallow /cgi-bin-csrv/
Disallow /test/

*

Rule Path
Disallow /res/thomas/
Disallow /univ/mis/cfm/gradhunt/
Disallow /ServerSupport/LinkChecker/
Disallow /ServerSupport/LinkChecker.TEST/
Disallow /ServerSupport/UsageStats/
Disallow /ServerSupport/UsageStats.TEST/
Disallow /cgi-bin/
Disallow /cgi-bin-auth/
Disallow /cgi-bin-csrv/
Disallow /ftparchive/
Disallow /test/
Disallow /~
Disallow /~
Disallow /~
Disallow /usagestats/
Disallow /admin/aso/
Disallow /admin/sro/
Disallow /admin/uao/ugrad/courses/cupid/?
Disallow /archaeology/intranet/
Disallow /commercialservices/catering_and_bars/
Disallow /newstudent/
Disallow /study-new/
Disallow /study/undergraduate/courses-2017/
Disallow /univ/coll/
Disallow /univ/mis/cfm/
Disallow /univ/mis/fm/
Disallow /univ/mis/fmport/
Disallow /univ/mis/fmstudent/
Disallow /admin/yimshelp/
Disallow /search/
Disallow /preview/
Disallow /phd-adapting-to-the-challenges-of-a-changing-environment
Disallow /phd-applied-linguistics
Disallow /phd-applied-linguistics
Disallow /phd-archaeology
Disallow /phd-archaeology-distance-learning
Disallow /phd-biological-chemistry
Disallow /phd-biology
Disallow /phd-biology-acce
Disallow /phd-biomedical-science
Disallow /phd-chemistry
Disallow /phd-cognitive-neuroscience-and-neuroimaging
Disallow /phd-computer-science-distance-learning
Disallow /phd-conservation-studies
Disallow /phd-economics
Disallow /phd-education
Disallow /phd-education-distance-learning
Disallow /phd-electronic-engineering
Disallow /phd-english
Disallow /phd-english-distance-learning
Disallow /phd-english-with-creative-writing
Disallow /phd-english-with-creative-writing-distance-learning
Disallow /phd-environment-and-politics
Disallow /phd-environment-and-politics
Disallow /phd-environmental-economics-and-environmental-management
Disallow /phd-environmental-geography
Disallow /phd-environmental-science
Disallow /phd-global-development
Disallow /phd-health-economics
Disallow /phd-health-economics
Disallow /phd-health-sciences
Disallow /phd-history
Disallow /phd-history-of-art
Disallow /phd-history-of-art-distance-learning
Disallow /phd-human-geography-and-environment
Disallow /phd-language-and-communication
Disallow /phd-law
Disallow /phd-linguistics
Disallow /phd-management
Disallow /phd-mathematics
Disallow /phd-mathematics-with-foundation-phase
Disallow /phd-mechanistic-biology
Disallow /phd-medieval-studies
Disallow /phd-music
Disallow /phd-music-technology
Disallow /phd-philosophy
Disallow /phd-physics
Disallow /phd-plasma-science-and-fusion-energy
Disallow /phd-politics
Disallow /phd-postwar-recovery-studies
Disallow /phd-ppe
Disallow /phd-psycholinguistics
Disallow /phd-psychology
Disallow /phd-social-policy-and-social-work
Disallow /phd-sociology
Disallow /phd-teaching-english-to-speakers-of-other-language
Disallow /phd-tfti
Disallow /phd-womens-studies
Disallow /phd-computer-science
Disallow /phd-tfti-creative-practice
Disallow /mphil-archaeology
Disallow /mphil-biology
Disallow /mphil-chemistry
Disallow /mphil-cognitive-neuroscience-and-neuroimaging
Disallow /mphil-conservation-studies
Disallow /mphil-economics
Disallow /mphil-electronic-engineering
Disallow /mphil-english
Disallow /mphil-environment-and-politics
Disallow /mphil-environmental-economics-and-environmental-management
Disallow /mphil-environmental-geography
Disallow /mphil-environmental-science
Disallow /mphil-health-sciences
Disallow /mphil-history
Disallow /mphil-history-of-art
Disallow /mphil-human-geography-and-environment
Disallow /mphil-language-and-communication
Disallow /mphil-law
Disallow /mphil-mathematics
Disallow /mphil-music
Disallow /mphil-music-technology
Disallow /mphil-philosophy
Disallow /mphil-ppe
Disallow /mphil-psychology
Disallow /mphil-social-policy-and-social-work
Disallow /mphil-sociology
Disallow /mphil-womens-studies
Disallow /mres-social-policy
Disallow /mres-social-work
Disallow /ma-analytic-theology
Disallow /ma-applied-human-rights
Disallow /ma-applied-linguistics-english-teaching
Disallow /ma-applied-linguistics-language-teaching
Disallow /ma-archaeological-studies-by-research
Disallow /ma-community-music
Disallow /ma-comparative-international-social-policy
Disallow /ma-comparative-international-social-work
Disallow /ma-comparative-syntax-semantics
Disallow /ma-conflict-governance-development
Disallow /ma-conservation-studies
Disallow /ma-conservation-studies-by-research
Disallow /ma-conservation-studies-historic-buildings
Disallow /ma-contemporary-history-international-politics
Disallow /ma-criminology-social-research
Disallow /ma-cultural-heritage-management
Disallow /ma-culture-society-globalisation
Disallow /ma-culture-thought
Disallow /ma-early-modern-history
Disallow /ma-early-prehistory-human-origins
Disallow /ma-education
Disallow /ma-educational-practice
Disallow /ma-eighteenth-century-studies
Disallow /ma-english-building-history
Disallow /ma-english-by-research
Disallow /ma-english-literary-studies
Disallow /ma-field-archaeology
Disallow /ma-film-literature
Disallow /ma-film-television-production-cinematography
Disallow /ma-film-television-production-directing
Disallow /ma-film-television-production-producing
Disallow /ma-film-television-production-sound
Disallow /ma-film-television-production-visual-effects
Disallow /ma-filmmaking-by-research
Disallow /ma-funerary-archaeology
Disallow /ma-global-crime-justice
Disallow /ma-global-international-citizenship-education
Disallow /ma-global-literature-culture
Disallow /ma-global-social-policy
Disallow /ma-historic-buildings
Disallow /ma-historical-archaeology
Disallow /ma-history-by-research
Disallow /ma-history-of-art
Disallow /ma-history-of-art-architectural
Disallow /ma-history-of-art-british-art
Disallow /ma-history-of-art-by-research
Disallow /ma-history-of-art-medieval
Disallow /ma-history-of-art-modern-contemporary
Disallow /ma-interactive-media-by-research
Disallow /ma-international-political-economy
Disallow /ma-international-relations
Disallow /ma-iron-age-celtic-archaeology
Disallow /ma-law-by-research
Disallow /ma-linguistics
Disallow /ma-linguistics-by-research
Disallow /ma-literature-romantic-period
Disallow /ma-management-creative-cultural-industries
Disallow /ma-material-culture-experimental-archaeology
Disallow /ma-medical-history-humanities
Disallow /ma-medieval-archaeology
Disallow /ma-medieval-history
Disallow /ma-medieval-literatures-languages
Disallow /ma-medieval-studies
Disallow /ma-mesolithic-archaeology
Disallow /ma-modern-contemporary-literature-culture
Disallow /ma-modern-history
Disallow /ma-music-by-research
Disallow /ma-music-education
Disallow /ma-music-english-church-music
Disallow /ma-music-music-psychology
Disallow /ma-music-musicology
Disallow /ma-music-performance-practice
Disallow /ma-music-piano-studies
Disallow /ma-music-production
Disallow /ma-music-solo-voice-ensemble-singing
Disallow /ma-music-technology-by-research
Disallow /ma-music-vocal-studies
Disallow /ma-peace-and-conflict-studies
Disallow /ma-philosophy
Disallow /ma-philosophy-artificial-intelligence
Disallow /ma-philosophy-by-research
Disallow /ma-phonetics-phonology
Disallow /ma-playwriting
Disallow /ma-poetry-poetics
Disallow /ma-political-legal-philosophy
Disallow /ma-political-theory
Disallow /ma-psycholinguistics
Disallow /ma-public-administration-policy
Disallow /ma-public-history
Disallow /ma-railway-studies
Disallow /ma-renaissance-and-early-modern-studies
Disallow /ma-renaissance-literature
Disallow /ma-roman-archaeology
Disallow /ma-screenwriting-by-research
Disallow /ma-social-justice-education
Disallow /ma-social-media-social-research
Disallow /ma-social-policy
Disallow /ma-social-research
Disallow /ma-social-work
Disallow /ma-sociolinguistics
Disallow /ma-sociology-by-research
Disallow /ma-stained-glass-conservation-heritage-management
Disallow /ma-sustainability-studies
Disallow /ma-tesol
Disallow /ma-theatre-by-research
Disallow /ma-theatre-making
Disallow /ma-victorian-literature-culture
Disallow /ma-women-violence-and-conflict
Disallow /ma-womens-and-gender-studies-gemma
Disallow /ma-womens-studies
Disallow /ma-womens-studies-by-research
Disallow /ma-womens-studies-humanities
Disallow /msc-green-chemistry
Disallow /msc-accounting-financial-management
Disallow /msc-advanced-clinical-practice
Disallow /msc-advanced-computer-science
Disallow /msc-audio-music-technology
Disallow /msc-bioarchaeology
Disallow /msc-biology-by-research
Disallow /msc-biomedical-sciences
Disallow /msc-biomedical-sciences-by-research
Disallow /msc-biotechnology
Disallow /msc-chemistry-by-research
Disallow /msc-communications-engineering
Disallow /msc-corporate-sustainability-environment
Disallow /msc-cyber-security
Disallow /msc-development-economics-emerging-markets
Disallow /msc-digital-archaeology
Disallow /msc-digital-heritage
Disallow /msc-digital-systems-engineering
Disallow /msc-early-prehistory-human-origins
Disallow /msc-econometrics-economics
Disallow /msc-economic-evaluation-health-technology
Disallow /msc-economics
Disallow /msc-economics-finance
Disallow /msc-economics-public-policy
Disallow /msc-electronic-engineering-by-research
Disallow /msc-embedded-wireless-systems
Disallow /msc-engineering-management
Disallow /msc-environment-by-research
Disallow /msc-environmental-economics-management
Disallow /msc-environmental-science-management
Disallow /msc-film-television-production-sound
Disallow /msc-film-television-production-visual-effects
Disallow /msc-finance
Disallow /msc-finance-econometrics
Disallow /msc-financial-engineering
Disallow /msc-financial-engineering
Disallow /msc-funerary-archaeology
Disallow /msc-fusion-energy
Disallow /msc-global-marketing
Disallow /msc-green-chemistry-sustainable-industrial-tech
Disallow /msc-health-economics
Disallow /msc-health-research
Disallow /msc-health-sciences
Disallow /msc-human-centred-interactive-technologies
Disallow /msc-human-resource-management
Disallow /msc-intelligent-robotics
Disallow /msc-interactive-media-by-research
Disallow /msc-international-business
Disallow /msc-international-business-strategic-management
Disallow /msc-international-humanitarian-affairs
Disallow /msc-international-strategic-management
Disallow /msc-management
Disallow /msc-management-business-finance
Disallow /msc-material-culture-experimental-archaeology
Disallow /msc-mathematical-finance
Disallow /msc-mathematical-finance-online
Disallow /msc-mathematical-sciences
Disallow /msc-mathematics-by-research
Disallow /msc-mental-health-research
Disallow /msc-mental-health-wellbeing-education
Disallow /msc-music-technology-by-research
Disallow /msc-nursing-adult
Disallow /msc-nursing-mental-health
Disallow /msc-physics-by-research
Disallow /msc-project-analysis-finance-investment
Disallow /msc-psychology-education-conversion-programme
Disallow /msc-safety-critical-systems-engineering
Disallow /msc-social-media-and-management
Disallow /msc-social-media-interactive-technology
Disallow /msc-statistics-and-computational-finance
Disallow /msc-sustainability-science
Disallow /msc-sustainable-business
Disallow /msc-system-safety-engineering-with-automotive
Disallow /msc-zooarchaeology
Disallow /msc-forensic-speech-science
Disallow /msc-computer-science-by-research
Disallow /mpa-applied-social-public-policy-evaluation
Disallow /mpa-international-development
Disallow /mpa-public-administration
Disallow /llm-art-law
Disallow /llm-international-corporate-commercial-law
Disallow /llm-international-human-rights-law-practice
Disallow /llm-law
Disallow /llm-law-juris-doctor
Disallow /llm-legal-political-theory
Disallow /mph-public-health
Disallow /pgce-biology
Disallow /pgce-chemistry
Disallow /pgce-english
Disallow /pgce-geography
Disallow /pgce-history
Disallow /pgce-mathematics
Disallow /pgce-modern-languages-french
Disallow /pgce-modern-languages-german
Disallow /pgce-modern-languages-spanish
Disallow /pgce-physics
Disallow /pgcert-general-practice-in-nursing
Disallow /pgcert-health-economics-health-care-professionals
Disallow /pgcert-health-research-and-statistics
Disallow /pgcert-system-safety-engineering
Disallow /pgdip-advanced-clinical-practice
Disallow /pgdip-astronomy
Disallow /pgdip-educational-practice
Disallow /pgdip-health-economics-health-care-professionals
Disallow /pgdip-safety-critical-systems-engineering
Disallow /services/careers/app/profiles2/?sector=
Disallow /services/careers/app/profiles2/?department=
Disallow /services/careers/app/profiles2/?qualification=
Disallow /services/careers/app/profiles2/?profile=
Disallow /services/careers/app/profiles2/messages/
Disallow /services/careers/app/profiles2/?page=
Disallow /services/careers/app/profiles2/?sort_type=

siteimprovebot-crawler

Rule Path
Allow /

Comments

  • robots.txt for http://www.york.ac.uk/
  • see <http://info.webcrawler.com/mak/projects/robots/norobots.html> for info.
  • Disallow shortcut URLs from PG prospectus
  • Disallow paginated sub-pages of profiles and mentors app