Log In Sign Up

kcl.ac.uk
robots.txt

Robots Exclusion Standard data for kcl.ac.uk

Resource Scan

Scan Details

Site Domain kcl.ac.uk
Base Domain kcl.ac.uk
Scan Status Ok
Last Scan2024-06-30T12:05:11+00:00
Next Scan 2024-07-30T12:05:11+00:00

Last Scan

Scanned2024-06-30T12:05:11+00:00
URL https://kcl.ac.uk/robots.txt
Redirect https://www.kcl.ac.uk/robots.txt
Redirect Domain www.kcl.ac.uk
Redirect Base kcl.ac.uk
Domain IPs 137.73.130.135
Redirect IPs 185.18.139.104
Response IP 185.18.139.104
Found Yes
Hash 80320e6d7837806a1c8464b414fd8d491e0e0d1a883445c73683197dae1d4eb6
SimHash ee4bd49401aa

Groups

*

Rule Path Comment
Disallow *?ContensisTextOnly=true* -
Disallow /search/ Stop Google indexing our search
Disallow /business/people/*-* -
Disallow /monitoring/ -
Disallow /home -
Disallow /it/owa.aspx -
Disallow /onlinecourses/ -
Disallow /health/research/specialistcentres/cib/people/bbsrcfellows/flatterssarah* -
Disallow /archive/events/* -
Disallow /online/postgraduate-loans* -
Disallow /form-containers* -
Disallow /study-at-kings/sitemap -
Disallow /study/modules* -
Disallow /study-at-kings/ -
Disallow /study-at-kings/landing/your-step-by-step-guide-to-kings -
Disallow /archive/news/kings/newsrecords/2011/11november/indiainstituteavanthachair -
Disallow /*ContensisTextOnly%3D* -
Disallow /*search_keywords%3D* -
Disallow /study/undergraduate/events/on-demand/watch -
Disallow /study/postgraduate-taught/events/on-demand/watch -
Disallow /sport-and-wellness/ -
Disallow /study/postgraduate-taught/prospectus-thankyou -
Disallow /study/undergraduate/prospectus-thankyou -
Disallow /offer-holders -
Disallow *page%3D* -
Disallow *categories%3D* -
Disallow *s%3D* -
Disallow *cat%3D* -
Disallow *term%3D* -
Disallow *researchCategories%3D* -
Disallow *utm_* -
Disallow *fbclid%3D* -
Disallow *platform%3D* -
Disallow *letter%3D* -
Disallow *unibuddy%3D* -
Disallow *alp_source%3D* -
Disallow *keyword%3D* -
Disallow *pageIndex%3D* -
Disallow *li_fat_id%3D* -
Disallow /campuslife/acservices/accessibilitytestcompleted.aspx -
Disallow /accessibilitytest -
Disallow /research/accessibility-test-for-compliance-with-wcag-2.2-aa-standards -
Disallow /events/test-accessibility-for-compliance-with-wcag-2.2-standards -
Disallow /study/accessibility-test-for-wcag-2.2-aa-compliance-standards -
Disallow /jobs/role/accessibility-test-for-wcag-2.2-standards-compliance -

siteimprovebot-crawler

Rule Path
Allow /

fast-webcrawler/0.2

Rule Path
Disallow /

yanga worldsearch bot v1.1/beta

Rule Path
Disallow /

slurp

Rule Path
Disallow /wrap/pmwiki/

twiceler-0.9

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.kcl.ac.uk/sitemap.xml
sitemap https://www.kcl.ac.uk/sitemap1.xml
sitemap https://www.kcl.ac.uk/sitemap2.xml
sitemap https://www.kcl.ac.uk/sitemap3.xml
sitemap https://www.kcl.ac.uk/sitemap4.xml
sitemap https://www.kcl.ac.uk/sitemap5.xml
sitemap https://www.kcl.ac.uk/sitemap6.xml
sitemap https://www.kcl.ac.uk/sitemap7.xml
Back to top

Comments

  • Disallow: /offer-holders/
  • IN01723209 - hide the thank you pages
  • User-agent: Baiduspider
  • Disallow: /
Back to top