Log In Sign Up

nationalarchives.gov.uk
robots.txt

Robots Exclusion Standard data for nationalarchives.gov.uk

Resource Scan

Scan Details

Site Domain nationalarchives.gov.uk
Base Domain nationalarchives.gov.uk
Scan Status Ok
Last Scan2024-11-02T16:58:33+00:00
Next Scan 2024-11-16T16:58:33+00:00

Last Scan

Scanned2024-11-02T16:58:33+00:00
URL https://nationalarchives.gov.uk/robots.txt
Redirect https://www.nationalarchives.gov.uk/robots.txt
Redirect Domain www.nationalarchives.gov.uk
Redirect Base nationalarchives.gov.uk
Domain IPs 137.221.134.194
Redirect IPs 54.230.112.20, 54.230.112.44, 54.230.112.74, 54.230.112.87
Response IP 3.164.240.108
Found Yes
Hash a08fac1b3135f9ad805d6b33ba4ed886108b8062b9c5e9c3a17960119aaf3c52
SimHash f3d81b7ecfe8

Groups

googlebot

Rule Path
Disallow /help-with-your-research/research-guides/birth-marriage-death-england-and-wales
Disallow /help-with-your-research/research-guides/birth-marriage-and-death-certificates
Disallow /help-with-your-research/research-guides/adoptions
Disallow /help-with-your-research/research-guides/electoral-registration
Disallow /help-with-your-research/research-guides/living-people
Disallow /help-with-your-research/research-guides/historical-nhs-patient-records/

bingbot

Rule Path
Disallow /help-with-your-research/research-guides/birth-marriage-death-england-and-wales
Disallow /help-with-your-research/research-guides/birth-marriage-and-death-certificates
Disallow /help-with-your-research/research-guides/adoptions
Disallow /help-with-your-research/research-guides/electoral-registration
Disallow /help-with-your-research/research-guides/living-people
Disallow /help-with-your-research/research-guides/historical-nhs-patient-records/

slurp

Rule Path
Disallow /help-with-your-research/research-guides/birth-marriage-death-england-and-wales
Disallow /help-with-your-research/research-guides/birth-marriage-and-death-certificates
Disallow /help-with-your-research/research-guides/adoptions
Disallow /help-with-your-research/research-guides/electoral-registration
Disallow /help-with-your-research/research-guides/living-people
Disallow /help-with-your-research/research-guides/historical-nhs-patient-records/

duckduckbot

Rule Path
Disallow /help-with-your-research/research-guides/birth-marriage-death-england-and-wales
Disallow /help-with-your-research/research-guides/birth-marriage-and-death-certificates
Disallow /help-with-your-research/research-guides/adoptions
Disallow /help-with-your-research/research-guides/electoral-registration
Disallow /help-with-your-research/research-guides/living-people
Disallow /help-with-your-research/research-guides/historical-nhs-patient-records/

*

Product Comment
* applies to all robots
Rule Path
Disallow /_404-tests
Disallow /1939-data-room
Disallow /08-10-13scripts
Disallow /08-10-13styles
Disallow /abc123
Disallow /about/record-transfer-spring-2015
Disallow /academic
Disallow /accessions/accprograms/
Disallow /advanceorders
Disallow /advanceorders/includes/
Disallow /advanceorders/js/
Disallow /amp-blog.htm
Disallow /archives/
Disallow /beta/
Disallow /book-a-reading-room-visit/
Disallow /bookshop/account.aspx
Disallow /bookshop/App_Code/
Disallow /bookshop/basket.aspx
Disallow /bookshop/Bin/
Disallow /bookshop/Controls/
Disallow /bookshop/coupn.aspx
Disallow /bookshop/digital_receipt.aspx
Disallow /bookshop/ordercomplete.aspx
Disallow /bookshop/process_order.aspx
Disallow /bookshop/receiptfailure.aspx
Disallow /bookshop/searchresult.aspx
Disallow /bookshop/titleshow.asp
Disallow /bookshop/xml/
Disallow /Bridging
Disallow /business
Disallow /cab/
Disallow /cabinetpaperssearch
Disallow /catalogue
Disallow /celebratingcensus
Disallow /census
Disallow /chat
Disallow /clickandbuy
Disallow /conferences/
Disallow /conferencesclosed
Disallow /contact/form/
Disallow /contact-us/*/form
Disallow /cover-it-live/
Disallow /currency/
Disallow /customerrors/
Disallow /dc-framework/
Disallow /dc-guidance/
Disallow /dc-riskassessment/
Disallow /dc-service/
Disallow /dc-training/
Disallow /design-guide/
Disallow /digexpress
Disallow /digitalcontinuity
Disallow /digitalexpress/
Disallow /discovery
Disallow /document_handling
Disallow /documentsonline
Disallow /documentphotos/
Disallow /documents/archives/ArchivesObsolete
Disallow /documents/information-management/imr-guidance-and-appendices.pdf
Disallow /documents/mog.pdf
Disallow /documents/old/
Disallow /dol
Disallow /droid/
Disallow /e179/includes/
Disallow /e179/scripts/
Disallow /ecards
Disallow /editorial/
Disallow /educationservice
Disallow /edwardii-conference
Disallow /emergency.htm
Disallow /empire-at-war
Disallow /enewsletter
Disallow /enewsletter/global/
Disallow /enewsletter/includes/
Disallow /equity
Disallow /equity/includes/
Disallow /equity/scripts/
Disallow /ero
Disallow /ERORecords
Disallow /eventsbooking
Disallow /eventsbooking/email_templates/
Disallow /eventsbooking/includes/
Disallow /eventsbooking/scripts/
Disallow /failover
Disallow /Filestore/
Disallow /foi/pubscheme/
Disallow /gazettestender
Disallow /gettingstarted
Disallow /gcs
Disallow /gwdb
Disallow /ia/
Disallow /hospitalrecords/includes/
Disallow /househistory
Disallow /ichora5
Allow /images/news/
Allow /images/news/112/
Allow /images/about/
Allow /images/home/
Allow /images/visit/
Disallow /ImageViewer/
Disallow /information-principles
Disallow /information-management/manage-information/places-of-deposit/
Disallow /includes/
Disallow /irlist/includes/
Disallow /labs-maintenance.htm
Disallow /labs-unavailable.htm
Disallow /legion
Disallow /livingin1901
Disallow /livingthepoorlife
Disallow /localhistory
Disallow /media
Disallow /militaryhistory
Disallow /museum
Disallow /museum/includes/
Disallow /mypage
Disallow /news/999-snow.htm
Disallow /news/372.htm
Disallow /online/includes
Disallow /pas-198
Disallow /podcasts
Disallow /poor_law
Disallow /portal
Disallow /preservation
Disallow /presspreview/
Disallow /pronom
Disallow /pronom/product/
Disallow /pronom/vendor/
Disallow /recordsmanagement
Disallow /records/old/
Disallow /registration
Disallow /registration/includes/
Disallow /registration/js/
Disallow /religiousarchives
Disallow /rt2014v2/
Disallow /search/
Disallow /search.htm
Disallow /search/name_search.aspx
Disallow /searchthearchives
Disallow /sepia/images
Disallow /SFACS
Disallow /shop/record-copying-beta.htm
Disallow /shop/record-copying-faqs-beta.htm
Disallow /spanish-civil-war
Disallow /streamline
Disallow /TNACRL
Disallow /towton
Disallow /transfer
Disallow /video
Disallow /webarchive/orphans/
Disallow /wdytya
Disallow /xml
Disallow /yourarchives
Disallow /zzz/
Back to top

Other Records

Field Value
sitemap http://www.nationalarchives.gov.uk/sitemaps/sitemap-index.xml
Back to top

Comments

  • Tells Scanning Robots Where They Are And Are Not Welcome
Back to top