heads of knowledge and information management and...

Heads of Knowledge and Information Management and Departmental Record Officer Training 2019

25th November 2019

Delivered and supported by a variety of teams across The National ArchivesFacilitator - Katie Kinkead

Welcome to the Learning Room

Objectives

By the end of the training you will be able to:

• Explain how your role and TNA roles and responsibilities

are affected by PRA and Information Rights Legislation.

• Recognise how to further and better collaborate with TNA.

• Understand and clarify TNA’s key functions that affect your role.

Public Records Act 1958

The National Archives

Public Record Bodies

Responsibilities under Public Records Act:

The National Archives

Provide guidance and supervision to publicrecord bodies on the safekeeping and selectionof public records.

Preserve transferred records. Provide facilities for the public to see and

obtain copies of transferred records, unlessthe records are closed or retained because anexemption in the Freedom of Information Actapplies.

Oversee the place of deposit system on behalfof the Secretary of State for Digital, Culture,Media and Sport.

Return records temporarily at the request ofthe transferring organisation.

Care and preserve records.

Responsibilities under Public Records Act:

Public Record Bodies

Selection of records for permanent preservationunder the guidance and supervision of the Keeper of Public Records.

Safe-keeping of records. Transfer of records to The National Archives or

an approved place of deposit by the due date unless they need to be retained, in which case the Secretary of State for Digital, Culture, Media and Sport’s approval must be obtained.

Carry out formal applications for retention and closure through The National Archives which are then reviewed by The Advisory Council on National Archives and Records.

Disposal of records not selected for preservation, by destruction or presentation to another institution.

Department for Digital, Culture, Media and Sport

DCMS – Secretary of State

The National Archives

Jeff James - Chief Executive and Keeper

Lucy Fletcher - Director for Government

John Sheridan - Digital Director

Government Audience

Digital Archiving

Public Record Bodies

How we all fit together…

Cabinet Office (CO)

Department for Digital, Culture, Media and Sport

DCMS – Secretary of State

The National Archives

Public Record Bodies

• Information Management Report (IMR)• Information Management Assessment

(IMA)• Training• Advice and Guidance• Events/User-groups

How we all fit together…

Department for Digital, Culture, Media and Sport

DCMS – Secretary of State

The National Archives

Public Record Bodies

The Advisory Council on National

Records and Archives (ACNRA)

Places of Deposit (POD)

Information Commissioner’s

Office (ICO)

GKIM Professions

What can support you

• Office 365 GKIM user group – governmentaudience@nationalarchives.gov.uk

• Google GKIM user group – governmentaudience@nationalarchives.gov.uk

• ADRO – Jason.King@thecrownestate.co.uk

• KIM Leaders – david.smith@communities.gov.uk

• IRMS – irms.org.uk

• ARA – archives.org.uk

• CILIP – cilip.org.uk

• Civil Service Learning – https://civilservicelearning.civilservice.gov.uk/

• Crown Commercial Service –https://www.crowncommercial.gov.uk/agreements/rm3781

• The National Archives Training and Events –http://www.nationalarchives.gov.uk/information-management/training/

FOI - Responding to historical information requests (Access to Public Records)

Facilitated by Conor Gregan and Christine Peters, FOI Managers, The National Archives

GDPR from the archival perspective

Linda Stewart

Data Protection Officer

A long tradition of making data available

1981,1984, 1998, 2018

Research, history 33.—(1) In this section—

and statistics. “research purposes” includes statistical or historical

purposes; “the relevant conditions”, in relation to any processing of

personal data, means the conditions—

(a) that the data are not processed to support measures or

decisions with respect to particular individuals, and (b) that the data

are not processed in such a way that

substantial damage or substantial distress is, or is likely to be,

caused to any data subject….19Processing for archiving, research and statistical purposes: safeguards

This section has no associated Explanatory Notes

(1)This section makes provision about—

(a)processing of personal data that is necessary for archiving purposes in the public interest,

(b)processing of personal data that is necessary for scientific or historical research purposes, and

(c)processing of personal data that is necessary for statistical purposes.

(2)Such processing does not satisfy the requirement in Article 89(1) of the GDPR for the processing to be subject to appropriate safeguards

for the rights and freedoms of the data subject if it is likely to cause substantial damage or substantial distress to a data subject.

2005: Access under

PRA becomes access

under FOIA = access

via DPA 1998

European

Convention

108 1981

Data Protection Act 1984

Mention Archiving in Privacy Notice

The GDPR has extra and more granular requirements to inform data subjects of the processing undertaken.

Departments are encouraged to mention transfer to the National Archives in their privacy notices, and to provide a link to the National Archives’ website.

Why has this not happened?

John Smith

Note that the name of a person may not in itself beenough to make the person identifiable and it usuallydepends on the context in which it appears or thepresence of supplementary information enabling aperson to be identified.

However if the record is being made available online and will be exposed to search engines, the name will enable linking to other information that could identify the person, and should therefore be redacted.

John Smith’s Agricultural Census Return

Any records (paper or digital) that are indexed byname or other identifier - generally case files,should be closed until DPA no longer applies.

Life expectancyremainsat 100 years…

…or earlier if events intervene

The hundred year ‘rule’

How do you approach sensitivity review?

Closed or redacted

Generally for 100 years

Finding aids, generally open, with caveats:

How do you approach sensitivity review?

Where ?

Who?

Why?

What happens to personal data in your records once they are here?

Archival records can still be used to take decisions about people

Data subject’s information rights

Sharing with other government departments

Digitisation and online access

GDPR,AIPI and BAU

Qualifications -

- Evidence of status

Rights – Subject Access Requests

Open records - we have to supply if we’re given enough information to locate –not just everything about me in the archives

Closed records – no obligation to supply but we usually do; we were set up to provide evidence.

Right to be forgotten

Erasure

Never destroy

Remove from access under our takedown and reclosure policy

Most people just don’t want their name to come up in search engine returns out of context.

If we decide not to remove online content we advise people of their right to make a separate erasure request to the search engines.

Right to Rectification

Corrections

The National Archives cannot alter an archived public record

S 9 of the PRA 1958 – record is what it is

Evidence that a government department had wrong information about them is of value to the data subject.

We will consider annotating the record, leaving the original unchanged - if it is technically possible

Disclosing to police

Can see open archival documents - just like anyone else

Can see who has accessed records if we are satisfied it is necessary

If they wish to see a closed record, we ask them to ask their DRO to contact you. If you’re satisfied, you request the closed record back in the normal way.

s. 29 DPA 1998 is now Sch 2. part 1. para 2.(1) DPA 2018

The Crown is not indivisible

Personal Data Online - The privacy deathstar

Personal Data in the Archives

Engage with government

Transfer to TNA

Online

‘Personal Data’ Online

Records about living people

not publically available online

Known to be

deceased

100 year

‘rule’

Limited access –

academic paywall

Researchers’ DP responsibilities

Researchers are Data Controllers for any personal data they access – we have notices in reading rooms and in our rules this effect.

Although the data they see is open, they still have a duty not to use it in a manner that would cause substantial damage or substantial distress

The National Archives does not give privileged access to closed personal data (permitted under DP but not allowed under FOI)

Recap GDPR mostly BAU for transferring personal data to TNA

It is still transferring dept’s responsibility to locate and reviewpersonal data that could identify living individuals

Data subjects have limited rights: redaction not erasure;annotation not correction

‘Crown Indivisibility’ does not apply to data protection: TNAwill not share details of people who have accessed archivalrecords without statutory authority or a specific court order.

Police must access closed records via transferring department

Availability of personal data impacts likelihood of distress

Researchers responsible as Data Controllers for any use theymake of personal data

Intellectual Property

Lynn Swyny

Copyright Manager

Only one thing is

impossible to God - to

find any sense in any

copyright law on the

planet.

What do you know about…

CopyrightLicensing

Re-use

CopyrightCrown copyright

Non-crown copyright

Third party copyright

DRO responsibilities

Understand your organisation’s copyright situation, including any delegation

Maintain information about 3rd party copyright that you hold

If you have one, make contact with your organisation’s commercial licensing team

Familiarise yourself with available guidance from TNA and IPO

Copyright 2TNA responsibilities

The Keeper manages Crown copyright and database right under Letters Patent from the Queen

Authorises all licensing under Open Government Licence terms

In rare circumstances, can approve assignment of Crown copyright to a non-Crown body, or assignment to the Crown

Will give a Delegation of Authority to a Crown body if it is able to license its own material

Advice and guidance on copyright and licensing

Re-use of Public Sector InformationRe-use of PSI Regulations 2015

Managing Public Money

DRO Responsibilities

Generally, comply with the Regulations

Understand what material you hold that is available for re-use, & how it is being re-used

Have in place a complaints handling process

Understand your department’s public task

If you have one, make contact with your organisation’s commercial licensing team

Re-use of Public Sector Information 2

TNA responsibilities

Lead department for re-use – source of guidance on re-use for public sector bodies

Guidance on defining your public task

Advice from the policy team

Licensing

Open Government Licence – default for Crown bodies, recommended for all UK public sector bodies

Crown bodies can only vary from OGL if they have a delegation

Other government licences are available if you have a delegation or are a non-Crown body

But remember, Government policy remains that wherever possible, public sector information should be made available for re-use under OGL terms; and the Regulations allow relatively little scope for restrictive licensing

Licensing 2DRO responsibilities

If you have one, make contact with your organisation’s commercial licensing team

Ensure that your information is available for re-use under OGL terms wherever possible

TNA responsibilities

Maintaining the UK Government Licensing Framework

Advice and guidance

Any Questions?

Mark Twain image by skeeze from PixabayAll other images © Crown copyright

UK Government Web Archive

Tom Storrar

Head of Web Archiving

Who has heard of web archiving?Who has heard of UK Government Web

Archive?

1996…

…2019

Collecting the government

web estate…

…including socialmedia

The National Archives and Web Archiving

Comprises more than 25,000 crawls of over 4,500 websites (1996-present)

Is approximately 155tb in size, nearly 6 billion resources

Has a powerful full text search tool

Is fully open online and accessed by millions of users.

It is a vehicle for Collection, Preservation and for Access for digital records.

It is also a tool for contextualisation of records past, present and future.

The UK Government Web Archive

What do we capture?>800 distinct websites and social media accounts are regularly archived: those of central govt. public bodies, delivery channels (e.g. GOV.UK) national NHS, public inquiries, some inquests.

On these, we take everything we can on the target website: Publications, datasets, other documents Images Video, animations And all other files that make up the website

…but, web archiving operates within technical constraints. Content must be: Publicly-available Reachable by robots / crawlers

For more information, see our guidance at http://www.nationalarchives.gov.uk/webarchive/guidance/

Finding content in the web archiveSearch (https://webarchive.nationalarchives.gov.uk/search/)

- Search across the full text of UKGWA, with features such as

restrict to website, date range, format

A to Z (http://www.nationalarchives.gov.uk/webarchive/atoz/)

- Most of the websites we have archived are listed here

Know the URL? Use the /*/ index

- e.g. https://webarchive.nationalarchives.gov.uk/*/https://www.gov.uk/

- Shows all the dates at which a resource was captured.

Discovery (https://discovery.nationalarchives.gov.uk/)

- Most websites are catalogued at series level

How do you think we can work together?

Consider the web archive as a vehicle for digital transfer.

Make sure that the content is “crawlable”! Check our guidance and, if in doubt, ask our advice.

Copyright: ensure that the website copyright statement is clear. This supports future reuse and we need to know of any 3rd party content.

Review our Takedown Policy:

nationalarchives.gov.uk/legal/takedown-and-reclosure-policy/

Tell us about any new websites/web platforms/social media.

Leave content available on the web for at least six months and consider archiving timescales – it is not an instantaneous process!

Check capture before removing it from the live web.

How we can work together

webarchive@nationalarchives.gov.uk

Introduction to Transfer Digital Records (TDR)

Kirsten Arnold, Service Owner: Digital Selection & TransferNovember 2019

User engagement

Guidance for ongoing and upcoming transfers

Establishing & delivering service improvements

What we do in Digital Selection & Transfer:

Current Digital Transfer Journey

Formed of a number of steps:

Appraisal & Selection

Digital Sensitivity Review

Transfer(records reach TNA)

Ingest into TNA Systems

Preservation

Access (via Discovery)

Transfer Steps

We currently ask the transferring body to:

1. Complete a digital transfer form

2. Install and utilise three software tools

3. Generate a number of reports and files including a metadata schema which is bespoke to each transferring department

4. Place a copy of their files on an encrypted hard drive and send to TNA

Why do we need to improve on what we are currently offering?

NEED VALUE

Statutory Trust and reputation

Users Easier and simpler transfers

Digital by design Improves efficiency

TDR AGILE APPROACH

https://govdesign.tumblr.com/image/164061029093

TDR GOVERNANCE – BALANCE & CHECKS

BUSINESS OBJECTIVES

USER NEEDS

TDR REQUIREMENTS

Security (Cloud)

Data Protection (GDPR)

Accessibility (WCAG)

GDS Guidance

Audits (BDO)

Agile

However…TDR is not a new concept!

TB1 TB2 TB3 TB4 TB5 TB6 TB7 TB8 TB9 TB10 TB11

Stage Min Max Average TDR

Tools installation (DROID etc.)

Digital Transfer Form

request & completion

Technical checkpoint:

(DROID export)

Metadata for records:

Interim metadata CSV

Metadata for series

(creation/check)

Summary for closure to

Advisory Council

Advisory Council meeting:

(and approval)

Metadata for

series:(validation/confirmation)

Metadata for records:

Final metadata CSV

"Physical" transfer

Subtotals of Stages

Duration (days)

STAGE DURATIONS IN CURRENT PROCESS

STAGE DURATIONS IN CURRENT PROCESSTB1 TB2 TB3 TB4 TB5 TB6 TB7 TB8 TB9 TB10 TB11

Stage Min Max Average TDR

Tools installation (DROID etc.) 337.00 87.00 13.00 67.00 390.00 273.00 404.00 0.50 0.50 0.50 0.50 404.00 157.25 YES

Digital Transfer Form

request & completion 0.50 0.50 0.50 17.00 1.00 0.50 17.00 3.90 YES

Technical checkpoint:

(DROID export) 36.00 0.50 50.00 0.50 0.50 0.50 0.50 21.00 21.00 1.00 0.50 50.00 13.15 YES

Metadata for records:

Interim metadata CSV 27.00 0.50 21.00 59.00 40.00 39.00 310.00 10.00 10.00 4.00 0.50 310.00 52.05 YES

Metadata for series

(creation/check) 43.00 0.50 0.50 41.00 0.50 62.00 0.50 4.00 5.00 5.00 76.00 0.50 62.00 21.64 YES

Summary for closure to

Advisory Council 305.00 0.50 0.50 0.50 52.00 52.00 0.50 37.00 0.50 305.00 56.00 NO

Advisory Council meeting:

(and approval) 50.00 37.00 0.50 68.00 52.00 13.00 63.00 13.00 63.00 33.00 5.00 0.50 68.00 36.14 NO

Metadata for

series:(validation/confirmation) 1.00 2.00 0.50 88.00 0.50 0.50 0.50 0.50 0.50 0.50 88.00 10.44 YES

Metadata for records:

Final metadata CSV 11.00 134.00 0.50 245.00 0.50 7.00 0.50 0.50 0.50 245.00 49.88 YES

"Physical" transfer 7.00 4.00 0.50 0.50 36.00 27.00 57.00 0.50 57.00 18.86 YES

Subtotals of Stages 817.50 266.50 87.00 570.00 589.00 422.00 831.00 50.50 157.50 80.50 81.00 419.30

Duration (days)

DIG

Gateways’ structure

Interaction start point

Transferring bodies and selection and appraisal

BREAKING ASSUMPTIONS

Transfer Digital Records (TDR) is a TNA service for transferring bodies who have selected their born digital records for transfer.

TDR is a cloud based collection, preparation and transfer service that will enable the upload of these records and facilitate a more streamlined transfer, as an improvement to the current manual process.

USER PERSONAS

TDR BRAINKirsten Arnold Product OwnerService Owner (Digital Selection & Transfer)

Laura DamianLead Developer

Ian HoyleSnr Developer

Sam Palmer Snr Developer

Suzanne Hamilton Snr Developer

Tom KingDeveloper

Sarra Hamdi Lead User Researcher

David Clipsham Technical Architect

Pia Lembke UI Designer/Developer

Linda Dodd Delivery Manager (Interim)

Mark Kingsbury Developer

TDR ALPHA STAGE – June – October 2019

Opportunity to try out different solutions

Building prototypes and testing different ideas

Challenge the status quo – explore new approaches

Produced throw away code

Test riskiest assumptions

Focus on the areas which will bring the most challenges

By the end of Alpha we decided which of the ideas we’ve tested we want to take forward as an MVP for the Beta stage

TDR ALPHA SCOPE

To demonstrate how technical design, infrastructure and tools can resolve these key user needs with a working proof-of-concept prototype to represent three of the four defined TDR Agile Epics:

UPLOAD – I as a User can get my content into TDR ‘easily’

PROCESSING –TDR will add data & complete specified checks for me

TRANSFER/EXPORT – My content/data reaches TNA with its integrity intact

AUDIT – data model approach only; out of Alpha scope for prototyping

PAIN POINTS

1.Complete a digital transfer form

Integrated into TDR

2. Install and utilise three software tools

TDR will remove the need for users to install most if not all of these tools including DROID

3. Generate a number of reports and files including a metadata schema which is bespoke to each transferring department

TDR will reduce the need for the exchange of spreadsheets

4. Place a copy of their files on an encrypted hard drive and send to TNA

Users will use a web based browser to upload their transfer consignments

It does not include appraisal, selection, sensitivity review or any other aspects related to ‘pre’ transfer.

Rather it looks to automate operational aspects of the transfer process itself

Remove the need for tool installation and editing of metadata spreadsheets in Excel.

IN SUMMARY

No more tools to download

Cloud-based (AWS)

Automated metadata creation

Automated ‘sanitisation’ –antivirus, file format identification

Interactive, guided ‘fix-up’ to finalise your transfers

TDR DELIVERS …

Your Time

Thank you for attending the training

Explain how your role and TNA roles and responsibilities are affected by PRA and Information Rights Legislation.

Recognise how to further and better collaborate with TNA.

Understand and clarify some of TNA’s key functions that affect your role.

By the end of the training you will be able to:

GovernmentAudience@nationalarchives.gov.ukKatie.Kinkead@nationalarchives.gov.uk

Behind the Scenes Tour