The Archive, Big Data, and Security

Or

Why do dusty documents really matter?

Tim GollinsHead of Digital Preservation

(The National Archives)&

Honorary Research Fellow (Glasgow University School of Computing Science)

Outline● The Archive is Big Data

● The Archive is about our Security

● Sensitivity Review of Digital Records– Both a Threat and an Opportunity to

The Archive and thus to our Security

The Archive is Big Data● Big data is not new

– Data of a volume that is transformative ● Medieval Times

– The Master of The Rolls● Over 1 Billion sheets of paper (records)● Already over 1.5 Billion web pages● Capacity for over 13 Petabytes of Digital

Records

The Archive is Security● Security rests on the Citizen's Trust in the State● The Archive underpins the fabric of our society● Enables Trust

– Is the impartial witness– Holds the executive to account - the court of history

● Fundamental to The Rule of Law– Underpins many of Lord Bingham’s principles– E.g. “Ministers and pubic officers at all levels must

exercise the powers conferred on them in good faith, fairly, for the purpose for which the powers were conferred, without exceeding the limits of such powers and not unreasonably”

Transfer To The Archive● Complex and Opaque process

– Decisions can appear perverse– Checks and Balances– Involving an “Advisory Council”– The “Lord Chancellor's Blanket” (Blue?)

● Journalists and Eminent Historians are questioning the process

● Conspiracy Theorists Ply Their Trade

Digital Sensitivity Review● Threat

– Volume & Resources – Complexity - Content and Containers– Risk – specifics are now easy to find– Decisions – The Rule of Law– Timing – transition from 20 to 30 years

● Opportunity– Some things are easier – but search can overload– Constancy– Efficiencies possible– Technically Assisted Digital Sensitivity Review

Conclusion - The Right Balance● Freedom of Information not just openness ● Openness & Transparency of process● Calls for Privacy - keep “my” data private● Calls for Openness - what is done in “my” name ● Need for limits

– National Security– Protection of individuals from harm

● Digital Records Makes it Much harder● Clear need for Research

– and the means to conduct it