building systems with integrity

Paul DowneyTechnical Architect Government Digital Service @psd

GDS@psd

Confidentiality Availability Integrity

Integrity of InformationMost of the public discussion regarding cyber threats has focused on the confidentiality and availability of information; cyber espionage undermines confidentiality, whereas denial-of-service operations and data deletion attacks undermine availability. In the future, however, we might also see more cyber operations that will change or manipulate electronic information in order to compromise its integrity (i.e. accuracy and reliability) instead of deleting it or disrupting access to it. Decision making by senior government officials (civilian and military), corporate executives, investors, or others will be impaired if they cannot trust theinformation they are receiving.

— Worldwide Threat Assessment of the US Intelligence Community, Senate Armed Services Committee, February 26, 2015

GDS@psd

Blood group

GDS@psd

“Integrity, i.e. accuracy and reliability”

GDS@psd

Molly Dishman & Martin Fowler on agile architecture:

GDS@psd

“Architecture is about things which are hard to change”

GDS@psd

“complexity comes from irreversibility”

GDS@psd

“remove things from the system which are hard to change”

It’s quite difficult to unboil an egg

GDS@psd

Integrity: things you want to be difficult to change break

GDS@psd

We need to talk about rotting …

memorandum mori

GDS@psd

Big data Open data Linked data Raw data

GDS@psd

Data proliferation (1977)

GDS@psd

Data ≈ stuff

GDS@psd

Sturgeon’s revelation (law)

GDS@psd

Data warehousing

GDS@psd

Software ≈ stuff you can easily change that does stuff to stuff

GDS@psd

Work out what’s needed

GDS@psd

1. Start with needs 2. Do less 3. Design with data 4. Do the hard work to make it simple 5. Iterate. Then iterate again. 6. This is for everyone 7. Understand context 8. Build things people can build on 9. Be consistent, not uniform 10. Make things open: it makes things better

Design Principles

Start with needs*

* user needs, not government needs

https://www.gov.uk/service-manual

36

GDS@psd

Registers: important lists of things

GDS@psd

Government has a lot of registers:

GDS@psd

companies, charities, trade unions, courts, schools, universities, hospitals, zoos, circuses, inspections, licences, certificates, births, marriages, deaths, electoral roll, insolvencies, bankruptcies, passports, animal passports, drivers, vehicles, land parcels, land ownership, land use, legal boundaries, awards, tax rates, benefits, livestock movements, flood risk, river levels, companies, fish caught, patents, trademarks, designs, non-native invasive plants, bank holidays, clock changes …

GDS@psd

things people need to be able to trust

GDS@psd

and government is the canonical official source

GDS@psd

“An alternative to scurrilous gossip & rumour”

GDS@psd

Integrity: being honest and having strong principles

GDS@psd

The systems we build should reflect our values

GDS@psd

“As Chief Registrar of Foo, I need to know the Foo system of record hasn’t been tampered with”

GDS@psd

IOW: Should be tamper proof

GDS@psd

https://www.gov.uk

https://www.gov.uk/ after-a-death

https://www.gov.uk/ info/after-a-death

Design with Data

gov.uk/performance

Psychic paper

GDS@psd

“As someone with an interest in Foo I need to know a Foo record came from the Foo Registry and it hasn’t been tampered with”

GDS@psd

IOW: provenance is important

A digital signature is …

I am me I agree!&

Simon Wardley http://blog.gardeviance.org/

http://blog.gardeviance.org/2013/03/basics-repeated-again.html

http://blog.gardeviance.org/2013/03/basics-repeated-again.html

GDS@psd

Choosing technology

GDS@psd

The main thing is you must be able to change your mind

GDS@psd

The Web is rotting

GDS@psd

The Web is links

GDS@psd

Link rot

GDS@psd

Url shortenersAbuseTransnational lawBlockingAdvertisingPrivacy and securityAdditional layer of complexity

GDS@psd

Purl — Persistent uniform resource locator

GDS@psd

http://

ourincrediblejourney.tumblr.com

https://adactio.com/journal/tags/preservation

It must be gov, yeah, yeah!

No link left behind!

Do less

Use the HTTP:

GDS@psd

Stuff rots

Writing law demands a certain level of commitment from goats, calves and sheep

GDS@psd

Horcruxes?

GDS@psd

Backups!

GDS@psd

Physical media

My precious!

GDS@psd

Robot tape libraries

GDS@psd

Computers rot and fail

GDS@psd

Bitrot

GDS@psd

Digital dark ages

GDS@psd

Digital obsolescence

GDS@psd

Digital dark ages

BBC Doomsday project

https://github.com/digital-preservation

GDS@psd

Rotten data

GDS@psd

Cruft?

GDS@psd

People inject entropy

“People stuff up, but if you really want to stuff up you need a computer”

– Anon

GDS@psd

“If you really want to stuff things up, add more people”

– (paraphrasing Fred Brooks)

GDS@psd

P.E.B.C.A.K

GDS@psd

GDS@psd

Can you turn it on and off again?

Autonomy Mastery Purpose

GDS@psd

Learn from your* mistakes

* collective noun

GDS@psd

“Anyone who’s worked with technology at any scale is familiar with failure. Failure cares not about the architecture designs you slave over, the code you write and review, or the alerts and metrics you meticulously pore through.”

— John Allspaw, Blameless Post Mortems and a Just Culture

GDS@psd

Do things that scare you, often

GDS

gdstechnology.blog.gov.uk/2015/02/06/running-a-game-day-for-gov-uk/

GDS@psd

Fight the entropy!

GDS@psd

Ship of Theseus

ISE Shrine– Clay Shirky, Here comes everybody

GDS@psd

Facts don’t rot!

GDS@psd

Architecture

Do the hard work to make it simple

Knocking down the Towers of SIAM

Register appliance

Certificate transparency

Merkle tree magic

redecentralize.org

Build in the context of your domain, organisation The Web

Paul DowneyTechnical Architect Government Digital Service @psd