building systems with integrity
TRANSCRIPT
Paul DowneyTechnical Architect Government Digital Service @psd
GDS@psd
Confidentiality Availability Integrity
Integrity of InformationMost of the public discussion regarding cyber threats has focused on the confidentiality and availability of information; cyber espionage undermines confidentiality, whereas denial-of-service operations and data deletion attacks undermine availability. In the future, however, we might also see more cyber operations that will change or manipulate electronic information in order to compromise its integrity (i.e. accuracy and reliability) instead of deleting it or disrupting access to it. Decision making by senior government officials (civilian and military), corporate executives, investors, or others will be impaired if they cannot trust theinformation they are receiving.
— Worldwide Threat Assessment of the US Intelligence Community, Senate Armed Services Committee, February 26, 2015
GDS@psd
Blood group
GDS@psd
“Integrity, i.e. accuracy and reliability”
GDS@psd
Molly Dishman & Martin Fowler on agile architecture:
GDS@psd
“Architecture is about things which are hard to change”
GDS@psd
“complexity comes from irreversibility”
GDS@psd
“remove things from the system which are hard to change”
It’s quite difficult to unboil an egg
GDS@psd
Integrity: things you want to be difficult to change break
GDS@psd
We need to talk about rotting …
memorandum mori
GDS@psd
Big data Open data Linked data Raw data
GDS@psd
Data proliferation (1977)
GDS@psd
Data ≈ stuff
GDS@psd
Sturgeon’s revelation (law)
GDS@psd
Data warehousing
GDS@psd
Software ≈ stuff you can easily change that does stuff to stuff
GDS@psd
Work out what’s needed
GDS@psd
1. Start with needs 2. Do less 3. Design with data 4. Do the hard work to make it simple 5. Iterate. Then iterate again. 6. This is for everyone 7. Understand context 8. Build things people can build on 9. Be consistent, not uniform 10. Make things open: it makes things better
Design Principles
Start with needs*
* user needs, not government needs
36
GDS@psd
Registers: important lists of things
GDS@psd
Government has a lot of registers:
GDS@psd
companies, charities, trade unions, courts, schools, universities, hospitals, zoos, circuses, inspections, licences, certificates, births, marriages, deaths, electoral roll, insolvencies, bankruptcies, passports, animal passports, drivers, vehicles, land parcels, land ownership, land use, legal boundaries, awards, tax rates, benefits, livestock movements, flood risk, river levels, companies, fish caught, patents, trademarks, designs, non-native invasive plants, bank holidays, clock changes …
GDS@psd
things people need to be able to trust
GDS@psd
and government is the canonical official source
GDS@psd
“An alternative to scurrilous gossip & rumour”
GDS@psd
Integrity: being honest and having strong principles
GDS@psd
The systems we build should reflect our values
GDS@psd
“As Chief Registrar of Foo, I need to know the Foo system of record hasn’t been tampered with”
GDS@psd
IOW: Should be tamper proof
GDS@psd
https://www.gov.uk
Design with Data
Psychic paper
GDS@psd
“As someone with an interest in Foo I need to know a Foo record came from the Foo Registry and it hasn’t been tampered with”
GDS@psd
IOW: provenance is important
A digital signature is …
I am me I agree!&
Simon Wardley http://blog.gardeviance.org/
http://blog.gardeviance.org/2013/03/basics-repeated-again.html
http://blog.gardeviance.org/2013/03/basics-repeated-again.html
GDS@psd
Choosing technology
GDS@psd
The main thing is you must be able to change your mind
GDS@psd
The Web is rotting
GDS@psd
The Web is links
GDS@psd
Link rot
GDS@psd
Url shortenersAbuseTransnational lawBlockingAdvertisingPrivacy and securityAdditional layer of complexity
GDS@psd
Purl — Persistent uniform resource locator
GDS@psd
http://
https://adactio.com/journal/tags/preservation
It must be gov, yeah, yeah!
No link left behind!
Do less
Use the HTTP:
GDS@psd
Stuff rots
Writing law demands a certain level of commitment from goats, calves and sheep
GDS@psd
Horcruxes?
GDS@psd
Backups!
GDS@psd
Physical media
My precious!
GDS@psd
Robot tape libraries
GDS@psd
Computers rot and fail
GDS@psd
Bitrot
GDS@psd
Digital dark ages
GDS@psd
Digital obsolescence
GDS@psd
Digital dark ages
BBC Doomsday project
https://github.com/digital-preservation
GDS@psd
Rotten data
GDS@psd
Cruft?
GDS@psd
People inject entropy
“People stuff up, but if you really want to stuff up you need a computer”
– Anon
GDS@psd
“If you really want to stuff things up, add more people”
– (paraphrasing Fred Brooks)
GDS@psd
P.E.B.C.A.K
GDS@psd
GDS@psd
Can you turn it on and off again?
Autonomy Mastery Purpose
GDS@psd
Learn from your* mistakes
* collective noun
GDS@psd
“Anyone who’s worked with technology at any scale is familiar with failure. Failure cares not about the architecture designs you slave over, the code you write and review, or the alerts and metrics you meticulously pore through.”
— John Allspaw, Blameless Post Mortems and a Just Culture
GDS@psd
Do things that scare you, often
GDS
gdstechnology.blog.gov.uk/2015/02/06/running-a-game-day-for-gov-uk/
GDS@psd
Fight the entropy!
GDS@psd
Ship of Theseus
ISE Shrine– Clay Shirky, Here comes everybody
GDS@psd
Facts don’t rot!
GDS@psd
Architecture
Do the hard work to make it simple
Knocking down the Towers of SIAM
Register appliance
Certificate transparency
Merkle tree magic
redecentralize.org
Build in the context of your domain, organisation The Web
Paul DowneyTechnical Architect Government Digital Service @psd