engineering privacy engineering
TRANSCRIPT
..
Engineering Privacy Engineering
Dr. Ian OliverSecurity ResearchNokia Networks
16 April 2015
1 © Nokia Solutions and Networks
A confession...
2 © Nokia Solutions and Networks
I am an engineer
3 © Nokia Solutions and Networks
This Talk
An appreciation about how an engineer conceptualises a system engineering problem
or
How do we bring engineers and privacy professionals together?
or
How do we make privacy engineering a discipline?
4 © Nokia Solutions and Networks
Question
You have a data breach - of any kind - whose fault was it?
..1 the privacy officer?
..2 the privacy policy itself?
..3 the engineers for not complying?
5 © Nokia Solutions and Networks
Question
You have a data breach - of any kind - whose fault was it?
..1 the privacy officer?
..2 the privacy policy itself?
..3 the engineers for not complying?
5 © Nokia Solutions and Networks
Question
You have a data breach - of any kind - whose fault was it?
..1 the privacy officer?
..2 the privacy policy itself?
..3 the engineers for not complying?
5 © Nokia Solutions and Networks
Question
You have a data breach - of any kind - whose fault was it?
..1 the privacy officer?
..2 the privacy policy itself?
..3 the engineers for not complying? ← HINT!
6 © Nokia Solutions and Networks
Question - Why?
Was your data breach due to:
..1 A failure in security? (typical RCA result)
..2 Bad programming?
..3 Bad design?
..4 Wrongly worded privacy policy?
..5 Miscommunication?
7 © Nokia Solutions and Networks
Speaking with Engineers
..1 Conversation 1• Privacy Officer: “Does your system collect PII?”• Engineer: “No!”
..2 Conversation 2• Privacy Officer: “Does your system collect GPS coordinates?”• Engineer: “Maybe...”
..3 Conversation 3• Privacy Officer: “What is that struct{float x, float y} thing?”• Engineer: “OK, you got me!”
8 © Nokia Solutions and Networks
Speaking with Engineers
..1 Conversation 1• Privacy Officer: “Does your system collect PII?”• Engineer: “No!”
..2 Conversation 2• Privacy Officer: “Does your system collect GPS coordinates?”• Engineer: “Maybe...”
..3 Conversation 3• Privacy Officer: “What is that struct{float x, float y} thing?”• Engineer: “OK, you got me!”
8 © Nokia Solutions and Networks
Speaking with Engineers
..1 Conversation 1• Privacy Officer: “Does your system collect PII?”• Engineer: “No!”
..2 Conversation 2• Privacy Officer: “Does your system collect GPS coordinates?”• Engineer: “Maybe...”
..3 Conversation 3• Privacy Officer: “What is that struct{float x, float y} thing?”• Engineer: “OK, you got me!”
8 © Nokia Solutions and Networks
Complexity• DNT
• 0 or 1
• Cookies• Context, Usage and Type vs Bytes
• Cloud• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics
• Notice and Consent• Policies
• Legalese vs Mathematics• POLICY ∈ P
{Type× Usage× Purpose× Provenance× . . .
}− ∅
• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .
9 © Nokia Solutions and Networks
Complexity• DNT
• 0 or 1• Cookies
• Context, Usage and Type vs Bytes
• Cloud• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics
• Notice and Consent• Policies
• Legalese vs Mathematics• POLICY ∈ P
{Type× Usage× Purpose× Provenance× . . .
}− ∅
• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .
9 © Nokia Solutions and Networks
Complexity• DNT
• 0 or 1• Cookies
• Context, Usage and Type vs Bytes• Cloud
• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics
• Notice and Consent• Policies
• Legalese vs Mathematics• POLICY ∈ P
{Type× Usage× Purpose× Provenance× . . .
}− ∅
• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .
9 © Nokia Solutions and Networks
Complexity• DNT
• 0 or 1• Cookies
• Context, Usage and Type vs Bytes• Cloud
• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics
• Notice and Consent
• Policies• Legalese vs Mathematics• POLICY ∈ P
{Type× Usage× Purpose× Provenance× . . .
}− ∅
• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .
9 © Nokia Solutions and Networks
Complexity• DNT
• 0 or 1• Cookies
• Context, Usage and Type vs Bytes• Cloud
• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics
• Notice and Consent• Policies
• Legalese vs Mathematics• POLICY ∈ P
{Type× Usage× Purpose× Provenance× . . .
}− ∅
• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .
9 © Nokia Solutions and Networks
Towards a Solution
• Terminology and Semantics• Modelling and Context• Requirements• Risk• Compliance and Auditing
10 © Nokia Solutions and Networks
The Semantics of PII
11 © Nokia Solutions and Networks
Understanding PII/Personal Data
12 © Nokia Solutions and Networks
Terminology• Terminological or Ontological Structures
• Security classification• Information type (versus machine type or PII/personal data)• Provenance, Jurisdiction• Usage, Purpose• Controller, Processor and Data Subject• Identity and Authority, Notice and Consent• Requirements and Risk
• System Architecture, as a data flow• Data Processing and Storage, Environment and Breaches• Data Subject• Data Transformation• Architectural Partitioning and System Boundaries• Boundary Crossings
• Inference Rules and Calculi• ⋄
(x : MachineAddress ⊢ x : Location
)• Policy Calculation, Refinement and Compliance Checking
13 © Nokia Solutions and Networks
Terminology• Terminological or Ontological Structures
• Security classification• Information type (versus machine type or PII/personal data)• Provenance, Jurisdiction• Usage, Purpose• Controller, Processor and Data Subject• Identity and Authority, Notice and Consent• Requirements and Risk
• System Architecture, as a data flow• Data Processing and Storage, Environment and Breaches• Data Subject• Data Transformation• Architectural Partitioning and System Boundaries• Boundary Crossings
• Inference Rules and Calculi• ⋄
(x : MachineAddress ⊢ x : Location
)• Policy Calculation, Refinement and Compliance Checking
13 © Nokia Solutions and Networks
Terminology• Terminological or Ontological Structures
• Security classification• Information type (versus machine type or PII/personal data)• Provenance, Jurisdiction• Usage, Purpose• Controller, Processor and Data Subject• Identity and Authority, Notice and Consent• Requirements and Risk
• System Architecture, as a data flow• Data Processing and Storage, Environment and Breaches• Data Subject• Data Transformation• Architectural Partitioning and System Boundaries• Boundary Crossings
• Inference Rules and Calculi• ⋄
(x : MachineAddress ⊢ x : Location
)• Policy Calculation, Refinement and Compliance Checking
13 © Nokia Solutions and Networks
Modelling
The purpose of any model is to explore and answer questions about a system
or
So you really want to talk to an engineer?
14 © Nokia Solutions and Networks
Modelling
15 © Nokia Solutions and Networks
Notices and Consent
16 © Nokia Solutions and Networks
Requirement Aspect
17 © Nokia Solutions and Networks
Requirement Structure
18 © Nokia Solutions and Networks
Risk
after Solove, Anton-Earp, et al
19 © Nokia Solutions and Networks
Analysing The Model
• Let C be the set of components of a system,. . .
• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .
• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the
whole system• Why?!
• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}
• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk
20 © Nokia Solutions and Networks
Analysing The Model
• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .
• Let S be a system with the signature (C, I),. . .
• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the
whole system• Why?!
• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}
• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk
20 © Nokia Solutions and Networks
Analysing The Model
• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .
• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the
whole system• Why?!
• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}
• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk
20 © Nokia Solutions and Networks
Analysing The Model
• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .
• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the
whole system• Why?!
• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}
• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk
20 © Nokia Solutions and Networks
Analysing The Model
• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .
• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the
whole system• Why?!
• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}
• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk
20 © Nokia Solutions and Networks
Metricisation
• Risk → (Cost× V alue), Cost ∝ V alue
• Requirements→ Risk
•(InformationType× Usage× . . .
)→ Requirements⇔ POLICY
• Policy ⊑ Architecture ⊑ Design ⊑ Code
21 © Nokia Solutions and Networks
Metricisation
• Risk → (Cost× V alue), Cost ∝ V alue
• Requirements→ Risk
•(InformationType× Usage× . . .
)→ Requirements⇔ POLICY
• Policy ⊑ Architecture ⊑ Design ⊑ Code
21 © Nokia Solutions and Networks
Metricisation
• Risk → (Cost× V alue), Cost ∝ V alue
• Requirements→ Risk
•(InformationType× Usage× . . .
)→ Requirements⇔ POLICY
• Policy ⊑ Architecture ⊑ Design ⊑ Code
21 © Nokia Solutions and Networks
Metricisation
• Risk → (Cost× V alue), Cost ∝ V alue
• Requirements→ Risk
•(InformationType× Usage× . . .
)→ Requirements⇔ POLICY
• Policy ⊑ Architecture ⊑ Design ⊑ Code
21 © Nokia Solutions and Networks
Quick Summary
• Terminology is the key - no more talking about PII in isolation
• Infomation Type, Usage, Purpose, Provenance etc
• Model your system
• Formalise requirements - polices are too abstract
• Risk management through metrics derived from the system model
• System model as a data flow becomes the binding and communication medium
22 © Nokia Solutions and Networks
Quick Summary
• Terminology is the key - no more talking about PII in isolation
• Infomation Type, Usage, Purpose, Provenance etc
• Model your system
• Formalise requirements - polices are too abstract
• Risk management through metrics derived from the system model
• System model as a data flow becomes the binding and communication medium
22 © Nokia Solutions and Networks
Quick Summary
• Terminology is the key - no more talking about PII in isolation
• Infomation Type, Usage, Purpose, Provenance etc
• Model your system
• Formalise requirements - polices are too abstract
• Risk management through metrics derived from the system model
• System model as a data flow becomes the binding and communication medium
22 © Nokia Solutions and Networks
Quick Summary
• Terminology is the key - no more talking about PII in isolation
• Infomation Type, Usage, Purpose, Provenance etc
• Model your system
• Formalise requirements - polices are too abstract
• Risk management through metrics derived from the system model
• System model as a data flow becomes the binding and communication medium
22 © Nokia Solutions and Networks
Quick Summary
• Terminology is the key - no more talking about PII in isolation
• Infomation Type, Usage, Purpose, Provenance etc
• Model your system
• Formalise requirements - polices are too abstract
• Risk management through metrics derived from the system model
• System model as a data flow becomes the binding and communication medium
22 © Nokia Solutions and Networks
Quick Summary
• Terminology is the key - no more talking about PII in isolation
• Infomation Type, Usage, Purpose, Provenance etc
• Model your system
• Formalise requirements - polices are too abstract
• Risk management through metrics derived from the system model
• System model as a data flow becomes the binding and communication medium
22 © Nokia Solutions and Networks
Putting it all Together
?
23 © Nokia Solutions and Networks
Putting it all Together
Culture
24 © Nokia Solutions and Networks
Lessons - Privacy as a Safety Critical Aspect
• Aviation• Medicine: Surgery, Anaesthesia• Civil Engineering
25 © Nokia Solutions and Networks
The End
Thankyou
26 © Nokia Solutions and Networks