deepsec 2014 - the measured cso
TRANSCRIPT
![Page 1: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/1.jpg)
THE MEASURED CSOALEX HUTTON -‐ A TOO BIG TO FAIL BANK
@ALEXHUTTON
![Page 2: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/2.jpg)
SECTION 1: BACKGROUND Who am I? What is this topic? Where are we? How did we get here?
SECTION 2: ON THE ROLE OF THE CSO What is a CSO? What do they do? What is success? How do they get there?
SECTION 3: BECOMING MEASURED What does that mean? What do we need? How do we do it? Where shall we go?
![Page 3: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/3.jpg)
SECTION 1: BACKGROUND Who am I? What is this topic? Where are we? How did we get here?
SECTION 2: ON THE ROLE OF THE CSO What is a CSO? What do they do? What is success? How do they get there?
SECTION 3: BECOMING MEASURED What does that mean? What do we need? How do we do it? Where shall we go?
![Page 4: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/4.jpg)
SECTION 1: BACKGROUND Who am I? What is this topic? Where are we? How did we get here?
SECTION 2: ON THE ROLE OF THE CSO What is a CSO? What do they do? What is success? How do they get there?
SECTION 3: BECOMING MEASURED What does that mean? What do we need? How do we do it? Where shall we go?
![Page 5: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/5.jpg)
SECTION 1: BACKGROUND Who am I? What is this topic? Where are we? How did we get here?
![Page 6: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/6.jpg)
1.1 WHO AM I
![Page 7: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/7.jpg)
• Security Engineer
• Security Product Management
• E-Commerce Site Design / Manager
• Risk Consultant
• OCTAVE / NIST
• FAIR
• Verizon DBIR
• IANS Faculty
• Director, Operations / Technology Risk
• Director, Information Security
1.1 WHO AM I
![Page 8: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/8.jpg)
1.2 WHAT IS THIS TOPIC
![Page 9: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/9.jpg)
“…when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”
William Thomson, 1st Baron Kelvin & Measurement Badass
![Page 10: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/10.jpg)
The Journey Towards Knowledge (and therefore, security)
1.2 WHAT IS THIS TOPIC
![Page 11: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/11.jpg)
WHERE ARE WE (OUR INDUSTRY)
![Page 12: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/12.jpg)
Security is now so essential a concern that
we can no longer use adjectives and adverbs
but must instead use numbers.
Dan Geer, Security Badass
![Page 13: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/13.jpg)
Unfortunately…
![Page 14: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/14.jpg)
Science is based on inductive observations to derive meaning and understanding and measurement on quality (ratio) scales, so what about InfoSec?
Where do we sit in the family of sciences?
![Page 15: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/15.jpg)
We’re the Crazy Uncle with tinfoil hat antennae used to talk to the space aliens of Regulus V, has 47 cats, and who too frequently (but benignly) forgets to wear pants.
![Page 16: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/16.jpg)
Take, for example, CVSS
![Page 17: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/17.jpg)
“the Base Equation multiplies Impact by 0.6 and Exploitability by 0.4”
![Page 18: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/18.jpg)
= ShinyJet Engine X Peanut Butter
![Page 19: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/19.jpg)
“the Base Equation multiplies Impact by 0.6 and Exploitability by 0.4”
![Page 20: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/20.jpg)
20
adding onewilly-nilly doesn’t suddenly transformordinal rankings into ratio values.
decimals aren’t magic.
![Page 21: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/21.jpg)
At our present skill in measurement of security, we
generally have an ordinal scale at best, not an interval scale
and certainly not a ratio scale. In plain terms, this means we
can say whether X is better than Y but how much better and
compared to what is not so easy.
– Again, Baddss Dan Geer
![Page 22: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/22.jpg)
State of the Industry- proto-science - somewhat random fact
gathering (mainly of readily accessible data)
- a“morass”of interesting, trivial, irrelevant observations
- a variety of theories (that are spawned from what he calls philosophical speculation) that provide little guidance to data gatheringThomas Kuhn Philosophy of Science Badass
![Page 23: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/23.jpg)
1.3 HOW DID WE GET HERE
![Page 24: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/24.jpg)
1.3 HOW DID WE GET HERE
The tragedy of two mistakes
![Page 25: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/25.jpg)
FIRST MISTAKE: LIMITING OURSELVES(security is an engineering issue?)
![Page 26: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/26.jpg)
• OSI Model (original version)
![Page 27: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/27.jpg)
• OSI Model (SOA Remix)
![Page 28: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/28.jpg)
• OSI Model (Mika’s 12” Extended Dance Version)
10: Religion Operator Layer
![Page 29: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/29.jpg)
SECOND MISTAKE: BLIND LEADING THE BLIND
![Page 30: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/30.jpg)
BLIND MAN 1: THE FUD FACTORY
![Page 31: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/31.jpg)
FUD FACTORY EXAMPLE - MOBILE VS WEB
![Page 32: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/32.jpg)
Google Trend: Web Security Mobile Malware
![Page 33: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/33.jpg)
#RSAC
36
Clustering of over 5,000 incidents
Espionage
Point of Sale
Skimming Devices
Theft/Loss
Error
EmployeeMisuse
Web Applications
DBIR Top Patterns:
![Page 34: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/34.jpg)
Web Only:
Web Applications
![Page 35: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/35.jpg)
In FinServ vs. All Industries
![Page 36: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/36.jpg)
DBIR Global Representation of Assets in Cases:
![Page 37: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/37.jpg)
DBIR Global Representation of Assets in Cases:
NHTCU investigation into groups using mobile malware showed that in less than a year’s time, five variations of mobile malware for one specific bank could be detected. Modest estimates suggest that criminals gained around €50,000 per week using this specific form of mobile malware, harvesting over 4,000 user credentials from 8,500 infected bank customers in just a few months. Mobile malware does not move the needle in our stats as we focus on organizational security incidents as opposed to consumer device compromises.
![Page 38: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/38.jpg)
DBIR Global Representation of Assets in Cases:
NHTCU investigation into groups using mobile malware showed that in less than a year’s time, five variations of mobile malware for one specific bank could be detected. Modest estimates suggest that criminals gained around €50,000 per week using this specific form of mobile malware, harvesting over 4,000 user credentials from 8,500 infected bank customers in just a few months. Mobile malware does not move the needle in our stats as we focus on organizational security incidents as opposed to consumer device compromises.
![Page 39: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/39.jpg)
BLIND MAN 2: THE ACCOUNTING-CONSULTANCY INDUSTRIAL COMPLEX
![Page 40: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/40.jpg)
![Page 41: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/41.jpg)
![Page 42: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/42.jpg)
Complex (adaptive)Systemsa system composed of interconnected parts that as a whole exhibit one or more properties not obvious from the properties of the individual parts
![Page 43: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/43.jpg)
These “risk” statements you’re making...
I don’t think you’re doing it right.
- (Chillin’ Friederich Hayek)
![Page 44: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/44.jpg)
BLIND MAN 3: OUR BROKEN MODELS
![Page 45: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/45.jpg)
“the Base Equation multiplies Impact by 0.6 and Exploitability by 0.4”
![Page 46: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/46.jpg)
ROYTMAN: ON VULNERABILITIES
![Page 47: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/47.jpg)
ROYTMAN: ON VULNERABILITIES
![Page 48: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/48.jpg)
A CSO MUST BECOME “MEASURED” TO ESCAPE THE MISTAKES OF THE PAST AND PUSH INTO THE FUTURE
![Page 49: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/49.jpg)
SECTION 2: ON THE ROLE OF THE CSO What is a CSO? What do they do? What is success? How do they get there?
![Page 50: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/50.jpg)
• What Is a CISO (throne of blood image
WHAT IS A CSO
![Page 51: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/51.jpg)
• What Is a CISO (throne of blood image
WHAT IS A MEASURED CSO
![Page 52: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/52.jpg)
![Page 53: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/53.jpg)
![Page 54: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/54.jpg)
W.E. DEMING
Father of Total Quality Management and inspiration that drove the Japanese “post-war economic miracle.”
![Page 55: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/55.jpg)
IT WAS NO MIRACLE. What Deming taught the Japanese was “management by fact.”
![Page 56: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/56.jpg)
• Improvements to the system are never ending.
• The only people who really know where the real potentials for improvement are the workers.
• The system is always changing.
• There are countless ways for the system to go wrong.
• Statistics (metrics) are used to focus the conversation on fact and improvement
• Goals for quality are cross-silo
• Theories for improvements are implemented and tested.
• The management uses the workers as essential "instruments" in understanding what is.
![Page 57: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/57.jpg)
A MEASURED CSO:
• Relies on metrics, data, intel for good decisions,
• Invests in improvements to People, Process and Technology,
• Puts innovation for improvements to the system (improvements = security, cost) in the hands of the operator,
• Ensures that there is a feedback loop for effectiveness initiatives, and
• Works tirelessly within the bureaucracy to improve all aspects of the system.
![Page 58: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/58.jpg)
THE MEASURED CSO’S MISSION:
• To provide the best and least-cost security for shareholders, and continuity of employment for his workers.
• We, as an industry, know that “best” and”least-cost” are not necessarily contradictory
• We also have a HUGE continuity issue
![Page 59: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/59.jpg)
THE MEASURED CSO USES METRICS TO IMPROVE THE SYSTEM.
![Page 60: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/60.jpg)
WHAT IS THAT SYSTEM - That which Defends (Detects, Responds, & Prevents).
![Page 61: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/61.jpg)
THE MEASURED CSO USES METRICS TO:
• Develop and improve the People, Process, and Technology to Defend
• Plan / Build / Manage those defenses
![Page 62: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/62.jpg)
THE SYSTEMS FOR DEVELOPING METRICS ARE MORE IMPORTANT THAN THE SYSTEMS OF DOGMA THAT DEFINE “STANDARDS” OF OPERATION.
![Page 63: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/63.jpg)
THE SYSTEMS FOR DEVELOPING METRICS ARE MORE IMPORTANT THAN THE SYSTEMS OF DOGMA THAT DEFINE “STANDARDS” OF OPERATION.
Sorry, ISACA
![Page 64: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/64.jpg)
THE SYSTEMS FOR DEVELOPING METRICS ARE MORE IMPORTANT THAN THE SYSTEMS OF DOGMA THAT DEFINE “STANDARDS” OF OPERATION.
• There are two systems which the CSO must manage across (at least 4 audiences)
• Those that support “defend”
• Those that support Plan/Build/Manage
![Page 65: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/65.jpg)
MEASURED CSO SYSTEM 1: THE METRICS AND MODELS THAT “DEFEND”
![Page 66: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/66.jpg)
EPIDEMIOLOGY
![Page 67: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/67.jpg)
EPIDEMIOLOGYRisk Factors (Determinants) Variables associated with increased frequency of event.
Risk Markers Variable that is quantitatively associated with a disease or other outcome, but direct alteration of the risk marker does not necessarily alter the risk of the outcome.
Correlation vs. Causation Risk factors or determinants are correlational and not necessarily causal, because correlation does not prove causation.
![Page 68: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/68.jpg)
EPIDEMIOLOGYRisk Factors (Determinants) Variables associated with increased frequency of event.
Risk Markers Variable that is quantitatively associated with a disease or other outcome, but direct alteration of the risk marker does not necessarily alter the risk of the outcome.
Correlation vs. Causation -Risk factors or determinants are correlational and not necessarily causal, because correlation does not prove causation.
THE MEANS TO FIND PATTERNS
![Page 69: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/69.jpg)
Example of a medical approach:Dr. Peter Tippett & Verizon DBIR
![Page 70: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/70.jpg)
A security incident (or threat scenario) is modeled as a series of events. Every event is comprised of the following 4 A’s:
Agent: Whose actions affected the asset Action: What actions affected the asset Asset: Which assets were affected Attribute: How the asset was affected
VERIS (Vocabulary for Event Recording & Incident Sharing)
70
![Page 71: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/71.jpg)
![Page 72: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/72.jpg)
72
![Page 73: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/73.jpg)
Object-Oriented Modeling
VERIS (Vocabulary for Event Recording & Incident Sharing)
73
1 2 3 4 5 >" >" >" >"Incident as a chain of events >"
![Page 74: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/74.jpg)
Object-Oriented Modeling
VERIS (Vocabulary for Event Recording & Incident Sharing)
74
1 2 3 4 5 >" >" >" >"Incident as a chain of events >"
A “Pattern”
![Page 75: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/75.jpg)
VERIS: Classification of Events by Risk Factor
![Page 76: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/76.jpg)
Complex System?
VERIS FOUND PATTERNS!
![Page 77: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/77.jpg)
#RSAC
36
Clustering of over 5,000 incidents
Espionage
Point of Sale
Skimming Devices
Theft/Loss
Error
EmployeeMisuse
Web Applications
DBIR Top Patterns:
![Page 78: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/78.jpg)
THE KEY TO THE MEASURED CSO SYSTEM 1: FRAMEWORK, DATA, MODELS
![Page 79: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/79.jpg)
√∫∑
Framework
Models Data=
∩
VERIS+
![Page 80: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/80.jpg)
actor information
asset information
impact information
controls information
risk
Classifying sets of security information
![Page 81: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/81.jpg)
√∫∑
Framework
Models Data=
∩Data Warehousing+
![Page 82: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/82.jpg)
82
Apache Storm
![Page 83: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/83.jpg)
83
Data MapReduce Process Analytics & Reporting
Threat Intel FeedsControl DataControl LogsSystem Logs
Event History & Loss Loss Distribu8on Dev. B.I.A.
Control DataControl LogsSystem Logs
Configuration DataVulnerability DataHR InformationProcess Behaviors
XMLCSVEDI
LOGSQL
JSONText
BinaryObjects
create map
reduce
TraditionalRDBMSSystems
Workflow
Analytics
Reporting
![Page 84: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/84.jpg)
![Page 85: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/85.jpg)
![Page 86: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/86.jpg)
![Page 87: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/87.jpg)
Models suggesting IO
C= true
![Page 88: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/88.jpg)
88
1 2 3 4 5 >" >" >" >"Incident as a chain of events >"
![Page 89: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/89.jpg)
89
1 2 3 4 5 >" >" >" >"Incident as a chain of events >" X X X
![Page 90: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/90.jpg)
90
Example of data enrichment:
Asset Intel : Vendor-owned SaaS application
![Page 91: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/91.jpg)
√∫∑
Framework
Models Data=
∩
![Page 92: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/92.jpg)
MEASURED CSO SYSTEM 1: THE METRICS AND MODELS THAT “DEFEND” AGAINST THREAT PATTERNS.
(real and anticipated or forecasted)
![Page 93: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/93.jpg)
MEASURED CSO SYSTEM 2: THE METRICS NEEDED TO PLAN/BUILD/MANAGE SYSTEMS (OPERATIONS)
![Page 94: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/94.jpg)
THE MEASURED CSO MUST ALSO INCLUDE A KEEN UNDERSTANDING AND PARTNERSHIP WITH IT OPERATIONS
![Page 95: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/95.jpg)
THE MICROMORT A one in a million chance of death Ronald A. Howard
![Page 96: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/96.jpg)
Activities that increase the death risk by roughly one micromort, and their associated cause of death (wikipedia):
Traveling 6 miles by motorbike (accident) Traveling 17 miles by walking (accident) Traveling 10 miles by bicycle (accident) Traveling 230 miles (370 km) by car (accident) Traveling 1000 miles (1600 km) by jet (accident) Traveling 6000 miles (9656 km) by train (accident) Traveling 12,000 miles (19,000 km) by jet in the United States (terrorism) Increase in death risk for other activities on a per event basis:
Hang gliding – 8 micromorts per trip Ecstacy (MDMA) – 0.5 micromorts per tablet (most cases involve other drugs)
![Page 97: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/97.jpg)
Modern Risk Management is not only bad at describing risk, but it also is focused on reporting its own version “micromorts”Inefficiently.
Traveling 10 miles by bicycle (accident)
![Page 98: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/98.jpg)
Modern Risk Management is not only bad at describing risk, but it also is focused on reporting its own version “micromorts”Inefficiently.
Traveling 10 miles by bicycle (accident)
Ecstacy (MDMA) – 0.5 micromorts per tablet (most cases involve other drugs)
![Page 99: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/99.jpg)
Modern Risk Management is not only bad at describing risk, but it also is focused on reporting its own version “micromorts”Inefficiently.
Traveling 10 miles by bicycle (accident)
![Page 100: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/100.jpg)
Modern Risk Management is not only bad at describing risk, but it also is focused on reporting its own version “micromorts”Inefficiently.
Traveling 10 miles by bicycle (accident)
![Page 101: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/101.jpg)
The Measured CSO must know where IT is overweight, smoking ecstasy, while riding a rocket-powered bicycle on the railing of a bridge.
![Page 102: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/102.jpg)
DATA: VISIBLE OPS FOR SECURITY
![Page 103: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/103.jpg)
![Page 104: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/104.jpg)
104
Example of data enrichment:
Asset Intel : Vendor-owned SaaS application
![Page 105: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/105.jpg)
SECTION 3: BECOMING MEASURED What does that mean? What do we need? How do we do it? Where shall we go?
![Page 106: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/106.jpg)
106
MOST METRICS PROGRAMS
![Page 107: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/107.jpg)
If we consider a single metric as a building block
![Page 108: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/108.jpg)
108
It should be used by the CSO to paint a picture of the security program
![Page 109: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/109.jpg)
109
Whose context is the whole of IT.
![Page 110: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/110.jpg)
110
But because we gather what is most readily available - most metrics programs look like my living room.
How does the measured CSO get context?
![Page 111: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/111.jpg)
GOAL, QUESTION, METRICConceptual level (goal) goals defined for an object for a variety of reasons, with respect to various models, from various points of view.
Operational level (question)
questions are used to define models of the object of study and then focuses on that object to characterize the assessment or achievement of a specific goal.
Quantitative level (metric)
metrics, based on the models, is associated with every question in order to answer it in a measurable way.
Victor Basili
![Page 112: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/112.jpg)
GQM FOR FUN & PROFIT
Goals establishwhat we want to accomplish.
Questions help us understand how to meet the goal. They address context.
Metrics identify the measurements that are needed to answer the questions.
Goal 1 Goal 2
Q1 Q2 Q3 Q4 Q5
M1 M2 M3 M4 M5 M6 M7
![Page 113: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/113.jpg)
Execution
Models
Data
Goal 1 Goal 2
Q1 Q2 Q3 Q4 Q5
M1 M2 M3 M4 M5 M6 M7
GQM FOR FUN & PROFIT
![Page 114: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/114.jpg)
GQM EXAMPLE: PATCH MANAGEMENT
Patching Scorecard
Goal 1: Comprehensive
Goal 2: Timely
Goal 3: Cost Efficient
![Page 115: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/115.jpg)
GQM EXAMPLE: PATCH MANAGEMENT
Patching Scorecard
Goal 1: Comprehensive
Goal 2: Timely
Goal 3: Cost Efficient
% Coverage by Business Units
%Coverage by Asset category
%Coverage by Risk
Unix
Windows Server
DesktopOS
Components
Likelihood
Impact
Most Significant Failures
Repeat Offenders
By Asset Category
By Location (DMZ, Semi-Pub, Internal)
By Business Unit
By Asset Category
By Location (DMZ, Semi-Pub, Internal)
By Business Unit
![Page 116: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/116.jpg)
GQM EXAMPLE: PATCH MANAGEMENT
Patching Scorecard
Goal 1: Comprehensive
Goal 2: Timely
Goal 3: Cost Efficient
What should our Priorities be for timeliness?
What is Policy for timeliness?
What other Considerations for Timeliness?
What is time to patch like for assets with worst Likelihoods?
What is time to patch like for assets with worst Impacts?
What % are Late by
What are our Repeat Offenders?
likelihood
Impact
by asset category
by business unit
by risk
UNIX
Windows Server
Desktop
likelihood
impact
![Page 117: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/117.jpg)
GQM EXAMPLE: PATCH MANAGEMENT
Patching Scorecard
Goal 1: Comprehensive
Goal 2: Timely
Goal 3: Cost Efficient
Cost
Risk Reduction
Hour per Asset spent PatchingBy Asset Category
By Location (DMZ, Semi-Pub, Internal)
By Cost Per Hour
Hour per Asset, by ALE per Hour
Hour per asset category
![Page 118: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/118.jpg)
GQM EXAMPLE: PATCH MANAGEMENT
• The Measured CSO creates a scorecard of KRI’s & KPI’s that Includes:
• Historical values
• “Triggers”
• “Thresholds”
(each of these?) aren’t perfect, but establish a hypothesis for testing & optimization.
![Page 119: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/119.jpg)
Now you’re ready to come correct, my Bias!
- (Chillin’ Friederich Hayek)
![Page 120: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/120.jpg)
MEASURED CSO FRAMEWORK FOR GQM: NIST CSF
NIST CSF
Identify
Protect
Detect
Respond
Recover
Asset Management
Business Environment
risk assessment
risk management strategy
Governance
Access Control
Awareness and Traininig
Data Security
Information Protection Processes and Procedures
Maintenance
Protective Technology
Anomalies and Events
Security Continuous Monitoring
Detection Processes
Response Planning
Response Communications
Response Analysis
Response Mitigation
Response Improvements
Recovery Planning
Improvements
Communications
![Page 121: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/121.jpg)
SECTION 3: BECOMING MEASURED What does that mean? What do we need? How do we do it? Where shall we go?
![Page 122: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/122.jpg)
√∫∑
Framework
Models Data=
∩
![Page 123: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/123.jpg)
![Page 124: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/124.jpg)
124
Example of data enrichment:
Asset Intel : Vendor-owned SaaS application
![Page 125: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/125.jpg)
ETL AND STORE ALL THE THINGS!!!
![Page 126: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/126.jpg)
126
Data MapReduce Process Analytics & Reporting
Threat Intel FeedsControl DataControl LogsSystem Logs
Event History & Loss Loss Distribu8on Dev. B.I.A.
Control DataControl LogsSystem Logs
Configuration DataVulnerability DataHR InformationProcess Behaviors
XMLCSVEDI
LOGSQL
JSONText
BinaryObjects
create map
reduce
TraditionalRDBMSSystems
Workflow
Analytics
Reporting
![Page 127: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/127.jpg)
Models suggesting IO
C= true
![Page 128: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/128.jpg)
![Page 129: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/129.jpg)
“If you do not know how to ask the right question, you discover nothing.”
![Page 130: DeepSec 2014 - The Measured CSO](https://reader030.vdocument.in/reader030/viewer/2022032420/55a5af441a28abd0618b45b8/html5/thumbnails/130.jpg)
RESOURCESFOR GQM AND MICROMORTS -‐ WIKIPEDIA FOR DBIR DATA, THE VERIZON DBIR FOR DEMING QUOTES, THE WORKS OF MYRON TRIBUS:
http://www.qla.com.au/papersTribus/Oslo3.pdf http://www.unreasonable-‐learners.com/wp-‐content/uploads/2011/03/Germ-‐Theory-‐of-‐Management-‐Myron-‐Tribus1.pdf
http://www.qla.com.au/papersTribus/DEMINGS_.PDF