how to get promoted - first · threatconnect.comthreatconnect.com copyright © 2019 threatconnect,...
TRANSCRIPT
![Page 1: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/1.jpg)
ThreatConnect.comCopyright © 2019 ThreatConnect, Inc.
How to Get PromotedDeveloping metrics to show how threat intel works
![Page 2: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/2.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Who are we?
Toni Gidwani @t_gidwani
Director of Research
Side gig as a Georgetown professor
Maker of gelato
2
Marika Chauvin @MarSChauvin
Senior Threat Intelligence Researcher
Research junkie
Stress baker
![Page 3: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/3.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Contents
3
The Problem: Showing value
Classes of metrics
Examples by maturity
![Page 4: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/4.jpg)
Copyright © 2019 ThreatConnect, Inc.
Problem
How do I show that threat intel provides value to my org?
![Page 5: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/5.jpg)
ThreatConnect.com Copyright © 2019 ThreatConnect, Inc.5
“Building a Threat Intel Programme” Survey Respondents
![Page 6: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/6.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Most Important Success Factor
6
❏ Remove risks from cybercrime activities
❏ Protect personal client information
❏ Protect monetary assets of the organization
❏ Increase productivity for other parts of the organization
❏ Revenue generated for the organization
❏ Prevent service interruption for core business functions
❏ Avoid embarrassing public disclosures of information
![Page 7: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/7.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.7
Disconnect: Executives Self-rate Maturity Much Higher
7
![Page 8: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/8.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
The Problem When We’re Not on the Same Page...
8
![Page 9: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/9.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
“Metrics”
9
![Page 10: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/10.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Metrics: Can’t live with them, can’t live without them
Good metrics
● Clear
● Measurable
● Correlate to business outcomes
10
Common pitfalls
● What we can count
● Output, not impact
● Too tactical for your boss’ boss
![Page 11: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/11.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.11
Types of Metrics
Measures of Performance
Measures task completion and efficiency
Am I doing this right?
Measures of Effectiveness
Measure what is accomplished and whether goals are being met
Am I doing the right things?
![Page 12: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/12.jpg)
ThreatConnect.com Copyright © 2019 ThreatConnect, Inc.12
Measures of Performance
Useful for:
● Impact of automation/efficiencies
● Process improvement
● Utilization of resources
● Incentivising a baseline step
Examples:
● Total alerts issued
● Total items reviewed/parsed
● % of malware samples detonated
● IOCs shared with community
![Page 13: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/13.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
… But
Limitations:
● Less useful for senior leaders
● Risk incentivizing poor behavior
● Less useful over long-term
13
![Page 14: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/14.jpg)
ThreatConnect.com Copyright © 2019 ThreatConnect, Inc.14
Measures of Effectiveness
Useful for:
● Conveying program value to senior
leaders
● Can be qualitative or quantitative
● Drive data collection
● Drive process development
Examples:
● Incidents discovered from TI
● Countermeasures enacted
● Total proactive blocks
● Mean time to detection
● Savings generated
![Page 15: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/15.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.15
...But
Cons:
● More difficult to generate
● Not as easily countable
● Often require interaction
and input from other teams
![Page 16: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/16.jpg)
Copyright © 2019 ThreatConnect, Inc.
Key Takeaway
Measures of Effectiveness are more compelling to your boss’ boss
![Page 17: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/17.jpg)
Copyright © 2019 ThreatConnect, Inc.
Showing Value at Different Maturity Levels
...because I can’t wait 5 years
![Page 18: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/18.jpg)
ThreatConnect.com Copyright © 2019 ThreatConnect, Inc.18
Self-Reported Money Saved
60% saved a significant sum of money in the last year
● Least mature: ~ £333
● Mid-level programmes: £5.9 million
● Well-defined programmes: £14.5 million
![Page 19: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/19.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.19
Schrodinger’s Breach: When Getting Better Looks Worse
Gains for lower maturity programs come first from:
● Improving visibility
● Understanding the threat
● Enhanced detection
![Page 20: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/20.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Metrics to Tell if Improving or Everything is on Fire
Getting started?
● IOCs observed
● Incidents discovered from TI
● Qualitative feedback loop
● Countermeasures enacted
20
![Page 21: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/21.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Metrics to Tell if Improving or Everything is on Fire
More mature?
● False positive ratio
● Impact year over year
○ Mean time to detection
○ Mean time to respond
● New intelligence from cases
● Incident criticality impacted by TI
21
![Page 22: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/22.jpg)
ThreatConnect.com Copyright © 2019 ThreatConnect, Inc.22
Quantifying value
● Mean cost of breach
○ Downtime
○ Additional resources to address breach
(consultants, identity theft protection, etc)
● Feedback loop can be used to justify salary,
team budget, and direct analysis efforts
● IBM Cost of a Data Breach Calculator
![Page 23: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/23.jpg)
ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc.
Metrics to Tell if Improving or Everything is on Fire
23
Easy
Difficult
Least Valuable Most Valuable
● Mean time to discovery● Mean time to mitigation
● New intelligence from cases
● IOCs observed
● Feedback loop
● Number of IOCs● Number of ingested feeds
● Incidents worked● AV detections
● Countermeasures enacted
● False positive ratio
● Incident criticality impacted by TI
● Mean cost of breach
● Revenue saved
● New incidents from TI● Number of reports
![Page 24: How to Get Promoted - FIRST · ThreatConnect.comThreatConnect.com Copyright © 2019 ThreatConnect, Inc. Metrics to Tell if Improving or Everything is on Fire 23 Easy Difficult](https://reader030.vdocument.in/reader030/viewer/2022040912/5e863576be9c730e5557dd63/html5/thumbnails/24.jpg)
ThreatConnect.com
Copyright © 2019 ThreatConnect, Inc.
Thank You