data science atl meetup - risk i/o security data science
DESCRIPTION
This is a talk about data science operations and the applications of Risk I/Os insights to the security industry - how we went about mining insights from our large datasetTRANSCRIPT
What Your Security Data Isn’t
Telling You
@mroytman
Michael RoytmanData Scientist, Risk I/OM.S. Operations Research, Georgia Tech
PART 1: !
DATA SCI OPS: !
LESS IS MORE !
LESS TOOLS
LESS DATALESS MODEL COMPLEXITY
MORE IMPACT
LESS DATA SCIENTISTS
SAY “BIG DATA”
ONE MORE TIME
EVERYONE IS A DATA SCIENTIST
TAKE ONLY WHAT YOU NEED
PART 2: !
FIX WHAT MATTERS
Remove the Threat
RemediationAccept the Risk
Repair the Vulnerability
“It is a capital mistake to theorize before one has data.
!
!
!
!
Insensibly, one begins to twist facts to suit theories, instead of
theories to suit facts.”
C(ommon) V(ulnerability) S(coring) S(ystem)
“CVSS is designed to rank information system vulnerabilities”
Exploitability/Temporal (Likelihood)
Impact/Environmental (Severity)
The Good: Open, Standardized Scores
FAIL 1: A Priori Modeling“Following up my previous email, I have tweaked my equation to try to achieve better separation between adjacent scores and to have CCC have a perfect (storm) 10 score...There is probably a way to optimize the problem numerically, but doing trial and error gives one plausible set of parameters...except that the scores of 9.21 and 9.54 are still too close together. I can adjust x.3 and x.7 to get a better separation . . .”
2: Data FundamentalismSince 2006 Vulnerabilities have declined by 26 percent.” http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf !
!
The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ” http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf
3: Attackers Change Tactics Daily
Repair the Vulnerability
I Love It When You Call Me Big Data50,000,000 Live Vulnerabilities
1,500,000 Assets
2,000 Organizations
I Love It When You Call Me Big Data
15,000,000 Breaches
Baseline AllthethingsProbability (You Will Be Breached On A Particular Open Vulnerability)?
=(Open Vulnerabilities | Breaches Occurred On Their CVE) /(Total Open Vulnerabilities)
2%
Probability A Vuln Having Property X Has Observed Breaches
RANDOM VULN
CVSS 10
CVSS 9
CVSS 8
CVSS 6
CVSS 7
CVSS 5
CVSS 4
Has Patch
0.000 0.010 0.020 0.030 0.040
Counterterrorism
Known Groups
Surveillance
Threat Intel, Analysts
Targets, Layouts
Past Incidents, Close Calls
Uh, Sports?
Opposing Teams, Specific Players
Gameplay
Scouting Reports, Gametape
Roster, Player Skills
Learning from Losing
Defend Like You’ve Done It Before
Groups, Motivations
Exploits
Vulnerability Definitions
Asset Topology, Actual Vulns on System
Learning from Breaches
Probability A Vuln Having Property X Has Observed Breaches
Random Vuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0.0 0.1 0.2 0.2 0.3
Data is Everything and Everything is DataSpray and Pray = 2%
CVSS 10 = 4%
Metasploit and Exploit DB = 30%
www.risk.io/jobs
@mroytman
THANKS!