enterprise level digital threat modeling at the a case ... · digital threat modeling at the...
TRANSCRIPT
![Page 1: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/1.jpg)
1
The Battle for New York:A Case Study of Applied Digital Threat Modeling at the Enterprise Level
Rock Stevens, Daniel Votipka, Elissa Redmiles, Michelle Mazurek | University of Maryland
Patrick Sweeney | Wake Forest University
Colin Ahern | New York City Cyber Command
![Page 2: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/2.jpg)
Threat Modeling◂ What is it?◂ Why do it?◂ Where’s the proof?
2
![Page 3: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/3.jpg)
1.Threat ModelingThis study is the first empirical evaluation of a digital threat modeling framework at the enterprise level
![Page 4: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/4.jpg)
4
Environment
![Page 5: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/5.jpg)
Study Methods
Six-part process over the span of 120 days
5
Baseline 120-day Analysis
30-day Follow-up
Individual Sessions
Post-training Survey
Educational Intervention
![Page 6: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/6.jpg)
Baseline Survey
6
Baseline
![Page 7: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/7.jpg)
1-hour Educational Intervention
7
Baseline Educational Intervention
![Page 8: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/8.jpg)
8
Baseline Educational Intervention Center of Gravity
Actionable Defense Plan
![Page 9: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/9.jpg)
9
Baseline Educational Intervention Center of Gravity
![Page 10: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/10.jpg)
1-on-1 TMF Application Session
10
Baseline Individual Sessions
Educational Intervention
![Page 11: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/11.jpg)
Immediate Post-training Survey
Export Output for Validation
11
Baseline Individual Sessions
Post-training Survey
Educational Intervention
![Page 12: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/12.jpg)
30-Day Follow-up Survey
12
Baseline 30-day Follow-up
Individual Sessions
Post-training Survey
Educational Intervention
![Page 13: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/13.jpg)
120-day Analysis of Logs
13
Baseline 120-day Analysis
30-day Follow-up
Individual Sessions
Post-training Survey
Educational Intervention
![Page 14: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/14.jpg)
14
Baseline 120-day Analysis
30-day Follow-up
Individual Sessions
Post-training Survey
Educational Intervention
Perceived EfficacyAccuracyActual AdoptionActual Efficacy
![Page 15: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/15.jpg)
What did participants think about the threat modeling framework?
15
Perceived Efficacy
![Page 16: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/16.jpg)
Did participants produce relevant mitigating strategies?
16
Accuracy
![Page 17: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/17.jpg)
What remained with the organization beyond initial training?
17
Actual Adoption
![Page 18: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/18.jpg)
What impact did changes have on the enterprise?
18
Actual Efficacy
![Page 19: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/19.jpg)
Results
19
![Page 20: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/20.jpg)
Baseline◂ 25 participants (37% of
workforce)◂ Commercial services◂ Compliance standards◂ Industry best practices
20
![Page 21: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/21.jpg)
Perceived Efficacy◂ 12/25 identified new aspects
never before considered◂ More confident in their abilities◂ Empowered to communicate
21
![Page 22: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/22.jpg)
“ “Plan effectively, document, track, monitor progress, and essentially understand our security posture”
22
![Page 23: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/23.jpg)
Accuracy◂ 96% ADP accuracy◂ 147 unique mitigation
strategies (64% new)◂ 16/25 ADPs ready for
immediate implementation23
![Page 24: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/24.jpg)
Accuracy◂ No work role, amount of
education, IT experience, or combination thereof enjoyed a statistically significant advantage
24
![Page 25: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/25.jpg)
Actual Adoption
25
• Securing accounts• Crowdsourcing assessments• Improving sensor coverage• Reducing human error
![Page 26: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/26.jpg)
26
Actual Adoption
![Page 27: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/27.jpg)
Actual Efficacy
27
• Securing accounts• Crowdsourcing assessments• Improving sensor coverage• Reducing human error
Blocked account hijackings of five privileged user accounts
![Page 28: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/28.jpg)
Actual Efficacy
28
• Securing accounts• Crowdsourcing assessments• Improving sensor coverage• Reducing human error
Discovered and remedied three vulnerabilities in public-facing web servers
![Page 29: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/29.jpg)
Actual Efficacy
29
• Securing accounts• Crowdsourcing assessments• Improving sensor coverage• Reducing human error
Blocked 541 unique intrusion attempts
![Page 30: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/30.jpg)
Limitations
30
• No TMF comparison• Demand characteristics• Representative environment?
![Page 31: Enterprise Level Digital Threat Modeling at the A Case ... · Digital Threat Modeling at the Enterprise Level > Questions / Feedback? rstevens@cs.umd.edu | @ada95ftw •](https://reader033.vdocument.in/reader033/viewer/2022042221/5ec7ee4b189b200b744bd55e/html5/thumbnails/31.jpg)
31The Battle for New York: A Case Study of Applied Digital Threat Modeling at the Enterprise Level
> Questions / Feedback? [email protected] | @ada95ftw
• <2-hr training made an immediate impact without additional costs
• Identified 147 unique mitigation strategies• Quantitatively improved security over 120 days• Useful for empowering and communicating
Summary