automated attack surface approximation [fse - src 2015]
TRANSCRIPT
Christopher Theisen
Automated Attack Surface Approximation
1/11
Background
Attack Surface?
Ex. early approximation of attack surface – Manadhata [1]:Only covers API entry points
…easy to say, hard to define (practically).
OWASP defines Attack Surface as the paths in and out of a system, the data that travels those paths, and the code that protects both
2/11
[1] Manadhata, P., Wing, J., Flynn, M., & McQueen, M. (2006, October). Measuring the attack surfaces of two FTP daemons. In Proceedings of the 2nd ACM workshop on Quality of protection (pp. 3-10). ACM
The goal of this research is to aid software engineers in prioritizing security efforts by approximating the attack surface of a system via crash dump stack trace analysis.
3/11
Proposed Solution
Crashes represent user activity that puts the system under stress
We *know* external input touched the entities on the stack trace
Are there security implications?
H1: Crash dumps localize vulnerabilities
4/11
foo!foobarDeviceQueueRequest+0x68foo!fooDeviceSetup+0x72foo!fooAllDone+0xA8bar!barDeviceQueueRequest+0xB6bar!barDeviceSetup+0x08bar!barAllDone+0xFFcenter!processAction+0x1034center!dontDoAnything+0x1030
OverviewCatalog all code that appears on stack traces
5/11
OverviewCatalog all code that appears on stack traces
5/11
OverviewCatalog all code that appears on stack traces
5/11
Attack Surface Analysis
Windows 8 [2] Fuzzing User Crashes*%binaries 0.9% 48.4%
%vulnerabilities 14.9% 94.6%
*Stack traces from dogfood testing crashes and field crashes
6/11[2] C. Theisen, K. Herzig, P. Morrison, B. Murphy, and L. Williams, “Approximating Attack Surfaces with Stack Traces,” in Companion Proceedings of the 37th International Conference on Software Engineering, 2015
Mozilla Firefox User Crashes%files 8.4%
%vulnerabilities 72.1%
Stack traces highlighted where security vulnerabilities were.
Vulnerability Prediction Models
Generate VPM based on 29 metrics (Churn, LoC, etc.) [3]
Run the VPM with all files considered as possibly vulnerable
Repeat, but remove code not found on stack traces
Vulnerability Prediction Model (VPM)
Precision improved from 0.5 to 0.69 Recall improved from 0.02 to 0.05
Statistical improvement? Yes. Practical? No.
Results [2]
[3] T. Zimmermann, N. Nagappan and L. Williams, "Searching for a Needle in a Haystack: Predicting Security Vulnerabilities for Windows Vista," in Software Testing, Verification and Validation (ICST), 2010 Third International Conference on, 2010
7/11[2] C. Theisen, K. Herzig, P. Morrison, B. Murphy, and L. Williams, “Approximating Attack Surfaces with Stack Traces,” in Companion Proceedings of the 37th International Conference on Software Engineering, 2015
Firefox Analysis
More crashes = more vulnerabilities?
More stack traces, less files, higher flaw density!Lose coverage as you increase stack trace cutoff
Priority: Bottom upIntroduction | Methodology | Results and Discussion | Future Work | Conclusion
Files Flaws %Files %Vuln Precision Recall>= 1 4998 282 8.4% 72.1% 0.056 0.721>= 30 1853 210 3.1% 53.7% 0.113 0.537
>= 140 969 162 1.6% 41.4% 0.167 0.414All 59437 391 - - - -
8/11
Future Work
Introduction | Methodology | Results and Discussion | Future Work | Conclusion 9/11
Temporal Analysis
Initial attack surface approximation ...old nodes removed, new nodes added
Are new files now on the attack surface?Are legacy files files now on the attack surface?
Preliminary: Win 10 files dropped over time, but (old) items added back!
Future Work
Introduction | Methodology | Results and Discussion | Future Work | Conclusion 10/11
Few to Many Many to Many Many to Few
What are the security impact of these shapes?
Preliminary: 65% of entities have less than 5 links
Shape Analysis
A AA
Introduction | Methodology | Results and Discussion | Future Work | Conclusion
foo!foobarDeviceQueueRequest+0x68foo!fooDeviceSetup+0x72foo!fooAllDone+0xA8bar!barDeviceQueueRequest+0xB6bar!barDeviceSetup+0x08bar!barAllDone+0xFFcenter!processAction+0x1034center!dontDoAnything+0x1030
Thanks to…
11/11
Laurie WilliamsBrendan MurphyKim HerzigWindows Product Teams…and many more