maureen doyle, james walden northern kentucky university students: grant welch, michael whelan...
TRANSCRIPT
![Page 1: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/1.jpg)
Security of Open Source Web Applications
Maureen Doyle, James WaldenNorthern Kentucky University
Students: Grant Welch, Michael WhelanAcknowledgements: Dhanuja Kasturiratna
![Page 2: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/2.jpg)
1. Research Objective2. Evolution of Web App Security3. Security Resource Indicator4. Vulnerability Type Analysis5. Code Metrics6. Analysis Limitations7. Conclusion
10/12/2009University of Kentucky 2
Outline
![Page 3: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/3.jpg)
Goal: Identify predictors for vulnerability density (VD) and change in VD for open source web applications.
Research questions:1. Can software security practices predict
evolution of VD over time?2. Can code size or complexity predict VD?3. Can code change metrics predict VD?
10/12/2009University of Kentucky 3
Research Objective
![Page 4: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/4.jpg)
Static Analysis◦ Nagappan and Ball, ICSE 2005a◦ Coverity Open Source Report 2009◦ Fortify Open Source Security Study 2008
Complexity and Change Metrics◦ Nagappan and Ball, ICSE 2005b◦ Nagappan, Ball, and Zeller, ICSE 2006◦ Shin and Williams, QoP 2008
10/12/2009University of Kentucky 4
Related Work
![Page 5: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/5.jpg)
Reported Vulnerabilities in NVD or OSVD◦ Coarse-grained time evolution.◦ Difficult to correlate with revision.◦ Undercounts actual vulnerabilities.
Dynamic Analysis◦ Expensive.◦ False positives and negatives.◦ Requires installation of application.
Static Analysis◦ Expensive.◦ False positives and negatives.◦ Static Analysis Vulnerability Density =
vulns/kloc.
10/12/2009University of Kentucky 5
Measuring Vulnerabilities
![Page 6: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/6.jpg)
Selection process◦ PHP web applications from freshmeat.net.◦ Subversion repository with 100 weeks of
revisions.
Revisions◦ One revision selected per week for analysis.
Range of projects◦ 14 projects met selection criteria.◦ 5,800 to 388,000 lines of code (2008).◦ Removing highest & lowest, SLOC range of 25-150 kloc.
10/12/2009University of Kentucky 6
Samples
![Page 7: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/7.jpg)
Overall security improvement.◦ first week average: 8.88 vulns/kloc◦ final week average: 3.30 vulns/kloc
Average SAVD high vs. Coverity’s 0.30 SAVD.◦ Language differences: C/C++ vs. PHP.◦ Vulnerability differences buffer overflows vs XSS/SQL.
No correlation with NVD vulnerabilities.◦ NVD correlated with freshmeat popularity.
10/12/2009University of Kentucky 7
General Results
![Page 8: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/8.jpg)
10/12/2009University of Kentucky 8
Evolution of SAVD: 2006-2008
![Page 9: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/9.jpg)
10/12/2009University of Kentucky 9
Variation between Web Apps
![Page 10: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/10.jpg)
10/12/2009University of Kentucky 10
Example: Addressing Security Issues
![Page 11: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/11.jpg)
Public security resources on project site◦ URL for installation or configuration security◦ Dedicated address to report security bugs◦ Database of known security vulnerabilities◦ Documentation of secure development practices
Results◦ Correlation of r = 0.67 (p < 0.05) with D SAVD
10/12/2009University of Kentucky 11
Security Resource Indicator
![Page 12: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/12.jpg)
10/12/2009University of Kentucky 12
Security Resource Indicator
![Page 13: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/13.jpg)
Vulnerability Type Analysis
2006 2008
10/12/2009University of Kentucky 13
![Page 14: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/14.jpg)
10/12/2009University of Kentucky 14
Type Changes: 2006-2008
![Page 15: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/15.jpg)
University of Kentucky 10/12/2009 15
Changes by Vulnerability Type
![Page 16: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/16.jpg)
10/12/2009University of Kentucky 16
Severity Class Changes: 2006-2008
![Page 17: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/17.jpg)
Size measure◦ Source Lines of Code (SLOC)
Complexity measures◦ Cyclomatic Complexity◦ Nesting Complexity◦ Maximum, average, total
Change measures◦ Churn = lines added + changed◦ Lines deleted
10/12/2009University of Kentucky 17
Code Metrics
1
21. do loop2. stmt3. end loop
3
CC = E – N + 2 P = 3 – 3 + 2*1
![Page 18: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/18.jpg)
10/12/2009University of Kentucky 18
Code Metrics (All releases)
![Page 19: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/19.jpg)
10/12/2009University of Kentucky 19
Code Metrics (1st, Final release)
![Page 20: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/20.jpg)
May not apply to apps that didn’t meet criteria◦ Non-PHP applications◦ No SVN repository with two years of history
False positives◦ 18.1% rate from two sample applications◦ Coverity found a rate under 14% for their study
SAVD will differ between static analysis tools
10/12/2009University of Kentucky 20
Analysis Limitations
![Page 21: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/21.jpg)
OS PHP web app security improved:◦ 8.88 to 3.30 SAVD from 2008 to 2006.◦ But 8 of 14 apps increased SAVD over period.
SRI can indicate which apps will improve.
No single code metric is predictive for SAVD.◦ Complexity is an indicator for SAVD.◦ Churn is not an indicator for SAVD.
10/12/2009University of Kentucky 21
Conclusions
![Page 22: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/22.jpg)
Why does app security vary so much?◦ Analyze security processes for each app.
How do we validate SAVD measurement?◦ NVD count correlates with popularity.
Java web applications◦ How does Java SAVD compare with PHP SAVD?◦ How do trends compare between Java and PHP?◦ More software metrics available for Java.
10/12/2009University of Kentucky 22
Future Work
![Page 23: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/23.jpg)
10/12/2009University of Kentucky 23
Extra Slides
![Page 24: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/24.jpg)
10/12/2009University of Kentucky 24
SAVD vs Time and Size
![Page 25: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/25.jpg)
10/12/2009University of Kentucky 25
SAVD vs. Nesting
![Page 26: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/26.jpg)
10/12/2009University of Kentucky 26
SAVD vs. Churn
![Page 27: Maureen Doyle, James Walden Northern Kentucky University Students: Grant Welch, Michael Whelan Acknowledgements: Dhanuja Kasturiratna](https://reader030.vdocument.in/reader030/viewer/2022032702/56649f455503460f94c66295/html5/thumbnails/27.jpg)
University of Kentucky 10/12/2009 27