c overflows vulnerabilities exploit taxonomy and evaluation on static analysis tools - mock viva for...

C Overflow Vulnerabilities Exploit Taxonomy and

Evaluation on Static Analysis Tools

20th April 2014

Master of Sciences

Nurul Haszeli AhmadStudent Id: 2009625912

Supervisor: Dr Syed Ahmad Aljunid (FSMK, UiTM Shah AlamCo-Supervisor: Dr Jamalul-Lail Ab Manan (MIMOS Berhad)

Works on reliable and trustworthy system starts since late 70s Extensively after Morris Worm After three decades, reliable and trustworthy system are yet to achieve

with software vulnerabilities still being reported and C overflow vulnerabilities is still one of the top ranked vulnerabilities

This thesis focus on understanding vulnerabilities via taxonomy as one of ways to improve system reliability and trustworthy

The thesis◦ Taxonomy and Criteria of well-defined taxonomy◦ Method to evaluate taxonomy◦ Method to evaluate OS reliability◦ Result of evaluating static analysis tools

Abstract

Introduction

Review of Literature

Research Methodology

Results and Discussion

Conclusion and Recommendation

Q & A

Presentation Content

Background of Study Problem Statement RQ & RO Research Significance Assumptions Scope and Limitations

Introduction

Introduction – Background of Study

• First vulnerability was discovered unintentionally by Robert Morris in

1988

• However the hype of the vulnerabilities only starts after 1996

after an article written and published by hackers known as Aleph

One.

• Since then, the vulnerabilities and exploitation moves to different

stage.

• 87% increase in terms of exploitation on vulnerabilities (CSM, 2009)

• the intensity of attack on web application vulnerability (Cenzic,

2009)

• 90% of web application is vulnerable with Adobe ranked the top

contributor (Cenzic, 2010)

• First sophisticated malware exploited Windows vulnerability

reported (Symantec Corporation, 2010), (Falliere, et. al., 2011),

(Chen, 2010)

• The malware evolved



1988



One.


stage.



2009)





(Chen, 2010)


2006 2007 2008 2009 20100

1000

2000

3000

4000

5000

6000

7000

4842 4644

5562

4814

6253

No. of VulnerabilitiesReferences: Symantec Corporation, Internet Security Threat Report, Volume 16, 2011



1988



One.


stage.



2009)





(Chen, 2010)


# o

f vu

lnera

bili

ties

Year

References: Stefan Frei (NSS Labs), Analyst Brief: Vulnerability Threat Trends – A decade in view, Transition on the way, Feb 04, 2013


References: Microsoft Corporation, Software Vulnerability Exploitation Trends, Technical Report, 2013. http://www.microsoft.com/en-us/download/details.aspx?id=39680


References: MITRE Corporation, CVE Details – Vulnerabilities by type and Year [Online], Published at http://www.cvedetails.com/vulnerabilities-by-types.php, 2014. Retrieved on April 20th, 2014.

http://www.cvedetails.com/vulnerabilities-by-types.php

http://www.cvedetails.com/vulnerabilities-by-types.php


• Vulnerabilities in applications is higher compared to hardware and network

(Sans Institute, Top Cyber Security Risks - Vulnerability Exploitation Trends, 2009)

• 80% is ranked with high and medium severity (Microsoft Corporation, Microsoft Security Intelligence Report, 2011)

• 50% is contributed in major-used applications compared to web-based vulnerabilities

(IBM, 2011; HP, 2011)

• 3 – 15% (or more) vulnerabilities are capable to trigger memory overflow (Cenzic, 2010; Cisco, 2010; HP, 2011; SANS Institute, 2010; NIST,

2011)


References: Stefan Frei (NSS Labs), Analyst Brief: Vulnerability Threat Trends – A decade in view, Transition on the way, Feb 04, 2013


Reference: MITRE Corporation, Current CVSS Score Distribution For All Vulnerabilities, CVE Details [online]. 2014, Accessed on April 20, 2014 at http://www.cvedetails.com/


Reference: Microsoft Corporation, Software Vulnerability Exploitation Trends, 2013

Introduction – Problem Statement

• Vulnerabilities continue to persist• Requires only 0.1% from total vulnerabilities to cause problem to computer

system (Symantec Corporation, 2013)


• Vulnerabilities continue to persist• Requires only 0.1% from total vulnerabilities to cause problem to computer

system (Symantec Corporation, 2013)

• Vulnerabilities exist in software, hardware and networks.• Software vulnerabilities are due to programming errors, misconfiguration,

invalid process flow and because the nature of the programs it self (Beizer, 1990), (Aslam, 1995), (Longstaff et al, 1997), (Howard and Longstaff, 1988), (Krsul, 1988), (Piessens, 2002), (Vipindeep and Jalote, 2005), (Alhazmi et al, 2006), (Moore, 2007)

• There are many programming errors• Impact of this errors are software abnormal behavior, integrity error or being

utilize for exploitation• The most persistent is the classical C overflow vulnerabilities.• How serious it is?

• Viega & McGraw dedicated a chapter• Howard, LeBlanc and Viega discussed in two books• MITRE Corporation and SANS release a report• Microsoft and Apple stress this on their development process.

• Work to resolve this started by Wagner and progressive since then• But C overflow vulnerabilities continue to persist.


C overflow vulnerabilities continue to persist (more than 3 decades) despite

many works have been done (in the area of analysis tools, policies and knowledge

improvement).

Therefore, there are still gaps in this few area which requires further analysis and proposed solution with the objective of

solving this prolonged security nightmare.

Introduction … continue

Research Question Research Objective

1. Why C overflow vulnerabilities still persist although it is common knowledge, and there are numerous methods and tools available to overcome them?

1. To identify the reasons why C overflow vulnerabilities, despite more than three decades, still persist although there are various methods and tools available.

2. How to improve the understanding and knowledge of software developer on C overflow vulnerabilities from source code perspective?

2. To construct a well-defined C overflow vulnerabilities exploit taxonomy from source code perspective.

3. How to evaluate the well-defined C overflow vulnerabilities taxonomy from source code perspective?

3. To evaluate the constructed taxonomy against the well-defined criteria.

4. What is the effectiveness of static analysis tools in detecting the C overflow vulnerabilities exploit based on the well-defined taxonomy?

4. To evaluate the effectiveness of static analysis tools in detecting C overflow vulnerabilities based on the classes in the constructed taxonomy.

5. Which Windows-based operating systems is critical and vulnerable to exploit using C overflow vulnerabilities?

5. To evaluate the criticality of windows-based operating system concerning on its capability to avoid C overflow vulnerabilities exploit.


Research Significance:

C overflow vulnerabilities is still relevant

C language is widely used in various mission-critical applications

It ease the understanding of software developers for writing secure codes and security analyst to analyse codes

Direct the focus security research initiatives on most critical and relevant types of C overflow vulnerabilities

Determine the identified static analysis tools capability, strength and weaknesses


Assumptions:

Most exploits happen on 32-bit OS especially Windows

Number of exploits on

Linux or Unix vulnerabilities are relatively

small

Major concern is on 32-bit OS as 64-bit

OS is claimed more secure

Other programming language are

highly not critical


Scope and Limitation:

1. Focus on Windows 32-bit

2. Limited to Windows XP and 7 based on market share

3. Focus on C programming language

4. Evaluation is limited to five freely available open-source code applications

5. Evaluation is limited to five static analysis tools freely available

6. Focus on static analysis

Software Vulnerabilities C Overflow vulnerabilities Static Analysis and Methods Dynamic Analysis: Another Perspective Structuring the Knowledge of C Overflow

Vulnerabilities Conclusion

Review of Literature

Review of Literature – Software Vulnerabilities

• Software vulnerabilities is security flaws that exist within software due to use of unsafe function, insecure coding, insufficient validation, and or improper exception handling (Stoneburner, Goguen, & Feringa, 2002), (Viega & McGraw, 2002), (Seacord R. , 2005) and (Kaspersky Lab ZAO, 2013)

• A vulnerable software can cause harm or maliciously used to cause harm especially to human beings (Kaspersky Lab ZAO, 2013), (OWASP Organization, 2011), as happened in Poland (Baker & Graeme, 2008), Iran (Chen T. M., 2010), and in the case of Toyota break failure (Carty, 2010)

• There are many types of software vulnerabilities depending on perspectives (Beizer, 1990), (Aslam, 1995), (Longstaff, et al., 1997), (Krsul, 1998), (Piessens, 2002), (Vipindeep & Jalote, 2005), (Alhazmi, Woo, & Malaiya, 2006), (Moore H. D., 2007), (Howard, LeBlanc, & Viega, 2010), (SANS Institute, 2010), (OWASP Organization, 2011), (Kaspersky Lab ZAO, 2013), (MITRE Corporation, 2013), (Wikipedia Foundation, Inc, 2011)

• The most mentioned and most critical – Programming errors (Cenzic Inc., 2010), (Secunia, 2011), (NIST, 2014), (OSVDB, 2014)

• Impact of programming errors - causing abnormal behaviour, display incorrect result, Denial of Service (DoS) and/or memory violation (Chen T. M., 2010), (IBM X-Force, 2011), (HewlettPackard, 2011), (Baker & Graeme, 2008), (Carty, 2010), (Telang & Wattal, 2005)

References: (Microsoft, 2009), (Android Developer, 2012), (Stack Exchange Inc, 2012), (Oracle Corporation, 2012), (Apple Inc, 2012), (Symantec Corporation, 2013), (Kaspersky Lab, 2013), (Gong, 2009), (Fritzinger & Mueller, 1996), (Chechik, 2011), (Mandalia, 2011), (Kundu & Bertino, 2011),

Review of Literature – Software Vulnerabilities

• Many ways of exploiting programming errors – XSSi, SQLi, Command Injection, Path Traversal and Overflow exploitation (Martin B. , Brown, Parker, & Kirby, 2011), (Siddharth & Doshi, 2010), (Shariar & Zulkernine, 2011), (IMPERVA, 2011)

• Overflow happen in PHP, Java and persistently in C (Cenzic Inc, 2009), (MITRE Corporation, 2012), (Mandalia, 2011), (TIOBE Software, 2012), (Tiwebb Ltd., 2007), (DedaSys LLC, 2011) and (Krill, 2011)

Comparison Matrix C Java

Exploitation Impact Use in mission critical system and proven track record

Used in various application but yet to see any serious exploitation impact

Persistent and Dominance Since late 80s Since early year 2000Number of exploitation = low

Severity Perspective 60% is medium to critical (MITRE, IBM, Secunia & NIST)

70% medium to critical (OSVDB)

Defense and Preventive Insecure functionsLimitation on defensive and preventive mechanismAll preventive tools developed to detect and prevent Overflow vulnerabilities

JVM, applet & type safetyFrequent patches/fixAll preventive tools focus on coding standard and policy

References: (Microsoft, 2009), (Android Developer, 2012), (Stack Exchange Inc, 2012), (Oracle Corporation, 2012), (Apple Inc, 2012), (Symantec Corporation, 2013), (Kaspersky Lab, 2013), (Gong, 2009), (Fritzinger & Mueller, 1996), (Chechik, 2011), (Mandalia, 2011), (Kundu & Bertino, 2011),

Review of Literature – C Overflow Vulnerabilities

• C Overflow vulnerabilities is persistent with proven track record• The first known C overflow vulnerabilities exploit technique – format string attack (One, 1996),

(Longstaff et al, 1997)• Well known for more than 3 decades BUT still persist (SANS Institute, 2010), (Martin, Brown, Paller,

& Kirby, 2010) and (MITRE Corporation, 2013)• The first identified C overflow vulnerabilities exploit class – unsafe functions (Wagner D. A., 2000),

(Zitser, 2003), (Chess & McGraw, 2004), (Kratkiewicz K. J), (Seacord R. , 2005), (Howard, LeBlanc, & Viega, 2010), etc.






Expert UF AOB RIL IO MF FP VTC UV NT

Wagner et. al. (2000) ( ϴ

Viega et.al. (2000) ϴ √

Grenier (2002) ϴ √ √ √ Zitser (2003) ϴ √

Chess and McGraw ((2004) √

Tevis and Hamilton (2004) ϴ √ √ √

Zhivich (2005) ϴ √ Kratkiewicz (2005) ϴ √ Sotirov (2005) ϴ √ √ Tsipenyuk et.al. (2005) ϴ √ √ ϴ √ √ √

Alhazmi et. al. (2006) √

Kolmonen (2007) √ √ √ √ Moore (2007) ϴ √ √ Akritidis et.al. (2008) √ √ √

Pozza and Sisto (2008) ( √ √ √ √ √ Nagy and Mancoridis (2009) √ √

Shahriar and Zulkernine (2011) ϴ √






• Most of these 9 classes carries at least medium severity and impact.


Class of C Overflow Vulnerability

Severity Complexity Impact

Unsafe Function Critical Low CompleteArray out-of-bound Critical Low Complete

Return-into-libC Critical High PartialInteger Overflow Critical Low CompleteMemory Functions Critical Medium Partial

Function Pointer Moderate Medium PartialVariable Type Conversion Moderate Medium Complete

Uninitialized Variable Moderate Low Complete

Null Termination Moderate Low Partial






• Most of these 9 classes carries at least medium severity and impact.• Additional class is Pointer Scaling/Mixing. (NIST, 2007), (Searcord R. C., 2008), (Black et al, 2011)

(OWASP, 2012)

Review of Literature – Static Analysis Tools and MethodsPr

ogre

ss

Year1970

Kings, 1970- Debugging and understanding

program- Pattern-matchingLexical analysis (first method)- Implemented in grep

Cousots, 1977- Implements Abstract Syntax

Tree (Abstract Interpretation)- Purpose – debug & understand

Andersens, 1994- Introduce Inter-Procedural &

Pointer Analysis- Purpose – debug & understand

1990 2000 2010

Wagner, 2000- Integer Range Analysis- Array out-of-bound / Unsafe

functions- BOON

Viega et al, 2000- Lexical Analysis- Brief Data Flow and- Unsafe Functions- ITS4

Larochelle & Evans, 2001- Annotation-based- All overflow class

Ball & Rajamani, 2002- AI (utilize CFG, SDG and PDG)- All overflow class- SLAM

Cousots, 2005- Advance of AI (include complex

mathematical model)- All overflow class- ASTREE

Venet, 2005- Symbolic Analysis + IRA- Only utilize CFG- All overflow class

Burgstaller, 2006- Symbolic Analysis + new rules and

algorithm- All overflow class

Ball, 2006- Software Model Checking (based

on SA)- Utilize CFG + SDG and built

software model- All overflow class

Akritidis et al, 2008- AI (point-to analysis)- Embed identified potential

vulnerabilities during compile with protection

- All overflow class

Pozza and Sisto, 2008- DFA + IRA- Combine static and

dynamic- All overflow class

Wang et al, 2009- DFA + AB- Combine static and

dynamic- All overflow class

NEC Lab, 2004 – 2014- AI, IRA, SMC, Bounded

Model Checking (SAT capability)

- All overflow class

Review of Literature – Static Analysis Tools and MethodsTechnique

Researcher

LA Inter-PA Intra-PA AI DFA SA IRA AB TM SMC BMC

Wagner (2000)

√

Viega et. al. (2000)

√ √ √

Larochelle and Evans (2001)

√

Venet (2005) √ √

Ball et. al. (2006)

√ √

Akritidis et. al. (2008)

√

Pozza and Sisto (2008)

√ √ √ √

Wang et. al. (2009)

√ √

Nagy and Mancoridis (2009)

√ √ √ √

NEC Lab. (2004-2014)

√ √ √ √ √

Review of Literature – Static Analysis Tools and MethodsNo. Tool Latest Version Latest Release

DateTechnique Language COV Coverage

1 ITS4 (Cigital, 2011)

1.1.1 February 17, 2000 LA C / C++ Unsafe Function Return-into-lib

2 BOON (Wagner D., 2002)

1.0 July 5, 2002 IRA C Array Out-of-boundUnsafe Function

3 ARCHER Not available 2003 SA, DFA, InterPA C Array Out-of-boundUnsafe Function

4 CSSV (Dor, Rodeh, & Sagiv, 2003)

Not Available 2003 DFA, InterPA, IRA, AB C Array Out-of-bound, Unsafe Function

5 CQual (Foster, 2004)

0.981 June 24, 2004 DFA, AB, InterPA C Unsafe Function, Function Pointer,

6 MOPS (Chen & Wagner, 2002)

0.9.1 2005 SMC C No specific (rules dependent)

7 PScan (DeKok, 2007)

1.3 January 4, 2007 LA C Unsafe Function (limited to printf)

8 FlawFinder (Wheeler, 2007)

1.27 January 16, 2007 LA C / C++, Java No specific (list dependent)

9 UNO (Bell Labs, 2007)

2.13 October 26, 2007 DFA, SA, SMC C (ANSI C) Uninitialized variablesFunction pointerArray Out-of-boundMemory Function

10 Saturn (Aiken,et al., 2008)

1.2 2008 InterPA, Intra-PA, DFA, SA

C Function Pointer



11 GCC Security Analyzer (Pozza & Sisto, 2008)

Not available 2008 DFA, InterPA, IntraPA, AB, IRA

C Unsafe Function, Array Out-of-Bound, Integer Overflow, Variable Type Conversion, Memory Function

12 C Global Surveyor (Brat & Thompson, 2008)

Not Available 2008 AI, DFA, Program Slicing

C Uninitialized Variable, Function Pointer, Array Out-of-bound

13 BLAST (Henzinger, Beyer, Majumdar, & Jhala, 2008)

2.5 July 11, 2008 DFA, SMC C No specific (model dependent)

14 PC-Lint (Gimpel Software, 2013)

9.0 September, 2008 LA, DFA, AB C / C++ No specific (rule dependent)

15 Splint (National Science Foundation, 2010)

3.1.2 August 5, 2010 AB, InterPA ANSI C No specific (annotation dependent)

16 RATS (HP Fortify, 2011)

2.3 2011 LA C / C++, Perl, PHP, Python

Array Out-of-boundUnsafe Function



17 PolySpace (MathWorks, Inc, 2014)

V8.2 (R2011b) 2011 AI C Integer OverflowArray Out-of-bound, Uninitialized Variable, Functio Pointer / Pointer Aliasing, and Variable Type Conversion

18 F-Soft (platform) (NEC Laboratories, 2012)

Not Available 2011 AI, DFA, BMC, SA, SMC C / C++ Function Pointer,Memory Function,Return-into-lib,Variable type conversion,Null termination,Unsafe function

19 VARVEL (Hashimoto & Nakajima, 2009)

Not Available 2011 DFA, SMC C / C++ Function Pointer,Array Out-of-bound, Unsafe Function, Memory Function



20 CodeSonar (GrammaTech, Inc, 2011)

Not Available September 27, 2011

DFA, SMC, InterPA C Function pointer, Integer overflow, Memory function, Array out-of-bound, Unsafe function, Variable Type Conversion, Uninitialized Variable

21 CodeWizard (Parasoft, 2011)

9.2.2.17 December 12, 2011 LA, DFA C / C++ All (No specific)

22 ASTREE (Cousot P. , Cousot, Feret, Miné, & Rival, 2006)

11.12 December 15, 2011 AI, InterPA C / C++ Integer Overflow, Array Out-of-bound, Function Pointer,Uninitialized Variable

23 SLAM (Microsoft Corporation, 2011)

Not Available 2011 SMC C/C++ No specific (rule dependent)

Review of Literature – Static Analysis Tools and MethodsNo. Methods Strength Weaknesses1. LA Basic and simple method Ignore program semantics and pattern/rules

dependent2. InterPA Understand function relationship Dependent on other method

3. IntraPA Understand the internal process Ignore program semantics and dependent on other

4. AI 1. Semantics based2. Use approximation to reduce analysis time

1. Complex algorithm and process hence ignore some properties during the analysis.2. Approximation may lead to false alarm.3. Is not scalable for large programs.

5. DFA 1. Understand the flow 2. Provide a clear direction of vulnerabilities flow path and reduce false alarm

1.Suffer overhead due to extensive in-depth analysis,2.Ignore some properties or vulnerabilities and hence reduce precision.

6. SA Represent the actual program in symbolic execution form and implements refinement process.

1 Input and model dependent 2. Overhead due to many refinement processes.

7. IRA 1.Straight forward. 2.Strong constraint-based algorithm.

1.Ignores program semantics 2.Limited scope on C overflow vulnerabilities.

References: (Aho, Lam, Sethi, & Ullman, 1986), (Andersen, 1994), (Bae, 2003), (Ball, et al., 2006), (Balakrishnan, Sankaranarayanan, Ivančić, & Gupta, 2009), (Bakera, et al., 2009), (Biere, Cimatti, Clarke, & Strichman, 2003), (Burgstaller, Scholz, & Blieberger, 2006), (Chess & McGraw, 2004), (Clarke E. M., 2006), (Clarke E., 2009), (Clarke, Biere, Raimi, & Zhu, 2001), (Cousot & Cousot, 1977), (Cousot P. , et al., 2005), (D’Silva, Kroening, & Weissenbacher, 2008), (Deursen & Kuipers, 1999),(Dor, Rodeh, & Sagiv, 2001), (Erkkinen & Hote, 2006), (Evans, Guttag, Horning, & Tan, 1994), (Ferrara, 2010), (Ferrara, Logozzo, & Fanhdrich, 2008), (Gopan & Reps, 2007), (Haghighat & Polychronopoulos, 1993), (Holzmann G. J., 2002), (Ivančić, et al., 2005), (Jhala & Majumdar, 2009), (Kolmonen, 2007), (Kunst, 1988), (Ku, Hart, Chechik, & Lie, 2007), (Larochelle & Evans, 2001), (Li & Cui, 2010), (Lim, Lal, & Reps, 2009), (Logozzo, 2004), (Parasoft Corporation, 2009), (Pozza & Sisto, 2008),(Qadeer & Rehof, 2005), (Rinard, Cadar, Dumitran, Roy, & Leu, 2004), (Sotirov, 2005), (Tevis & Hamilton, 2004), (Vassev, Hinchey, & Quigley, 2009, (Venet, 2005), (Viega J. , Bloch, Kohno, & McGraw, 2000), (Wang, Guo, & Chen, 2009), (Wang, Zhang, & Zhao, 2008), (Wagner, Foster, Brewer, & Aiken, 2000), (Wenkai, 2002), (Zafar & Ali, 2007), (Zaks, et al., 2006)

Review of Literature – Static Analysis Tools and MethodsNo. Methods Strength Weaknesses8. AB 1.Utilize the annotation written in

the code and hence reduce the complexity to convert and analyse.

1.Dependent on correct syntax of annotation2.Depends on willingness of developer to annotate.

9. Type Matching

Semantically understand the type conversion

1. Depends on valid source code abstraction. 2. Limited to two types of C overflows vulnerabilities that relates to type conversion.

10. SMC Semantically understand the programs.Precise analysis with counterexample algorithm.

1. Suffer state explosion and model dependent.2. Not scalable for large or complex source code.Limited coverage on C overflow vulnerabilities

11. BMC 1.Semantically understand the programs.2.Precise analysis with counterexample capability.3.Less state to analyse compare to software model checking.4.Reduce state explosion

1.Still has the possibility of state explosion.2.Model dependent.3.Not scalable for large and complex source code.4.Limited coverage on C overflow vulnerabilities

References: (Aho, Lam, Sethi, & Ullman, 1986), (Andersen, 1994), (Bae, 2003), (Ball, et al., 2006), (Balakrishnan, Sankaranarayanan, Ivančić, & Gupta, 2009), (Bakera, et al., 2009), (Biere, Cimatti, Clarke, & Strichman, 2003), (Burgstaller, Scholz, & Blieberger, 2006), (Chess & McGraw, 2004), (Clarke E. M., 2006), (Clarke E., 2009), (Clarke, Biere, Raimi, & Zhu, 2001), (Cousot & Cousot, 1977), (Cousot P. , et al., 2005), (D’Silva, Kroening, & Weissenbacher, 2008), (Deursen & Kuipers, 1999),(Dor, Rodeh, & Sagiv, 2001), (Erkkinen & Hote, 2006), (Evans, Guttag, Horning, & Tan, 1994), (Ferrara, 2010), (Ferrara, Logozzo, & Fanhdrich, 2008), (Gopan & Reps, 2007), (Haghighat & Polychronopoulos, 1993), (Holzmann G. J., 2002), (Ivančić, et al., 2005), (Jhala & Majumdar, 2009), (Kolmonen, 2007), (Kunst, 1988), (Ku, Hart, Chechik, & Lie, 2007), (Larochelle & Evans, 2001), (Li & Cui, 2010), (Lim, Lal, & Reps, 2009), (Logozzo, 2004), (Parasoft Corporation, 2009), (Pozza & Sisto, 2008),(Qadeer & Rehof, 2005), (Rinard, Cadar, Dumitran, Roy, & Leu, 2004), (Sotirov, 2005), (Tevis & Hamilton, 2004), (Vassev, Hinchey, & Quigley, 2009, (Venet, 2005), (Viega J. , Bloch, Kohno, & McGraw, 2000), (Wang, Guo, & Chen, 2009), (Wang, Zhang, & Zhao, 2008), (Wagner, Foster, Brewer, & Aiken, 2000), (Wenkai, 2002), (Zafar & Ali, 2007), (Zaks, et al., 2006)

Review of Literature – Static Analysis Tools and MethodsNo. Tool Strength Limitation COV Coverage1 FlawFinder Faster and supported with risk level

assessmentHigh false alarms and limited type of C overflows vulnerabilities

Unsafe functions (database dependent)

2 RATS Faster, supported multiple language, and memory location identification

High false alarms and limited type of C overflows vulnerabilities

Unsafe functions (database dependent)

3 BOON Efficient in range analysis of variable and does not dependent on database or list for detection.

High false alarm due to ignorance of program semantics

Unsafe functions and array out-of-bound.

4 ARCHER Symbolically understand the source code and ability to analyze large source code.

Still produce false alarm, does not understand C language completely, and limited vulnerabilities

Unsafe functions and array out-of-bound

5 MOPS Analyzing model of program which constructed to the closest possible of execution form for precise detection

Complexity of modelling No specific C overflows vulnerabilities (model dependent)

6 UNO Efficient in analyzing complex program and low false alarm with counterexample capability

State explosion and limited vulnerabilities

Uninitialized variables, function pointer, and array out-of-bound

7 F-SOFT Combines many technique that increase detection capability and continuous improvement from NEC Labs.

Still suffer state explosion and produce false alarms.Time consuming due to many analysis stage.

Unsafe functions, function pointers, memory functions, return-into-libC, variable type conversion, and null termination.

References: (Tevis & Hamilton, 2004), (Kolmonen, 2007), (Zafar & Ali, 2007),(Viega J. , Bloch, Kohno, & McGraw, 2002), (Sotirov, 2005), (HP Fortify, 2011), (Chess & McGraw, 2004), (Zitser, Lippmann, & Leek, 2004), (Xie, Chou, & Engler, 2003), (Lim, Lal, & Reps, 2011), (Pozza & Sisto, 2008), (Engler & Musuvathi, 2004), (Li & Cui, 2010), (Holzmann G. J., 2002), (Jhala & Majumdar, 2009), (Balakrishnan, Sankaranarayanan, Ivančić, & Gupta, 2009), (Ganai, et al., 2008), (Balakrishnan, et al., 2010) and (Ivančić, et al., 2005), (Xie & Aiken, 2005), (Wang, Gupta, & Ivančić, 2007), (Ku, Hart, Chechik, & Lie, 2007), (Jhala & McMillan, 2005), (Henzinger T. A., Jhala, Majumdar, & Qadeer, 2003), (Beyer, Henzinger, Jhala, & Majumdar, 2007), (Marchenko & Abrahamsson, 2007), (Plösch, Gruber, Pomberger, Saft, & Schiffer, 2008) and (Anderson J. L., 2005), (Wang, Ding, & Zhong, 2008), (Evans & Larochelle, 2002), (Karthik & Jayakumar, 2005), (Venet, 2005), (Ball & Rajamani, 2002), (Microsoft Corporation, 2014), (Microsoft Corporation, 2011), (NIST, 2012).

Review of Literature – Static Analysis Tools and MethodsNo. Tool Strength Limitation COV Coverage9 GCC Security

AnalyzerReduce time by implementing the analysis in compiler and high detection rate.

Compilers overhead and still produce false alarm.Limited type of C overflows vulnerabilities.

Unsafe function, array out-of-bound, integer overflow, variable type conversion, and memory function

10 BLAST Combines variety of abstraction method, refinement methodology, and analysis algorithm.

Suffer state explosion, produce false alarm, and dependent on defined properties.

No specific C overflows vulnerabilities (model dependent)

11 PC-Lint Lots of error message to help user understand the errors found and implementation of data flow analysis and annotation based to reduce false alarms.Faster analysis time.

Dependent on rules and standard defined to verify the source code.Limited capability due to inheritance of lexical analysis weaknesses.

No specific C overflows vulnerabilities (rule dependent)

12 SPLINT Very fast and low false alarm (with proper implementation of code annotation)

Limited vulnerabilities (dependent on rule and annotation).False alarm can be very high when code is incorrectly annotated or limited rule.

No specific C overflows vulnerabilities (rule and annotation dependent)

13 ASTREE Precisely detecting four types of C overflows vulnerabilities.Low false alarm.

Scalability issues and limited to only four types of C overflows vulnerabilities.

Integer overflow, array out-of-bound, function pointer, and uninitialized variable

14 SLAM Very precise in detecting violation of device drivers.

Model dependent and limited to small size programs.

No specific C overflows vulnerabilities (model dependent)

References: (Tevis & Hamilton, 2004), (Kolmonen, 2007), (Zafar & Ali, 2007),(Viega J. , Bloch, Kohno, & McGraw, 2002), (Sotirov, 2005), (HP Fortify, 2011), (Chess & McGraw, 2004), (Zitser, Lippmann, & Leek, 2004), (Xie, Chou, & Engler, 2003), (Lim, Lal, & Reps, 2011), (Pozza & Sisto, 2008), (Engler & Musuvathi, 2004), (Li & Cui, 2010), (Holzmann G. J., 2002), (Jhala & Majumdar, 2009), (Balakrishnan, Sankaranarayanan, Ivančić, & Gupta, 2009), (Ganai, et al., 2008), (Balakrishnan, et al., 2010) and (Ivančić, et al., 2005), (Xie & Aiken, 2005), (Wang, Gupta, & Ivančić, 2007), (Ku, Hart, Chechik, & Lie, 2007), (Jhala & McMillan, 2005), (Henzinger T. A., Jhala, Majumdar, & Qadeer, 2003), (Beyer, Henzinger, Jhala, & Majumdar, 2007), (Marchenko & Abrahamsson, 2007), (Plösch, Gruber, Pomberger, Saft, & Schiffer, 2008) and (Anderson J. L., 2005), (Wang, Ding, & Zhong, 2008), (Evans & Larochelle, 2002), (Karthik & Jayakumar, 2005), (Venet, 2005), (Ball & Rajamani, 2002), (Microsoft Corporation, 2014), (Microsoft Corporation, 2011), (NIST, 2012).

Review of Literature – Static Analysis Tools and Methods In Summary

1. There are 11 methods known so far 2. More than 40 static analysis tools developed3. C overflow vulnerabilities continue to persist

1. All static analysis methods and tools focusing on solving the symptoms of C overflow vulnerabilities rather than the root cause of it.

2. Although there are tools interpret the source code, but, due to failure to understand the root cause and the structures of C, the tools is yet to understand the actual problem

3. Impacting the precision @ effectiveness of the methods and tools.

Can we solve the problem with Dynamic Analysis?

Review of Literature – Dynamic AnalysisDynamic analysis:1. Analyze program during program’s execution (Li and Chiueh, 2007)2. Due to that, it only analyze only executed path (Ernst, 2003)3. Tools that implement dynamic analysis – SigFree (Wang et al 2008), StackGuard (Lhee &

Chapin, 2002), StackShield (Lhee & Chapin, 2002), CCured (Sidiroglou & Keromytis, 2004), etc.

Strength:1. Source Code independent (Ernst, 2003), (Goichi et al, 2005), (Cornell, 2008) (Wang et al,

2008)2. Due to that and analyzed only executed path - analyze only important path (Ernst, 2003).

Hence, it reduces the time and false positive (Graham, Leroux, & Landry, 2008), (Cornell, 2008) and (Goichi et al, 2005).

3. Input dependent analysis and therefore not specifics to any C overflow vulnerabilities classes (Haugh & Bishop, 2003) and (Liang & Sekar, 2005)

Review of Literature – Dynamic AnalysisDynamic analysis:1. Analyze program during program’s execution (Li and Chiueh, 2007)2. Due to that, it only analyze only executed path (Ernst, 2003)3. Tools that implement dynamic analysis – SigFree (Wang et al 2008), StackGuard (Lhee &

Chapin, 2002), StackShield (Lhee & Chapin, 2002), CCured (Sidiroglou & Keromytis, 2004), etc.

Weakness:1. Frame-pointer dependent; for instance LibSafe (Lhlee & Chaplin, 2002) and (Sidiroglou &

Keromytis, 2004).2. Share the same issues as static analysis such as:

1. Annotation dependent (Newsome & Song, 2005) and (Zhivich, Leek & Lippmann, 2005)2. Limited to certain C overflow vulnerabilities classes (Sidiroglou & Keromytis, 2004),

(Akritidis et al, 2008), (Zhivich et al, 2005), (Sidiroglou & Keromytis, 2004), (Wang et al, 2008) and (Newsome & Song, 2005), (Lhee & Chapin, 2002), (Liang & Sekar, 2005), (Kratkiewicz & Lippmann, 2005), (Haugh & Bishop, 2003) and (Cornell, 2008).

3. Overhead analysis (Li & Chiueh, 2007), (Wang et al, 2008), (Rinard et al, 2004), (Lhee & Chapin, 2002), (Liang & Sekar, 2005), (Sidiroglou & Keromytis, 2004), (Aggarwal & Jalote, 2006), (Akritidis et al, 2008), (Graham et al, 2008), (Ernst, 2003), (Goichi et al, 2005) and (Zhivich et al, 2005).

4. Contributes to DoS or DDoS attack (Wang et al, 2008) and (Evans and Larochelle, 2002).5. Test Case dependent (Aggarwal & Jalote, 2006), (Graham et al, 2008) and (Cornell, 2008)6. Detection is based on known execution path (Ernst, 2003), (Goichi et al, 2005)

Review of Literature – Static versus Dynamic AnalysisNo. Static Dynamic Choosen

1. Ability to analyze unfinished code, fraction of complete system or performs unit test

Able to analyze complete compiled code or finished system.

Static

2. Detect at early stage and modified code before completion (Shapiro, 2008)

Detect after system is released and requires complete cycle of system modifications

Static

3. Cost effective (Shapiro, 2008), (Verifysoft Technology GmbH, 2013) and (Terry & Levin, 2006)

Cost of changing code is 100 times (Shapiro, 2008) and (Graham et al, 2008)

Static

4. Detection effectiveness is still an issue especially with numerous weaknesses in the methods

Yet to prove its ability and there is no guarantee on its effectiveness (Goichi et al, 2005) and (Ernst, 2003)

Draw

• Static Analysis is preferred analysis technique, hence this research focus on issues behind static analysis

Methods Tools Understanding/Knowledge

Review of Literature – Taxonomy and Classifications

1. Vulnerabilities understanding is the process of educating and building the knowledge on vulnerabilities (Krsul, 1998).

2. A major step towards enhancement of tools and implementation for better defense mechanism (Krsul, 1998) and (Tsipenyuk, Chess, & McGraw, 2005).

3. Three areas related to improving understanding and knowledge:1. Guidelines2. Books3. Taxonomy

4. The taxonomy and classifications on software vulnerabilities (previously known as bugs) was started based on RISOS (Research in Secure OS) project by National Bureau of Standards (Abbot, et al., 1976) and PA (Program Analysis) by Information Science Institute, University of California (Bisbey & Hollingworth, 1978).

5. Well-defined taxonomy fulfills the set of well-defined criteria and be able to define the objects and field of studies without doubt (Igure & Williams, 2008) and (Axelsson, 2000)

A well-defined taxonomy has significant impact in having good guidelines and books especially to understand C overflow vulnerabilities (Igure & Williams, 2008)

Review of Literature – Taxonomy and Classifications- Criteria of well-defined taxonomy

1996

Bishop & Bailey1. Deterministic2. Specificity

1998

Howard & Longstaff1. Mutually Exclusive2. Exhaustive3. Unambiguous4. Repeatable5. Accepted6. Useful

Ivan Krsul1. Objectivity2. Determinism3. Repeatability4. Specificity.

1999

Bishop1. Mutually exclusive2. Exhaustive3. Unambiguous4. Repeatable5. Accepted Useful

2001

Lough1. Accepted [Howa1997]2. Appropriateness [Amor1994]3. Based on the code, environment, or other technical details [Bish1999]4. Comprehensible [Lind1997]5. Completeness [Amor1994]6. Determinism [Krsu1998]7. Exhaustive [Howa1997, Lind1997]8. Internal versus external threats [Amor1994]9. Mutually exclusive [Howa1997, Lind1997]10. Objectivity [Krsu1998]11. Primitive [Bish1999]12. Repeatable [Howa1997, Krsu1998]13. Similar vulnerabilities classified similarly [Bish1999]14. Specific [Krsu1998]15. Terminology complying with established security terminology [Lind1997]16. Terms well defined [Bish1999]17. Unambiguous [Howa1997, Lind1997]18. Useful [Howa1997, Lind1997]

2003

Vijayaraghavan1. Appropriate2. Comprehensible3. Specific4. Useful5. Expandable

Hannan, Turner & Broucek6. Specificity7. Exhaustive8. Deterministic9. Accepted10. Terminology sompliant

Hansmann11. Accepted12. Comprehensible13. Completeness14. Determinism15. Mutual Exclusive16. Repeatable17. Terminology compliant18. Terms19. Well-defined20. Unambiguous21. Useful

2004

Killourhy, Maxion and Tan1. Mutually Exclusivity2. Exhaustivity3. Replicability(based on Lough)

2005

Polepeddi1. Mutually exclusive2. Exhaustive3. Unambiguous4. Repeatability5. Acceptability6. Utility / Useful

Tsipenyuk et al7. Simplicity8. Mutually exclusive9. Intuitive10. Category name is less generic11. Specific

Berghe et al12. Mutually exclusive13. Deterministic14. Exhaustive15. Objective16. Useful

2006

Alhazmi, Woo and Malaiya1. Mutual Exclusiveness2. Clear and Unique definition3. Repeatability4. Covers all vulnerabilities (Comprehensive)

2008

Igure and Williams1. Specificity2. Ambiguous3. Layered or hierarchy4. Unique definition on each level

Review of Literature – Taxonomy and Classifications- Criteria of well-defined taxonomy

No. Researchers Definition of well-defined taxonomy Comments1 Bishop and Bailey

(1996)A taxonomy that is well-structured with well-defined procedures. Too few

2 Krsul (1998) A taxonomy that has characteristics to ensure classifications can be successfully done

Lack of completeness

3 Howard and Longstaff ( 1998)

Not available There are redundant and arguable criterion

4 Bishop (1999) Not available Criterions conflict each other, hence inconsistent.

5 Lough (2001) Refer to Krsul definition of well-defined taxonomy Collection of previous criteria, which consist of repetitive or irrelevant to certain fields of taxonomy.

6 Vijayaraghavan (2003) A good taxonomy should have a characteristics specific to its purposes.

Specific to e-commerce environment and ignore mutually exclusive criterion.

7 Hannan, Turner, and Broucek (2003)

A good forensic taxonomy should be able to leverage the strength of multiple disciplines on any forensic issues.

Focusing on forensic issue and ignore repeatability and mutually exclusive.

8 Hansmann (2003) Not available. Based on Lough’s criteria. Same issues as Lough

9 Killhourhy et. al. (2004) Should have sensible criteria but consistent with general terms of taxonomy.

Lack of objectivity and specificity.

10 Polepeddi (2005) Should reduce difficulties to users to study and manage the objects of studies

Lack of deterministic and objectivity.

11 Tsipenyuk et. al. (2005) Well-defined taxonomy must be simple and easy to understand. Lack of deterministic, objectivity and repeatability.

12 Berghe et. al. (2005) Not Available Lack of specificity, repeatability and accepted criterion.

13 Igure and Williams (2008)

A good taxonomy provides a common language for the study of the field and classified according to set of rules.

Allowed ambiguity which causing non-repetitive implementation.

Review of Literature – Taxonomy and Classifications- Previous works on taxonomy or classificationsNo Experts Purpose Criterions failed to

fulfilImpact C overflows

vulnerabilities1 Lough (Lough,

2001)To ease security experts to design security protocol for network communications.

Unambiguous and repeatability

Confusing to user and hence resulting in different understanding for the same vulnerabilities.

Not available

2 Piessens (2002) As guidelines to software developers, tester and security implementer.

Unambiguous, specificity and repeatability


Not available

3 Vijayaraghavan (2003)

To educate and help security practitioners in developing a suitable test case

Unambiguous, specificity, useful and repeatability,

Does not specified web applications vulnerabilities and not practical due to confusing and ambiguity

Not available

4 Hannan et al (2003)

To leverage multi-disciplines in information security for forensic process.

Obviousness and completeness.

Not useful and difficult to apply in forensic computing

Not applicable.

5 Hansman (2003)

As guidelines for security expert

Specificity and repeatability

Confuse and causing non-repeatable result.

Generic

6 Seacord and Householder (2005)

As guidelines to classify vulnerabilities based on behaviour for countermeasures

Specificity, repeatability, unambiguous and useful.

Not repetitive in vulnerability classifications process and does not improve understanding on vulnerabilities.

Generic

7 Weber et al (2005)

As reference for security developers to develop security analysis tool.

Unambiguous, useful and repeatability

Difficult to understand and repeating the results.

Generic

8 Tsipenyuk et al (2005)

As reference for software developers to avoid vulnerabilities in coding

Unambiguous, repeatability and specific.

Too general and covers too many languages and therefore affecting repeatability.

Generic but covers almost all C overflow vulnerabilities classes



vulnerabilities1 Lough (Lough,

2001)To ease security experts to design security protocol for network communications.

Unambiguous and repeatability


Not available

2 Piessens (2002) As guidelines to software developers, tester and security implementer.

Unambiguous, specificity and repeatability


Not available

3 Vijayaraghavan (2003)

To educate and help security practitioners in developing a suitable test case

Unambiguous, specificity, useful and repeatability,

Does not specified web applications vulnerabilities and not practical due to confusing and ambiguity

Not available

4 Hannan et al (2003)

To leverage multi-disciplines in information security for forensic process.

Obviousness and completeness.

Not useful and difficult to apply in forensic computing

Not applicable.

5 Hansman (2003)

As guidelines for security expert

Specificity and repeatability


Generic

6 Seacord and Householder (2005)

As guidelines to classify vulnerabilities based on behaviour for countermeasures

Specificity, repeatability, unambiguous and useful.

Not repetitive in vulnerability classifications process and does not improve understanding on vulnerabilities.

Generic

7 Weber et al (2005)

As reference for security developers to develop security analysis tool.

Unambiguous, useful and repeatability

Difficult to understand and repeating the results.

Generic

8 Tsipenyuk et al (2005)

As reference for software developers to avoid vulnerabilities in coding

Unambiguous, repeatability and specific.

Too general and covers too many languages and therefore affecting repeatability.

Generic but covers almost all C overflow vulnerabilities classes

No. Experts Purpose Criterions failed to fulfil

Impact C overflows vulnerabilities

9 Sotirov (2005)

To construct and evaluate static analysis tools.

Unambiguous, repeatability and completeness

Failed to improve understanding on vulnerabilities and behaviour that triggers exploitation.

Out-of-bound, format string, and integer overflow, memory function, variable conversion (signed to unsigned)

10 Berghe et al (2005)

To evaluate constructed methodology for producing vulnerabilities taxonomy

Unambiguous, repeatability and completeness

Different users produce different result and affecting usefulness of the taxonomy.

Generic

11 Gegick and Williams (2005)

To identify and classify vulnerabilities in report or advisories

Repeatability and useful. User will have difficulties in classifying vulnerabilities as there is no specific class and affecting classifying consistency.

Generic

12 Alhazmi et. al. (2006)

To identify and classify exploitable vulnerabilities for security enhancement

Completeness Users tends to wrongly classify the studied object or failed to classify at all. This will affect their understanding and knowledge.

Generic

13 Bazaz and Arthur (2007)

To analyze software security

Repeatability, completeness, unambiguous and obvious.


Generic

14 Shahriar and Zulkernine (2011)

To ease monitoring program vulnerability exploitation

Repeatability, completeness, unambiguous and obvious.


Generic



vulnerabilities1 M. Zitser (2003)

To evaluate static analysis tool

Unambiguous, completeness and repeatability


Array out-of-bound and unsafe functions

2 K. Kratkiewicz (2005)





3 M. A. Zhivich (2005)

To evaluate dynamic analysis tool.


Limited vulnerabilities and for the purpose of assessing dynamic tools and not for understanding C overflow vulnerabilities.


4 A. I. Sotirov (2005)


Repeatability and completeness.

Lack of classes and has a class for unknown vulnerabilities which resulted inconsistency and hence does not ease the understanding.

Array out-of-bound, integer overflow and unsafe functions

5 H. D. Moore (2007)

For studies and understanding of overflow vulnerabilities

Specificity, completeness and repeatability


Array out-of-bound and integer overflow

Review of Literature – Summary of Review

1. There are gaps in software security areas pertaining to software vulnerabilities issues specifically to C Overflow Vulnerabilities:

1. Analysis methods and tools2. Security implementation and policies3. Understanding/knowledge on software vulnerabilities

2. As mentioned by Krsul (1998) and Tsipenyuk et al (2005), improving knowledge and understanding on vulnerabilities is a major step towards enhancement of tools and implementation for better defence mechanism. Hence, the focus is on taxonomy and classifications.

3. There are still no well-defined taxonomy constructed from source code perspective which consider developers point-of-view and covers all C overflow vulnerabilities classes

Research Question Research Objective



Review of Literature – Summary of ReviewResearch Question Research Objective



Review of Literature – Summary of Review

Research Framework and Phases Research Activities

Research Methodology

Research Methodology – Research Framework & Phases

Research Methodology – Research Framework & Phases

Theoretical Studies

Taxonomy Construction

Taxonomy Evaluation

Research Methodology – Research Framework & PhasesPhases

Section

Phase 1 – Theoretical Studies

Phase 2 – Taxonomy Construction

Phase 3 – Taxonomy Evaluation

Research Question (RQ)

RQ 1: Why C overflow vulnerabilities still persist although it is common and known for more than two decades?

RQ 2: How to identify and improve understanding of C overflow vulnerabilities and prevent from occurring again?

RQ 4: Is the taxonomy effectives in improving understanding of C overflow vulnerabilities?RQ 5: Is the taxonomy comprehensive and useful?

Research Objectives (RO)

RO 1: To identify strength and weaknesses of current mechanisms in detecting and preventing C overflow vulnerabilities from occurring RO 2: To identify list of C overflows vulnerabilities class

RO 3: To compile and construct criteria for well-defined taxonomy RO 4: To construct taxonomy specifically addressing C overflow vulnerabilities

RO 5: To evaluate taxonomy based on criteria RO 6: To evaluate the criticality and relevancies of C overflows vulnerabilities based on the taxonomy

Phase Deliverables / Output (RR)

RR 1: Strength and weaknesses of current detection and prevention mechanism RR 2: List of C overflow vulnerabilities classes

RR 3: Criteria of well-defined taxonomy RR 4: Taxonomy of C overflow vulnerabilities exploit

RR 5: Taxonomy validatedRR 6: Significant findings of the research

Research Methodology – Research Activities

1. Pre-analysis on vulnerabilities and information security issues

2. In-depth review on software vulnerabilities

3. Critical review:3.1 C overflow vulnerabilities and exploitation3.2 C overflow vulnerabilities defensive mechanism3.3 C overflow vulnerabilities detection and prevention mechanism

4. Critical review:4.1 Static Analysis techniques and tools4.2 Dynamic Analysis techniques and tools

5. Critical review:5.1 Vulnerabilities taxonomy5.2 Criteria of well-defined taxonomy

Start

Is the theoretical studies

comprehensive?

6. Conclude the theoretical studies phase.

End

No

Yes

Phase 1: Theoretical Studies

Research Methodology – Research Activities

Start

1. Critical review on relevant publications

2. Extracted the criteria for constructing taxonomy

3. Detail analysis on the identified criteria

4. Construct criteria for well-defined taxonomy

5. Review the constructed criteria for well-defined taxonomy

Completed review? Criteria

satisfied?

End

No

Yes

No

Yes

Phase 2: Taxonomy Construction – Construct well-defined criteria

Research Methodology – Research ActivitiesPhase 2: Taxonomy Construction – Construct Taxonomy

Start

1. Critical review on relevant reports

2. Formation of Classes

3. Detail analysis on related publications

4. Organized and constructed the taxonomy

5. Review the constructed taxonomy with all reports and publications

Taxonomy satisfied?

End

No

Yes

Start

1. Evaluate the taxonomy against the constructed well-defined criteria

2. Measure the criticality and significant of each identified class.

3. Measure the criticality of OS and vulnerabilities exploitation impact.

4. Evaluate the static analysis tools effectiveness in detecting the identified classes

5. Evaluate and verify the static analysis tools effectiveness and efficiencies in analyzing open-source code for comparative analysis against earlier works

Evaluation satisfied?

End

No

Yes

Research Methodology – Research ActivitiesPhase 3: Taxonomy Evaluation

Research Methodology – Research ActivitiesStart

1. Suitable tester is selected

2. Select 100 advisories/reports

3.1.2 Tester matched the vulnerability in advisories with class in taxonomy End

First iteration

?

3.1.1 Shared the taxonomy with tester

3. Collected and compiled the result

3. Measured and presented the result

Second iteration

?

Third iteration

?

3.2.1 Explain and educate the tester on the taxonomy

3.2.2 Tester matched the vulnerability in advisories with class in taxonomy

3.3.1. Select a different set of advisories or reports.

3.3.2 Tester used the same taxonomy and matched the vulnerabilities in the report. No guidance provided.

No

Yes Yes Yes

No No

Phase 3: 3.1.1 Evaluation on Taxonomy against the well-defined criteria - Measurement on Effectiveness and Completeness


1. Critical review on relevant reports

2. Formation of Classes

3. Detail analysis on related publications

4. Organized and constructed the taxonomy

5. Review the constructed taxonomy with all reports and publications

Taxonomy satisfied?

End

No

Yes

Phase 3: 3.1.2 Evaluation on Taxonomy against well-defined criteria - with Selected Well-known Online Vulnerability Database and Organization

Research Methodology – Research ActivitiesPhase 3: 3.2 Evaluation on the Significances and Relevancies of Classes Defined in the Taxonomy of C Overflow Vulnerabilities Exploit

Start

1. Get the class and its characteristics from the taxonomy

2. Searched the selected databases and organization based on the class name and its characteristics.

End

Found advisories or report?

More search

synonym?

3. Critical review on the advisory/report.

Complete 10

iteration?

4. Conclude and map the result in table.

Complete all class?

5. Compiled and analyzed the result.

Complete all databases or organization?

No

Yes

No

No

No

No

Yes

Yes

Yes

Yes

Research Methodology – Research ActivitiesPhase 3: 3.3 Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality

Start

1. Identify the relevant OS.

End

Complete execute all test

cases?

2. Develop vulnerable programs based on the classes in the taxonomy.

4. Execute the activities.

5. Compile result and perform analysis

No

Yes

3. Construct test activities for OS evaluation.


1. Critically reviewed and identified static analysis tools

End

Complete analyze all programs?

2. Select static analysis tools from the identified list

4. Execute the selected static analysis tool with the selected program as input.


No

Yes

3. Select a program to analyze.

All static analysis tool

used?

No

Yes

Phase 3: 3.4 Evaluate the static analysis tools effectiveness in detecting the identified classes


1. Critically reviewed and identified open-source codes/applications.

End

Complete analyze all programs?

2. Prepare the machine to be used for the testing

5. Execute the selected static analysis tool with the selected program as input.


No

Yes

4. Select a program to analyze from identified list.

All static analysis tool

used?

No

Yes

3. Select the static analysis tool from identified list

6. Get the result and normalize the machine.

Phase 3: 3.5 Evaluate the Static Analysis Tools Efficiencies in Analysing Open-source Code for Comparison with Previous Works

Research Methodology – Summary

1.Pre analysis on vulnerabilities and information security issues/cases.

2.In-depth review on software vulnerabilities

3.Critical review on C overflows vulnerabilities, its current detection and prevention mechanism

4.Critical review on program analysis techniques and tools focusing on static analysis

5.Critical review on vulnerabilities taxonomy

1.A thorough, careful and significant review on vulnerabilities taxonomy covering the:

i. criteria of well-defined taxonomy,

ii. vulnerabilities taxonomy, and

iii. C overflow vulnerabilities taxonomy

2.Identified and formulate the criteria of well-defined taxonomy

3.Construct ‘C Overflow Vulnerabilities Attack’ taxonomy from source code perspectives.

1.Evaluate the taxonomy against the well-defined criteria.

2.Measure the criticality and significant of each identified class.

3.Measure the criticality of OS and vulnerabilities exploitation impact.

4.Evaluate the static analysis tools effectiveness in detecting the identified classes

5.Evaluate and verify the static analysis tools effectiveness and efficiencies in analyzing open-source code for comparative analysis against earlier works.

6.Analyze the findings

Theoretical Studies

Taxonomy Construction

Taxonomy Evaluation

Taxonomy Construction Taxonomy Evaluation

Result and Discussions

Result and Discussions: Taxonomy Construction – Criteria for Well-Defined TaxonomyNo Criteria Description Purpose1. Simplicity Simplified collection of studied objects or subject into

readable diagram or structuresTo ease in understanding the studied objects or subject.

2. Organized structures

Organized into viewable, readable, and understandable format.

To demonstrate the relationship between objects and ease the process of understanding.

3. Obvious Objective is clear, measureable, and observable without doubt. Process flow is clear and easily followed. Structure or format is easily understood and able to stand on its own.

To ease the process of classifications.

4. Repeatability Result of classifying studied object by any independent user can easily be duplicated by others.

For consistency purposes and reliability of the result.

5. Specificity / Mutual exclusive / Primitive

Value of object must be specific and explicit. Object of study must belong to only one class.

To remove the ambiguity, ease the classification process and support repetitive result.

6. Similarity Object in the same class must have similar behavior or characteristics.

For consistency, repeatability and ease the understanding of studied objects.

7. Completeness Able to capture all objects studied without any doubt no matter when it is applied.

To ensure user are able to apply any new object into the classification whenever required without any doubt.

8. Knowledge compliant

Built using known existing terminology. To ease learning and classifying.

Result and Discussions: Taxonomy Construction – C Overflow Vulnerabilities Exploit Taxonomy

Taxonomy of C Overflow Vulnerabilities Exploit

Unsafe Functions

Array Out-of-Bound

Integer Range/Overflow

Return-into-libc

Memory Function

Function Pointer / Pointer Aliasing

Variable Type Conversion

Pointer Scaling / Pointer Mixing

Uninitialized Variable

Null Termination

Result and Discussions: Taxonomy Construction – C Overflow Vulnerabilities Exploit Taxonomy

No Unsafe Functions Array out-of-bound Return-into-libc1 Use of unsafe C/C++

functionsUse of array Use a pointer to string and

does not requires unsafe functions or array.

2 Used input either by user or by system that is pass into the function as input

Involves index which is supplied by an operation in a program

Use a pointer to point to a string of character

3 Unsafe function is executed to trigger the overflow

To trigger overflow, index used is beyond the upper and lower bound of the array

The string of character contains new system or function call or new memory addressing or large string to overflow the next reference address until it reach the new intended memory addressing which contains a function call based on available functions in libC.

Result and Discussions: Taxonomy EvaluationEvaluation 1: Result of evaluating the effectiveness and completeness in classifying vulnerabilities using the taxonomy

No. Methods Tester Result Successful mapping rate (%)

S M F

1 5 selected tester perform the classifications using the taxonomy without guidance on 100 vulnerabilities reports

Tester 1 60 5 35 0.60Tester 2 51 28 21 0.51Tester 3 71 14 15 0.71Tester 4 68 5 27 0.68Tester 5 80 9 11 0.80

Average successful rate of using taxonomy to classify vulnerabilities 0.662 Using the same reports and tester but

with guidance and explanation on taxonomy


Average successful rate of using taxonomy to classify vulnerabilities 0.9143 Using the same tester but the tester

was given different sets of vulnerabilities reports from previous test.


Average successful rate of using taxonomy to classify vulnerabilities 0.882Average success 0.819

Result and Discussions: Taxonomy EvaluationEvaluation 2: Result of Comparison of Taxonomy with Advisories/Reports for Completeness Measurement

Vulnerabilities Database

WebsiteClass of C OverflowVulnerabilities

NIST OWASP OSVDB Symantec Microsoft SecuniaDept of

Homeland Security

% Matches

Unsafe Functions Yes Yes Yes Yes Yes Yes Yes 100%Array Out-of-bound Yes Yes Yes Yes Yes Yes Yes 100%Integer Range/overflow

Yes Yes Yes Yes Yes Yes Yes 100%

Return-into-LibC Yes Yes Not Applicable

Not Applicable

Yes Not Applicable

Yes 100%

Memory Function Yes Yes Yes Yes Yes Yes Yes 100%Function Pointer / Pointer Aliasing

Yes Yes Not Applicable

Yes Yes Yes Yes 100%


Yes Yes Not Applicable

Not Applicable

Yes Yes Yes 100%


Yes Yes Yes Not Applicable

Yes Yes Yes 100%

Uninitialized Variable Yes Yes Yes Yes Yes Not Applicable

Yes 100%

Null Termination Yes Yes Yes Yes Yes Yes Yes 100%Unknown class (Unable to match with the taxonomy)

No No No No No No No 0%

Result and Discussions: Taxonomy EvaluationEvaluation 3: Evaluation on Relevancies and Significant of Classes in C Overflow Vulnerabilities Exploit Taxonomy - Severity


C overflow vulnerabilities class


Homeland Security

Unsafe functions High High High High High High HighArray out-of-bound High High High High High High HighInteger range/overflow

High High High High High High High

Return-into-LibC High Medium NA NA Medium NA MediumMemory Function High Medium Medium High Medium Medium HighFunction Pointer / Pointer Aliasing

Medium Medium NA Low Medium High Medium


Medium Medium NA NA Low Medium Medium


High Medium Low NA High High High


Medium Low Low Low Low Low Medium

Null Termination Medium Low Low Medium Low Medium Medium

Result and Discussions: Taxonomy EvaluationEvaluation 3: Evaluation on Relevancies and Significant of Classes in C Overflow Vulnerabilities Exploit Taxonomy - Persistency




Homeland Security

Unsafe functions 2013 2013 2013 2013 2013 2013 2013Array out-of-bound

2014 2013 2012 2012 2013 2013 2013

Integer range/overflow

2013 2012 2012 2012 2013 2013 2013

Return-into-LibC 2013 2012 NA NA 2013 NA 2010Memory Function 2014 2012 2012 2012 2013 2013 2012Function Pointer / Pointer Aliasing

2013 2010 NA 2010 2013 2013 2010


2013 2010 NA NA 2013 2010 2010


2010 2009 2009 NA 2009 2009 2009


2013 2012 2012 2012 2013 2013 2012

Null Termination 2009 2009 2009 2009 2009 2009 2009

Result and Discussions: Taxonomy EvaluationEvaluation 3: Evaluation on Relevancies and Significant of Classes in C Overflow Vulnerabilities Exploit Taxonomy - Impact




Homeland Security

Unsafe functions High High High High High High HighArray out-of-bound


Integer range/overflow


Return-into-LibC High Medium NA NA High NA MediumMemory Function High High High High High High HighFunction Pointer / Pointer Aliasing

Medium High NA Medium Medium Medium Medium


Medium Medium NA NA Medium Medium Medium


Medium Medium High NA High High High


Medium High Medium Medium Medium Medium Medium

Null Termination Medium Medium Medium High Medium Medium Medium

Result and Discussions: Taxonomy EvaluationEvaluation 4: Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality- Significance and Relevancies

Operating System


Windows XP 32-bits

Windows 7 32-bits

Windows 7 64-bits

Linux (Centos 5.5) 32-bits

Unsafe functions Yes Yes Yes Yes

Array out-of-bound Yes Yes Yes Yes

Integer range/overflow Yes Yes Yes Yes

Return-into-LibC Yes Yes Yes Yes

Memory Function Yes Yes Yes Yes

Function Pointer / Pointer Aliasing Yes Yes Yes Yes

Variable Type Conversion Yes Yes Yes Yes

Pointer Scaling / Pointer Mixing Yes Yes Yes Yes

Uninitialized Variable Yes Yes Yes Yes

Null Termination Yes Yes Yes Yes

Legend: Vulnerable (Is the OS vulnerable): Yes – vulnerable, No – Not vulnerableDifficulties (Level of difficulties to exploit): L – Low, M – Medium, H – Very difficult to exploit, NA – Not Applicable

Result and Discussions: Taxonomy Evaluation

Types of C overflows vulnerabilities attack

Windows XP 32-bits Windows 7 32-bits Windows 7 64-bits Centos 5.5 Linux 32-bits

V D V D V D V D

Unsafe Functions Yes L Yes L Yes M Yes LArray Out-of-bound Yes L Yes L Yes M Yes LInteger Range/overflow Yes L Yes L Yes M Yes LReturn-into-LibC Yes H Yes H Yes H Yes HMemory Function Yes H Yes H Yes H Yes HFunction Pointer / Pointer Aliasing

Yes M Yes H Yes H Yes M

Variable Type Conversion Yes M Yes M Yes H Yes MPointer Scaling / Pointer Mixing

Yes H Yes H Yes H Yes H

Uninitialized Variable Yes L Yes L Yes L Yes LNull Termination Yes L Yes L Yes M Yes L

Evaluation 4: Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality- Vulnerable and Difficulties


Result and Discussions: Taxonomy Evaluation


Evaluation 4: Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality- Input and Outcome for Windows XP 32-bitsClass of C overflows vulnerabilities exploit

Input characteristics Effect on system / outcome

Unsafe Functions Few characters more than given size

Overflow , loop without ending or throw memory exception.

Array Out-of-bound Few characters more than given size

Overflow happen but program continue. Printed empty space, adding last character ('\n') and print double spacing instead of one.

Integer Range/overflow

Variable except integer larger than n32.

Overflow happen.

Return-into-LibC No specific length Nothing happen and the program continues.

Memory Function Depends on type of functions

memset() triggers overflows whereas double free() will result in vulnerable for exploitation.


Few characters bigger than given size

Display weird characters. Length of second variable extended to the same size of first variable. Possible use for exploitation to trigger overflow


Larger integer to small character

Larger integer will force to display weird character. If the conversion is from int to char and then to int again; overflow happen and vulnerable for exploitation.


No specific length A memory violation is shown (overflow). For struct pointer, it will still assign memory address which later can be use to exploit by attackers.

Uninitialized Variable Not requires any length Assign a value to the variable and memory address, which can be used for exploitation.

Null Termination Length is equivalent to defined length

Nothing happen on the system and no additional characters added.

Result and Discussions: Taxonomy EvaluationEvaluation 4: Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality- Input and Outcome for Windows 7 32-bits

Class of C overflows vulnerabilities exploit

Length of attacks Effect on system / outcome

Unsafe Functions Few characters more than given size

Overflow, loop without ending or throw memory exception

Array Out-of-bound

Few characters more than given size

Overflow happen but program continue. Printed empty space, adding last character ('\n') and print double spacing instead of one.



Overflow happen.

Return-into-LibC No specific length Nothing happen and the program continues.Memory Function

Depends on type of functions

memset() triggers overflows whereas double free() will result in vulnerable for exploitation.



Display weird characters. Length of second variable extended to the same size of first variable. Possible use for exploitation to trigger overflow



Display weird character, convert to negative integer (unsigned value) thus becoming overflow and vulnerable for exploitation.


No specific length A memory violation is shown (overflow).


Not requires any length Assign a value to the variable and memory address, which can be used for exploitation.



Result and Discussions: Taxonomy EvaluationEvaluation 4: Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality- Input and Outcome for Windows 7 64-bits



Unsafe Functions Few characters more than the size of n64.

Overflow when the input to first variable is larger than n64 bit sizes.


Overflow happen but program continue.



Overflow happen.


Memory Function Depends on type of functions Memory error violation (overflow) and after few processes, the program aborted.



Nothing happen.



Display weird character, convert to negative integer (unsigned value) thus becoming overflow and vulnerable for exploitation.


No specific length The system will still assign value and memory address which later can be used to exploit by attackers.




Result and Discussions: Taxonomy EvaluationEvaluation 4: Evaluation on Significant and Relevancies of C Overflow Vulnerabilities Classes and Impact to OS Criticality- Input and Outcome for Linux (Centos 5.5) 32-bits



Unsafe Functions Few characters more than the size of n32.

Overflow only happen when the input to first variable is larger than n32 bit sizes.


Overflow happen but program continue.

Integer Range/overflow Variable except integer larger than n32.

Overflow happen.


Memory Function Depends on type of functions

Segmentation fault, which is equivalent to memory overflow. Program aborted.



Overflow on first variable after copy second variable with value from first variable



Conversion successful if the given integer is within range of character. Larger integer will force to display weird character.


No specific length The system will still assign value and memory address which later can be used to exploit by attackers.




Result and Discussions: Taxonomy EvaluationEvaluation 5: Evaluation on Static Analysis Tools Effectiveness in Detecting Vulnerabilities based on C Overflow Vulnerabilities Exploit Taxonomy

Vulnerability class Program Types SPLINT RATS ITS4 BOON BLASTUnsafe Function Intra-procedural Yes Yes Yes Yes Yes

Inter-procedural Yes Yes Yes Yes YesArray Out-of-bound Intra-procedural Yes Yes Yes Yes Yes

Inter-procedural Yes Yes Yes Yes YesInteger Ranger / Overflow

Intra-procedural Yes Yes Yes Yes YesInter-procedural Yes Yes Yes Yes Yes

Return-into-LibC Intra-procedural No No No No NoInter-procedural No No No No No

Memory Function Intra-procedural Yes Yes Yes Yes YesInter-procedural Yes Yes Yes Yes Yes


Intra-procedural No No No No YesInter-procedural No No No No Yes


Intra-procedural No No No No YesInter-procedural No No No No Yes


Intra-procedural No Yes No No YesInter-procedural No Yes No No Yes

Uninitialized Variable Intra-procedural No No No No NoInter-procedural No No No No No

Null Termination Intra-procedural No No No No NoInter-procedural No No No No No

Result and Discussions: Taxonomy EvaluationEvaluation 6: Evaluation on Static Analysis Tools Effectiveness in Detecting Vulnerabilities on Open-Source Code

Tools Program Time(minutes)

Detect? False alarm True alarm Detection rate

SPLINT Apache HTTPD 280 5 5 0 0%Google Chrome 180 8 7 1 12.5%MySQL CE 300 9 7 2 22.2%Open VM Tool 130 8 5 3 37.5%TPM Emulator 20 4 4 0 0%

RATS Apache HTTPD 270 6 6 0 0%Google Chrome 200 9 6 3 33.3%MySQL CE 330 15 13 2 13.3%Open VM Tool 110 5 4 1 20%TPM Emulator 35 6 6 0 0%

ITS4 Apache HTTPD 290 8 7 1 12.5%Google Chrome 230 11 10 1 9.0%MySQL CE 310 13 8 5 38.5%Open VM Tool 100 9 7 2 22.2%TPM Emulator 20 6 6 0 0%

Boon Apache HTTPD 315 6 6 0 0%Google Chrome 190 5 4 1 20%MySQL CE 400 9 6 3 33.3%Open VM Tool 170 13 12 1 7.7%TPM Emulator 25 13 11 2 15.4%

BLAST Apache HTTPD 650 24 21 3 12.5%Google Chrome 490 14 11 3 21.4%MySQL CE 410 29 19 10 34.5%Open VM Tool 320 23 8 15 65.2%TPM Emulator 80 14 4 10 71.4%

Result and Discussions: SummaryPhases

Section

Phase 2 – Taxonomy Construction Phase 3 – Taxonomy Evaluation



RQ 4: Is the taxonomy effectives in improving understanding of C overflow vulnerabilities?

RQ 5: Is the taxonomy comprehensive and useful?


RO 3: To compile and construct criteria for well-defined taxonomy

RO 4: To construct taxonomy specifically addressing C overflow vulnerabilities

RO 5: To evaluate taxonomy based on criteria

RO 6: To evaluate the criticality and relevancies of C overflows vulnerabilities based on the taxonomy


RR 3: Criteria of well-defined taxonomy

RR 4: Taxonomy of C overflow vulnerabilities exploit

RR 5: Taxonomy validated

RR 6: Significant findings of the research

1. Consolidate and construct a list of criterion for well-defined taxonomy

2. Construct C Overflow Vulnerabilities Exploit Taxonomy

3. Perform 5 Evaluation which presented in 6 evaluation results

Based on the evaluations, it is concluded that1. The taxonomy is well-defined and inline will the criteria

2. The evaluations proved that the classes in the C Overflow Vulnerabilities Exploit taxonomy is significant and relevant

Conclusion

Phases

Section

Phase 1 – Theoretical Studies Phase 2 – Taxonomy Construction

Phase 3 – Taxonomy Evaluation


RQ 1: Why C overflow vulnerabilities still persist although it is common and known for more than two decades?


RQ 4: Is the taxonomy effectives in improving understanding of C overflow vulnerabilities?

RQ 5: Is the taxonomy comprehensive and useful?


RO 1: To identify strength and weaknesses of current mechanisms in detecting and preventing C overflow vulnerabilities from occurring

RO 2: To identify list of C overflows vulnerabilities class

RO 3: To compile and construct criteria for well-defined taxonomy

RO 4: To construct taxonomy specifically addressing C overflow vulnerabilities

RO 5: To evaluate taxonomy based on criteria

RO 6: To evaluate the criticality and relevancies of C overflows vulnerabilities based on the taxonomy


RR 1: Strength and weaknesses of current detection and prevention mechanism

RR 2: List of C overflow vulnerabilities classes

RR 3: Criteria of well-defined taxonomy

RR 4: Taxonomy of C overflow vulnerabilities exploit

RR 5: Taxonomy validated

RR 6: Significant findings of the research

Conclusion

Conclusion1. C Overflow Vulnerabilities is still relevant2. There is NO well-defined taxonomy specifically focusing on complete C Overflow

Vulnerabilities from source-code perspective for improvement of understanding and knowledge of C developers which looks into the root cause of the problem. Therefore, this is a taxonomy; “C Overflow Vulnerabilities Exploit” Taxonomy; that is proven from the evaluation done to be helpful and useful.

3. 5 evaluations done that shows the significant and relevancies of each classes in the constructed taxonomy

Recommendation1. To further develop a method using the taxonomy as guidelines for static analysis

tools improvement2. To further evaluate the effectiveness of the taxonomy

Nurul Haszeli AhmadStudent Id: 2009625912FSMK, UiTM Shah [email protected]

Supervisor: Dr Syed Ahmad Aljunid (FSMK, UiTM Shah Alam)Co-supervisor: Dr Jamalul-Lail Ab Manan (MIMOS Berhad)

mailto:[email protected]

c overflows vulnerabilities exploit taxonomy and evaluation on static analysis tools - mock viva for...

Education

vulnerabilities references

ranked vulnerabilities

software vulnerabilities

vulnerabilities csm

understanding vulnerabilities

cve details vulnerabilities

c overflow vulnerabilities

vulnerability threat