detecting and exploiting integer overflows

OutlineIntroduction to integer overflows

Automated detectionConclusion

Detecting and exploiting integer overflows

Guillaume TOURON

Laboratoire Verimag, Ensimag - Grenoble INPMarie-Laure Potet, Laurent Mounier

20/05/11

1 / 18

Introduction to integer overflowsContextBinary representationIntegers misinterpretation

Automated detectionStatic binary analysisData flow analysisImplementation

Conclusion

2 / 18

ContextBinary representationIntegers misinterpretation

Work subject

Subject

Binary code static analysis for vulnerabilities detection

I Focus on arithmetic problems

Application security is critical for information systems

I Programming bad practices

I Work with a professional environment : IDA Pro

I Develop some analysis to make easier vulnerabilities detection

3 / 18

Work subject

Subject

3 / 18

Work subject

Subject

3 / 18

Buffer overflow

4 / 18

Buffer overflow vulnerabilities

Exploitability

Integer overflow can lead to buffer overflowBuffer overflow can lead to arbitrary code execution

Integer overflows and buffer overflows top ranked by CWEExploitability (CWE):

I Buffer overflow: High to Very High (3rd)

I Integers overflow: Medium (16th)

Conclusion

We have to care about arithmetic overflow and avoid them

5 / 18

Exploitability

Conclusion

5 / 18

Exploitability

Conclusion

5 / 18

x86 integers binary representation

Basic C types on x86 32 bits:

char short int long intsigned [-128,127] [-32,768,32,767] [−231,231 − 1] [−263,263 − 1]unsigned [0,255] [0,65535] [0,232 − 1] [0,264 − 1]

Signed values representation

For negative values, MSB = 1 (2’s complement representation)

e.g −1 = 0xFFFFFFFF

6 / 18

x86 integers binary representation

Basic C types on x86 32 bits:

char short int long intsigned [-128,127] [-32,768,32,767] [−231,231 − 1] [−263,263 − 1]unsigned [0,255] [0,65535] [0,232 − 1] [0,264 − 1]

Signed values representation

For negative values, MSB = 1 (2’s complement representation)

e.g −1 = 0xFFFFFFFF

6 / 18

Dangerousness of misinterpreting

First issue

Small negative integers can be interpreted as huge integers

Dangerous cases:

I Sanity checks

I Copy operations

I Array indexations

Dangerous functions

Some famous functions: strncpy, strncat, snprintf, memcpy...These functions take a length unsigned parameter

7 / 18

First issue

Dangerous cases:

I Sanity checks

I Copy operations

I Array indexations

Dangerous functions

7 / 18

First issue

Dangerous cases:

I Sanity checks

I Copy operations

I Array indexations

Dangerous functions

7 / 18

First issue

Dangerous cases:

I Sanity checks

I Copy operations

I Array indexations

Dangerous functions

7 / 18

First issue

Dangerous cases:

I Sanity checks

I Copy operations

I Array indexations

Dangerous functions

7 / 18

memcpy example

void *memcpy(void *dest, const void *src, size t n);

⇒ What happens if this value is user-controlled?

Let’s take an example

#de f i n e LEN 512. . .v o i d vu l n ( char ∗ s r c , i n t s ) {

char ds t [ LEN ] ;i n t s i z e = s ;i f ( s < LEN) {

memcpy( dst , s r c , s i z e ) ;}

}. . .v u l n ( ”Test ” , −1);

8 / 18

memcpy example

void *memcpy(void *dest, const void *src, size t n);

⇒ What happens if this value is user-controlled?

Let’s take an example

#de f i n e LEN 512. . .v o i d vu l n ( char ∗ s r c , i n t s ) {

char ds t [ LEN ] ;i n t s i z e = s ;i f ( s < LEN) {

memcpy( dst , s r c , s i z e ) ;}

}. . .v u l n ( ”Test ” , −1);

8 / 18

Analysis

We have size = −1 (0xFFFFFFFF )CPU compares size and 512 as signed values

⇒ size < 512 == True

Vulnerability

But memcpy takes a unsigned argument, so size = 232 − 1By consequences, a buffer overflow occurs

A potential attacker can take control of flow execution

9 / 18

Analysis

Vulnerability

9 / 18

Analysis

Vulnerability

9 / 18

Static binary analysisData flow analysisImplementation

Pattern matching

Patterns

We look for interesting (= dangerous) patterns

Some patterns:I Calls to dangerous functions (memcpy, strncpy...)

I Search signed comparisons on unsigned parameters

I Dangerous instructions

r ep movsd

I Array indexation

movl $0x2a ,−0x2c(%ebp ,%eax , 4 )

10 / 18

Pattern matching

Patterns

r ep movsd

I Array indexation

10 / 18

Pattern matching

Patterns

r ep movsd

I Array indexation

10 / 18

Pattern matching

Patterns

r ep movsd

I Array indexation

10 / 18

Data dependencies

Looking for interesting data dependencies

I Sensitive parameters (e.g size from memcpy)

I Counter registers (e.g %ecx for rep prefixed instructions)

Analysis steps

I Scan code to find interesting dataI Sensitive parameters (e.g size for memcpy)

I Backtrack these data for dependenciesI Apply code patterns to exhib vulnerabilities

I Misinterpretation (e.g comparison as signed values)

11 / 18

Data dependencies

Analysis steps

11 / 18

Data dependencies

Analysis steps

11 / 18

Data dependencies

Analysis steps

11 / 18

Backward analysis

Dependencies

For a block B we have: OUT (B) =⋃

∀S∈Successors(B) IN(S)

12 / 18

Backward analysis

Transfer function

Computes new tainted variables set for a basic block B:

IN(B) = F B(StmSeq,OUT (B))

We must define a subset of x86 (grammar)⇒ Focus on instructions that imply dependencies

Examples:

I mov [ε|s|sx |zx ]

I Binary operations (add, addc, sub, sbb, and, xor, or...)

13 / 18

Backward analysis

Transfer function

Examples:

13 / 18

Backward analysis

Transfer function

Examples:

13 / 18

Environment

Several tools used:I Binary analysis environment

I IDA ProVery used in security industryPowerful, many features available

I CFG displayI Several plugins

I APII First, IDAPython

API for Python script in IDA ProI Then, Paimei Framework

Layer above IDAPython (easier to use)

14 / 18

Output example

Example on CVE-201-3970

15 / 18

Results

I Automation

I Customization

I False positive

Improvements:I Improve data-flow analysis

I Symbolic computation engine ?

I Add more dangerous code patternsI Allow users to write their own patterns

I Simple generic description language

16 / 18

Results

I Automation

I Customization

I False positive

16 / 18

Results

I Automation

I Customization

I False positive

16 / 18

General conclusion

Great subject, interesting people

First approach in researchI Documentation stage

I Backward analysisI Vulnerabilities examples

I Implementation experimentation

Use new tools, techniques and frameworks

17 / 18

18 / 18

detecting and exploiting integer overflows

Documents

arithmetic overflow

buffer overflowbuffer

buffer overflows

rdi integers overflow

professional environment

ida proi

cweexploitability cwe

laurent mounier2005111