Predicting Vulnerable Software Components

Stephan Neuhaus

Thomas ZimmermannAndreas Zeller

Security Advisory 2005-12

Title: Livefeed bookmarks can steal cookiesImpact: HighProducts: FirefoxDescription: Earlier versions of Firefox allowed javascript: and data: URLs as Livefeed bookmarks. When they updated the URL would be run in the context of the current page and could be used to steal cookies or data displayed on the page. If the user were on a page with elevated privileges (for example, about:config) when the Livefeed was updated, the feed URL could potentially run arbitrary code on the user's machine.

Security Advisory 2005-13

Title: Window Injection SpoofingSeverity: LowProducts: Firefox, Mozilla SuiteDescription: A website can inject content into a popup opened by another site if the target name of the popup window is known. An attacker who knows you are going to visit that other site could spoof the contents of the popup.

Security Advisory 2005-14

Title: SSL "secure site" indicator spoofingSeverity: ModerateProducts: Firefox, Mozilla SuiteDescription: Various schemes were reported that could cause the "secure site" lock icon to appear and show certificate details for the wrong site. These could be used by phishers to make their spoofs look more legitimate, particularly in windows that hide the address bar showing the true location.

Security Advisory 2005-15Title: Heap overflow possible in UTF8 to Unicode conversionSeverity: HighProducts: Firefox, Thunderbird, Mozilla SuiteDescription: It is possible for a UTF8 string with invalid sequences to trigger a heap overflow of converted Unicode data. Exploitability would depend on the attackers ability to get the string into the buggy converter. General web content is converted elsewhere but we can't rule out the possibility of a successful attack.

Security Advisory 2005-16Title: Spoofing download and security dialogs with overlapping windowsSeverity: HighProducts: Firefox, Mozilla SuiteDescription: Michael Krax demonstrates that the download dialog and security dialogs can be spoofed by partially covering them with an overlapping window. Some users may not notice the OS window border and browser statusbar bisecting what appears to be a single dialog, and be convinced by the spoofing text of the top-most window to click on the "Allow" or "Open" button of the window below.

Vulnerabilities0

Security Advisory 2005-41Title: Privilege escalation via DOM property overridesSeverity: CriticalProducts: Firefox, Mozilla SuiteDescription: moz_bug_r_a4 reported several exploits giving an attacker the ability to install malicious code or steal data, requiring only that the user do commonplace actions like click on a link or open the context menu. The common cause in each case was privileged UI code ("chrome") being overly trusting of DOM nodes from the content window.

Security Advisory 2006-76Title: XSS using outer window's Function objectImpact: HighProducts: Firefox 2.0Description: moz_bug_r_a4 demonstrated that the Function prototype regression described in bug 355161 could be exploited to bypass the protections against cross site script (XSS) injection, which could be used to steal credentials or sensitive data from arbitrary sites or perform destructive actions on behalf of a logged-in user.

Is this new component likely to be vulnerable?What other components are vulnerable?

Vulnerability Database

Version Archive

CodeCodeCodeCode

Component Component Component

Code

PredictorVulture

Code

Programmer Code Complexity

Language

Look for features that areinvariant under evolution

GUI Database Certificates OS

Imports

nsIContent.h

nsIContentUtils.h

nsIScriptSecurityManager.h

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘✔

nsIPrivateDOMEvent.h

nsReadableUtils.h

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘

✘✘

Research Questions

• How well do imports predict vulnerabilities?

• Can imports be used for classification (vulnerable or not) and for regression (number of vulnerabilities)?

Case Study: Mozilla

• CVS from January 4, 2007

• 14,368 C/C++ files

• 134 Security Advisories since January 2005

• Only 424 vulnerable components (4.05%)

⇒ Prediction is challenging

components in Mozilla

vulnerable components424

10,452

4.05%

Mozilla Vulnerabilities

security

nss

lib

libpkix

pkix_pl_nss

modu pki sy

pkix

top uti r

incl

ckfw

builtins ca

ns

freebl

mpi ecl

softoken

ssl

pk11wr

util certd smim

crmf

pki1

pki

pkcs12

certhig

pkcs de

jar cry

bas asn

cmd

zlib lib m pk si fips

crlu

pk

blt

ce c

S

t

manager jss

org

s

layout

generic

base

style xul

base

src

tr

tables

svg

base re

mathml

base

src

forms

prin in ht bu

js

src

xpconnect

src test

l

liveco

fdlib

tamarin

core

pcre code MM

shell pl e

jsd

mailnews

base

src util

search

src

imap

src

addrbook

src

mime

src

compose

src

import

outl

eud

src

oex

local

src

mapi

old ma

news

src

exten

palm

b s

db b

modules

oji

tests

src

JNI

C Arr

Ac

A C

sr

pu

plugin

tools

test s s

base

src

sam

def

S

libimg

png

rdf

src

libfont

jmcge

src

softupd

src

libpr0n

dec s

zlib

src

libre

src

libjar

libp

src

libb

src

xml

pro

s li

gfx

src

ps

gtk

xlib

windo

mac

os2

x11sh

theb xlib

qt phot

be

xpr

xp sh

f

cairo

cairo

src

glitz

src

libpixma

src

thebe

src

publi

publ

content

base

src p

html

content

src

doc

src

xslt

src

xslt xpath

xul

temp

src

doc

src

cont

svg

content

src

events

src

xbl

src

xml

d

xtf

can

extensions

canvas3d

src

xforms

webservice

soap

sche

pro

w i

java

xpcom

xmlterm

base line

python

xpco

spellch

src

walle

src

univ

src

sche

src

met

sr

sql

b

pre ins typ

aut

coo

xm

w s p

s

l

nsprpub

pr

src

md

wi uni ma

o b

misc

io

pthre

thr cp

li

tests

include

md p

li

t

li

xpcom

io glue

reflect

xptcal

src

x

s

ds

obsolete

c

string

pu sr

typelib

xpi x

base tests

build

MoreFi

compo

thr pr

s

widget

src

mac gtk2

windows

gtk

os2 beos

xpwi

xlib

qt ph

co g g

embedding

browser

activex

src

co

plu

co

pl

gtk

src

powerp

phot

web

qt

compon

printin

win

web

fi

qa

teste

tests

mfc w

directory

c-sdk

ldap

libraries

libldap

clie

exa

i

suncsdk

c-sdk

ldap

libraries cli i

xp

b

db

sqlite3

src

mork

src

tri

sr

md

ef

Compiler

Code

md

x

Front

Primi

Utilitie

Gener

zli qa

c

Runtim

Syste

C N C

gc

sr i

Tools

Pack

co D

Ex D

xpinstall

wizard

windows

setup uni

os2

setup

libxpne

GUSI

mac

unix

src

intl

uconv

ucvlat src

ucvcn

uti ucv

u

u

u t

unichar

src

locale

src

chardet

src

ctl

src

l s

netwerk

base

src

protocol

http

src

ftp

streamco

cache

test

dns

co

bui

s

java

webclient

src_moz

xpcom

te ja u

pluggab

wf

do

jni

plu

editor

libeditor

html base

text

txm

com

txtsv

toolkit

components

place

src

his s

do pa

airbag

airbag

xre m

xpfe

compone

sear boo

hi

st

bootstra

app

appshel

calendar

libical

src

libical

au

libic

test

base

js2

src

re

browser

components

places

src

migrat

boo s

parser

htmlparser

src p

expa

lib

tools

trace-

li

codes re

re

le

d

jp

f p

accessible

src

atk bas ht

ms

xu

dom

src

base

other-license

7zst

src

7zi

libart_

atk-1.

jpeg

msgsdk

C

protocol

cck

expat

driver

muc

ib

gc

boehm

c

plugin

oji

MRJ

plu

MRJ

pl

rdf

base

src

chro

d

docshell

base s

uriloader

extha b

camino

src

ipc

ipcd

e

lib

mac

profile

sr

config

mston

src

view

src

mail

com

buil

win

caps

src

dbm

s i

sun

stu

web

w

sto

sr

gcon

chro

mini

Mozilla Vulnerabilities

security

nss

lib

libpkix

pkix_pl_nss

modu pki sy

pkix

top uti r

incl

ckfw

builtins ca

ns

freebl

mpi ecl

softoken

ssl

pk11wr

util certd smim

crmf

pki1

pki

pkcs12

certhig

pkcs de

jar cry

bas asn

cmd

zlib lib m pk si fips

crlu

pk

blt

ce c

S

t

manager jss

org

s

layout

generic

base

style xul

base

src

tr

tables

svg

base re

mathml

base

src

forms

prin in ht bu

js

src

xpconnect

src test

l

liveco

fdlib

tamarin

core

pcre code MM

shell pl e

jsd

mailnews

base

src util

search

src

imap

src

addrbook

src

mime

src

compose

src

import

outl

eud

src

oex

local

src

mapi

old ma

news

src

exten

palm

b s

db b

modules

oji

tests

src

JNI

C Arr

Ac

A C

sr

pu

plugin

tools

test s s

base

src

sam

def

S

libimg

png

rdf

src

libfont

jmcge

src

softupd

src

libpr0n

dec s

zlib

src

libre

src

libjar

libp

src

libb

src

xml

pro

s li

gfx

src

ps

gtk

xlib

windo

mac

os2

x11sh

theb xlib

qt phot

be

xpr

xp sh

f

cairo

cairo

src

glitz

src

libpixma

src

thebe

src

publi

publ

content

base

src p

html

content

src

doc

src

xslt

src

xslt xpath

xul

temp

src

doc

src

cont

svg

content

src

events

src

xbl

src

xml

d

xtf

can

extensions

canvas3d

src

xforms

webservice

soap

sche

pro

w i

java

xpcom

xmlterm

base line

python

xpco

spellch

src

walle

src

univ

src

sche

src

met

sr

sql

b

pre ins typ

aut

coo

xm

w s p

s

l

nsprpub

pr

src

md

wi uni ma

o b

misc

io

pthre

thr cp

li

tests

include

md p

li

t

li

xpcom

io glue

reflect

xptcal

src

x

s

ds

obsolete

c

string

pu sr

typelib

xpi x

base tests

build

MoreFi

compo

thr pr

s

widget

src

mac gtk2

windows

gtk

os2 beos

xpwi

xlib

qt ph

co g g

embedding

browser

activex

src

co

plu

co

pl

gtk

src

powerp

phot

web

qt

compon

printin

win

web

fi

qa

teste

tests

mfc w

directory

c-sdk

ldap

libraries

libldap

clie

exa

i

suncsdk

c-sdk

ldap

libraries cli i

xp

b

db

sqlite3

src

mork

src

tri

sr

md

ef

Compiler

Code

md

x

Front

Primi

Utilitie

Gener

zli qa

c

Runtim

Syste

C N C

gc

sr i

Tools

Pack

co D

Ex D

xpinstall

wizard

windows

setup uni

os2

setup

libxpne

GUSI

mac

unix

src

intl

uconv

ucvlat src

ucvcn

uti ucv

u

u

u t

unichar

src

locale

src

chardet

src

ctl

src

l s

netwerk

base

src

protocol

http

src

ftp

streamco

cache

test

dns

co

bui

s

java

webclient

src_moz

xpcom

te ja u

pluggab

wf

do

jni

plu

editor

libeditor

html base

text

txm

com

txtsv

toolkit

components

place

src

his s

do pa

airbag

airbag

xre m

xpfe

compone

sear boo

hi

st

bootstra

app

appshel

calendar

libical

src

libical

au

libic

test

base

js2

src

re

browser

components

places

src

migrat

boo s

parser

htmlparser

src p

expa

lib

tools

trace-

li

codes re

re

le

d

jp

f p

accessible

src

atk bas ht

ms

xu

dom

src

base

other-license

7zst

src

7zi

libart_

atk-1.

jpeg

msgsdk

C

protocol

cck

expat

driver

muc

ib

gc

boehm

c

plugin

oji

MRJ

plu

MRJ

pl

rdf

base

src

chro

d

docshell

base s

uriloader

extha b

camino

src

ipc

ipcd

e

lib

mac

profile

sr

config

mston

src

view

src

mail

com

buil

win

caps

src

dbm

s i

sun

stu

web

w

sto

sr

gcon

chro

mini

Mozilla Vulnerabilities

js

src

xpconnect

src

livecon

content

base

src public

html

content

src

documen

src

xbl

src

xul

document

src p

content

src

templates

src

events

src

public

xslt

src

xslt base

xml

document

src

content

src

svg

conte

canvas

src

layout

xul

base

src

tree

src

grid

generic base

forms tables style

build

svg

mathm

printin

html

security

nss

lib

util

softoken

pki

ssl

cryptohi

free

pk1

cert

smi

cmd

manager

boot ssl

dom

src

base js

e

public

widget

src

mac windows

gtk

gtk2

xlib os2

qt phot beos

xpwi

pu

netwerk

protocol

http

src

about view

base

src

streamc

convert

cache

src

dn

s

caps

src inclu

xpinstall

src

uriloader

exthandler

un

os2

mac

wi be

base

modules

plugin

base

libpr0n

decod

gif x

libjar

oji

xpcom

string

public src

io glue

parser

expat

lib

htmlp

src

docshell

base

mailnews

base

src se

addrb

src

mime

src

ne

sr

embedding

components

windoww

src

c

br

w

editor

libeditor

html ba

te

c

xpfe

appshell

src

co

se

hi

extensions

xforms sq

b

toolkit

components

satch hi au

gfx

cairo

cairo

sr

intl

unich

sr ut

uc

sr

view

src p

acces

src

b x

rdf

base

chro

storage

src

chrome

src

db calen

brows

camin

ipc

webs

Mozilla Vulnerabilities

js

src

xpconnect

src

livecon

content

base

src public

html

content

src

documen

src

xbl

src

xul

document

src p

content

src

templates

src

events

src

public

xslt

src

xslt base

xml

document

src

content

src

svg

conte

canvas

src

layout

xul

base

src

tree

src

grid

generic base

forms tables style

build

svg

mathm

printin

html

security

nss

lib

util

softoken

pki

ssl

cryptohi

free

pk1

cert

smi

cmd

manager

boot ssl

dom

src

base js

e

public

widget

src

mac windows

gtk

gtk2

xlib os2

qt phot beos

xpwi

pu

netwerk

protocol

http

src

about view

base

src

streamc

convert

cache

src

dn

s

caps

src inclu

xpinstall

src

uriloader

exthandler

un

os2

mac

wi be

base

modules

plugin

base

libpr0n

decod

gif x

libjar

oji

xpcom

string

public src

io glue

parser

expat

lib

htmlp

src

docshell

base

mailnews

base

src se

addrb

src

mime

src

ne

sr

embedding

components

windoww

src

c

br

w

editor

libeditor

html ba

te

c

xpfe

appshell

src

co

se

hi

extensions

xforms sq

b

toolkit

components

satch hi au

gfx

cairo

cairo

sr

intl

unich

sr ut

uc

sr

view

src p

acces

src

b x

rdf

base

chro

storage

src

chrome

src

db calen

brows

camin

ipc

webs

Mozilla Vulnerabilities

js

src

xpconnect

src

livecon

content

base

src public

html

content

src

documen

src

xbl

src

xul

document

src p

content

src

templates

src

events

src

public

xslt

src

xslt base

xml

document

src

content

src

svg

conte

canvas

src

layout

xul

base

src

tree

src

grid

generic base

forms tables style

build

svg

mathm

printin

html

security

nss

lib

util

softoken

pki

ssl

cryptohi

free

pk1

cert

smi

cmd

manager

boot ssl

dom

src

base js

e

public

widget

src

mac windows

gtk

gtk2

xlib os2

qt phot beos

xpwi

pu

netwerk

protocol

http

src

about view

base

src

streamc

convert

cache

src

dn

s

caps

src inclu

xpinstall

src

uriloader

exthandler

un

os2

mac

wi be

base

modules

plugin

base

libpr0n

decod

gif x

libjar

oji

xpcom

string

public src

io glue

parser

expat

lib

htmlp

src

docshell

base

mailnews

base

src se

addrb

src

mime

src

ne

sr

embedding

components

windoww

src

c

br

w

editor

libeditor

html ba

te

c

xpfe

appshell

src

co

se

hi

extensions

xforms sq

b

toolkit

components

satch hi au

gfx

cairo

cairo

sr

intl

unich

sr ut

uc

sr

view

src p

acces

src

b x

rdf

base

chro

storage

src

chrome

src

db calen

brows

camin

ipc

webs

Distribution of MFSAs

Number of MFSAs

Num

ber o

f Com

pone

nts

1 3 5 7 9 11 13

12

520

5030

0

Distribution of Bug Reports

Number of Bug Reports

Num

ber o

f Com

pone

nts

1 3 5 7 9 13 17 24

12

520

5030

0

Imports

• 9,066 imports

• 79,541 import relations (x imports y)

• Takes about five minutes to compute

Resultssoon

Support VectorMachines

Support VectorMachines

Support Vectors

Support VectorMachines

ResultsNow!

Experiments

• 40 random splits6,968 rows in training set, 3,484 rows in validation set

• ClassificationTrain SVM, compute recall and precision

• RegressionTrain SVM, compute rank correlation on top 1%

• SVM: linear kernel with default parametersR implementation (up to 10GB of main memory)

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

0.55 0.60 0.65 0.70 0.75

0.35

0.40

0.45

0.50

0.55

(a) Precision and Recall

Recall

Prec

ision

0.2 0.3 0.4 0.5 0.6 0.7

0.0

0.2

0.4

0.6

0.8

1.0

(b) Rank Correlation

Rank Correlation

Cum

ulat

ive

Dist

ribut

ion

●●

●●●●

●●

●●

●●

●●●●

●●●●

●●●●●●●●●

●●●●●

●●●

●●

●2/3 of all vulnerable components detected

2/3 of all vulnerable components detected45% (about 1/2) of predictions correct

moderately strong correlation (mostly significant at p < 0.01)

Similar Results for Bugs

Packages + Import relationships(Schröter et al, ISESE 2006)

Precision: 66.7% Recall: 69.4%

Binaries + Dependencies(Zimmermann/Nagappan @ Microsoft Research, 2006)

Precision: 64.4% Recall: 75.3%

PredictedRank

Component ActualRank

1 nsDOMClassInfo 3

2 SGridRowLayout 95

3 xpcprivate 6

4 jsxml 2

5 nsGenericHTMLElement 8

6 jsgc 3

7 nsISEnvironment 12

8 jsfun 1

9 nsHTMLLabelElement 18

10 nsHttpTransaction 35