bila r phd thesis

7/30/2019 Bila r Phd Thesis

1/143

Quantitative Risk Analysis of Computer Networks

A Thesis

Submitted to the Facultyin partial fulfillment of the requirements for thedegree of

Doctor of Philosophy

by

Daniel Bilar

Thayer School of EngineeringDartmouth College

Hanover, New Hampshire

June, 2003

Examining Committee:

George Cybenko(Chairperson)

Robert Morris(Member 1)

Robert Gray(Member 2)

Sue McGrath(Member 3)

Carol L. FoltDean of Graduate Studies

c 2003 Trustees of Dartmouth College


2/143


3/143

ii

Thayer School of EngineeringDartmouth College

Quantitative Risk Analysis of Computer Networks

Daniel Bilar

Doctor of Philosophy

George CybenkoRobert MorrisRobert GraySue McGrath

ABSTRACT

Quantitative Risk Analysis of Computer Networks (QSRA) addresses the problem of risk opacity

of software in networks. It allows risk managers to get a detailed and comprehensive snapshot of

the constitutive software on the network, assess its risk with assistance of a vulnerability database,

and manage that risk by rank ordering measures that should be taken in order to reduce it, subject

to cost, functionality and risk constraints. A theoretical methodology is proposed and a prototype

implementation has been developed. Six out-of-the-box popular operating systems were studied using

the methodology and the prototype.

We find that around 75% of discovered vulnerabilities are patchable within two weeks, and around

90% within 40 days after initial discovery. We find a statistically significant time window difference

between security-audited and non-security audited software. Across the operating systems, the majority

of faults give rise to availability and full compromise consequences. There is a statistically significant

difference between fault types: Input validation faults are proportionally over-represented. There is a

statistically significant difference between consequence types: Full compromise consequences are propor-

tionally over-represented. There is, however, no statistically significant fault or consequence proportion

difference between the audited systems.

QSRAs risk assessment model calculated that for all audited systems, four to six months after their

respective release date, the probabilities are very high (66% to 99%) that an attacker can conduct a full

consequence compromise, remotely and locally. Risk management analysis for remote risk probabilities

indicates that, given a moderate fault count, QSRAs highest risk analytic risk mitigation strategy

consistently outperforms the simpler strategy of choosing software with the highest vulnerability count.

Highest risk outperforms the undifferentiated highest count strategy for at least four out of the six

tested operating systems and for four out of five fault consequences.


4/143

iii

I would like to thank everybody who helped me. George Cybenko as my adviser and mentor who

showed me that one can routinely have two brilliant ideas a day and still be a decent and kind human

being. My tender, kind and loving girlfriend Daniella Hirschfeld who brought me sandwiches when I was

starving writing this thesis. Biiiiig kiss, my sweetheart! NH Senator Gregg, Dean Duncan and George

again for getting Washington, DC to finance my excellent work environment. Susan McGrath who runs

the IRIA research group with great success and a winning presence. Bob Gray, a kind man for always

finding time to field questions as he is preparing four papers at the same time. Guofei Jiang whose

unassuming demeanor belies his immense work ethic and sparkling intelligence. Robert Morris as an

example that truth can be far stranger than fiction. The excellent teachers I had at Thayer, including

but not limited to Prof. Eric Hansen and Prof. Ulf Oesterberg. And last but not least, all those people

who follow Wordsworths dictum:

That best portion of a good mans life; his little, nameless, unremembered acts of kindness

and love

I remember.


5/143

Contents

1 Introduction 1

1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Feasibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.1 Fault Tree Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.2 Previous work on cybersecurity risk . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.4 Major findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4.1 Parameters and Vulnerability Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.4.2 Faults and consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.4.3 Design and Implementation of the QSRA system . . . . . . . . . . . . . . . . . . . 8

1.4.4 QSRA empirical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Design of the QSRA system 10

2.1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.1 Software quality, faults, failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Process sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4 Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.4.1 Preliminary example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.4.2 QSRA risk calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.5 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5.1 Optimization problem setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.6 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.6.1 ISTS ICAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.6.2 NETWORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

iv


6/143

CONTENTS v

2.7 A QSRA example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Implementation of QSRA system 28

3.1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 QSRA Event Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3.1 Vulnerability assessment: Software inventorying . . . . . . . . . . . . . . . . . . . . 32

3.3.2 Vulnerability assessment: Software identification . . . . . . . . . . . . . . . . . . . 34

3.3.3 Risk assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.3.4 Risk management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4 Analysis 37

4.1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.1.1 Parameters and Vulnerability Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.1.2 Faults and consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.1.3 QSRA empirical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.2 Parameter and vulnerability data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2.1 Vulnerability lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2.2 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.2.3 Timing data set findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.2.4 Attacker expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.3.1 Audited environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.3.2 Data and analytical framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.4 Vulnerability assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.4.1 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.4.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.5 Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.5.1 Risk calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.5.2 Fault and Consequences distributions . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.6 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

4.7 Possible improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

4.7.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

4.7.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79


7/143

CONTENTS vi

5 Future Work 82

5.1 Database update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.1.1 Vulnerability Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

5.1.2 Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.1.3 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

A Numeric analysis 90

A.1 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

A.2 ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

B Databases 92

B.1 ISTS ICAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

B.2 NETWORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

C Supplemental data 101

C.1 Average time window from discovery of the vulnerability to posted patch . . . . . . . . . 101

C.2 Software inventorying experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

C.3 Faults and Consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

C.3.1 Vulnerability count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

C.3.2 Fault count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

C.4 Risk management scenario results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

D Glossary 124

D.1 Boxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124


8/143

List of Tables

1.1 Virus spread profile 1990-2003 [Kes00][Bek03] . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Software fault densities [Bin97] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.1 Hypothetic risk of a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 Fault types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3 Consequence types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.4 Functionality (Class) groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.5 Vulnerabilities data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.6 Risk management results for George . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.7 Remote risk limits for George . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.8 Risk assessment: Consequence losses and consequence probabilities for George . . . . . . 27

2.9 Risk management: Consequence losses and consequence probabilities for George . . . . . 27

2.10 Vulnerabilities of Apache 2.0.35 and MySQL 4.0.5 . . . . . . . . . . . . . . . . . . . . . . 27

2.11 Vulnerabilities of Apache 1.3.26 and MySQL 3.23.50 . . . . . . . . . . . . . . . . . . . . . 27

3.1 QSRA event sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2 Sample uname and ver output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1 Questionnaire: Vulnerability lifecycle and attacker ability . . . . . . . . . . . . . . . . . . 40

4.2 Categorical and continuous data sources: decision criteria . . . . . . . . . . . . . . . . . . 41

4.3 Vulnerability lifecycle events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4.4 ANOVA results of time between discovery and patch [tdesc, tpatch] . . . . . . . . . . . . . . 44

4.5 Experts estimate for time window from discovered vulnerability to posted patch [tdesc, tpatch] 49

4.6 Overview of questionnaire results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.7 Vulnerability advisories vs attack in 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.8 Standard installed software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

vii


9/143

LIST OF TABLES viii

4.9 Experimental factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.10 Timing experiment: Software count, database size, OS . . . . . . . . . . . . . . . . . . . . 54

4.11 Timing experiment: Connection type, ARP cache . . . . . . . . . . . . . . . . . . . . . . . 55

4.12 Timing experiment: Database size, number of inventoried hosts . . . . . . . . . . . . . . . 55

4.13 Stepwise regression results, two factors: Number of software packages, database size . . . 57

4.14 Stepwise regression results, two factors: Number of hosts, database size . . . . . . . . . . 57

4.15 OS ANOVA: Software inventorying timing data . . . . . . . . . . . . . . . . . . . . . . . . 60

4.16 Size of software inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

4.17 Parameters: Attacker ability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.18 Parameters: Time window till posted exploit . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.19 OS consequence probability calculation results . . . . . . . . . . . . . . . . . . . . . . . . 65

4.20 Proportion of OS to consequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.21 Proportion of consequences to OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.22 Proportion of consequences to fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.23 Proportion of faults to consequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.24 Proportion of OS to fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.25 Proportion of faults to OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.26 ANOVA results of proportions of faults, OS and consequences . . . . . . . . . . . . . . . . 68

4.27 Fault type breakdown 2002 [oSN02] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

4.28 Consequence magnitudes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

4.29 Individual OS remote consequence risk probability scenario results . . . . . . . . . . . . . 72

4.30 Individual OS remote vulnerability count scenario results . . . . . . . . . . . . . . . . . . 73

4.31 OS family average remote risk probability scenario results . . . . . . . . . . . . . . . . . . 74

4.32 OS family average remote vulnerability count scenario results . . . . . . . . . . . . . . . . 74

4.33 Relevant libraries used by Apache 1.3.26 and Apache 1.3.27 . . . . . . . . . . . . . . . . . 81

5.1 Revised vulnerability lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

5.2 New vulnerability consequence types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

B.1 System aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92




B.2 Vulnerabilities aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95


10/143

LIST OF TABLES ix

B.2 Vulnerabilities aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

B.3 Relations aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

B.4 NCC aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

B.5 Risk Assessment aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

B.5 Risk Assessment aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

B.6 Risk Management aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

B.6 Risk Management aggregate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

C.1 Security advisories as of September 30th, 2002 . . . . . . . . . . . . . . . . . . . . . . . . . 101

C.2 Average time window (days) of discovered vulnerability to posted patch . . . . . . . . . . 103

C.3 ANOVA group mean rank comparison: OS . . . . . . . . . . . . . . . . . . . . . . . . . . 106

C.4 ANOVA results group mean rank comparison: consequences vs fault type . . . . . . . . . 112

C.5 ANOVA results group mean rank comparison: consequences vs OS . . . . . . . . . . . . . 112

C.6 ANOVA results group mean rank comparison: fault type vs OS . . . . . . . . . . . . . . . 113

C.7 Vulnerability count by OS, class and access: Debian 3.0 and Mandrake 8.2 . . . . . . . . . 115

C.8 Vulnerability count by OS, class and access: Suse 8.0, W2K SP2, OpenBSD 3.1 and

RedHat 7.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

C.9 Fault count of standard OS installations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118


11/143

List of Figures

1.1 Fault tree example of event No power on the 415V bus [www03] . . . . . . . . . . . . . 5

2.1 Software view of the network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 QSRA approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3 Fault tree for consequence c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4 Escalation exploits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.1 Inventorying the network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.2 Sample lsof output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.3 Sample fport output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4 Specifying the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5 Assessing the software risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6 Managing software risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.1 Timeline of events in vulnerability lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2 Time between discovery and patch [tdesc, tpatch] . . . . . . . . . . . . . . . . . . . . . . . . 45

4.3 Histogram of time between discovery and patch [tdesc, tpatch] . . . . . . . . . . . . . . . . . 46

4.4 Group mean comparison of [tdesc, tpatch] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.5 Scatterplot: Software inventorying time for one host . . . . . . . . . . . . . . . . . . . . . 56

4.6 Scatterplot: Software inventorying time for multiple hosts . . . . . . . . . . . . . . . . . . 57

4.7 Failure probability of independent components in series . . . . . . . . . . . . . . . . . . . 63

4.8 Fault count data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.1 Vulnerability database input mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

C.1 Boxplot of [tdesc, tpatch] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

C.2 Software inventorying time ANOVA: Boxplots for connection type and ARP size . . . . . 105

x


12/143

LIST OF FIGURES xi

C.3 Software inventorying time ANOVA: Boxplots for database size and software count . . . . 108

C.4 Software inventorying time ANOVA: Boxplots for database size and software count . . . . 109

C.5 Fault count ANOVA: Boxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

C.6 Risk probability scenario reduction: Consequences . . . . . . . . . . . . . . . . . . . . . . 120

C.7 Risk probability scenario reduction: Consequences . . . . . . . . . . . . . . . . . . . . . . 121

C.8 Risk probability scenario reduction: Individual OS . . . . . . . . . . . . . . . . . . . . . . 122

C.9 Risk probability scenario reduction: OS families . . . . . . . . . . . . . . . . . . . . . . . . 123

D.1 Sample boxplot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124


13/143

Chapter 1

Introduction

1.1 Problem statement

At the end of the 20th century, we have witnessed the massive transition from isolated, disconnected

computers to networked computer clusters all over the world. At the time of this writing (Spring 2002),

there are an estimated 195 million networked hosts world-wide [Tec02].

This global pervasive connectivity has been a boon for consumers, businesses and governments alike

due to the ease, convenience and speed of electronic data exchange. However, the ease of use and

relative anonymity that the Internet affords has been leveraged by criminal elements, as well. The

National Infrastructure Protection Centers (http://www.nipc.gov/) Congressional statement of May

25th, 2000, highlights the risks we face today:

Ninety percent of respondents detected security breaches over the last 12 months. At least

74% of respondents reported security breaches including theft of proprietary information,

financial fraud, system penetration by outsiders, data or network sabotage, or denial of

service attacks. Many companies experienced multiple attacks; 19% of respondents reported

10 or more incidents. Information theft and financial fraud caused the most severe financial

losses, estimated by the respondents at $68 million and $56 million respectively. The losses

from 273 respondents totaled just over $265 million. Notably, this survey does not include

harm caused by recent destructive episodes such as the Distributed Denial of Service attacks

on e-commerce sites in February, and the ILOVEYOUor Love Bug virus earlier this month.

No private, commercial or government agency is completely safe or has been unaffected by the

proliferation of this kind of cyber-crime. E-Commerce Times reported that the ILOVEYOUvirus affected

1


14/143

CHAPTER 1. INTRODUCTION 2

Table 1.1: Virus spread profile 1990-2003 [Kes00][Bek03]Virus Year Type Time to prevalence 1 Estimated damagesJerusalem, 1990 .exe File, Boot Sector 3 years $50MCascade,Form,Concept 1995 Word Macro 4 months $50MMelissa 1999 E-mail enabled, Word Macro 4 days $93M-$385MLove Bug 2000 E-mail enabled, .vbs 5 hours $700M - $6.7BSlammer 2003 self-propagating worm 10 minutes $1B

Table 1.2: Software fault densities [Bin97]Application Faults per thousand lines

of source codeNASA Space Shuttle Avionics 0.1

Leading-edge software companies 0.2Reliability Survey 1.4Best, Military system survey 5.0Worst, Military system survey 55.0

45 million hosts and inflicted monetary damages to the tune of estimated $2.6 billion [Eno00]. The

infamous Melissa macro virus caused an estimated $300 million in damage in 1999 and several prominent

e-commerce sites were hit by Distributed Denial of Service attacks in the beginning of 2000 [SoT00].

The estimated worldwide damage caused by automated digital attacks over $30 billion for 2002 [mi202b].

These estimated damage figures have to be taken with a grain of salt, but the trend is clear. Moreover,

in just a dozen years time, the propagation speed, as well as the estimated damages have increased by

five, and two orders of magnitude, respectively (see Table 1.1).

One of the persistent factors is faulty software. Software fault density remains between 0.1 and 55

faults per thousand lines of source code (see Table 1.2). Exacerbating circumstances include increases in

code size and in the number of interaction pathways between the various software subsystems. Perhaps

the most aggravating fact is the unwillingness and unawareness of designers and implementers alike to

use proven techniques to build secure code. The net result has been an increase in the number of poten-

tially serious software faults. In the absence of larger, long-term efforts and initiatives (vendor liability,

legal regulations, widespread adoption of safe design practices and responsible user behavior), the risk

posed by faulty software has to be managed somehow.

1Time it took to become the #1 most prevalent computer virus in history


15/143


1.2 Feasibility

There are two main objections that have been raised to the feasibility of cyberrisk analysis and manage-

ment:

Unknown enemies with unknown skills, knowledge, resources, authority, motives, and objectives

(SKRAMO) attacking unknown vulnerabilities make risk measurement and management impossi-

ble.

Dealing with known SKRAMO and known vulnerabilities does little to reduce risk.

The first argument is rather like Lord Kelvins 1895 remark in front of the Royal Society: Heavier

than air flying machines are impossible. Incomplete information does not mean no information at

all. There are studies of software defects in code [MV96][Rea03], theoretical modelling of software

defects [MIO90] and attack proportions/damages among operating systems [mi202c][mi202a]; as well as

extensive historical vulnerability databases [oSN02][Str02]. These sources can be used to hypothesize

about future unknown vulnerabilities, attack frequencies, and attack targets. It is correct, however,

that there is a conspicuous lack of attacker studies - their knowledge, resources, and skills. The short

answer is that in light of incomplete information, cyber risk, like all risks, cannot be eliminated, but

can be managed. Various organizations, diligent ones like the US DoD among them, have been doingthis for some time now [Gor03]. The issue is subtler when one tries to evaluate cyberrisk in the context

of formulating insurance policies and setting insurance premiums. Insurance companies have to guard

against two phenomena when setting premiums: adverse selection, when someone chooses to insure

against a particular loss because of private information not available to the insurance companies at the

time of contracting (like somebody getting life insurance before going to Liberia), and moral hazard,

which denotes the lack of incentives by the insured to take actions that reduce the probability of a loss

subsequent to purchasing the insurance [GLS03]. The moral hazard problem can be fruitfully addressed

by QSRA. Extensive software auditing and subsequent risk analysis can pinpoint problem areas to the

insurers, which in turn can offer discounts for companies that take (QSRA specified) security measures.

The short answer is that the QSRA methodology is not the whole picture, but is just part of the premium

assessing process.

The second argument has been shown to be false. Known vulnerabilities are the main entry point

into most systems, as security groups have maintained and recent incidents have shown [SASS02,

/top20][CC03, VU484891]. Mitigating these risks would go a long way towards improving security.

In addition, the issue is not known vulnerabilities per se. The issue is one of locating the software on the


16/143


network that exhibits these vulnerabilities and patching them. Few administrators are diligent about

installing the latest patches and upgrades. Almost no one has a good overview of the software running

on their network in the first place. An extensive software inventorying is a pre-requisite for dealing with

known vulnerabilities possible, and is one of the most useful features of the implemented QSRA system.

1.3 Risk

1.3.1 Fault Tree Analysis

Fault tree analysis was developed by Bell Telephone Laboratories in 1961, to improve the reliability of

the Minuteman Launch Control System [II99]. Since then, it has been extensively used for evaluatingsystem safety and concomitant risk in many engineering disciplines (power, nuclear, electric, source code

analysis) [oRBC75] [LM00]. Fault tree analysis is used to model failure situations in (typically) multi-

component systems. The goal of fault tree analysis is to investigate dependencies between failures of

components that lead to the fault trees top-event. A fault tree is a logical diagram representing the

consequences of component failures on the failure of the so-called top-event, the root of the tree. The

top-event occurrence depends on a specific combination of basic events, which are combined with two

primary types of gates, AND and OR.

An example of a fault tree is given in Figure 1.1. The top-event is 415VBUS, indicating that there

is no power on the 415V bus. The sub-events are ISO-A, ISO-D (and its negation NOT(ISO-D)), CON-A,

DIESEL, and EXT-GRID-REPAIR. The following Boolean expression illustrates their relationship:

415VBUS = ((ISO-A NOT(ISO-D)) + CON-A + DIESEL) (EXT-GRID-REPAIR + ISO-D) (1.1)

Systems

A systems reliability function r(t) is the probability that the top event does not occur over time period

(0, t). The systems failure function f(t) is equal to 1 r(t). Systems can be arranged in different

configurations. Hillier and Lieberman identify three structures: series, parallel, and k-out-of-n-systems

[HL90]. A serial system is a system which does not fail only if all components do not fail. A parallel

system is a system which does not fail only if at least one component does not fail. A k-out-of-n-system

fails if at least (n k) components fail.

For serial systems with n independent components, each with reliability ri(t), the systems reliability

rsystem(t) and the systems failure fsystem(t) are given by Equations 1.2 and 1.3.


17/143


Figure 1.1: Fault tree example of event No power on the 415V bus [www03]

rsystem(t) =

ni=1

ri(t) (1.2)

fsystem(t) = 1 ni=1

(1 fi(t)) (1.3)

This is the basic risk modelling setup that is going to be used by QSRAs calculations.

1.3.2 Previous work on cybersecurity risk

There exist a number of suites that attempt to evaluate and manage cybersecurity risk (Strategic Tech-

nology Institutes Cost-Benefit-Analysis Tool C-BRAT [Ins01], SAINT Corporations Security Admin-

istrators Integrated Network Tool [Cor01], Network Associates CyberCop Scanner [Ass01]), proposed

measures to quantify the risk of vulnerability exposure (Northcutts severity measure system [Nor99])

and approaches for assessing vulnerabilities in computer networks (Sandias computer-attack graph gen-

eration tool [PS98] [SPEC01]).

The risk calculations in the aforementioned approaches are either opaque (in the case of the com-


18/143


mercial products) or very crude (Northcutts measure), or gloss over how the probability/risk measure

will actually be calculated (Sandia). Furthermore, practical, specific, granular risk management is a

key aspect of an integrated methodology of mitigating cybersecurity risk - and there has been a paucity

of tools and approaches in that respect. Lastly, since todays networks are increasingly ad-hoc, mobile

and wireless (with the invariably concomitant increase in risk), no effective methodology can afford to

be static - it must constantly adjust itself or be quickly adjustable to reflect the addition, removal and

modifications of hosts and their constitutive software. Ideally, the assessment, analysis and management

recommendations must be real-time or near-real time (on the order of minutes). QSRA was designed

with six principles in mind:

1. Generality: QSRA is general in its design and can be deployed to assess and manage the risk posed

by any IP device on a network

2. Adaptiveness: QSRA can be run in a semi-automated or manual fashion on a network of interest;

mobile, wireless, static, or combinations

3. Comprehensiveness: QSRAs risk metric calculations take machine configuration, functionality

and value, attacker ability, software installed/running, time flow, and compromise dependencies

into account

4. Granularity: The granularity of the data used in the risk analysis step is on the level of the

individual program, broken down into different compromise consequences of different severity

5. Transparency: QSRA is an open methodology, as well as a toolset,that is largely platform inde-

pendent with exclusive use of COTS components

6. Solutions: QSRA offers risk management, i.e. rank ordered practical steps to reduce risk such

as upgrading this web server on IP 127.0.0.2 to a newer version, or disabling service A on IP

127.0.0.7 subject to cost, compatibility and functionality constraints.

1.4 Major findings

Vulnerability data was collected for six operating systems: Red Hat Linux 7.3, SuSe Linux 8.0, Mandrake

Linux 8.2, OpenBSD 3.1, Debian Linux 3.0, and Windows 2000 Professional SP2. The standard, out-of-

the-box workstation installation was chosen when installation options were presented. Since the Linux

and OpenBSD distributions come with a wealth of applications, MS Office was assumed to be part of

the standard out-of-the-box Windows 2000 distribution.


19/143


1.4.1 Parameters and Vulnerability Data

Vulnerability lifecycle

Empirical research on 58 vulnerabilities over a three months period showed that in over 60% of cases

where data were available, exploits were obtainable before and at the time the patch was released. Most

exploits were obtainable the day the patch was released, and some preceded the patch by as much as

two months.

Time window data finding

For six audited operating system, we find that around 75% of vulnerabilities are patchable within two

weeks, and around 90% within 40 days after initial discovery.

There is no statistical significant difference between locally and remotely exploitable vulnerabilities,

nor between vulnerability consequence types. No statistical inference can be drawn regarding fault types.

There is no statistical significant difference between open source and closed source software. There is,

however, a statistically significant difference between security-audited and non-security audited software.

Attacker expertise

Automated exploit tools and straight-forward exploit descriptions are readily found on the Internetand make acquisition of attacker skills easy. A corollary of this finding is that one cannot count on

vulnerabilities and associated exploits to remain secret.

1.4.2 Faults and consequences

Faults were mapped to a vulnerability consequence, a fault type and an operating system. The over-

whelming majority of fault occurrences lead to availability and full compromise consequences. 75% of

the 3-tuples (OS, consequence and fault type) have a count of 0, and around 25% fall within a count

range of four to one.

Within availability and full compromise consequences, around 15% percent have a count of one or

two, and around 25% fall within a count range of four to one. Input validation faults are proportionally

over-represented.

There is a statistically significant difference between consequence types. Full compromise conse-

quences are proportionally over-represented. There is no statistically significant fault or consequence

proportion difference between the audited hosts.


20/143


1.4.3 Design and Implementation of the QSRA system

Java as an implementation language proved its worth in cross-platform development. It cannot be

stressed enough how much time is saved by the lack of the tedious memory leaks and pointer manip-

ulations that plague C and C++. Text parsing, however, is not Javas forte, and hence QSRA client

development took a disproportional amount of time.

The decision to design a relational, as opposed to a flat database was found to have been very

fortunate. Having all data relations in third normal form is advisable, since this enables additions and

mutations to be accommodated with few if any changes to the existing design.

The ists_icat database holds data about software characteristics and its associated vulnerabilities.

It is used to provide choices for the software identification, data and parameters for risk assessment and

options for risk management. It has two main functional aggregates: Systems which holds data on

software programs, and Vulnerabilities, which holds data on the softwares associated vulnerabilities.

Its comprehensiveness is key. Lower estimate for necessary risk management data for 40 software class

categories with 5 substitutes programs each across 3 families (Windows, BSD, Linux) is 600 entries. The

lower estimate for risk analysis is far higher, since there are thousands of software packages for each

family. The lower bound record size of the Systems aggregate, which holds data data on software

programs, for production quality deployment is estimated to be around 3000 entries, namely for three

OS families with fifty functionality groups containing twenty records each. This number is necessary so

that the risk management optimizer has a sufficiently large pool from which to assemble an alternative

software makeup. In steady-state, and with an input mask, it takes no more than 5 minutes to amass

and assess one entry and another 3 minutes to write it to the database, making the time commitment

to populate the Systems database to be around 400 man-hours, ar at least two months for a single

person.

The time commitment to keep the Vulnerability aggregate current is continuous and substantial.

Researching and scrubbing a single vulnerability takes at least ten minutes, and in many cases may take

an hour or more. CERT reported over four thousand new vulnerabilities in 2002 [CC03], so at a rate of

eighty vulnerablities/week, one should schedule at a minimum of 12 man-hours a week, although twenty

to thirty man-hours a week would be more advisable.

1.4.4 QSRA empirical results

As implemented, software inventorying time grows linearly with the number of audited hosts and the

number of installed software packages and the number of records in the database networks which records


21/143


the software makeup of the network. Database throughput to networks becomes a non-negligible factor

when the number of records in the database is high.There is no statistically significant time difference

between auditing wireless hosts and landline hosts.

QSRAs risk assessment model calculated that across all the original audited operating systems, four

to six months after their respective release data, the probabilities are very high (66% to 99%) that

an attacker can conduct a full consequence compromise, remotely and locally. One or two faults are

enough to cause the probabilities to rise to 50% and more. Almost all, if not all, of the faults have to

be eliminated to have an appreciable effects on risk probabilities.

Risk management analysis for remote risk probabilities indicates that, given a moderate fault count,

QSRAs highest risk analytic risk mitigation strategy consistently outperforms the simpler strategy of

choosing software with the highest vulnerability count. Highest risk outperforms the undifferentiated

highest count strategy for at least four out of the six tested operating systems and for four out of five

fault consequences. The most compelling effects are found on the Windows system, probably due to the

comprehensiveness of Windows-style patches. The effects on Linux-family systems are less pronounced.

Both strategies have minimal effect on risk probability reduction across all audited hosts when the

vulnerability count is high.


22/143

Chapter 2

Design of the QSRA system

2.1 Findings

The ists_icat database holds data about software characteristics and its associated vulnerabilities. It

is used to provide choices for the software identification, data and parameters for risk assessment and

options for risk management. It has two main functional aggregates: Systems which holds data on

software programs, and Vulnerabilities, which holds data on the softwares associated vulnerabilities.

Its comprehensiveness is key. Lower estimate for necessary risk management data for 40 software class

categories with 5 substitutes programs each across 3 families (Windows, BSD, Linux) is 600 entries. The

lower estimate for risk analysis is far higher, since there are thousands of software packages for each

family. The lower bound record size of the Systems aggregate, which holds data data on software

programs, for production quality deployment is estimated to be around 3000 entries, namely for three

OS families with fifty functionality groups containing twenty records each. This number is necessary so

that the risk management optimizer has a sufficiently large pool from which to assemble an alternative

software makeup. In steady-state, and with an input mask, it takes no more than 5 minutes to amassand assess one entry and another 3 minutes to write it to the database, making the time commitment

to populate the Systems database to be around 400 man-hours, ar at least two months for a single

person.

The time commitment to keep the Vulnerability aggregate current is continuous and substantial.

Researching and scrubbing a single vulnerability takes at least ten minutes, and in many cases may take

an hour or more. CERT reported over four thousand new vulnerabilities in 2002 [CC03], so at a rate of

eighty vulnerablities/week, one should schedule at a minimum of 12 man-hours a week, although twenty

10


23/143

CHAPTER 2. DESIGN OF THE QSRA SYSTEM 11

to thirty man-hours a week would be more advisable.

2.2 Overview

Conceptually for QSRA, the network is not a collection of hardware. From QSRAs point of view, the

network is a collection of IP addresses with associated software. Figure 2.1 is an illustration of this view.

Figure 2.1: Software view of the network

The software on each IP-enabled device can be classified into three groups.

Services Services are running programs that have one or more sockets bound to listening ports. They

react to connection requests from the other devices. An example would be a httpd daemon listening on

port 80, the common web service port.


24/143


Applications The main difference between services and applications is that the latter are not listening

on any port. Rather, the term encompasses installed software on the device. This software may or may

not be actively executing. An example would be MS Office.

Operating system The operating system is the most important software on a device, since it provides

the operating environment for other software.

2.2.1 Software quality, faults, failures

Software code quality has not markedly improved. Although it is impossible, both on theoretical and

practical grounds, to build completely secure systems, commercial off-the-shelf (COTS) software has been

exhibiting the same source code fault density for the past twenty years. Software fault density remains

between 0.1 and 55 faults per thousand lines of source code (see Table 1.2 on page 2). Exacerbating

circumstances include increases in code size (Windows 2000 has an estimated 35 million lines of source

code [Whe02]) and in the number of interaction pathways between the various software subsystems.

Perhaps the most aggravating fact is the unwillingness of designers and implementers alike to use proven

techniques to build secure code [HL02, pp.19-57]. The net result has been an increase in the number of

potentially serious software faults.

A fault is a defect in a program that, given certain conditions, causes a failure [MIO90, p. 6]. A fault

may cause many failures; likewise, a failure may be caused by more than one fault. Some failures can

be exploited by attackers. Faults, as well as failures, may be classified according to type. Drawing from

the taxonomy used by ICAT, SecurityFocus, Landwehr and Bishop [oSN02] [Sec02] [LBMC94] [Bis00],

the following fault types can be distinguished:

1. Input validation error: A fault is characterized as an Input Validation Error if the input being

received by the software is not properly checked, thereby enabling a vulnerability to be exploited

by a certain input sequence. An example of this would be a buffer overflow. A different example

would be a boundary condition error - the canonical case being the Y2K problem, where the 2

digit year assumption was exceeded. Another example would be insufficient validation of HTTP

requests that may allow execution of code.

2. Access validation error: A fault is characterized as a Access Validation Error if software is vul-

nerable because the access control mechanism is faulty. The problem lies not with the configuration

of the access control mechanism but with the mechanism itself. An example would be an error

that enabled authentication to an FTP service using a non-existent username/password.


25/143


3. Exceptional condition handling error: A fault is characterized as an Exceptional Condition Han-

dling error if software becomes vulnerable due to an exceptional condition that has arisen. The

handling (or mishandling) of the exception by the software enables a vulnerability. An example of

this would be an error in router software that allows remote attackers to cause a denial of service

via a special BOOTP packet that is forwarded to broadcast MAC addresses.

4. Environmental error: A fault is characterized as an Environmental Error if unanticipated syn-

ergies causes the software to be vulnerable. This may be due, for example, to an unexpected

interaction between an application and the operating system or between two applications on the

same host. An example of this would be an interaction between encryption software in conjunction

with encrypted file systems, which result in the creation of cleartext temporary files that cannot

be wiped or deleted due to the strong permissions of the file systems.

5. Configuration error: A fault is characterized as a Configuration Error if user-controllable settings

cause the software to become vulnerable. This fault is caused by controllable configuration, not

inherent design errors. An example of this error would be unchanged, well-known default user

accounts on remotely accessible software.

6. Race condition: A fault is characterized as a Race condition if the non-atomicity of a security

check causes the existence of a vulnerability. An example would be software checking the validity

of an operation against a security model. However, between the time of the security check and

the actual operation, some changes occur that invalidate the security model. Attackers can then

perform illegal operations like writing to the password file.

7. Design error: A fault is characterized as a Design Error if there exists no errors in the software

implementation or configuration. Rather, the initial design is faulty. A example of this error would

be a default root password that cannot be changed.

The assumption is made that some fault types are harder to exploit than others; taking advantage of

a race condition requires on the whole more expertise and finesse than exploiting an input validation

error, for instance.

Faults cause failures, which will henceforth be called consequences. Consequences may be one of five

different types (see Table 2.3 on page 17) .

Each of these consequences entails a potential loss, which is dependent on the IP-enabled device on

which the software resides. The determinants are, intuitively, data and/or functionality of the IP-enabled

device. An availability breach on a public Web server may be much more serious than an identical breach


26/143


on an internal Web server, for instance. Similarly, a confidentiality breach on a database hosting customer

data may spell ruin, whereas the impact of the same confidentiality breach on a database hosting archival

information may be minimal. Presently, there are no known ways to automate the generation of the

loss functions; there has to be a risk manager who can gauge the value of each IP-enabled device for the

networks information/business/mission purpose and set the appropriate magnitudes for the consequence

losses.

2.3 Process sequence

The software information of the network is gathered and processed in three steps, as shown in Figure

2.2:

Figure 2.2: QSRA approach

1. Vulnerability Assessment: Conduct an exhaustive and detailed inventory and identification of

software on a network of interest (operating systems, services, and applications)


27/143


28/143


2.4 Risk Assessment

2.4.1 Preliminary example

Risk is commonly defined as the product of probability and severity of adverse effects, and the most

common approach to quantify risk is a single figure - its expected value [Hai98, p. 29]. Mathematically

speaking, given a random variable X with probability function px(x) and loss function lx(x), the expected

risk value in the discrete case is equal to

E(x) =

pX(x)lX(x) (2.1)

It is apparent that these are generic probability weighed averaging formulas. Its semantic specializa-

tion into an expected value of risk occurs through the loss function. The unit of the expected risk value

is the unit used by the loss function and could be downtime, cost, credibility, etc.

As a preliminary example, the simplified risk of attack consequences on a host that is running one

application is shown in Table 2.1. The assumption is that an attacker can exploit three vulnerabilities,

with the following consequences for the host: Full compromise (equivalent to root access), Availability

(commonly known as Denial of Service) and Confidentiality (as an example, reading a password file).

Let X be a discrete variable denoting the individual fault consequence. Let the probability function

be the probability that the vulnerability is targeted by the hacker. Let the loss function be defined as

general damage, in US$ to the function the host serves.

Table 2.1: Hypothetic risk of a hostX pX(x) lX(x) RiskX(x)Full 0.1 $100 $10Availability 0.8 $10 $8Confidentiality 0.1 $50 $5

With the probabilities and expected values shown in Table 2.1, the expected value of risk E(X) of ahacker attack is $10 + $8 + $5 = $23.

2.4.2 QSRA risk calculation

Since the unit of analysis is software, this step gauges the risk of the deployed software on the network.

The first section Basic Risk describes the software risk without taking interactions into account. Basic

remote and local consequence risk is calculated this way. The second section Extended Risk takes

interactions between programs into account. This refines the risk of remote attack risk by including


29/143


Table 2.2: Fault typesType DescriptionInput validation error improper input sequence validation

Buffer overflow error expected buffer length exceededBoundary condition error assumed boundary exceeded

Access validation error faulty access control mechanismExceptional condition mishandling incorrect exception handlingEnvironmental error discrepancy between test and installation en-

vironmentConfiguration error unsafe configuration by userRace condition non-atomicity of security checkDesign error faulty basic design

Table 2.3: Consequence types

Type DescriptionAvailability Some software or data is unavailable for legitimate useConfidentiality Unintended read privilege to dataIntegrity Unintended write privilege to dataProcess User access over software and dataFull Full access over all software and data on a device

interactions between services and applications.

Basic Risk

Basic risk calculates the basic remote and local risk, without taking interactions between programs into

account. The risk equations presented in this section focus on services (remotely accessible software),

but can be applied without loss of generality to applications (locally accessible software).

Let C be the set of consequences. Let V be the access venue (remotely exploitable or locally ex-

ploitable); V {vr, vl}. Let loss(i,c) be the consequence loss for a specific IP-enabled device i. Let

pv,c(t) be the probability of consequence c C, v V. Then, we have

Risk(v,i,c)(t) = pv,c(t) loss(i,c) (2.2)

where c C, i I

The consequence losses, loss(i,c), are domain-dependent on the data and the functionality of the IP-

enabled device and have to be set by the risk manager. The probability of consequence c is the com-

bined failure probability of the services residing on the device that have faults that potentially entails

consequence c. This probability is time dependent and assumed to be increasing monotonically.

For remotely exploitable risk (access venue v = vr), let Kc be the set of services with faults that

entail consequence c. Let pvr ,c(t), c C be the aggregate probability of the set of services Kc leading


30/143


Figure 2.3: Fault tree for consequence c

to consequence c. Let pc,k(t) be the exploit probability of consequence c of a specific service k. Then,

we have

pvr ,c(t) = 1 kKc

(1 pc,k(t)) (2.3)

where c C

Equation 2.3 represents the failure of independent components in series [Ros93, pp. 418, 434]. The

components are the software in Kc and 1

(1 pc,k(t)) is the combined failure probability of the

services residing on the device that have faults that potentially entail consequence c. This is intuitively

reasonable, since the faulty services constitute independent pathways toward consequences. The more

pathways you have, the more probable that this consequence will materialize. Figure 2.3 illustrates a

consequence fault tree with two service pathways toward consequence c.

The same pathway argument applies to multiple faults in a single service as well. Service k may

have many faults which may lead to consequence c. For instance, an input validation fault and a race

condition fault both could lead to a full (root) compromise. Let Fc,k be the set of faults in service k

that lead to consequence c. Let qf(t) be the exploit probability of fault f, f Fc,k, at time t . Then,

we have

pc,k(t) = 1

fFc,k

(1 qf(t)) (2.4)

(2.5)

qf(t) denotes the probability that an attacker will be able to exploit fault type f at time t. Let t0 be

the time when a fault has been discovered. pautomated(t,f) be a probability function that an automated

tool to exploit this fault has been developed since tdesc, the time the vulnerability was discovered. The


31/143


function is assumed to increase monotonically from time tdesc till tposted, the time the exploit tools are

publicly available and then stabilize. Let Fc,k be the set of faults in service k that lead to consequence

c Let pmanual(f) , f Fc,k, be the proportion of attackers that can exploit this fault without an

automated tool. These are skilled attackers who do not need an automated tools. Let ptool(f), f Fc,

be the additional proportion of attackers that can exploit this fault with an automated tool. As time

goes by, it becomes more likely that this set of attackers learn how to exploit this vulnerability. Then

we have

qf(t) = pautomated(t,f)proptool(f) + propmanual(f) (2.6)

The basic remote and local risk over all consequences can be calculated this way

Extended Risk: Services and Applications

So far, remote and local consequence risk have been calculated without taking interactions between

programs into account. Interactions between programs enable escalation exploits. Escalation exploits are

a manifestation of negative synergies between vulnerabilities. These are attacks that use vulnerabilities in

one software program - usually a service - as a stepping stone to exploit vulnerabilities in another software

program, usually an application. Figure 2.4 illustrates the pathways of two types of escalation exploits.Inclusion of escalation exploits affects the consequence probability pvr,full by potentially increasing the

risk of a remote full consequence attack.

Remote2Local-User2Root Also known as R2L-U2R, a pathway in which a service is remotely ex-

ploited that has a fault entailing a process, and/or integrity privilege compromise. This privilege is then

used by an attacker to access an application a with a fault that can entail a full consequence compromise.

Remote2Local-Remote2Root Also known as R2L-R2R, it represents a pathway in which a service

is remotely exploited that has a fault entailing a process and/or integrity privilege compromise. This

compromise enables a changes in the state of some service, creating a configuration error fault within

that service that enables an attacker to compromise service k2, leading to a full consequence breach.

Let Escalation be the set of the two Remote2Local-Remote2Root and Remote2Local-Remote2Root

escalation events. Let P I be the set of consequences that affect non-root level of access control; P I =

Process

Integrity. Let svr,full(t) be the remote full consequence probability, resulting from services

with configuration faults. Let tvr ,full(t) be the remote full consequence probability, excluding services

with configuration faults. We have


32/143


Figure 2.4: Escalation exploits

pEscalation(t) = pvr,PI(t)(pvl,full(t) + svr ,full(t) pvl,full(t) svr ,full(t)) (2.7)

Revising the full consequence probability pvr,full to include these escalation exploit pathways is

straightforward. Combined with the non-escalation attack event that leads to full compromise, we have

pvr,full(t) = tvr,full(t) + pEscalation(t) tvr,full(t) pEscalation(t) (2.8)

Aggregation Each consequence risk is measured in $US. The aggregate risk of the IP-enabled device

i is then the summation of 2.2 on page 17 for all consequences c C, C being the set of consequences

in Table 2.3 on page 17. The device risk is then

Risk(i) =cC

Risk(i,c) (2.9)

2.5 Risk Management

Once the risk metrics have been calculated for the networks constitutive IP-enabled devices, the next

step is to implement measures to reduce the networks risk profile. This is the risk management step.

Subject to cost, functionality, and risk limit constraints, the risk profile optimization procedure consists

of replacing, patching and/or removing software. The QSRA application, together with its optimization

module, works with the vulnerability database to assemble an alternative software makeup.

The conceived optimization problem is an instance of a Set Partitioning Problem (with base con-


33/143


Table 2.4: Functionality (Class) groupsaccess control admin tool applicationapplication server authentication basic OS toolsboot server bridge (hardware) bridge (software)bug tracking communication compressiondatabase dhcp encryptionfile file sharing firewallfirmware IDS internet clientISDN library logmail mail client mail servername NAT network access (Ethernet)network access (non-Ethernet) network storage newsnon-TCP server OS kernel packet filterpacket sniffer printer spooler programming languageremote access router (hardware) router (software)

sandbox security module server daemonsuperdaemon switch (hardware) switch (software)task scheduler text editor timeunspecified web web proxywireless access point . . . . . .

straints). See Eso for an introduction [Eso99]. The problem statement is formulated as an Integer

Linear Program (ILP) [BHM77, Ch. 9]. The ILP for one IP-enabled device a is described, without loss

of generality.

There are four general sets of constraints: OS compatibility, functionality, risk and cost limits. The

QSRA OS compatibility ensures that the QSRA application solver module only chooses compatible

software. Functionality ensures that the software set chosen by the solution retains pre-optimization

functionality. If host a had a web server, a database server and a ftp server before optimization, it has

to offer this functionality afterward, too. Table 2.4 lists the functionality (class in QSRA vernacular)

groups in the prototype QSRA implementation. This list is not complete and can be amended; it is merely

a listing of functionality of the inventoried software in section 4.3. Risk limits are set by the risk manager

and reflect the upper acceptable bounds for the risk consequences. Cost limits are the costs associated

with software - monetarily, they include installation, configuration, maintenance, upgrade, training,

and acquisition costs, expressed in US$. Timewise, they include downtime, installation, configuration,

training and/or maintenance costs, expressed in hours. They are set by the risk manager, and specify

an upper bound. The risk manager should be work in conjunction with a senior network administrator

and a senior executive in order to accurately assess the risk and cost dimensions.


34/143


35/143


subject to

F(1)

x(1)

f(1)

rhs (2.15)...

F(k)x(k) f(k)rhs

V(1)E(1)x(1) + . . . +V(k)E(k)x(k) vrhs (2.16)

R(1)x(1) + . . . +R(k)x(k) rrhs (2.17)

such that

xi binary, 1 i k

The feasible solutions are returned as indicator vectors xi, 1 i k. If no feasible solution can be

found, the risk and cost limit constraints (2.12) (2.16)(2.13)(2.17) may have to be relaxed.

2.6 Database

QSRAs knowledge repository consists of two relational databases: ists_icat and networks. Both

databases were designed to be modularly expandable, easily mutable and free of the data redundancies

found in other databases. Hence, all relations were designed to be, at a minimum, in third normal form.

For more information on relational database design and normalization, see Rolland and Gillenson [Rol98,

pp.72-85] [Gil87].

2.6.1 ISTS ICAT

ists_icat holds data about software and its associated vulnerabilities. It is used to provide data and

parameters for software identification, risk assessment and risk management. The database contains 40

tables which can b e separated into four functional aggregates.

System This set of 17 tables holds data on software programs known as Systems. The inventoried

services and application executables are mapped to these Systems, down to the patch level.

Vulnerabilities This set of 13 tables holds data on vulnerabilities.

Relations This set of 5 tables links the Systems to Vulnerabilities.


36/143


37/143


Table 2.5: Vulnerabilities data sourcesResource Name URL(e),(l) SecuriTeam http://www.securiteam.com

(e),(l) Packet Storm http://packetstormsecurity.nl/(e),(l) New Order http://neworder.box.sk/(e),(l) Security Tracker http://www.securitytracker.com/(t),(a),(c), (e),(l) Network Security News http://www.security-update.com(t),(a),(c), (l) SecurityFocus http://online.securityfocus.comportal for (t),(a),(c),(e),(l) CVE http://cve.mitre.org(t),(a),(c) ISS X-Force Database http://www.iss.net/security_center(t),(a),(c) ICAT Metabase http://icat.nist.gov(t),(a),(c),(l) MARC http://www.theaimsgroup.com(t),(a),(c), (e),(l) @stake http://www.atstake.com/researchLegend (t)ype, (a)ccessibility, (c)onsequences, (e)xploits,(l)ifecycle

Risk Assessment This set of 5 tables holds entries pertaining to the calculated risk measure of a

network: The individual software risk, the risk of the IP-enabled device hosting that software, and

the aggregate network risk.

Risk Management This set of 8 tables holds the entries pertaining to risk management scenarios and

the alternative software makeup on the network

This database holds extremely detailed information about the software setup on the network. In

effect it is a gold mine for any attacker. As such, access to this database should be restricted to theQSRA application and senior trusted personnel. As a technical security measure, the database contents

as well as the communication stream between the QSRA application and networks should be encrypted.

A detailed makeup of the databases can be found in appendix B on page 92.

2.7 A QSRA example

Assume we have a host George that functions as a web and database server for the PUSH company. The

PUSH companys risk manager Susan knows that an immaculate, omnipresent George is vital for PUSHs

public image. The integrity of the data residing on it is important as well - but not its confidentiality,

since the information is public anyway. Of course, Susan would not want to see the server processes

taken over (and perhaps redirect the user to a rival company) or worse yet, have George be taken over

completely and used as a zombie in a DDoS attack against PUSH or another company (and be subjected

to potential liability suits). Susan thus assesses the loss consequences for George for the company. QSRA

calculates the risk (attack) probabilities after conducting the software inventorying and identification.

Software inventorying shows that George is a Debian Linux 3.0 host with service Apache 1.3.26 running,


38/143


Table 2.6: Risk management results for GeorgeSoftware Functionality Replacement software Migration cost Remote riskMySQL 3.23.50 database server MySQL 4.0.5 $3,000 $5,700

Apache 1.3.26 web server Apache 2.0.35 $2,000 $15,200

Table 2.7: Remote risk limits for GeorgeConsequence Type Remote risk limit Risk for original software Risk for new softwareAvailability $3000 $9,900 $2,700Integrity $3000 $9,500 $0Confidentiality $500 $0 $0Process $8,000 $27,300 $5,700Full $14,000 $49,500 $12,500

tied to a back-end running MySQL 3.23.50 database server. The respective vulnerabilities of these

services is shown in Table 2.11. Risk assessment reveals that its attack probabilities are pretty high.

Table 2.8 shows the data from Susan and the QSRA process so far. The total remote and local attack

consequence risk is $96,200 and $61,000, respectively.

Susan wants to keep Georges functionality. After consulting with management and senior network

administration, she sets the acceptable risk limits for George (see Table 2.7). She is not that concerned

with insiders (personel hires trustworthy people), but is concerned with outside attackers. She is willing

to spend roughly half of her calculated remote attack risk fix to change her risk profile, around $50,000.

With the risk and cost limits set, Susan invokes the risk management process. The optimizer chooses

two new programs, Apache 2.0.35 and MySQL 4.0.5, to substitute for their respective (more vulnerable)

predecessors. The vulnerabilities of these new services is shown in Table 2.10. Their migration costs

and remote risk are shown in Table 2.6. The risk profile is calculated for these services, as shown in

Table 2.9. The total remote and local attack consequence risk is now $20,900 and $0, respectively. This

represents a 78% remote risk reduction.

All of Susans requirements and PUSHs organizational needs have thus been met: The costs of

migrating to the new software are $5,000, which meets the cost limit of $50,000. The consequencerisk limits have not been exceeded, as is shown in Table 2.7. Since functionality is preserved, PUSHs

managers are content with the decision, as well. If no feasible alternative software makeup had been

found, either the migration costs or the risk limits would have had to be relaxed, and the risk management

process re-invoked.


39/143


Table 2.8: Risk assessment: Consequence losses and consequence probabilities for GeorgeType Consequence loss remote attack local attack

for George probability risk probability riskAvailability $10,000 0.99 $9,900 0.95 $9,500Integrity $10,000 0.95 $9,500 0.40 $4,000Confidentiality $0 0.87 $0 0.40 $0Process $30,000 0.91 $27,300 $0Full $50,000 0.99 $49,500 0.95 $47,500

Table 2.9: Risk management: Consequence losses and consequence probabilities for GeorgeType Consequence loss remote attack local attack

for George probability risk probability riskAvailability $10,000 0.27 $2,700 0 $0Integrity $10,000 0 $0 0 $0Confidentiality $0 0 $0 0 $0Process $30,000 0.19 $5,700 0 $0Full $50,000 0.25 $12,500 0 $0

Table 2.10: Vulnerabilities of Apache 2.0.35 and MySQL 4.0.5Type Apache 2.0.35 MySQL 4.0.5

remote local remote localAvailability 3 Integrity Confidentiality Process 2 Full 1

Table 2.11: Vulnerabilities of Apache 1.3.26 and MySQL 3.23.50Type Apache 1.3.26 MySQL 3.23.50

remote local remote localAvailability 1 1 2 Integrity 1 Confidentiality 1 Process 2 3 1Full 1


40/143

Chapter 3

Implementation of QSRA system

3.1 Findings

Java as an implementation language proved its worth in cross-platform development. It cannot be

stressed enough how much time is saved by the lack of the tedious memory leaks and pointer manipulation

that plague C and C++. Text parsing, however, is not Javas forte, and hence QSRA client development

took a disproportional amount of time.

The decision to design a relational, as opposed to a flat database was found to have been very

fortunate. Having all data relations in third normal form is advisable, since this enables additions and

mutations to be accommodated with few if any changes to the existing design.

3.2 Overview

As a principle, the QSRA application, clients and SQL database were implemented using royalty-free

software, with the simplest tools available. The implementation was to be portable and small. As such,

the following products were used to implement the QSRA application and client:

Language Suns Java 1.3.1, ensuring portability and ease of implementation[Mic01b]

Database MySQL ABs MySQL 3.23, the worlds most widely used open-source database [AB00]

Tools Victor Abells lsof 4.47, an software-to-port mapper for Unix to identify service executables;

Foundstones fport 1.33, a software-to-port mapper for Windows NT/2000/XP to identify service

executables; Michel Berkelaars lp_solve 2.0, a Java implementation of a mixed integer linear

program solver [Abe00] [Fou00] [Ber96]

28


41/143

CHAPTER 3. IMPLEMENTATION OF QSRA SYSTEM 29

The current QSRA and database implementation was meant to set up the framework for the method-

ology, a proof-of-concept software engineering implementation. It took more than eighteen months of

developing, testing, redesigning and debugging. The application contains around 6000 lines of source

code (200 kB of data), and the client applications around 900 lines (30 kB of data). The whole archived

file with the GUI and other helper classes totals 980kB of data.

The database went through a few redesigns before and during implementation. Implementing the

schemata was straightforward; thanks to the initial design decision to put all the relations in third

normal form, additions and mutations in the schema were easier to make. Around 400 system and 200

vulnerability records were inserted into the prototype ists_icat database.

Not implemented QSRA was first implemented without escalation attacks (negative synergy effects

between services and applications) in mind. The necessity to introduce escalation attacks became ap-

parent in the course of the research as more attention was given to vulnerability exploits and their

interactions with installed applications. The decision was made to incorporate escalation attacks into

the methodology, but defer implementation in the prototype. A work-around numerical framework,

which realized the updated methodology, was designed and implemented using Microsoft Excel.

The prototype QSRA client implementation returns data on services and operating systems only,

applications have to be inventoried manually. Well-behaved applications follow recommended instal-lation procedures. In the case of Windows machines, proper installation means that meta-data about the

installed software is recorded in the Windows Registry under the key HKEY_LOCAL_MACHINE\SOFTWARE.

In the case of Linux and BSD machines, there are similar database-like structures (such as packages.rpm

for Linux or _pkg-name_ for BSD) that get updated when standard installation procedures are followed

properly.

As a concomitant to the client not auditing the applications, the prototype QSRA application does

not take escalation attacks into account in its risk analysis calculations. The lack of this feature in

the prototype did not limit the experimental approach, it just added manual labor to an otherwise

automated risk calculation process.

Implementation and testing of inventorying applications on a client should take four weeks. Tieing

it into the QSRA application and testing this feature is more time-consuming and is estimated to take

at least two months. For risk management, the prototype QSRA application implementation does not

take time cost constraints into consideration when performing the risk management optimization. The

reasons for this was the belated realization that time can just be another $US metric after conversion.

This avoided the added complication of having to weight two metrics in the formulation of the objective


42/143


43/143

VulnerabilityAssessment

PRTCL

IP

NAME

OS

PORT

tcp

1.1.1.2

httpd

Linux2

.4

80

tcp

1.1.1.2

xinted

Linux2

.4

21

tcp

1.1.1.4

svchost

Win2K

135

udp

1.1.1.4

lsass

Win2K

500

tcp

1.1.1.5

svchost

WinXP

5000

udp

1.1.1.5

WinXP

38037

msgsys

Unix

lsof

WinXP

fport

Win2K

fport

(TCP)4447

,

4448

(1)

(2)

(3)

MSIIS5

80

MSIIS5

.1

VulnerabilityAssignment

tcp

1.1.1.2

httpd

Linux2

.4

80

PRTCL

IP

NAME

OS

PORT

tcp

1.1.1.2

xinetd

Linux2

.4

21

port

version

patch

MSIIS3

80

MSIIS3

.0andprev.

Apache1

.3

80

Apache1

.3.2

0

MaphttpdtoApache1

.3.2

0

MySQL

ists

_icat

networks

SysVerPatch

commonP

ort

CP2SVP

IP2SVP

(1)

(2)

(4)

(3)

(5)(5)

MySQL

ists

_icat

networks

Vuln

Vuln2ExploitFactors

IPRisk

IndivRisk

RiskAssessment

NetworkRisk

IPRisk

IndivRisk

Full

Trust

Process

Integrity

Availability

Confidentiality

Consequence

Risk

340

200

12001

00 0 90

Full

Trust

Process

Integrity

Availability

Confidentiality

Consequence

Risk

600

260

2300

100

120

300

IPaddress

Risk

1.1.1.4

1290

1.1.1.2

3680

1.1.1.5

230

Apache1..3

..20

1.1.1.2

(1)

(2)

(4)

(3)

MySQL

ists

_icat

networks

Cost

Class

IPRisk

IndivRisk

RiskManagement

Scenario

IP

Risk

Improv

Scen2

1.1.1.2

3680

0%

Scen1

1.1.1.2

2750

25%

Scen1

1.1.1.4

800

38%

Full

Trust

Process

Integrity

Availability

Confidentiality

340 2001200

100 0 90

consequence

Lim

it

money

time

12000

148

resources

Lim

it

IP

Risk

1.1.1.2

3680

Scenarios

1.1.1.4

1290

1.1.1.5

230

Baseline

optimize

System

Scenario

(1)

(2)

(5)

(4)

(6)

(3)

QSRA

connectstoassessorinstalledinclient

onTCPport4447/4448.

(3)QSRA

parseslsof

andfportdataand

displaysthelistofser

vices(calledhere

CommunicationProcess,CP)forevery

IP-enableddeviceinventoried

Clients(IP-enab

le

devices)send

raw

lsof,

fport

data

about

open

ports

and

the

programsbound

tothem

(1)

Process

Port

Proto

Path

svchost

135

TCP

svchost.exe

persfw

44334

TCP

persfw.exe

(2)CommandDeviceSizeNodeName

xinetdTCP*:21

(LISTEN)

xinetdTCP*:44

3(LISTEN)

httpdTCP*:80

(LISTEN)

(1)QSRA

requestsselectionofsoftware

(calledhereSystemso

rSVPs)from

ists

icat.

(3)QSRA

displayslistofprogramsandwaits

forusertomapCPst

oSVPs

(4)Mappingdataissenttonetworks

(2)ists_

icat

returnsalistofprograms.

(5)networks

sto

resaScenario:

theCPs

andSVPs,aswellastheCP-to-SVP,SVP-

to-IP,IP-to-netw

orkmappings.

(1)QSRA

requeststh

enumericSVP

vulnerabilityinformat

ionforthecurrent

networkscenario.

(3)QSRA

calculatestheIndivRisk,IPRisk

andNetworkRiskofthescenario,usingdata

from

bila r phd thesis

Documents