Partial specification of risk models

Tim Bedford

Strathclyde Business School

Glasgow, Scotland

Contents...

Modelling considerations Information or KL divergence Vines Copula assessment Conclusions

Modelling goals

Decision Theory* perspective slightly different to Statistics*

Modelling aims How do we judge what model is appropriate?

– Type– Outputs– Level of detail for cost-effective modelling– Bottom-up or top-down

*Caveat: convenient labels for discussion only. Not intended to describe the views of any actual person.

What is a model?

Device for making predictions Creative activity giving understanding A statement of beliefs and assumptions

Input data Real system Output

Input data Model Output

Nominal versus non-nominal

What behaviour are we trying to capture in risk models?

Nominal / Non-nominal– EVT is trying to model the extremes of nominal

behaviour– Technical risk models try to find non-nominal

behaviour Discrete or continuous nature of departures

from nominal state is important from modelling perspective

Model as a statement of beliefs

Often many different plausible statistical models consistent with the data

May be identifiability problems Non-statistical validation is required

Providing finer grain structure to model arising from knowledge of context can be a way of providing validation

Model dimensions

George Mitchell describes 7 dimensions on which models can be compared– Actuality – abstract– Black box – structural– Off the shelf – purpose built– Absolute – relative– Passive – behavioural– Private – public– Subsystem – whole system

Model dimensions - 2

Black box Structural

FT ISRDStatisticalRegression

Predictive Explanatory

Instrumental Realistic

Macro Micro

Back of envelope

Large computer

Role of EJ

•Decision variables introduced in the model•Causal connections differ from correlated connections

Black box Structuralmodel

Example – common cause

Modelling dependent failure in NPPs etc Existing models used for risk

assessment use historical data but give no sights– Alpha factor model– Multiple greek letter model – Etc etc

Common cause model

Role of experts

Structuring – providing framework, specifying important variables, conditional independencies

Quantifying – providing assessments on quantities that they can reasonably assess

Essentially, all models are wrong, but some are useful

George Box

General research theme

Develop good ways to determine a decision model when only a partial specification is possible

Mainly working in technical/engineering applications

Expert assessment methods

Many methods for expert assessment of distributions - for applications in reliability/risk often non-parametric

Experts provide input by– Means, covariances..– Marginal quantiles, product-moment correlations– Marginal quantiles, rank correlations

Here look at methods for building up a subjective joint distribution

Building a joint distribution

Assume experts have given us information on marginals…

How do we build a joint distribution with information from experts?– Iman-Conover method– Markov trees– Vines

Copula

Joint distribution on unit square with uniform marginals Copula plus marginals specify joint distribution

– If X~F and Y~G then (F(X),G(Y)) is a copula

Any (Spearman) rank correlation is possible between –1, 1. But range of PM correlation depends on F and G

X

Y

(F(x),G(y))

G(Y)

F(X)

Information

Also known as – Relative entropy– Kullback Leibler

divergence Coordinate free

measure Requires specification

of background distribution– Another role for expert? – “Other things being

equal....”

dQ

dQ

dP

dQ

dPQPI log)|(

Minimum information copulae

Partially specify the copula, eg by (rank) correlation

Find “most independent” copula given information specified

Minimize relative information to independent copula= uniform distribution

Equivalent to min inf in original space

dudvvufvuffI )),(log(),()(

Markov trees and Vines

(Minimum information) copulae used to couple random variables

Marginals specified plus certain (conditional) rank correlations

Main advantage is no algebraic restrictions on correlations

Disadvantage is difficulty of assessing correlations

Decomposition Theorem

Markov tree example

1

Vine example

Vine example

Vine example

Vine example

Vine example

Information decomposition…

Information decomposition

Information calculation

So…

You can build up multivariate distributions from bivariate pieces

Minimum information pieces give global minimum information

But is it realistic to elicit correlations?

Observable quantities and expert assessment Experts are best able to judge observable

functions of data Distributions of such functions are not free,

restrictions depend on– marginals – other functions being assessed

REMM project feeds back “contradictory” information real-time to allow experts to reflect on inconsistencies

PARFUM method allows probabilistic inversion of random quantities to build joint

Formulation as minimal information problem Define domain specific functions of the

variables to obtain quantiles

X

Y

(F(x),G(y))

G(Y)

F(X) h

h(x,y)Expectations of regions are fixed by quantiles

Examples

Product Moment correlation– If marginals are known then can just consider

range of E(XY)– Equivalent to looking at E(F-1(X)G-1(Y))– All possible values of this can be reached by the

min information distributions– Can map out possible values using information

function as measure of distance to infeasibility

Differences – quantiles for |X-Y|

Example - exp marginals FR 1, 2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

-5 -4 -3 -2 -1 0 1 2 3 4 5

E(XY) as a function of lambda

Example - exp marginals FR 1, 2

-20

0

20

40

60

80

100

120

140

160

180

200

-5 -4 -3 -2 -1 0 1 2 3 4 5

Information as a function of lambda

Copula density for Lambda=2

1 3 5 7 9

11 13 15 17 19S1

S6

S11

S16

0

2

4

6

8

10

12

14

Copula density for Lambda=-2

1 3 5 7 9

11 13 15 17 19

S1

S6

S11

S16

0

0.5

1

1.5

2

2.5

3

3.5

Example – constraints on X-Y

1 3 5 7 9

11 13 15 17 19

S1

S6

S11

S16

0

2

4

6

8

10

12

14

Marginals as before

Expert assesses

P(X-Y<0.3)=0.3

P(X-Y<0.9)=0.7

Sequential – Step 1

Sequential – Step 2

Sequential – Step 3

Conclusions

General problem of eliciting dependencies from experts

Copulae are a good tool, but how do we select the copula?

Can use vines to get simple parameterisation of covariance matrix

Can tie together observables and copulae using interactive computer based methods

Questions

Eliciting dependence from experts – some work done but more needed

General guidance to use Expert Judgement to add structure and variables to existing models, or to link models

Find ways to incorporate “features” specified by experts

Methods must recognize limitations of experts– All the usual biases– Poor in the tails– Insight not much deeper than the data

Contributors/Collaborators

Roger Cooke Dorota Kurowicza Anca Hanea Daniel Lewandowski Lesley Walls John Quigley Athena Zitrou

D1AD2 algorithm - 1

If specify expectations of functions

or equivalently of

then minimally informative density has form

),('...,),,('),,(' 21 yxhyxhyxh k

),(...,),,(),,( 21 vuhvuhvuh k

),()()( 21 vuavdud

)),(...),(),(exp(),( 2211 vuhvuhvuhvua kk

D1AD2 algorithm - 2

View discretised density as matrix product D1AD2 where Di are diagonal

Iterative algorithm generates D1 and D2

normalising by

Iteration is contraction in hyperbolic metric