responsible model development

Responsible Model

Development

Sell Data Management

&

Sensitivity Analysis

When the client asks how wrong the model can be, we should be able to look into his eyes and say,

The recommendation based on the modeling results has an error of $0.5Million.

And that’s something we believe can sell.

Tell the client the model can be wrong is the right thing to do.

And to do the thing right, we need to tell how wrong it is.

Right and wrong are just the

flip sides of RISKs.

Business Risk Exposure =

Probability of Failure X

Consequences of Failure

Asset Management Guys tell us:

Business Risk Exposure =

Probability of Model being WrongX

Consequences of Model being Wrong

How Wrong the Model is?

An Example

Pipe 1 Pipe 3Pipe 2If GIS is right

$30,000

To Upsize

Pipe 1 Pipe 2If GIS is wrong

Pipe 3

No Improvement

Needed

Consequences of being wrong: spend $30,000 on a good pipe

The error of the recommendation is$30,000 * 70% =

$21,000

Probability of being wrong: GIS with no source attribution, 70% chance of a typo.

BRE = $30,000 * (0.2+0) = $6,000

More generic formula

It is the modeler’s responsibility to inform the client the limitation of the model, and the extent.

It is a good way to sell Data Management and Sensitivity Analysis services.

It is a good way to add true value and gain trust.

Model responsibly.

Reference: Rules for responsible modeling, James, William, CHI http://www.chiwater.com/Publications/Books/r184.asp