responsible model development
DESCRIPTION
TRANSCRIPT
Responsible Model
Development
Sell Data Management
&
Sensitivity Analysis
When the client asks how wrong the model can be, we should be able to look into his eyes and say,
The recommendation based on the modeling results has an error of $0.5Million.
And that’s something we believe can sell.
Tell the client the model can be wrong is the right thing to do.
And to do the thing right, we need to tell how wrong it is.
Right and wrong are just the
flip sides of RISKs.
Business Risk Exposure =
Probability of Failure X
Consequences of Failure
Asset Management Guys tell us:
Business Risk Exposure =
Probability of Model being WrongX
Consequences of Model being Wrong
How Wrong the Model is?
An Example
Pipe 1 Pipe 3Pipe 2If GIS is right
$30,000
To Upsize
Pipe 1 Pipe 2If GIS is wrong
Pipe 3
No Improvement
Needed
Consequences of being wrong: spend $30,000 on a good pipe
The error of the recommendation is$30,000 * 70% =
$21,000
Probability of being wrong: GIS with no source attribution, 70% chance of a typo.
BRE = $30,000 * (0.2+0) = $6,000
More generic formula
It is the modeler’s responsibility to inform the client the limitation of the model, and the extent.
It is a good way to sell Data Management and Sensitivity Analysis services.
It is a good way to add true value and gain trust.
Model responsibly.
Reference: Rules for responsible modeling, James, William, CHI http://www.chiwater.com/Publications/Books/r184.asp