managing & mitigating model riskaelices/teaching/2011_financial_risk... · 2012-03-19 ·...
TRANSCRIPT
![Page 1: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/1.jpg)
Managing & Mitigating Model RiskManaging & Mitigating Model Risk
Model Validation GroupModel Validation GroupArea of MethodologyArea of MethodologySantanderSantander
Alberto ElicesAlberto Elices
Financial Risk ManagementFinancial Risk ManagementMasters in Mathematical EngineeringMasters in Mathematical EngineeringUniversidad Complutense de MadridUniversidad Complutense de MadridMadrid, May 23rd – June 3rd, 2011Madrid, May 23rd – June 3rd, 2011
![Page 2: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/2.jpg)
2
Outline
Introduction. Taxonomy of model risk. Model validation. Reconcile FO and Risk interests: FVA (Fair Value Adjustment). Conclusions.
![Page 3: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/3.jpg)
3
Introduction
After the crisis in the 2nd half of 2007, a big concern about pricing models has been raised. Risk management and model validation raise now considerably more attention. Model validation: Validation of model implementation is no longer enough. Periodic and comprehensive review of pricing models. Estimation of model risk.
Risk management: Calculate and apply fair value adjustment (FVA). Limit model risk exposure (reduce volume of operations).
![Page 4: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/4.jpg)
4
Taxonomy of model risk
Bad implementation. Missing a key source of risk. Uncertain model parameters. Difficulty to estimate market data. Wrong use of model. Market evolution risk.
![Page 5: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/5.jpg)
5
Taxonomy of Model Risk: Bad implementation
Incorrect derivation of equations.
Bugs in the code.
Low performance.
Non-intuitive selection of time/space steps or MC paths.
Unstable or non-converging numerical schemes.
Absence of error trapping routines to avoid bad inputs.
![Page 6: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/6.jpg)
6
Taxonomy of Model Risk: Missing a key source of risk
Inappropriate underlying stochastic process.
Robust calibration to market but unrealistic evolution or vice versa.
Missing market variables which affect pricing.
Under-dimensioned model. Some factors are assumed to be deterministic instead of stochastic.
Incorrect assumptions about relations between underlying variables.
![Page 7: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/7.jpg)
7
Taxonomy of Model Risk: Uncertain model parameters
One or more model parameters cannot be properly estimated from liquid market prices.
Wrong set of underlying calibration instruments.
Smile not accounted for.
Inappropriate treatment of extreme cases.
![Page 8: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/8.jpg)
8
Taxonomy of Model Risk: Difficulty to estimate market data
Unavailable market quotes: data highly dependent on sample length.
Incomplete market quotes: inconsistent and non-arbitrage free data.
Complete market quotes: estimation of data from bid - ask.
![Page 9: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/9.jpg)
9
Taxonomy of Model Risk: Wrong use of model
Model used outside valid range.
Product priced with a model for which it was not designed.
Wrong configuration parameters (insufficient time steps or simulation runs).
![Page 10: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/10.jpg)
10
Taxonomy of Model Risk: Market evolution risk
An initially valid model can become inadequate due to:
Changing market conditions.
Changing market pricing consensus.
Market instability.
![Page 11: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/11.jpg)
11
35
3 Model Validation functionModel Validation
Background and motivation Model testing
• Model adequacy analysis• Test complex models in simple cases• Premium analysis• Greek and stability analysis
Integration in corporate systems Conclusions and recommendations
Validation Process
![Page 12: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/12.jpg)
12
35
• Why was this product developed?• Is it a product designed for a campaign or for a single operation?• What are the market conditions which motivate this product?• Is it a change or improvement of an existing product?• If the product is part of a more complicated deal, what is the whole picture?• Similarities to any other previously validated models
3 Model Validation function
Get important information before validation starts as it may considerably change the strategy of tests
Validation Process
Background and motivation
![Page 13: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/13.jpg)
13
35
• Why a particular model was chosen? Which simplifications are involved, are they reasonable?• Study the adequacy of the calibration and skew treatment• Valid range where inaccuracies or approximations are acceptable
3 Model Validation function
Adequacy of the model from a theoretical and practical point of view
Validation Process
Model Testing – Model adequacy analysis
![Page 14: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/14.jpg)
14
35
• Reduce the model under validation to vanilla products, comparing both premium and sensitivities
- Run a few cases manually varying product inputs- Run a more systematic analysis varying market data (e.g. spot
prices, volatility or interest rate levels)
3 Model Validation function
Test models against simpler known solutions
Validation Process
Model Testing – Complex model in simple cases
![Page 15: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/15.jpg)
15
35
• Define a set of scenarios used to test the premium:- Random: define limits to the deal and market input parameters- Manual: justify why they have been chosen
• Calibration tests• Analyze differences between Front Office and Validation models:
- Do not ignore small discrepancies. Track down their origin!• Display plots or tables with the results:
- A histogram might be appropriate for random scenarios- When a single parameter is varied at a time, a series of plots is
a simple and clear option- When two parameters are varied, display a series of tables
3 Model Validation functionValidation Process
Model Testing – Premium analysis
![Page 16: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/16.jpg)
16
35
• Convergence analysis:- Testing the accurateness of premium varying Monte Carlo
simulations or grid parameters for PDEs• Life cycle analysis:
- Test evolution of premium, cash flow account and convergence to the final payoff throughout the life of the option
• Robustness analysis under realistic market parameters
3 Model Validation function
Justifying differences achieves a common sense compromise between extreme and real market scenarios
Validation Process
Model Testing – Premium analysis
![Page 17: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/17.jpg)
17
35
Front OfficeModel
Risk Dept.Model 0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
0.0 2.0 4.0 6.0 8.0 10.0
Discrepancies? NOYES
Hypothesis
error
FVA Validation
3 Model Validation functionValidation Process
Model Testing – Premium analysis
![Page 18: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/18.jpg)
18
35
• Plot premium and sensitivities:- Varying spot prices, volatility levels, maturity, interest rate
levels, correlation, etc.- Testing their variation range and whether they evolve smoothly
• Compare actual premium change and its prediction using Taylor expansion• Life cycle analysis: Test smooth evolution of sensitivities
3 Model Validation function
Sensitivities need to be stable enough for hedging and risk calculation (e.g. gamma stability for VaR in Madrid)
Validation Process
Model Testing – Greek and stability analysis
![Page 19: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/19.jpg)
19
35
• Stress the initial market conditions to check its behaviour
3 Model Validation function
-1.000.000
-800.000
-600.000
-400.000
-200.000
0
200.000
400.000
600.000
800.000
-2 -1 0 1 2Vega
V ariación de la curva de tipos
Is the model stable?
NO if market moves to this area!
Validation Process
Model Testing – Greek and stability analysis
![Page 20: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/20.jpg)
20
35
• Different environments are used for validation:- Excel spreadsheet + add-in- IL text files
• It is necessary to ensure that the model is correctly integrated into the final platform
3 Model Validation function
Models are global and unique, but there are different corporate systems
Validation Process
Integration in corporate systems
![Page 21: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/21.jpg)
21
35
3 Model Validation function
Questions to be adressed:• Has the model passed validation?• Is the model fit for its intended purpose?• What are the boundaries within the model is suitable?• What are the weaknesses and limitations of the model?• What are the recommendations for best use?• If differences between models appear, are they acceptable?• Is the model stable and robust enough for daily management without support?• Is it necessary to take into account a Fair Value Adjustments?
Validation Process
Conclusions and recommendations
![Page 22: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/22.jpg)
22
How to reconcile FO and Risk department interests?: FVA
The FVA should cover the expected hedging loss and its uncertainty: When hedging is carried out with a model with aggresive prices, the expected hedging loss is the fair minus the aggresive price: that difference plus a cushion for its uncertainty is the FVA.
FVA as a means to approve campaigns using limited models with controllable risk: A FVA allows accomplishing campaigns which would not possible with a slower more sophisticated model.
FVA to foster improvement of FO models: Models with limitations should be given FVA which should be released the more the model is improved.
![Page 23: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/23.jpg)
23
How to reconcile FO and Risk department interests?: FVA
FVA calculation philosophy: They should be transparent, easy to compute. They should be dynamic, stable, with smooth evolution through time (they should decrease approaching expiry). They should balance risk limitation and trading mitigation. Front Office should be able to reproduce them.
![Page 24: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/24.jpg)
24
How to reconcile FO and Risk department interests?: FVA
How FVA can be calculated: Use FVA tables calculated from studies (either with real or toy models).
Use FO pricing models to estimate model risk:– Changing unobserved or non-calibrated model parameters (mean reversion, correlations, etc).
– Compare prices of deals valued with different FO models (better models might take too long on a daily basis).
Simulate or back test portfolio hedging: sometimes impractical.
![Page 25: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/25.jpg)
25
Example of provisions: liquidity of volatility surface.
Liquidity provisions should allow unwinding a position.Sensitivities are diffused through time (towards lower more liquid maturities) and moneyness (towards closer to ATM).
Diffusion of offsetting sensitivities for the 10 year tenor
with W=0.5
0
200
400
600
800
1,000
1,200
1 2 3 4 5 7 10 15 20 25 30 40 50 60 70
Tenor
Se
nsi
tivi
tie
s (£
k)
Original Sensitivity 1 tailed distribution2 tailed distribution
Diffusion of sensitivities for the 5% strike with W=0.05
ATM = 6%
0
200
400
600
800
1,000
1,200
2.50%
3.00%
3.50%
4.00%
4.50%
5.00%
5.50%
6.00%
6.50
%7.0
0%7.5
0%8.0
0%
Tenor
Sen
siti
viti
es (
£k)
2 tail diffusion 1 tailed diffusion
Original Sensitivities
![Page 26: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/26.jpg)
26
Example of provisions: liquidity of volatility surface.
Liquidity provisions in practice:Movements of implied volatility surface might be mainly explained by parallel and slope (skew) shifts.Parallel shifts: hedged with ATM options.Slope shifts: hedged with risk reversals sensitive to slope.
![Page 27: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/27.jpg)
27
Example of provisions: uncertainty of volatility.
A Heston model calibrated to market is considered:Confidence intervals for total variance & vol are calculated.Up and down increments of vol are calculated vs maturity.Provision: max(|up & down movement * vega|).
![Page 28: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/28.jpg)
28
Example of calculation of FVA: FX self-quanto options.
Self quanto options are the worst case for correlation is 1. Four valuation methods are compared:
Black Scholes with at-the-money (ATM) volatility (left equation).Black Scholes with volatility selection (VolSelection) to the strike level of the non-quanto option (left equation).
Vega-volga-vanna (VVV) method.Estimation of probability distribution of ST using Gatheral’s parametrization of implied volatility surface (right equation)
( )[ ]),( TtPKSprice EURQ
EUR T
+−= E
tttEURt
USDtQ
t
Qt dWdtrrSdS σσ ++−= )( 2
( )[ ]0
1),(S
TtPSKSprice USDTTEUR+−= E
ttEURt
USDt
t
t dWdtrrSdS σ+−= )(
ATM VolSelection Gatheral VVV0.3704 0.3712 0.3990 0.3950
![Page 29: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/29.jpg)
29
Examples of bespoke validation: FX self-quanto options.
Gatheral’s parametrization is fitted to market data:Left plot: market fitting (maximum difference is 5.65bp).Right plot: implied probability distribution.
Delta 0.05 0.1 0.15 0.2 0.25 0.3 0.5 0.7 0.75 0.8 0.85 0.9 0.95
Vol 35.87 33.80 32.79 32.13 31.65 31.25 30.55 30.59 30.77 31.03 31.47 32.22 33.80
![Page 30: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/30.jpg)
30
Examples of bespoke validation: FX self-quanto options.
A set of a hundred random scenarios changing dates, interest rates and strikes preserving shape of volatility surface is generated. Call option prices are compared.
-0.0600
-0.0400
-0.0200
0.0000
0.0200
0.0400
0.0600
0.0800
0 10 20 30 40 50 60 70 80 90 100
Gatheral-VVVGatheral-VolSelectGatheral-ATM
![Page 31: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/31.jpg)
31
Examples of bespoke validation: FX self-quanto options.
Put option prices give a lot less differences for same scenarios. Volatility selection or ATM methods can significantly diverge from fair value. Best method: volga-vanna.
-0.0100
-0.0050
0.0000
0.0050
0.0100
0.0150
0.0200
0 10 20 30 40 50 60 70 80 90 100
Gatheral-VVVGatheral-VolSelectGatheral-ATM
![Page 32: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/32.jpg)
32
Conclusion: Managing & Mitigating Model Risk
Model validation. Quantifying model risk of a desk. Periodic & comprehensive review of models. Limit calculations. Fair Value Adjustments (FVA). Premium sensitivity to non-calibrated parameters. Comparison with other models. Simulation of hedging strategies.
![Page 33: Managing & Mitigating Model Riskaelices/teaching/2011_Financial_Risk... · 2012-03-19 · Managing & Mitigating Model Risk Model Validation Group Area of Methodology Santander Alberto](https://reader034.vdocument.in/reader034/viewer/2022042301/5ecbc156f3b6ce0ec2303e13/html5/thumbnails/33.jpg)