model failures: lessons to be learned · the tacoma narrows bridge. 11 the tacoma narrows bridge....
TRANSCRIPT
Model Failures:
Lessons to be Learned
Prepared by: John Herzfeld, FCAS, MAAAMilliman, Inc.
2
3
Human judgment in actuarial modeling
§ Model selection and fitting
§ Model testing
§ Communication of model results
Almost everything an actuary does involves models
4
The human element introduces dangers
Overconfidence Gaming the System
Falling in Love with the Model
5
The 2007-2008 Financial Crisis
Local Banks,Local Risk
Mortgage “Originators”
&Securitization
Gaming of Rating
Agencies’ Models
Misplaced incentives everywhere
6
The 2007-2008 Financial Crisis
“The Wall Street firms learned how to massage these models, change one or two little inputs and then get a better rating as a result. They learned how to game the rating agency’s models so that they could put lesser quality bonds in these portfolios, still get a high rating, and then sell the junk that they might not otherwise have been able to sell.”
—Gretchen Morgenson, (New York Times reporter), “Examining Goldman Sachs,” NPR interview in Fresh Air, May 4, 2010
7
Weather Models
Once in 100
years
1% chance per year
8
Weather Models
9
Weather Models
Extreme events are not as “extreme” as we think
Real-life distributions are “thick in the tail”
Chances of a 1000-year flood occurring in an individual’s lifespan? Over 5%
10
The Tacoma Narrows Bridge
11
The Tacoma Narrows Bridge
12
The Tacoma Narrows Bridge
13
Reliance Insurance Company
• Principal rating agency for licensed Property Casualty Insurers in the US
A.M. Best
• One of the oldest insurers in the United States
Reliance Insurance Company
14
Reliance Insurance CompanyThe company was “A-” rated in June 2000
when it stopped writing new business
In August 2000 the rating was lowered to “C”
In January 2001 the rating was “E”
Reliance was bankrupt in early 2001
15
Conclusions
Incentives Matter
Strive for Clarity
Model Failure is a Learning Opportunity
16
Thank You
John HerzfeldMilliman, Inc.201 Edgewater DriveSuite 289Wakefield, MA 01880USA
+1 781 213 6261