The COTOR ChallengeThe COTOR ChallengeCommittee on the Theory of RiskCommittee on the Theory of Risk

November 2004 Casualty Actuarial Society November 2004 Casualty Actuarial Society Annual MeetingAnnual Meeting

Phil Heckman’s Remarks:Phil Heckman’s Remarks:

- Distribution of Sample Estimator- Distribution of Sample Estimator

- Fitting Mixtures- Fitting Mixtures

- Visualization Tools- Visualization Tools

Distribution of EstimatorDistribution of Estimator

Good to know: Distribution of sample Good to know: Distribution of sample estimator for 5x5 layer.estimator for 5x5 layer.

Use Stuart’s TIG(.9,2000,.9) Use Stuart’s TIG(.9,2000,.9) distribution to simulate 9,999 distribution to simulate 9,999 samples of 250 events each.samples of 250 events each.

Tabulate Sum[Med(0,X-5M,5M)]/250Tabulate Sum[Med(0,X-5M,5M)]/2502.5% to 97.5% => (0, 40,000)2.5% to 97.5% => (0, 40,000)

Distribution of Estimator Distribution of Estimator 22

Pr 5 x 5 (250)

0.0001

0.0010

0.0100

0.1000

1.0000

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000

Distribution of Estimator Distribution of Estimator 33

Note mass points: 5M/250 = 20,000.Note mass points: 5M/250 = 20,000. Sample estimator is most naïve Sample estimator is most naïve

approach. Not surprising if para-approach. Not surprising if para-metric methods do better.metric methods do better.

Subjectivity remains in confidence Subjectivity remains in confidence bounds. Question is what do we bounds. Question is what do we really know, how can we use that really know, how can we use that knowledge?knowledge?

Fitting MixturesFitting Mixtures Insurance data tend not to be Insurance data tend not to be pure textbook distributions: too pure textbook distributions: too much going on at once.much going on at once. Try linear mixtures. Intuition Try linear mixtures. Intuition says BI/PD & MO/Lost Time behave says BI/PD & MO/Lost Time behave differently. differently. A lognormal mixture may A lognormal mixture may succeed where single LN fails. succeed where single LN fails. E.g. COTOR Challenge 1. E.g. COTOR Challenge 1. EstimateEstimate mixing probabilities. mixing probabilities.

Mixture ExampleMixture Example

Next slide shows a log/logit plot of Next slide shows a log/logit plot of WC claim size probabilities. Data are WC claim size probabilities. Data are wild, not generated.wild, not generated.

Model is mixture of two lognormals. Model is mixture of two lognormals. Single LN is shown for comparison.Single LN is shown for comparison.

Prob Mu Sig Calc Mean0.7549 5.6154 1.0550 479.100.2451 9.0020 1.2846 18,529.051.0000 4,903.39

Sample Mean: 4,772.01

Mixture FitsMixture FitsLog/Logit Plot

-10.0

-8.0

-6.0

-4.0

-2.0

0.0

2.0

4.0

6.0

8.0

10.0

1 10 100 1,000 10,000 100,000 1,000,000

Mixture RemarksMixture Remarks

Mixture Model fits very closely Mixture Model fits very closely except for lowest points.except for lowest points.

Single LN misses badly.Single LN misses badly. Should have stats for parameter Should have stats for parameter

estimates, but don’t. (Sorry.)estimates, but don’t. (Sorry.) Added flexibility plus accord with Added flexibility plus accord with

intuition make this a useful method.intuition make this a useful method.

Visualization ToolsVisualization Tools

Nonparametric approaches are Nonparametric approaches are available for visualizing distributions.available for visualizing distributions.

Kaplan-Meier (Product Moment) Kaplan-Meier (Product Moment) estimator developed for survival estimator developed for survival analysis. Can be adapted for claim analysis. Can be adapted for claim emergence, censored losses, etc.emergence, censored losses, etc.

No need to preview, choose intervals, No need to preview, choose intervals, etc.etc.

Some other time.Some other time.

Consequences of Assuming Consequences of Assuming NormalityNormality

The frequency of extreme events is The frequency of extreme events is underestimated – often by a lotunderestimated – often by a lot

Example: Long Term CapitalExample: Long Term Capital• ““Theoretically, the odds against a loss such as Theoretically, the odds against a loss such as

August’s had been prohibitive, such a debacle August’s had been prohibitive, such a debacle was, according to mathematicians, an event so was, according to mathematicians, an event so freakish as to be unlikely to occur even once freakish as to be unlikely to occur even once over the entire life of the universe and even over the entire life of the universe and even over numerous repetitions of the universe”over numerous repetitions of the universe” When Genius FailedWhen Genius Failed by Roger Lowenstein, p. 159 by Roger Lowenstein, p. 159

Criteria for JudgingCriteria for Judging

New and creative way to solve the New and creative way to solve the problemproblem

Methodology that practicing Methodology that practicing actuaries can useactuaries can use

Clarity of expositionClarity of exposition Accuracy of known answerAccuracy of known answer Estimates of confidence intervalEstimates of confidence interval

Table of ResultsTable of ResultsResponderResponder MeanMean Lower CLLower CL Upper CLUpper CL MethodMethod

AA 9,500.009,500.00 450.00450.00 17,500.0017,500.00 Inverse Logistic SmootherInverse Logistic Smoother

BB 6,000.006,000.00 0.000.00 26,000.0026,000.00 Kernel Smoothing/BootstrappingKernel Smoothing/Bootstrapping

CC 12,533.0012,533.00 2,976.002,976.00 53,049.0053,049.00 Log Regression of Density Function on large Log Regression of Density Function on large claimsclaims

DD 2,400.002,400.00 ?? ?? Generalized ParetoGeneralized Pareto

EE 6,430.006,430.00 1,760.001,760.00 14,710.0014,710.00 Fit distributions to triple logged data. Used Fit distributions to triple logged data. Used Bayesian approach for mean and CIBayesian approach for mean and CI

F1F1 10,282.0010,282.00 2,089.002,089.00 24,877.0024,877.00 Scaled ParetoScaled Pareto

F2F2 30,601.0030,601.00 6,217.006,217.00 74,038.0074,038.00 ParetoPareto

GG 4,332.654,332.65 297.34297.34 7,645.867,645.86 Empirical Semi SmoothingEmpirical Semi Smoothing

H1H1 2,700.002,700.00 0.000.00 17,955.0017,955.00 Single Parameter Pareto/Simulation for Single Parameter Pareto/Simulation for Confidence IntervalsConfidence Intervals

H2H2 8,772.008,772.00 0.000.00 54,474.0054,474.00 Generalized Pareto/Bayesian SimulationGeneralized Pareto/Bayesian Simulation

True MeanTrue Mean 6810.006810.00

Observations Regarding Observations Regarding ResultsResults

These estimations are not easyThese estimations are not easy Nearly 13 to 1 spread between lowest and Nearly 13 to 1 spread between lowest and

highest meanhighest mean Only 10% of answers came within 10% of Only 10% of answers came within 10% of

right resultright result All responders recognized tremendous All responders recognized tremendous

uncertainty in results (range from upper to uncertainty in results (range from upper to lower CL went from 8 to infinity)lower CL went from 8 to infinity)

Our statistical expert could not understand Our statistical expert could not understand the description of the method of 30% of the description of the method of 30% of the respondentsthe respondents

ObservationsObservations All but 2 of the methods relied on approaches commonly All but 2 of the methods relied on approaches commonly

found in the literature on heavy tailed distributions and found in the literature on heavy tailed distributions and extreme valuesextreme values

It is clear that it is very difficult to get accurate estimates It is clear that it is very difficult to get accurate estimates from a small samplefrom a small sample

The real world is even more challenging than thisThe real world is even more challenging than this• 250 claims probably don’t follow any known distribution250 claims probably don’t follow any known distribution• TrendTrend• DevelopmentDevelopment• Unforeseen changes in environmentUnforeseen changes in environment• Consulting with claims adjusters and underwriters should Consulting with claims adjusters and underwriters should

provide valuable additional insightsprovide valuable additional insights

ObservationsObservations The closest answer was 5% below the true The closest answer was 5% below the true

meanmean Half of the responses below the true mean, Half of the responses below the true mean,

Half were aboveHalf were above Average response was 40% higher than the Average response was 40% higher than the

meanmean Average response (ex outlier) was within 2% of Average response (ex outlier) was within 2% of

the meanthe mean Read: Read:

““The Wisdom of Crowds: Why the Many are Smarter The Wisdom of Crowds: Why the Many are Smarter than the than the Few and How Collective Wisdom Shapes Few and How Collective Wisdom Shapes Business, Economics, Business, Economics, Societies and Nations”Societies and Nations”

by: James Surowieckiby: James Surowiecki

Implications for Insurance Companies?Implications for Insurance Companies?

SpeakersSpeakers

MeyersMeyers EvansEvans FlynnFlynn WoolstenhulmeWoolstenhulme HeckmanHeckman

Announcement of WinnersAnnouncement of Winners

Louise Francis – COTOR ChairLouise Francis – COTOR Chair

Possible Next StepsPossible Next Steps

Make the results of the challenge Make the results of the challenge available to the membershipavailable to the membership

COTOR subcommittee to evaluate COTOR subcommittee to evaluate how to make techniques readily how to make techniques readily availableavailable

Another round making the challenge Another round making the challenge more real worldmore real world

Include trend and development Include trend and development Give multiple random samplesGive multiple random samples