scioinspire corp proprietary & confidential. copyright 2008 september, 2008 so you’ve got a...

SCIOinspire Corp Proprietary & confidential. Copyright 2008

September, 2008

So You’ve Got a Predictive Modeling Tool Congratulations! Now What?


• Ian Duncan, FSA MAAA

President, Solucia Consulting

• Kate Hall, ASA, MAAA.

Vice President, Solucia Consulting.

Faculty

2


• Have you thought about how you are going to use it?

• Is your data source optimal?

• Predictive Modeling Project Planning for ROI

• Care Management

• Underwriting. • Implementation and Automation.

• How about going back to test whether results are as predicted?

Agenda Topics

3


Models and Uses

4

• Numerous uses. Models are not necessarily optimal for each use.


5

Why do it? Potential Use of Models

Program Management Perspective

Identifying individuals at very highrisk of an event (death, LTC, disability, annuity surrender, etc.).

Identify management opportunities and determine resource allocation/ prioritization.

Reimbursement

Predicting (normalized) resource use in the population.

Reimbursement by Episode. Reimbursement by risk level.

Program Evaluation

Predicting resource use based on condition profile.

Trend Adjustment.

Provider Profiling

Profiling of provider

Efficiency Evaluation

Provider & health plan contracting

Actuarial, Underwriting

Calculating new business and renewal premiums


6

Optimal Models

Reimbursement

Predicting (normalized) resource use in the population.

Reimbursement by Episode. Reimbursement by risk level.

Program Evaluation

Predicting resource use based on condition profile.

Trend Adjustment.

Any model needs to be stable over time.

Episode Treatment Groups. Risk scoring models.

Risk Score models

Keys to successful model implementation:• Stability over time;• High correlation between risk score and $’s;• Independence between risk score and any intervention that may be applied;


7



Identifying individuals at very highrisk of an event (death, LTC, disability, annuity surrender, etc.).

Identify management opportunities and determine resource allocation/ prioritization.


• Risk Scoring models;• Gaps-in-care models;• Self-reported risk factor models;• Intervenability assessment models.

Keys to successful model implementation. All the things on the prior slide, plus:

• In care management, program managers frequently use more than one model. Which model’s predictions are used in which situation (which one trumps?) is not a simple problem.

• How do you dynamically incorporate new targets and terminate old ones? How do you convert predictive model targets into a set of algorithms to apply real-time to your data?


8


Provider Profiling

Profiling of provider

Efficiency Evaluation

Provider & health plan contracting

Actuarial, Underwriting

Calculating new business and renewal premiums

Episode Treatment Groups.

Risk Scores.

Risk Scores

Predicted Costs

Self-reported conditions (HRAs)

For Provider profiling, the biggest issue will be assembling a credible database with adequate volumes of consistent provider data.

For Underwriting, many techniques have been developed to address the interaction between cost, data timeliness and predicted outcomes.


8 Simple Rules for Dating your Model:

1. Know what you are getting into – plan the project’s desired outcomes.

2. Know the existing workflow into which the model will fit, and plan any changes in workflow that will result from the new model.

3. Evaluate data sources carefully.

4. Evaluate model(s) against a known objective, as well as against “Business As Usual”.

Implementing a Model

9


8 Simple Rules for Dating your Model:

5. Make sure you understand the results of the model evaluation, and have a plan to optimize if inadequate.

6. Look for ways to automate the new workflow (the point of a model is to replace human intelligence, so don’t let the humans get in the way).

7. Pilot.

8. Evaluate outcomes: did the new system produce the high-risk targets that it was predicted to? How many of the targets were new/not found by existing methods? For an underwriting or provider reimbursement project, how did the results compare with the prior method? What can be done to enhance the results?

Implementing a Model (contd.)

10


Model Evaluation Examples

11

The next few slides are examples of projects in which we have evaluated a model(s) for implementation.

These are all examples of situations in which it was necessary to “Re-frame” the original predictive modeling question in order to understand the value of the model.


12

How well does the model perform?

All Groups

0

20

40

60

80

100

120

140

-100

%+

-90%

to -9

9%

-80%

to -8

9%

-70%

to -7

9%

-60%

to -6

9%

-50%

to -5

9%

-40%

to -4

9%

-30%

to -3

9%

-20%

to -2

9%

-10%

to -1

9%

0% to

-9%

0% to

9%

10%

to 19

%

20%

to 29

%

30%

to 39

%

40%

to 49

%

50%

to 59

%

60%

to 69

%

70%

to 79

%

80%

to 89

%

90%

to 99

%

Analysis 1: all groups. This analysis shows that, at the group level, prediction is not particularly accurate, with a significant number of groups at the extremes of the distribution.


13

How well does the model perform?

NodePREDICTED

Average Profit

PREDICTED Number in

Node

PREDICTED Number in Node

(Adjusted)

ACTUAL Number in

nodeACTUAL

Average Profit

Directionally Correct (+ or -)

Predicted to be

Profitable1 (3.03) 70 173 170 (0.60) 2 0.19 860 2,122 2,430 0.07 3 (0.20) 2,080 5,131 6,090 (0.06) 4 0.09 910 2,245 2,580 0.10 5 (0.40) 680 1,678 20 0.02 6 (0.27) 350 863 760 0.16 7 0.11 650 1,604 1,810 0.04 8 0.53 190 469 470 (0.01) 9 (0.13) 1,150 2,837 2,910 0.03

10 0.27 1,360 3,355 3,740 0.04 11 0.38 1,560 3,849 3,920 (0.07) 12 0.08 320 789 830 0.08 13 0.06 12,250 30,221 29,520 0.02 14 0.27 2,400 5,921 6,410 0.21 15 (1.07) 540 1,332 1,320 (0.03) 16 0.07 10,070 24,843 24,950 (0.08) 17 (0.33) 1,400 3,454 3,250 (0.10) 18 0.11 4,460 11,003 11,100 0.08 19 (0.13) 1,010 2,492 2,100 (0.11)

42,310 104,380 104,380 0.005 6 red13 green 11 nodes


14

Underwriting Decision-making

Underwriting Decision Total Profit Average Profit per

Case

Cases Written

Accept all cases as rated. 557.5 0.005 104,380

Accept all cases predicted to be profitable; reject all predicted unprofitable cases.

1,379.4 0.016 87,760

Accept all cases predicted to be profitable; rate all cases predicted to be unprofitable +10%.

2,219.5 0.021 104,380

Accept all cases for which the directional prediction is correct.

2,543.5 0.026 100,620

Accept all cases for which the directional prediction is correct; rate predicted unprofitable cases by +10%

3,836.5 0.038 100,620

Accept all cases for which the directional prediction is correct.

2,540.8 0.025 101,090


15

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

99 96 93 90 87 84 81 78 75 72 69 66 63 60 57 54 51 48 45 42 39 36 33 30 27 24 21 18 15 12 9 6 3 0 Model Percentile

Percent of Members w/ Hospitalization Identified

Model 2 Model 1

Lift Chart – Comparison between Two models

Care Management - Analysis


16

Background

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 Model Percentile

Percent of Members w/ Hospitalization Identified

Model 2 Model 1

Lift Chart – Comparison between Two models


The importance of Intervenability

17

Prediction is Not Enough

For example, are these conditions equally intervenable?

NAME LABEL CONDITION GROUP CCDXG060 10.01 colon cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG061 10.02 rectal cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG062 10.03 oth/unspec ca of digest organs/per Cancer Breast/Prostate/Colorectal/Other CancerDXG063 10.04 melanoma Cancer Breast/Prostate/Colorectal/Other CancerDXG064 10.05 breast cancer, age 45+ Cancer Breast/Prostate/Colorectal/Other CancerDXG065 10.06 cancer of uterus Cancer Breast/Prostate/Colorectal/Other CancerDXG066 10.07 cancer cervix/fem genital organs Cancer Breast/Prostate/Colorectal/Other CancerDXG067 10.08 prostate cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG068 10.09 cancer testis/male genital organs Cancer Breast/Prostate/Colorectal/Other CancerDXG069 10.10 ca bladder/ureter/urethra/oth urin Cancer Breast/Prostate/Colorectal/Other CancerDXG070 10.11 cancer of kidney and renal pelvis Cancer Breast/Prostate/Colorectal/Other CancerDXG071 10.12 cancer of the eye Cancer Breast/Prostate/Colorectal/Other CancerDXG072 10.13 thyroid/endocrine ca/exc adrenal/p Cancer Breast/Prostate/Colorectal/Other CancerDXG073 10.14 other/ill-defined site cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG078 10.19 breast cancer, age < 45 Cancer Breast/Prostate/Colorectal/Other Cancer

High Cost does not equal High Opportunity.


Segmentation and Operational Model

Population Segment Interventions

ICMIntensive Case Mgmt: High dollar cases (>$50,000 over 3 months excluding those with PCC) Specific diagn

Case Management Condition Management Wellness Coaching

CCChronic Conditions Asthma, COPD, CAD, CHF, Diabetes

ECElective/Other Manageable Conditions Maternity Depression Osteoarthritis Hips/Backs etc.

NTHCNon targeted health conditions (lower prevalence, lower cost and less modifiable conditions) -- those with claims beyond positive utilization for conditions other than those defined in other segments

ARAt risk on the basis of HRA -- those with no claims other than preventive

but with defined risks on basis of HRA .

AW Apparently well on basis of HRA with no claims other than preventive Targeted messaging highlighting resources to stay healthy

NUNonusers: No HRA or claims-based conditions

.Targeted messaging urging HRA and positive utilization

Wellness Coaching for those indicated on basis of HRA (High risk in correctable areas: Nutrition, Weight Reduction, Stress Mgmt, Physical Activity, Smoking Cessation w ith high motivation and high confidence)

Condition Management Wellness Coaching

Wellness Coach

Health Coach (RN)

Health Coach RN


Care Management Program Planning

19


20

Step 1The intake coordinator receives a daily admit notification.

Step 2For each patient the intake coordinator completes a brief admission survey.

Step 3ProGuide combines the admission survey with historical patient information and assigns the patient a risk ranking in real time on nurse C.M. task-list.

High Risk

Medium Risk

Low Risk

Step 4The intervening nurse contacts admitting hospital on behalf of high and medium risk patients. Tasks are prioritized according to Risk ranking.

High Risk

Medium Risk

Data Data WarehouseWarehouse

Implementation

Data Data WarehouseWarehouse


In future, integrated Systems are key to success

Electronic Patient/Provider Record

Transactional Systems Service-Level Data

Rx Claims

Medical Claims

Lab Values

Pre-authorization

Surveys

Demographics

*Actuarial Functionality

Price Programs

Plan Intervention Programs

Price Guarantees

Model Inter- vention Pgm

Analyze FinancialOpportunity

Analyzer

Workflow System

Reporting

RemoteReplication

Manage Patients

CM Database

Evaluation

Reconciliation

OutcomesManagement Identification

21


22

But so too are non-traditional functions

• Customer interaction: ability to attract and enroll patients;• Financial projections and guarantees: Care Management is

increasingly being sold as a Financial Product, rather than a clinical or administrative product;

• Designing incentives/disincentives to steer patients and change behavior.

• Data/information about best-practices, gaps and quality providers.


23

Care Management is evolving

“The trend that we are beginning to see with our large health plan clients is a re-examination of the outsourced model of DM that has been prevalent for the last 5 years in favor of an “insourced” or “assembled” strategy. Health plans are questioning the economics of outsourced DM and the silo effect of stand-alone DM programs. DM companies able to effectively address health plans’ concerns with cost, program integration and seamless member management, as well as more effectively engaging their provider networks, will be better positioned as this trend evolves.”

Ian Duncan, quoted in Disease Management News, February 10, 2006.


24

Predictive modeling can’t solve…

• How to integrate into health plan/provider processes;

• Involving the provider;• Timeliness of information;• How to integrate the medical record.


• Long lead time needed - - - due to renewal notification requirements & claims lag.

• Means little data available for first renewal – need to supplement predictive model information.

• Large group may use the predictive model results as an adjustment to the renewal – time lag could be longer.

• Rx data has less lag so incorporating up to date Rx data may have a benefit.

Unique circumstances of underwriting

25


• The loss ratio method or the build up method for renewals for a block of business is a reliable and proven method to assess the overall needed rate increase.

• Allocating the needed rate increase among small groups is an opportunity to introduce predictive modeling.

• Rather than using the predicted cost as an absolute use it to stratify groups and apply the rate increase to groups based on where their risk falls compared to the mean for the block.

Predictive Modeling & Renewal Rating

26


Small Group Example

• Determine the average risk score for your in-force business

• Calculate the average rate up for your in- force business

• Find where the threshold is for best rate.

Predictive Modeling & Renewal Rating

27


• New Business Underwriting & Predictive Modeling

• Example:

Predictive Modeling & New Business Rating

28


What did we do?

Stage 1

• Convert written condition responses from the underwriting questionnaires into identifiable DxCG conditions.

• This was an extensive manual process (although automatable in the future).

• Some degree of subjectivity was involved, though no more than in the standard underwriting process.

New Business Underwriting Project

29


What did we do?

Stage 2

• Convert DxCG conditions into DCG condition-based scores: automatic process.

• Calculate DxCG age/gender scores. • Summarize at a group level.• Calculate Relative Risk Score (RRS) as condition-based

score divided by age/gender score. We are trying to isolate the deviation from the age/gender norm (because the differences due solely to age/gender are accounted for in the premium).


30


What did we do?

Stage 3

• Compare the RRS by group based on the predictive model and implied RRS from the underwriting formula to the actual first year experience.

A couple of notes:• Children not included in the analysis - insufficient data

to generate predictive values;• Some conditions not mapped – may require original

source document to understand the written conditions;• Need to further develop the “with

complications”/severity information.• Need to incorporate information contained in the drug

data.


31


Current manual approach and predictive modeling approach initially produce similar results.

Given the similar results:• The data limitations of the automated approach;• The potential for refinements to the data collection

and• The potential for refinements to the model

Give the opportunity to improve the accuracy of the underwriting process AND reduce the manual effort.

Preliminary Results

32


Ranking of Calculated Underwriting LoadManual Loss Ratio* Percentile Ranking Below 15th

Between 15th

& 25th

Between 25th

& 50th

Between 50th

& 75th

Between 75th

& 90th Over 90th Grand Total

Correct

Grouping

Under

Predicted

Over

Predicted

Below 15th 48 15 27 27 7 8 132 36.4% 63.6%

Between 15th & 25th 19 13 33 16 4 2 87 14.9% 21.8% 63.2%

Between 25th & 50th 30 29 68 49 27 16 219 31.1% 26.9% 42.0%

Between 50th & 75th 14 15 57 67 45 21 219 30.6% 39.3% 30.1%

Between 75th & 90th 8 10 21 37 35 21 132 26.5% 57.6% 15.9%

Over 90th 13 5 13 23 14 20 88 22.7% 77.3%

Grand Total 132 87 219 219 132 88 877

Ranking of Relative Risk ScoreManual Loss Ratio* Percentile Ranking Below 15th

Between

15th & 25th

Between 25th

& 50th

Between 50th

& 75th

Between 75th

& 90th Over 90th Grand Total

Correct

Grouping

Under

Predicted

Over

Predicted

Below 15th 49 15 24 22 14 8 132 37.1% 62.9%

Between 15th & 25th 15 13 26 17 9 7 87 14.9% 17.2% 67.8%

Between 25th & 50th 31 24 73 48 26 17 219 33.3% 25.1% 41.6%

Between 50th & 75th 15 22 58 61 33 30 219 27.9% 43.4% 28.8%

Between 75th & 90th 13 7 25 40 32 15 132 24.2% 64.4% 11.4%

Over 90th 9 6 13 31 18 11 88 12.5% 87.5%

Grand Total 132 87 219 219 132 88 877

* Incurred claims divided by “manual” premium (actual without rate up).

Preliminary Results

33


• Self-reported data

• FSA/HSA Disbursement data

• Rx data from a service

• Other consumer data

• Other?

Possibilities

34


Discussion

35

scioinspire corp proprietary & confidential. copyright 2008 september, 2008 so you’ve got a...

Documents