ncdr.14 risk adjusted datadata- ---ws # 1ws # 1 risk adjusted datadata- ---ws # 1ws # 1 ... •...

3/24/2014

1

NCDR.14 Risk Adjusted NCDR.14 Risk Adjusted NCDR.14 Risk Adjusted NCDR.14 Risk Adjusted

DataDataDataData---- WS # 1WS # 1WS # 1WS # 1

Dr. Michael Kontos Dr. Michael Kontos Dr. Michael Kontos Dr. Michael Kontos ---- Director, Pauley Heart Center Coronary Intensive Care Unit , VCU Medical Center

Kim Hustler Kim Hustler Kim Hustler Kim Hustler – Clinical Quality Consultant, American College of Cardiology

The following relationships exist:

Dr. Kontos: Provenceo, Wellpoint/Anthem,

Astellas

Kim Hustler: No Disclosures

Beth Pruski: No Disclosures

ObjectivesObjectivesObjectivesObjectives

• Explain the value of the InInInIn----Hospital Hospital Hospital Hospital Risk Adjusted Risk Adjusted Risk Adjusted Risk Adjusted

Mortality and Bleeding Models (RAM and RAB) Mortality and Bleeding Models (RAM and RAB) Mortality and Bleeding Models (RAM and RAB) Mortality and Bleeding Models (RAM and RAB) for

the ACTION Registry - GWTG

• Discuss the Inclusion/Exclusion Criteria

• Identify the related detail line items in the ACTION

Registry – GWTG Outcome Report

3/24/2014

2

ObjectivesObjectivesObjectivesObjectives

• Use the bleeding risk model to predict and identify

patients at risk for in-hospital bleeding events

• Identify opportunities for quality improvement

utilizing Risk Adjusted Bleeding (RAB) data on the

Outcomes Report

Risk Adjusted Model

ARS Question # 1ARS Question # 1ARS Question # 1ARS Question # 1

Have you used the ACTION Registry – GWTG Mortality

Risk Model to identify the variables associated with

higher mortality for your AMI patients?

1. No

2. Yes

Risk Adjusted Model

ARS Question # 1ARS Question # 1ARS Question # 1ARS Question # 1

Have you used the ACTION Registry – GWTG Mortality

Risk Model to identify the variables associated with

higher mortality for your AMI patients?

1. No

2. Yes

3/24/2014

3

Risk AdjustmentRisk AdjustmentRisk AdjustmentRisk Adjustment

� Unadjusted rate(s): Unadjusted rate(s): Unadjusted rate(s): Unadjusted rate(s):

�Do NOT account for differences in:

�Patient age

�Overall co-morbid conditions

�Variations among hospital types

�Sites with lower or higher risk patients or lower volume of

patients

Risk Adjustment

� Risk Adjusted rate(s): Risk Adjusted rate(s): Risk Adjusted rate(s): Risk Adjusted rate(s):

�Analyzes outcomes based on specific patient risk factors

�Provides estimate of risk:

�Specific to individual patient

�Specific to the hospital’s own performance

�Allows for ‘apples to apples’ comparison

3/24/2014

4

ASSUMPTIONS: ASSUMPTIONS: ASSUMPTIONS: ASSUMPTIONS: LOGISTIC VS.

HIERARCHICAL LOGISTIC MODEL

Logistic RegressionLogistic RegressionLogistic RegressionLogistic Regression

• Outcomes for patients are independentindependentindependentindependent:

– Outcomes for patients treated within same hospital and by different hospitals are independent

– “Hospital” effects are independent & fixed

Reference: Normand, S.L.

ASSUMPTIONS: ASSUMPTIONS: ASSUMPTIONS: ASSUMPTIONS: LOGISTIC VS. HIERARCHICAL LOGISTIC MODEL

Hierarchical Logistic RegressionHierarchical Logistic RegressionHierarchical Logistic RegressionHierarchical Logistic Regression

• Outcomes are conditionallyconditionallyconditionallyconditionally independent:independent:independent:independent:

– Outcomes for different patients are independent givenhospital effect****

– Observations for different patients treated in different hospitals are conditionally independent

– “Hospital” effects are independent


In-Hospital Risk Adjusted

Mortality Model - Variables

• Age

• Heart Rate on

Admission

• Baseline troponin ratio

• ECG findings

• Systolic blood pressure

• Initial serum creatinine

• History of peripheral

arterial disease

• Heart failure OR

cardiogenic shock on

admission

Exclusions: Exclusions: Exclusions: Exclusions: Transferred out patients

3/24/2014

5

Mortality Variables Odds Ratio (95% CI)

C – Statistic

• Measure of a model’s discrimination value, or

concordanceconcordanceconcordanceconcordance

– Sensitivity and specificity of each value (or variable) in

the model

• Determines whether the model is predictive

– Value of 0.5 = no discrimination (flip a coin)

– Value of 1 = ‘perfect’ discrimination

• Clinical factors vs. administrative variables

Outcome Report Detail Lines-

Unadjusted vs. Risk Adjusted Mortality

• Line 1326 Unadjusted Mortality (Death)

• Line 1332 Risk Adjusted Mortality (Death)

3/24/2014

6

Outcome Report Detail Lines- Unadjusted vs. Risk Adjusted Mortality



3/24/2014

7

Risk Adjusted Mortality (RAM)

DocumentsDocumentsDocumentsDocuments

• Site’s mortality rate skyrocketed for Q3

• 1 death

• Suspect this metric increased due to a higher incidence of

HF symptoms at arrival

• Entered HF if documented that the patient is SOB on light

exertion and dependent lower limb edema

• Concerned about the new mortality score of 6.3% with 1

incidence contributing

RAM in Outcomes Report

ARS Question #2

What value is the Overall RAM value for

this site?

1. 1.4%

2. 1.5%

3. 3.6%

4. 6.3%

3/24/2014

8

Risk Adjusted Mortality (RAM)Documentation:Documentation:Documentation:Documentation:

• Sites mortality rates skyrocketed for Q3, 1 death

• Suspect rise due to a higher incidence of HF at arrival

• Entered HF if documented that the patient is SOB on light

exertion and dependent lower limb edema

• Concerned about the new mortality score of 6.3% with 1

incidence contributing

What What What What value is the Overall RAM value for this sitevalue is the Overall RAM value for this sitevalue is the Overall RAM value for this sitevalue is the Overall RAM value for this site????

1. 1.4%

2. 1.5%

3. 3.6%

4. 6.3%

Answer: #4 (3.6) RAM

RAM details

• Unadjusted Death- 6.3% “actual” deaths - 1 of 16 patients

• RAM calculated over R4Qs

• Risk Adjusted Death- 3.6%

• Comparison Group- benchmark 4.3%

• US National- benchmark- 4.2%

• For the population included, with the variables specific to

this population, the value of 3.6% shows you performed

well based on the acuity of your population. Entry of HF was

appropriate

• The 3.6% score is lower than the comparison group or US-

better performer

• Low volume (1 death) affects result

3/24/2014

9

ARS Question # 3

Have you used the ACTION Registry – GWTG Bleeding

Risk Model?

1. No

2. Yes

ARS Question # 3

Have you utilized the ACTION Registry – GWTG

Bleeding Risk Model?

1. No

2. Yes

3/24/2014

10

Definitions: Suspected Bleeding

Event vs. In-Hospital Major Bleeding

Suspected Bleeding EventSuspected Bleeding EventSuspected Bleeding EventSuspected Bleeding Event

• Hemoglobin drop > 3 g/dl

• Transfusion of whole blood or packed red blood

cells

• Procedural intervention or surgery at bleeding site

to reverse/stop or correct bleeding

Definitions: Suspected Bleeding

Event vs. In-Hospital Major Bleeding

In In In In –––– Hospital Major BleedingHospital Major BleedingHospital Major BleedingHospital Major Bleeding

• Absolute Hemoglobin decrease of > 4 g/dl

• Intracranial hemorrhage

• Documented or suspected retroperitoneal bleed

• Any red cell transfusion with Hgb < 9 g/dl, OR

• Any red cell transfusion with Hgb > 9 g/dl and a

suspected bleeding event

In-Hospital Major Bleeding Risk

Model- Variables• Age

• Diabetes

• ECG findings

• Heart rate on admission

• Systolic BP on

admission

• Heart failure OR

cardiogenic shock on

admission

• Gender

• Weight

• Home Warfarin

• Initial Hemoglobin

(Hgb)

• Initial Serum

Creatinine

• Prior peripheral arterial

disease

3/24/2014

11


Model- Exclusions

• Transferred out patients

• Patients who die within 24 hours

• Missing age/gender

• Bleeding during or after CABG

• Initial Hgb recorded afterafterafterafter PRBC transfusion

In – Hospital Major Bleeding

Variables Odds Ratio (95% CI)

In – Hospital Major Bleeding

Variables Odds Ratio (95% CI)

3/24/2014

12

Outcome Report Detail Lines:Unadjusted vs. Risk Adjusted Major Bleeding

• Line 1348 Unadjusted bleeding

• Line 1354 (RAB)

Outcome Report Detail Lines- Unadjusted

vs. Risk Adjusted In-Hospital Major

Bleeding



Bleeding

3/24/2014

13



Bleeding

Why Measure and Report Bleeding?

• Increasingly recognized as adverse outcome

– Increased hospital mortality

– Increased long term mortality

• Increased mortality a result of:

– Severe bleeding

– Stopping anti-platelet medications

– Increased inflammation

Why Measure and Report Bleeding?

• Is a potentially modifiable risk factor

– Change medications (eg, clopidogrel vs prasugrel; bivalirudin vs GP IIb/IIIa inhibitors)

– Change access approach (radial vs femoral)

• Clinical trials underestimate bleeding:

– Selected population

– Lower risk pts

– Treatment standardized

3/24/2014

14

Using ACTION Registry – GWTG

Prediction Score for In-Hospital Major

Bleeding



Bleeding



Bleeding

3/24/2014

15

Risk Adjusted Bleeding (RAB)

Documents:Documents:Documents:Documents:

• 84 y/o female presented symptoms of ACS, from

nursing home. HR 87, SBP 137, weight 75 kg

• History dyslipidemia

• Ruled in NSTEMI, Diagnostic cath, no PCI, for CABG

• Creatinine 0.8, initial Hemoglobin 13.8 g/dl

• Documented bleeding at access site

• Lowest Hemoglobin 6.8 g/dl

• 1 unit PRBC transfused

ARS Question # 4

Does this RAB score show better performance than

the Comparison group and US benchmark?

1. No

2. Yes

Risk Adjusted Bleeding (RAB)Documents:Documents:Documents:Documents:

• 84 y/o female, HR 87, SBP 137, weight 75 kg

• History dyslipidemia

• Ruled in NSTEMI, Diagnostic cath, no PCI, CABG

• CR 0.8, Hgb 13.8 g/dl, Lowest Hgb 6.8 g/dl

• Documented bleeding at access site, 1 unit PRBC

Does this RAB score show better performance than the Does this RAB score show better performance than the Does this RAB score show better performance than the Does this RAB score show better performance than the

Comparison group and US benchmark?Comparison group and US benchmark?Comparison group and US benchmark?Comparison group and US benchmark?

1. No

2. Yes

3/24/2014

16

Answer: #1 (No)

• A higher RAB value signifies lower performance

• The variance in benchmarking is minimal

• Pt risk factors- age, gender, initial hemoglobin

Look at your detail lines for more information

• Site had only 1 suspecting bleeding event R4Q

• 24 24 24 24 patients with >3g/dl drop in hemoglobin

• 2 patients with Initial Hgb >=9

• 0 patients with Initial Hgb <9

• 2 Transfusions

Reasons included as Major BleedReasons included as Major BleedReasons included as Major BleedReasons included as Major Bleed

Criteria for Major Bleed:Criteria for Major Bleed:Criteria for Major Bleed:Criteria for Major Bleed:

• Major bleeding was defined as an absolute Hgb Major bleeding was defined as an absolute Hgb Major bleeding was defined as an absolute Hgb Major bleeding was defined as an absolute Hgb

decrease of 4 g/dl or greaterdecrease of 4 g/dl or greaterdecrease of 4 g/dl or greaterdecrease of 4 g/dl or greater

• Intracranial hemorrhage

• Documented or suspected retroperitoneal bleed

• AAAAny red cell blood transfusion with baseline ny red cell blood transfusion with baseline ny red cell blood transfusion with baseline ny red cell blood transfusion with baseline HgbHgbHgbHgb

>>>>9 g/dl9 g/dl9 g/dl9 g/dl

• Any red cell transfusion with Hgb <9 g/dl and a

suspected bleeding event.

3/24/2014

17

Resources

� Resources → Documents → Quality Tools and Resources → Documents → Quality Tools and Resources → Documents → Quality Tools and Resources → Documents → Quality Tools and

Reference DocumentsReference DocumentsReference DocumentsReference Documents

�ACTION Mortality Model Document

�ACTION In-Hospital Major Bleeding Document

� Resources → EducationResources → EducationResources → EducationResources → Education

Questions ?

3/24/2014

18

Case Studies on Risk Adjusted Case Studies on Risk Adjusted Case Studies on Risk Adjusted Case Studies on Risk Adjusted

Mortality and BleedingMortality and BleedingMortality and BleedingMortality and Bleeding

Kim Hustler – Clinical Quality

Consultant, American College of

Cardiology

RAM & RAB

“The secret of getting ahead is

getting started. The secret of

getting started is breaking

your complex, overwhelming

tasks into small manageable

tasks, and then starting on

the first one.”

Mark Twain



• 80 y/o presented with symptoms of ACS

• ECG ST depression

• Troponin 1.2

• History of peripheral artery disease

• HR 110

• SBP 155

• Creatinine 1.9

3/24/2014

19


Predicted score of in-hospital mortality


PointsPointsPointsPoints

Age 80 13

Baseline Creatinine 1.9 3

SBP 155 9

Baseline Troponin 1.2 1

Initial HR 110 5

HF on presentation no 0

ECG findings ST depression 3

Prior peripheral artery

disease

yes 2

Total 36

ARS Question # 5

With a predictive score of 36363636, what is the risk of in-

hospital mortality for this patient?

1. 0.05%

2. 0.1%

3. 0.5%

4. 5%

3/24/2014

20




1. 0.05%

2. 0.1%

3. 0.5%

4. 5%

Answer: #4 (5%)

• Patient’s variables taken

into account provides a

predicted probability of

in-hospital death

• Highest risk was age,

then SBP

• All other variables were

low risk

• Providing the low risk

probability



• 65 y/o presented with symptoms of ACS

• ECG ST elevation

• Troponin 0.8

• History of peripheral artery disease, current Dialysis

• HR 110

• SBP 88

• Cardiogenic shock

3/24/2014

21





Age 65 10

Baseline Creatinine On dialysis 13

SBP 88 19

Baseline Troponin 0.8 0

Initial HR 110 5

Cardiogenic shock on

presentation

yes 18

ECG findings ST elevation 6


disease

yes 2

Total 73

ARS Question # 6



1. 0.07%

2. 0.78%

3. 7.8%

4. 78%

3/24/2014

22




1. 0.07%

2. 0.78%

3. 7.8%

4. 78%

Answer: #4 (78%)

• Patient’s variables taken

into account provides a

predicted probability of in-

hospital death at 78%

• Despite the lower age this

patient was at much higher

risk of death

High predictive variables


• Dialysis

• Low SBP



• 80 y/o male presented with symptoms of ACS

• ECG ST depression

• Initial Hemoglobin 14.0

• History of peripheral artery disease, DM

• HR 110

• SBP 155

• Creatinine 1.9

• Weight 72 kg

3/24/2014

23

Predicted score of in-hospital major bleeding



Age 80 4

Gender Male 0

Baseline Creatinine 1.9 2

SBP 155 0

Baseline Hemoglobin 14 6

Initial HR 110 8

ECG findings ST depression 3


disease

yes 3

Diabetes yes 3

Weight 72 kg 3

Total 32

ARS Question # 7

With a predicted score of 32, what is the risk of in-

hospital major bleed for this patient?

1. 0.01% 3. 8.0%

2. 0.1% 4. 12.0%

3/24/2014

24




1. 0.01% 3. 8.0%

2. 0.1% 4. 12.0%

Answer: #4 (12%)

• Patient’s variables taken into account provides a

predicted probability of in-hospital major bleeding

• Highest risk was HR, Hgb, age, PAD

• Other variables were lower risk

• Providing the low risk probability



• 65 y/o female presented with symptoms of ACS

• ECG ST elevation

• Hemoglobin 10.5

• History of peripheral artery disease, current Dialysis

• HR 110

• SBP 88


• Weight 65 kg

3/24/2014

25


Predicted score of in-hospital bleeding


Age 65 3

Gender Female 4

Baseline Creatinine On dialysis 11

SBP 88 4

Baseline Hemoglobin 10.5 12

Initial HR 110 8

ECG findings ST elevation 7


disease

yes 3

Weight 65 kg 4

Total 56

ARS Question # 8



1. 0.01% 3. 8.0%

2. 0.1% 4. 12.0%

3/24/2014

26




1. 0.05% 3. 50.0%

2. 0.5% 4. 52.0%

Answer: #4 (52%)

• Patient’s variables taken into account provides a

predicted probability of in-hospital major bleeding

• Highest risk was Hgb, Dialysis, SBP, Weight, Female,

STEMI,PAD

• Other variables were lower risk

• Providing the low risk probability

Risks for Mortality & Major Bleeding65 y/o Mortality65 y/o Mortality65 y/o Mortality65 y/o Mortality PointsPointsPointsPoints 65 y/o Bleeding65 y/o Bleeding65 y/o Bleeding65 y/o Bleeding PointsPointsPointsPoints

Age 10 Age 3

Creatinine/Dialysis 13 Creatinine/Dialysis 11

SBP- Low 19 SBP 4

Initial HR 5 Initial HR 8

HF HF

ECG findings- STEMI 6 ECG findings 7

Prior PAD 2 Prior PAD 3

Cardiogenic shock 18 Weight 4

Baseline Troponin 0 Baseline Hemoglobin 12

Female 4

Warfarin/ Diabetes

Total/ Risk 73/ 78% 56/ 52%

3/24/2014

27

Section H- In-Hospital Events

Seq. #9040 Suspected Bleeding Event

Documentation:Documentation:Documentation:Documentation:

• Presentation as STEMI

• History of anemia

• To cath lab for primary PCI

• RN documents bleeding at access site,

hemostasis established

• The Hgb dropped 3 g/dl

• Physician- no documentation of a bleeding event

ARS Question # 9

How How How How would you answer Suspected Bleeding Event would you answer Suspected Bleeding Event would you answer Suspected Bleeding Event would you answer Suspected Bleeding Event

for this scenario?for this scenario?for this scenario?for this scenario?

1. No

2. Yes

Seq. #9040 Suspected Bleeding Event





• RN documents bleeding at access site, hemostasis

established


How How How How would you answer Suspected Bleeding Event for would you answer Suspected Bleeding Event for would you answer Suspected Bleeding Event for would you answer Suspected Bleeding Event for

this scenariothis scenariothis scenariothis scenario????

1. No

2. Yes

3/24/2014

28

Answer: #2 (Yes)

• You did have documentation of a bleed at access site

• The 3 g/dl drop in hemoglobin would meet the supporting

definition

Risk Adjusted Bleeding Model

Major Bleeding





• RN documents bleeding at access site,

hemostasis established

• The hgb dropped 3 g/dl


ARS Question # 10

Would this same patient meet the definition for Would this same patient meet the definition for Would this same patient meet the definition for Would this same patient meet the definition for

Major Bleed, and be included in Risk Adjusted Major Bleed, and be included in Risk Adjusted Major Bleed, and be included in Risk Adjusted Major Bleed, and be included in Risk Adjusted

Bleeding model?Bleeding model?Bleeding model?Bleeding model?

1. No

2. Yes

3/24/2014

29

Major Bleeding





• RN documents bleeding at access site, hemostasis

established


Would this same patient meet the definition for Major Would this same patient meet the definition for Major Would this same patient meet the definition for Major Would this same patient meet the definition for Major

Bleed, and be included in Risk Adjusted Bleeding Bleed, and be included in Risk Adjusted Bleeding Bleed, and be included in Risk Adjusted Bleeding Bleed, and be included in Risk Adjusted Bleeding

model?model?model?model?

1. No

2. Yes

Answer: #1 (No)

• You did have documentation of a bleed at access site

• The 3 g/dl drop in hemoglobin would meet the supporting

definition for suspected bleeding event

• Major Bleeding event requires drop in hemoglobin 4 g/dl

Article located at ncdr.com

Resources/Documents v2

3/24/2014

30

RAM & RAB Metrics will be coming

Now is no time to think of what you do not

have. Think of what you can do with what there

is.

Ernest Hemingway

Questions ?

3/24/2014

31


Model- Variables• AgeAgeAgeAge

• Diabetes

• ECG findings

• Heart rate on admissionHeart rate on admissionHeart rate on admissionHeart rate on admission

• Systolic BP on Systolic BP on Systolic BP on Systolic BP on

admissionadmissionadmissionadmission

• Heart failure OR Heart failure OR Heart failure OR Heart failure OR

cardiogenic shock on cardiogenic shock on cardiogenic shock on cardiogenic shock on

admission admission admission admission

• Gender

• Weight

• Home Warfarin

• Initial Hemoglobin

(Hgb)

• Initial Serum Initial Serum Initial Serum Initial Serum

CreatinineCreatinineCreatinineCreatinine

• Prior peripheral arterial Prior peripheral arterial Prior peripheral arterial Prior peripheral arterial

diseasediseasediseasedisease

LOGISTIC VS. HIERARCHICAL LOGISTIC

MODELING

• Modeling (prediction or adjustment) take into account:– patient risk factors

– clinical presentation characteristics

• Outcomes are influenced by where the patient is treated (“clustering”)

• Affects adjusted mortality estimates, particularly for hospitals at extremes of risk

• Can result in inappropriate classification


LOGISTIC VS. HIERARCHICAL LOGISTIC

MODELING

• Logistic modeling – Ignores hospital effects

– Assumes hospital patient treated doesn’t affect outcomes

• Hierarchical modeling– Includes hospital effects (“clustering”)

– Patient outcomes in part based on the hospital where the patient is treated
