CB Analysis – 2

© Allen C. Goodman, 2013

Old Wine in New Bottles?

Lots of Questions About CB

• Traditional CB looked at issues like land use. For example, should we build a dam or not?

• Incremental Benefits?– Some land, that we had been using, became more

productive.– Some land, that we couldn’t use before, now could

be used.• Incremental Costs?

– Costs of the building the dam itself.– Costs attributed to the dam regarding land that

was currently being used.– Costs attributed to the dam regarding land that

was brought into use.

Questions About Health-Related CB

• Health-related CB looks at similar issues. For example, should we give kidney dialysis?

• Incremental Benefits?– People, who have been ill from kidney disease, are

now less ill, and presumably more productive.– People, who would have died from kidney disease, will

live longer and be more productive.

• Incremental Costs?– Starting kidney dialysis is expensive.– Continuing kidney dialysis involves additional costs.– Person who lives longer may need a kidney transplant

down the road and that’s REALLY expensive.

Other substantive issues

• What discount rate do we use? It makes a difference!

• How do we measure willingness-to-pay (benefits) for improvement?

• Do we include all costs, no matter what?• If we count future benefits don’t we also

have to count future costs?

Good example

Manns, Meltzer, Taub, Donaldson, Health Economics 12, 949-958, 2003, “Illustrating the impact of including future costs in economic evaluations: an application to end-stage renal disease.”

(1) How does high cost of ongoing dialysis affect cost/QALY?

(2) What impacts do “future costs” have?(3) Do QALYs represent adequate measures of

benefits for ESRD (end-stage renal disease)?

Gold Standard

• Double Blind.

• Can we double blind a dialysis test?

• Why or why not?

Future Costs for ESRD

Ongoing dialysis 3 times per week and/or transplantation.

Unrelated medical and non-medical expenditures.

Both could end up being big!

They looked at a comparison between synthetic (new) hemodialysers and cellulose (older) hemodialysers.

Discounted QALYs at 5% per year.

Various Analyses

(1) inclusion of the cost of the dialyser only, excluding the cost of related medical care;

(2) inclusion of the cost of the dialyser and of related medical costs, such as dialysis and transplantation, assuming that all patients are treated with in-center hemodialysis;

(3) inclusion of the cost of the dialyser and both related and unrelated medical costs; and

(4) inclusion of the cost of the dialyser and all related and unrelated medical costs and nonmedical expenditures.

Items 3 and 4

(3) The cost of unrelated health care (including the annual cost of non-kidney failure-related medications and the annual mean cost of non-kidney failure related hospitalisation) for hemodialysis patients using synthetic dialysers was estimated from a local study (Table 1).

(4) Non-medical expenditures were estimated by calculating lifetime total net resource use for patients by adding age-specific estimates of average consumption [28,29] net of earnings [6,30]. Average consumption by age was estimated with data from the Canadian Survey of Household Spending and included the annual consumption of non-medical and medical goods.

ITEM: In baseline analyses, the estimates considered for each of the above variables were for 60-year-old men.

Major findings

Over a 19 year horizon, synthetic dialysers an extra 0.38 QALYs/patient compared to cellulose. They were also more expensive.

Looking ONLY at intervention (dialyser only), cost/Q gained = $5,036.

When Related Costs included (costs of dialysis and transplantation*), cost/QALY gained = $83,501

*Includes nursing salary, physician charges,overhead, cost of kidney failure-related admissions, and cost of erythropoietin.

When all future costs included, cost/QALY gained = $121,124!

Magnitude of increases was largely related to high costs of future dialysis and possible transplantation.

Extra Y

Extra C

time

These are HIGH!!!

• “Critical value” is often taken as $50 000/Q.• Why do we provide hemodialysis if costs are so high?

– Cost estimates for future care are critical. How good are they?– Are some QALYs different than others?

• W/O dialysis, patients with ESRD WILL DIE! Possibly a QALY that prevents certain death is more valuable than one resulting from an improvement in the probability of survival or of experiencing a higher quality of life.

• AG: Also, if you’re going to add future costs to the formula, you’ll have to increase the “critical value.”

Sensitivity and Specificity

• If you get involved in health sector, you read a lot about these two terms. What do they mean?

• Let’s look at a screening test.

True incidence

Test

No(1)

Yes(2)

No

Yes

Specificity:Finding “no” whenTrue = “no”

Sensitivity:Finding “yes” whenTrue = “yes”

A perfect test yields

0

0

Sensitivity and Specificity

• If you get involved in health sector, you read a lot about these two terms. What do they mean?

• Let’s look at a screening test.

True incidence

Test

No(1)

Yes(2)

No

Yes

Specificity:Finding “no” whenTrue = “no”

Sensitivity:Finding “yes” whenTrue = “yes”

False Positive

False Negative

The PERFECT Screen

• Sensitivity = 100%

• Specificity = 100%

• Problem– Screens aren’t perfect. Increased

sensitivity usually comes and the cost of decreased specificity, and vice versa.

Sensitivity and Specificity

• Sensitivity = (True +) / (True + plus False -) -- Column 2

True incidence

Test No

Yes

Specificity

SensitivityFalse Positive

False Negative

• Specificity = (True -) / (True - plus False +) -- Column 1

No(1)

Yes(2)

Example; N = 400

• Sensitivity = (True +) / (True + plus False -) = 200/250 = 0.80

True incidence

Test No

Yes

100

20050

• True incidence: 250 + ; 150 - ; Test: 250 + ; 150 -

No(1)

Yes(2)

50

• Specificity = (True -) / (True - plus False +) = 100/150 = 0.67

Sensitivity and Specificity• True positives (sensitivity) give us benefits.

• Greater specificity implies fewer false positives, reducing unnecessary treatment expenses.

• False negatives have no impact in net benefit equation, because what happens to them is assumed to be no different than what would have occurred w/o screening.

True incidence

Test

No Yes

No

Yes

Specificity

SensitivityFalse Positive

False Negative

Educated Guesses

Louise RussellBecoming VERY dated, but it asks some important questions!

Becoming VERY dated, but it asks some important questions!

Screening

• Must balance the benefits of screening against the costs.

• She looks at:– cervical cancer– prostate cancer– high blood cholesterol

• We’ll concentrate on cervical cancer, because the effectiveness of screening and of treatment are both well-established.

• Pap test -- Scrape cells from the cervix onto a glass slide. Smear is sent to a lab, where it is examined for abnormal cells.

Screening

• Pap tests detects precursors to cervical cancer as well as the cancer itself. Early abnormalities can be followed up and treated with relatively simple outpatient procedures.

• Screening and early treatment can reduce incidence of cervical cancer by over 90%.

• STANDARD ADVICE -- Used to be to get a Pap smear each year.

• This has changed, even since 1980, when American Cancer Society recommended that if the first 2 annual tests were negative, subsequent tests could be given every 3 years.

There’s now a vaccine, although it does not negate the need for Pap tests

There’s now a vaccine, although it does not negate the need for Pap tests

So, what are the problems?Cervical cancer is sometimes missed -- possibly as often

as 20% of the time.1. Scraping may not include abnormal cells, EVEN if they are

present.2. Even if they are there, technician may miss them.3. Cancer may develop between tests.

To counter risk that a cancer will be missed, women can take test MORE OFTEN. Take the 20% miss rate.

If 100 women with cervical abnormalities are screened, 20/100 will be missed by the first test. If screened again, (20/100) * (100 - 80) = 4 will be missed again.

So after 2 tests, you’ve gotten 96/100 with cervical cancer – missed 4.

False Positives

• Just as you are likely to find those who have cancer, which is good, you are also likely to diagnose some as having cancer …

WHO DON’T HAVE IT“Common conditions like inflammation or injury to the cervix produce tissues so similar in appearance to dysplastic tissue that they can lead to a mistaken diagnosis when the sample is examined.”

• Good estimate of the FALSE positive rate is about 5%.

• PROBLEM -- the more often you examine, the more likely that you’ll get a false positive

Key aspects

• False positives may occur only 5% of the time, BUT• ALL WOMEN MAY EXPERIENCE THEM

• Without screening– chance of developing invasive cervical cancer is 2.5%

over the lifetime.– Chance of dying from it is 1.2%– Thus, at most, 2 or 3 of every 100 women will eventually

develop cervical cancer that could be missed by the test WHILE

– ALL 100 ARE VULNERABLE TO THE POSSIBILITY OF A FALSE POSITIVE RESULT

So, with false positives:

• Chance = 0.05 + 0.05*(1 - 0.05) + 0.05*(1-0.05)2 + …• If you work that out, the chance of having at least

1 false positive is 40% over 10 tests.• What can happen??

ANSWER> Unnecessary treatmentStress, worryWASTED treatment because the condition may have never developed into a cancer

Benefits

• Screening every 4 years average woman’s life expectancy by 94 days.

• Screening every 3 years average woman’s life expectancy by 95.5 days. Not much in the way of incremental benefits.

• Tangible gains appear from movement from 0 to 1 test for women who had had NO screening (older and Hispanic women in her study).

Costs

• (At the time) about $75 for visit and test.• Treatment costs of $300 to $1,300.

BOTTOM LINE

Early detection saves money, but the saving only partially offsets the costs of the tests and of treating false positives and conditions that would have regressed on their own.

Recommendations – American Cancer Society – 2012

• All women should start screening at age 21. No longer is screening recommended three years after starting vaginal intercourse.

• Women aged 21 to 29 should get a Pap test (conventional or liquid-based) every three years. The statement specifically recommends against annual Pap testing.

• For women 30 and over, Pap tests should be done every three years. The guidelines recommend against annual or more frequent Pap testing for this age group.

• Combining the Pap test with HPV testing every three to five years is the preferred strategy for women aged 30 and older.

• Screening is not recommended for women 65 or older who have had three or more normal Pap tests in a row and no abnormal Pap test results in the past 10 years, or who have had two or more negative HPV tests in the past 10 years.

http://www.cnn.com/2012/03/14/health/brawley-cervical-cancer-screenings/index.html

Go to link for comments.

Go to link for comments.

Update – Does this still occur?

• I gave “grand rounds” at Indiana University Medical School in February 2006 – it was a variation on the screening lecture.

• One faculty member allowed that they had much better tests now, and that the screening problems that I had alluded to were not problems any more.

Yes, it does!Letter from Fred Stehman MD, host and

Clarence E. Ehrlich Professor and Chair“As I promised I have included a couple of

articles for your information/files”“The first is from the American Cancer Society,

their current guidelines for Pap test frequency. These were updated in 2002 but are a continuation of the 1980s guidelines.” [1]

[1] Smith, Robert A. Cokkinides, Vilma, Eyre, Harmon J., “American Cancer Society Guidelines for the Early Detection of Cancer, 2006,” CA Cancer J Clin 2006; 56: 11-25.

• “The 2nd article from the American Journal of Obstetrics and Gynecology in 2004 indicates that in a closed system at Kaiser, half of the positive Pap smears were false-positives! This is an alarming figure in light of your statement that everything below this line is bad.” [2]

• “The 3rd paper, also from AJOG, shows that American obstetrician-gynecologists are perfectly capable of ignoring guidelines and continuing to screen too often and too long. Incentives matter! While we don’t get paid a lot for doing a Pap smear, we do get paid some, and it looks like my colleagues will continue to recommend more of our own services.”[3]

[2] Insinga, Ralph P., Glass, Andrew G. and Rush, Brenda B., “Diagnoses and outcomes in cervical cancer screening: A population-based study,” AJOG (2004) 191, 105-13.

[3] Saint, Mona, Gildengorin, Ginny, and Sawaya, George F., “Current cervical neoplasia screening practices of obstetrician/gynecologists in the US,” AJOG (2005) 192, 414-21.

Proposed Framework

Magnifyto 200%

So … why screen?

• Some conditions are asymptomatic. For example, kidney mass, prostate abnormality, breast cancer.

• Some risky behaviors (like smoking) are identifiably risky behaviors.

• Some people have personal or family history of illness.

http://www.scpr.org/programs/airtalk/2013/05/14/31786/the-medical-science-behind-angelina-jolie-s-decisi/

Some discussion

Supplementary Material

Guaiac Test

• Neuhauser/Lewicki looked at test w/ 6 sequential stool tests for occult blood. N = 278 people.

• 24 had positive results (any one of 6 tests were positive)

• 254 had negative results (all 6 tests were negative)

• Upon further assessment only 2 of 24 positives were found to have cancer. None of the 254 (-) were tested further.

• For 2 diseased cases, 11 of 12 (0.917) were positive.

• For the 22 non-diseased cases 46 of 126 (6 are unaccounted for) were positive (36.5% false positive). 80 were negative.

• For 254 non-diseased cases, 1524 were negative.

Not covered in class – Students not responsible for it

Not covered in class – Students not responsible for it

Guaiac Test (2)

• Idea. You’re seeking true positives. You do more tests to find more true positives, BUT at a cost.

• N-L assumed that 72 persons out of a population of 10,000 who are screened will get colonic cancer.

• On first test, then, of the 72, you’ll get 0.917 (fraction) of them, or about 66.

• On second test, you’ll get 0.917*(72 - 66) = 5.5

• GIVING 71.5 of the 72, after 2 tests !!!

• Incremental impact gets very small.

• Incremental cost gets VERY LARGE.

Not covered in class

Not covered in class

Table 24.3

Table 4.3 - Neuhauser/LewickiDetection Costs

No. No. I/G Total Inc. Mgl. Ave.

1 65.95 65.95 77511 77511 1175 11752 71.44 5.50 107690 30179 5492 15073 71.90 0.46 130199 22509 49146 18114 71.94 0.04 148116 17917 470262 20595 71.94 0.00 163141 15025 4695313 22686 71.94 0.00 176331 13190 43966667 2451

Not covered in class

Not covered in class

Guaiac Test (3)

• Major point is the distinction between mgl. and average.• Test-specific numbers depend on independence across

patients (believable) and across tests (less so).• One useful way is to look at first test, and then at battery of

the next 5. Even this gives you a marginal cost of $16,000+.• Suppose you had followed everyone, and ultimately found

that 1 person w/ negatives had gotten cancer. Sensitivity goes from 11 of 12 (0.917) to 11 of 18 (0.611).

• Also, some of the big numbers depend on very small denominators.

Not covered in class

Not covered in class

Which costs do we look at?

• Consider a world with only 2 diseases D1 and D2. Half get D1, half get D2.

• At age 60, individuals get D1 or D2.• They face a 5% chance of dying from D1 or a 5% chance

of dying from D2, or 10% in all.• Mortality risk can be reduced (that year) by 100% with

either treatment T1 (for D1) or T2 (for D2). • After 10 years, they die suddenly and unavoidably.• So (spreadsheet):Treatment T1 T2Treatment Cost 5000 5200 Model Input

Not covered in class

Not covered in class

Which costs do we look at?

Treatment T1 T2Treatment Cost 5000 5200 Model Input

Evaluating the cost-effectiveness ratio.

Effectiveness is the reduction in the probability of death * incremental lifespan. For each treatment, it is

0.05 * 10 years = 0.5 years.

Effectiveness 0.5 years 0.5 yearsThis will be the

denominator

Not covered in class

Not covered in class

If only T1 (D1), T2 (D2)-related future costs are included

Treatment T1 T2Treatment Cost 5000 5200 Model Input

Effectiveness 0.5 years 0.5 years Model Input

Palliative treatmentper year *10 9000 1000

These costs are conditional.

Unconditional costs

Pal * 0.95 * 10 8550 950

Not covered in class

Not covered in class

If only T1, T2-related future costs are included

Treatment T1 T2Treatment Cost 5000 5200 Model Input

Effectiveness 0.5 years 0.5 years Model Input

Palliative treatmentper year *10 9000 1000

These costs are conditional.

Unconditional costs

Pal * 0.95 * 10 8550 950

Total Exp. Costs13550 6150

Must compare to what would occur with no intervention.

Not covered in class

Not covered in class

If NO Intervention?• Expected future costs would be the palliative treatment that they

would be having if they lived past age 60 and had either D1 or D2. These costs equal:

0.9 (prob. of getting past age 60) * pal costs/yr. * 10 years

Pal * 0.9 * 10 8100 900

So, incremental costs are:Total Exp. Costs13550 6150

less 8100 900

Incremental Costs 5450 5250

Cost/yr. improv. 10900 10500

Treatment 2 is cheaper than Treatment 1.

>

Not covered in class

Not covered in class

Suppose we include UNRELATED COSTS

• Here’s the idea– If a person gets palliative treatment for both D1 and D2,

or $10,000. Some of the costs must be unrelated to the treatment, because the person can get intervention for D1 OR D2 -- not both.

Treatment T1 T2

Treatment Cost 5000 5200 Model Input

10,000 * 0.95 * 10 9500 9500

Total Exp. Costs14500 14700

Not covered in class

Not covered in class

If NO Intervention?• Expected future costs would be the palliative treatment that they

would be having if they lived past age 60 and had care for both D1 and D2. These costs equal:

0.9 (prob. of getting past age 60) * pal costs/yr. * 10 years

Pal * 0.9 * 10 9000 ! 9000 !

So, incremental costs are:Total Exp. Costs 14500 14700

less 9000 9000

Incremental Costs 5500 5700

Cost/yr. improvement 11000 11400

Treatment 1 is cheaper than Treatment 2!<

Not covered in class

Not covered in class

Why?

• Including the “unrelated” future costs tilts the balance against intervening in the relatively less expensive disease (because

you are adding LARGE additional expenses), and:

• Tilts the balance toward intervening in the relatively more expensive disease -- you’re adding small additional expenses.

Not covered in class

Not covered in class

Issues!

• Consider interventions. What are the future costs and benefits related to heart disease interventions?

… seatbelt interventions?

Most of the future costs may be unrelated to the particular intervention, but what about costs (like stroke or disease) which may be related to side effects of intervention?

Not covered in class

Not covered in class

What to include

• Weinstein-Manning suggest that you use either ALL the costs or NONE of them.

• Meltzer uses a human capital approach that says that individuals whose lives are extended should be charged for being more costly,

but credited for being more productive. This suggests that it may be economically more beneficial to save the lives of more productive members of society.

Not covered in class

Not covered in class