![Page 1: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/1.jpg)
Screening and Prognostic Tests
Thomas B. Newman, MD, MPH
October 20, 2005
![Page 2: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/2.jpg)
Overview Questions from last time; administrative stuff Screening tests
– Introduction– Biases in observational studies– Biases in randomized trials– Conclusion – ecologic view
Prognostic tests– Differences from diagnostic tests and risk
factors– Quantifying prediction: calibration and
discrimination – Value of information– Common problems
![Page 3: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/3.jpg)
TN Biases “When your only tool is a hammer, you
tend to see every problem as a nail.”
Biggest gains in longevity have been PUBLIC HEALTH interventions, not interventions aimed at individuals
Biggest threats are still public health threats
Interventions aimed at individuals are overemphasized
![Page 4: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/4.jpg)
Cultural characteristics
"We live in a wasteful, technology driven, individualistic and death-denying culture."
--George Annas, New Engl J Med, 1995
![Page 5: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/5.jpg)
What is screening? Common definition: testing to detect
asymptomatic disease Better definition*: application of a test to
detect a potential disease or condition in people with no known signs or symptoms of that disease or condition.– Disease vs condition– Asymptomatic vs no known signs or
symptoms
*Common screening tests. David M. Eddy, editor. Philadelphia, PA: American College of Physicians, 1991
![Page 6: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/6.jpg)
Types of screening
Unrecognized symptomatic disease screening: what IS making the person sick.
Disease screening: what WILL make the person sick.
Risk factor screening: what MIGHT make the person sick.
![Page 7: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/7.jpg)
Examples and overlap Continuum related to both certainty and timing of
symptoms May vary with age Unrecognized symptomatic disease: vision and
hearing problems in young children; iron deficiency anemia, depression
Presymptomatic disease: neonatal hypothyroidism, syphilis, HIV
Risk factor: hypercholesterolemia, hypertension Somewhere between: prostate cancer, breast
carcinoma in situ, more severe hypertension
![Page 8: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/8.jpg)
Disease vs. Risk factor screening. 1
(Unrecognized) Symptomatic Disease
# Labeled Few# Treated FewDuration of treatment
Varies
NNT LowEase of showing benefit
Easy
Potential for harm
False positives
![Page 9: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/9.jpg)
Disease vs. Risk factor screening. 2
(Unrecognized) Symptomatic Disease
Pre-symptomatic
Disease # Labeled Few Few# Treated Few FewDuration of treatment
Varies Varies, may be short
NNT Low LowEase of showing benefit
Easy Often difficult
Potential for harm
False positives False positives, pseudodisease
![Page 10: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/10.jpg)
Disease vs. Risk factor screening. 3
*May be political as well as scientific decision
(Unrecognized) Symptomatic Disease
Pre-symptomatic
Disease
Risk factor
# Labeled Few Few High*# Treated Few Few High*Duration of treatment
Varies, may be short
Long
NNT Low Low HighEase of showing benefit
Easy Often difficult Usually very difficult
Potential for harm
False positives False positives, pseudodisease
Harmful treatment,
delayed effects
![Page 11: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/11.jpg)
Possible harms from screening
To all tested To those with negative results To those with positive results To those not tested See course book
![Page 12: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/12.jpg)
Forces behind excessive screening -1 Companies selling machines to do the
test Companies selling the test itself Companies selling products to treat the
condition Clinicians who treat the condition Politicians who are (or want to appear)
sympathetic
![Page 13: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/13.jpg)
Forces behind excessive screening -2 Disease research and advocacy groups Academics who study the condition Clinicians doing or interpreting the test Managed care organizations The public
![Page 14: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/14.jpg)
E-mail excerpt -1 > PLEASE, PLEASE, PLEASE TELL ALL YOUR FEMALE FRIENDS AND RELATIVES TO INSIST ON A CA-125 BLOOD TEST EVERY YEAR AS PART OF THEIR ANNUAL PHYSICAL EXAMS. Be forewarned that their doctors might try to talk them out of it, saying, "IT ISN'T NECESSARY." > > …Insist on the CA-125 BLOOD TEST; DO NOT take "NO" for an answer!
![Page 15: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/15.jpg)
Biases in Observational Studies of Screening Tests
Volunteer bias Lead time bias Length time bias Stage migration bias Pseudodisease
![Page 16: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/16.jpg)
Volunteer Bias
People who volunteer for studies differ from those who do not
Examples– HIP Mammography study: women who
volunteered for mammography had lower heart disease death rates
– Coronary drug project: Men who took their medicine had about half the mortality of men who didn't, whether they were on drug or placebo
![Page 17: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/17.jpg)
Lead time bias
Source: EDITORIAL: Finding and Redefining Disease. Effective Clinical Practice, March/April 1999. Available at: ACP- Online http://www.acponline.org/journals/ecp/marapr99/primer.htm accessed 8/30/02
![Page 18: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/18.jpg)
Length Bias (Different natural history bias)
Screening picks up prevalent disease Prevalence = incidence x duration Slowly growing tumors have greater duration
in presymptomatic phase, therefore greater prevalence
Therefore, cases picked up by screening will be disproportionately those that are slow growing
![Page 19: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/19.jpg)
Length bias
Source: EDITORIAL: Finding and Redefining Disease. Effective Clinical Practice, March/April 1999. Available at: ACP- Online http://www.acponline.org/journals/ecp/marapr99/primer.htm
![Page 20: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/20.jpg)
Length Bias
Early detection Higher cure rate
Slower growing tumor with better prognosis
?
![Page 21: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/21.jpg)
Stage migration bias
Stage 0
Stage 1
Stage 2
Stage 3
Stage 4
Stage 0
Stage 1
Stage 2
Stage 3
Stage 4
Old tests New tests
![Page 22: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/22.jpg)
Stage migration bias
Also called the "Will Rogers Phenomenon"– "When the Okies left Oklahoma and moved to
California, they raised the average intelligence level in both states."
-- Will Rogers Documented with colon cancer at Yale Other examples abound – the more you look
for disease, the higher the prevalence and the better the prognosis
More generally, be careful with stratified analyses
Best reference on this topic: Black WC and Welch HG. Advances in diagnostic imaging and overestimation of disease prevalence and the benefits of therapy. NEJM 1993;328:1237-43.
![Page 23: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/23.jpg)
A more general example of Stage Migration Bias
VLBW (< 1500 g), LBW (1500-2499g) and NBW (>= 2500g) fetuses exposed to Factor X all have decreased mortality compared with those not exposed
Is factor X good? Maybe not! Factor X could be cigarette
smoking! – Smoking moves babies to lower birthweight strata– Compared with other causes of LBW (i.e.,
prematurity) it is not as bad
![Page 24: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/24.jpg)
Pseudodisease A condition that looks just like the disease,
but never would have bothered the patient In an individual treated patient it is impossible
to distinguish pseudodisease from successfully treated asymptomatic disease
Existence of pseudodisease can only be detected in groups of treated patients
Treating pseudodisease can only cause harm because (by definition) it is unnecessary
![Page 25: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/25.jpg)
Example: Mayo Lung Project (MLP) RCT of lung cancer screening Enrollment 1971-76 9,211 male smokers Two study arms
– Intervention arm: chest x-ray and sputum cytology every 4 months for 6 years (75% compliance)
– Usual care (control) arm: at trial entry only, a recommendation to receive same tests annually
![Page 26: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/26.jpg)
MLP Extended Follow-up Results*
Intervention group: more cancers diagnosed at early, resectable stage
Better survival of those with lung cancer
*Marcus et al., JNCI 2000;92:1308-16
![Page 27: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/27.jpg)
MLP Extended Follow-up Results* Slight increase in lung-cancer mortality (P=0.09 by
1996)
*Marcus et al., JNCI 2000;92:1308-16
![Page 28: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/28.jpg)
What happened?
Lead-time bias? Length bias? Volunteer bias? Overdiagnosis (pseudodisease)
Black, WC. Overdiagnosis: An unrecognized cause of confusion and harm in cancer screening. JNCI 2000;92:1280-1
![Page 29: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/29.jpg)
NHLBI National Lung Screening Trial
46,000 participants randomized in 2 years
Equal randomization Three annual screens Spiral CT versus chest x-ray!
![Page 30: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/30.jpg)
Each year, 182,000 women are diagnosed with breast cancer and 43,300 die. One woman in eight either has or will develop breast cancer in her lifetime...
If detected early, the five-year survival rate exceeds 95%. Mammograms are among the best early detection methods, yet 13 million women in the U.S. are 40 years old or older and have never had a mammogram.
39,800 Clicks per mammogram (Sept, ’04)
![Page 31: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/31.jpg)
RCTs of screening tests, Example: Mammography
New York TimesExpert Panel Cites Doubts On Mammogram's Worth
Washington Post Mammography Review Shatters the Status Quo
Doubts About Its Value Alarm Many
![Page 32: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/32.jpg)
Is screening for breast cancer with mammography justifiable?* Meta-analysis of randomized trials Methodologic issues raised
Quality of randomization Post-randomization exclusions Choice of outcome variable: Breast cancer
mortality vs. total mortality
*Gotzsche P,Olsen O. Lancet 2000;355:1293
![Page 33: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/33.jpg)
Poor Quality Randomization. Example: Edinburgh trial Randomization by practice (N=87?), not
by woman 7 practices changed allocation status Highest SES
– 26% of women in control group– 53% of women in screening group
26% reduction in cardiovascular mortality in mammography group
![Page 34: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/34.jpg)
Example 2: Biased post-randomization exclusion for previous beast cancer
New York Trial N=853 in screened group N=336 in control group Breast cancer mortality difference at 18 years: 44
deaths Edinburgh trial
N=338 in screened group N=177 in control group
![Page 35: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/35.jpg)
Explanation for differences in NY Trial* In screened group women with previous breast
cancer excluded at entry In control group, women with previous breast
cancer excluded only if they developed breast cancer
Thus, women with previous breast cancer in who did NOT develop breast cancer were included in the denominator of the control group but not the mammography group
Therefore, bias against mammography
* Fletcher SW, Gilmore JG. Mammography screening for breast cancer. NEJM 2003;348:1672-80. (Appendix 2)
![Page 36: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/36.jpg)
Problems with breast cancer mortality as an endpoint Assignment of cause of death is
subjective– Unblinded in NY, Two-county trials
Treatment may have effects on other causes of death
![Page 37: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/37.jpg)
Meta-analysis of radiotherapy for early breast cancer* Meta-analysis of 40 RCTs Central review of individual-level data; N
= 20,000 Breast cancer mortality reduced (20-yr
ARR 4.8%; P = .0001) Mortality from other causes increased
(20-yr ARR -4.3%; P = 0.003)
*Early Breast Cancer Trialists Collaborative Group. Lancet 2000;355:1757
![Page 38: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/38.jpg)
Mastectomies
Radiotherapy
![Page 39: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/39.jpg)
13-year total mortality, > 50 y.o.
Breast cancer deaths, 7 yr
![Page 40: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/40.jpg)
NCI Position* “NCI recommends mammography for women
starting in their 40s” -- Dr. Peter Greenwald, NCI director of cancer prevention
"Everyone agrees that mammography detects breast cancer when it's smaller, when it's earlier. There's no debate about that," Greenwald added. "And everybody agrees mammography detects more cancers.
"The debate is whether that has an impact on mortality later on. It is the only real method that we have, other than clinical exam, that's useful as screening for early detection in healthy women."
*Washington Post, January 24, 2002
![Page 41: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/41.jpg)
Cancer mortality vs Total mortality in RCTs
![Page 42: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/42.jpg)
TN Conclusions on Screening Screening decisions are heavily influenced by
politics, economics, emotion and wishful thinking Most screening occurs without informed consent High quality RCTs are needed Low power to discern effect on total mortality Big debate about efficacy. But even if
proponents are right, much screening is not cost-effective and its disadvantages are consistently downplayed
![Page 43: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/43.jpg)
Cost per QALY Mammography, age 40-50: $105,000* Mammography, age 50-69: $21,400* Smoking cessation counseling: $2000** HIV prevention in Africa: $1-20***
*Salzman P et al. Ann Int Med 1997;127:955-65 (Based on optimistic assumptions about mammography.)
**Cromwell J et al. JAMA 1997;278:1759-66
***Marseille E et al. Lancet 2002; 359: 1851-56
![Page 44: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/44.jpg)
Return to George Annas*
Need to begin to think differently about health. Two dysfunctional metaphors:– Military metaphor – battle disease, no
cost too high for victory, no room for uncertainty
– Market metaphor -- medicine as a business; health care as a product; success measured economically
*Annas G. Reframing the debate on health care reform by replacing our metaphors. NEJM 1995;332:744-7
![Page 45: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/45.jpg)
Ecology metaphor
Sustainability Limited resources Interconnectedness More critical of technology Move away from domination, buying,
selling, exploiting Focus on the big picture
–Populations rather than individuals–Causes rather than symptoms
![Page 46: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/46.jpg)
Assessment of Prognostic Tests
Difference from diagnostic tests and risk factors
Quantifying accuracy Value of prognostic information Common problems
![Page 47: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/47.jpg)
Potential confusion: “cross-sectional” means 2 things
Cross-sectional sampling means sampling does not depend on either the predictor variable or the outcome variable. (E.g., as opposed to case-control sampling)
Cross-sectional time dimension means that predictor and outcome are measured at the same time -- opposite of longitudinal
![Page 48: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/48.jpg)
Difference from Diagnostic Tests
Longitudinal rather than cross-sectional time dimension
Incidence rather than prevalence Sensitivity, specificity, prior probability
confusing Time to an event may be important Harder to quantify accuracy in individuals
– Exceptions: short time course, continuous outcomes
![Page 49: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/49.jpg)
Difference from Risk Factors Causality not important Absolute risk very important
– Sampling scheme makes a much bigger difference because absolute risks are less generalizable than relative risks
– Can be informative even if no bad outcomes!
![Page 50: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/50.jpg)
How accurate are the predicted probabilities?– Assemble a group– Compare actual and predicted probabilities
Calibration is important for decision making and giving information to patients
Like absolute risk in this way – less generalizable
Quantifying Prediction 1: Calibration
![Page 51: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/51.jpg)
How well can the test separate subjects in the group from the mean probability to values closer to zero or 1?
May be more generalizable Often measured with C-statistic
Quantifying Prediction 2: Discrimination
![Page 52: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/52.jpg)
Illustration
Perfect calibration, no discrimination:– Predicted = actual 5-year mortality = 45%
(for everyone) Perfect discrimination, poor calibration
– Every patient that dies has a predicted mortality of 51% and every patient who survives has a predicted mortality of 49%
![Page 53: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/53.jpg)
Quantifying Discrimination:
Dichotomize outcome at time t Then can calculate
– Sensitivity and specificity– Likelihood ratios– ROC curves, c-statistic– Can provide these for multiple time points.
In each case, probabilities are for an event on or before time t.
![Page 54: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/54.jpg)
Quantifying prediction. Analyze as a risk factor
Risk ratios (for cumulative incidence) Odds ratios (from logistic regression) Hazard ratios (for time to an event)
![Page 55: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/55.jpg)
Doctors and patients like prognostic information
But hard to assess its value Most objective approach is decision-
analytic. Consider: – What decision is to be made– Costs of errors– Cost of test
Value of Prognostic Information
![Page 56: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/56.jpg)
DECISION: Treat with more aggressive regimen
BEFORE test: 5-year mortality = 25% AFTER test: 5-year mortality either 10% or 50% BUT: do we know how bad it is:
– To treat patient with 10% mortality with more aggressive regimen?
– To treat patient with 50% mortality with less aggressive regimen?
Example
![Page 57: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/57.jpg)
Common Problems with Studies of Prognostic Tests- 1 Referral/selection bias – e.g. too many
studies from tertiary centers Effects of prognosis on treatment and
effects of treatment on prognosis– Effective treatments blunt relationships– End-of-life decisions may accentuate
relationships
![Page 58: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/58.jpg)
Common Problems with Studies of Prognostic Tests- 2 Loss to follow-up
– Can do sensitivity analysis Lack of blinding
– Especially important for subjective outcomes, e.g., physician decisions, cause of death
![Page 59: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/59.jpg)
Common Problems with Studies of Prognostic Tests- 3 Overfitting – given enough variables and a
small enough number of outcomes, can predict almost perfectly– Need separate validation
Inadequate sample size– Unlike situations where relative risk is
important, for absolute risk DENOMINATOR as important as numerator.
![Page 60: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/60.jpg)
On Cost-Effectiveness Analyses
"The essential purpose of a cost-effectiveness analysis is to calculate the net benefit or harm to a population if resources are put into one activity rather than another. But that question does not even arise if you do not look past the one activity that interests you...From this narrowed perspective, the results of cost-effectiveness analysis are not only moot, they are an irritant.”
-- David Eddy
![Page 61: Screening and Prognostic Tests Thomas B. Newman, MD, MPH October 20, 2005](https://reader030.vdocument.in/reader030/viewer/2022032606/56649e885503460f94b8c5b2/html5/thumbnails/61.jpg)
Questions?