division of geriatrics using secondary data analysis for outcomes research epi 211 april 2011...

Division of Geriatrics

Using Secondary Data Analysis for Outcomes Research

Epi 211April 2011

Michael Steinman, MD


Disclosures:

None

Acknowledgements:

J. Michael McWilliams

Ann Nattinger

SGIM Research Committee

Shameless plug for CER

http://ctsi.ucsf.edu/research/cer

Disclosures and acknowledgements


Question:

• You are a fellow / junior faculty member interested in studying...– Impact of nurse-led HTN clinics on clinical

outcomes in patients with HTN– Impact of implementing EMRs on appropriate

prescribing in ambulatory surgical patients– Whether quality measures of asthma control

in children correlate with actual clinical outcomes in this population


Question:

• Here’s your choice:

– A. Get a multimillion dollar grant to conduct a multi-center, multi-year RCT

– B. Analyze existing data


• Appreciate key conceptual and methodologic issues involved in outcomes research employing secondary data analysis

• Identify and use online tools for locating and learning about datasets relevant to your research

Learning objectives


• Working with secondary data– Conceptual and methodologic issues

• Overview of high-value datasets and web-based resources

• Q&A

Overview


Working with Secondary Data


Key Take-Home Points

• Secondary data analysis is rigorous research – Not throwing data on a wall and seeing what sticks

• RQ must meet FINER criteria and be interesting a priori

• Know the data as if it were your own– How was it collected; limitations (including validity)

• Read the codebooks and any/all documentation; validation studies; speak with PIs.

– Perfect enemy of good (but so is crap)


• Which comes first: question or dataset?a. Research question firstb. Dataset first

Conceiving a Project


• Which comes first: question or dataset?a. Research question firstb. Dataset first

• Hybrid approach1. Identify research focus, broad question2. Consider candidate datasets3. Hone question4. Iterate between 2 and 3

Conceiving a Project


• Data that have been collected but not for you

• Survey• Administrative (claims)• Discharge• Medical chart / EMR • Disease registries • Aggregate (ARF, US Census)• Combinations and linkages

Types of Secondary Data


• Compatibility with research question(s)

• Availability and expense

• Sample: representativeness, power

• Measures of interest present and valid– Predictors, outcomes, confounders

• Messiness and missingness

• Local expertise

• Linkages

Selecting a Database


1. Causal inference• Inherently limited with observational data• Does not preclude quasi-experimental

designs to recover causal effects• Core of comparative effectiveness research• Value of these approaches highly dependent on

expected confounders• For example, study of medical management vs.

catheterization for AMI

Challenges and Pitfalls


2. Validity of measures– Beware of assumptions– Problems: coding, reporting, recall biases– Carefully read the codebooks and

documentation about the study• How variables measured• (Who was included in study)

– Solutions: direct validation in subgroup or another data source, literature review, sensitivity analyses



• Want to measure financial resources• Explanation for underuse of health services, poor outcomes?

• Have measures of income.

• Are the two equivalent?

• Might financial resources also depend on:• Other assets – especially retired persons?• Family and community resources

What You Want and What You Have


• Want to measure presence of a chronic disease

• Have ICD9 codes from Medicare billing claims.

• Will this work?

• Accuracy of ICD9 claims may depend on:• Type of disease – specificity of symptoms, “dominance” in

clinical visit, accuracy of clinician diagnosis• Coding incentives – upcoding in Medicare, undercoding in VA• How codes operationalized – which codes to use; require 1 or 2

separate codes; what time period; etc.

What You Want and What You Have


3. Complexity of file structure– Row in dataset may not be unit of analysis– Skip patterns, proxy respondents


Ask: IF ((piRTab1X007AFinFam = FAMILYR) OR (piRTab1X007AFinFam = FINANCIAL_FAMILYR)) AND ((ACTIVELANGUAGE <> EXTENG) AND (ACTIVELANGUAGE <> EXTSPN)) AND (piInitA106_NumContactKids > 0) AND (piInitA100_NumNRKids > 0)

JE012 CHILDREN LIVE WITHIN 10 MILES Section: E Level: Household Type: Numeric Width: 1 Decimals: 0 CAI Reference: SecE.KidStatus.E012_ 2000 Link: G1980 2002 Link: HE012

IF {R DOES NOT HAVE SPOUSE/PARTNER and DOES NOT STILL HAVE HOME OUTSIDE NURSING HOME {(CS11/A028=1) and (CS26/A070 NOT 1)}} or {R & SPOUSE/PARTNER} LIVE IN SAME NURSING HOME (CS11/A028=1 and CS12/A030=1): [Do any of your children who do not live with you/Does CHILD NAME] live within 10 miles of you (in R's NURSING HOME CITY, STATE (CS25b/A067))?

OTHERWISE: [Do any of your children (who do not live with you)/Does CHILD NAME] live within 10 miles of you (in MAIN RESIDENCE [CITY/CITY, STATE STATE])?

6802 1. YES 4720 5. NO 32 8.DK (Don't Know); NA (Not Ascertained) 4 9. RF (Refused) 2087 Blank. INAP (Inapplicable)

A Simple Question?

* From the Health and Retirement Study


4. Data mining / overfitting• Is urine cortisol associated with Catholicism?• But…

• “Just because you were too stupid to think of the question in advance doesn’t mean it’s not important”

- Warren Browner



5. Representativeness of Sample• External validity (generalizability)• Internal validity (selection bias)• Example: comparing outcomes for insured and

uninsured patients using hospital discharge data• Must be hospitalized to enter sample• Not only limits generalizability (to outpatients)• But inferences about the sample may be wrong

– Sample would need to include uninsured who would have been hospitalized if insured



Finding the Right Dataset


Finding the Right Dataset

• Contain variables of interest – predictor, outcome, confounders

• Relevant time frame– Cross-sectional, longitudinal

• Feasible– Access: time, bureaucracy, cost– Usable

• No perfect datasets -> hybrid approach of developing research question


Administrative Data (VA)

• VA has multiple high-value administrative databases– Outpatient visit information

• Visit date, type of clinic, provider, ICD9 diagnoses

– Inpatient information• Admitting dx(s), discharge dx(s), CPT codes, bed section, meds

administered

– Lab data• >40 labs

– Pharmacy data• All inpatient and outpatient fills

– Academic affiliation– etc



• Huge bureaucracy and paperwork



• Messy data

• Huge size– 2 TB server

• Data analyst


Survey Data (NHANES)

• National Health and Nutrition Examination Survey (NHANES)– Nationally representative sample of >10K

patients every 2 years– Extensive interview data on clinical history

(including diseases, behaviors, psychosocial parameters, etc.)

– Physical exam information (e.g. VS)– Labs, biomarkers


Survey Data (NHANES)

• Free and easy to download• (Relatively) easy to use

– Although requires careful reading of documentation

• Serial cross-sectional • Disease data self-report• Very limited information about providers and

systems of care


Survey Data (NAMCS)

• National Ambulatory Medical Care Survey (NAMCS) and National Hospital Ambulatory Medical Care Survey (NHAMCS)

• Nationally representative sample of ~70K outpatient and ED visits per year

• Physician-completed form about office visit


Survey Data (NAMCS)

• Data more from physician perspective (diagnoses, treatments Rx’ed, etc) and some info on providers (e.g., clinic organization, use of EMRs, etc)

• Serial cross-sectional– Visit-focused– Not comprehensive, ? value for chronic diseases


Discharge Data (NIS)

• National Inpatient Sample (NIS)– Database of inpatient hospital stays collected from ~20% of US

community hospitals by AHRQ– Diagnoses and procedures, severity adjustment elements,

payment source, hospital organizational characteristics– Hospital and county identifiers that allow linkage to the American

Hospital Association Annual Survey and Area Resource File


Discharge Data (NIS)

• Relatively easy to access (DUA, $200/yr)

• Relatively easy to use– Though need close attention to

documentation

• Limited data elements

• Huge data files


Web-Based Resources

• Society of General Internal Medicine (SGIM) Research Dataset Compendium– www.sgim.org/go/datasets

• UCSF CELDAC– http://ctsi.ucsf.edu/research/celdac

• UCSF K-12 Data Resource Center– http://www.epibiostat.ucsf.edu/courses/

RoadmapK12/PublicDataSetResources/


Finding Additional Resources• National Information Center on Health Services Research and Health Care

Technology (NICHSR)• Inter-University Consortium for Political and Social Research (ICPSR)• Partners in Information Access for the Public Health Workforce• Roadmap K-12 Data Resource Center (UCSF)• List of datasets from the American Sociologic Association• Canadian Research Data Centers – Data Sets and Research Tools (Canada)• Directory of Health and Human Services Data Resources • Publicly Available Databases from National Institute on Aging (NIA)• Publicly Available Databases from National Heart, Lung, & Blood Institute (NHLBI)• National Center for Health Statistics (NCHS) Data Warehouse• Medicare Research Data Assistance Center (RESDAC); and Centers for Medicare

and Medicaid Services (CMS) Research, Statistics, Data & Systems• Veterans Affairs (VA) data

(all available at www.sgim.org/go/datasets)


National Information Center on Health Services Research and Health Care Technology (NICHSR)

•Databases, data repositories, health statistics•Fellowship and funding opportunities•Glossaries, research and clinical guidelines•Evidence-based practice and health technology assessment

•Specialized PubMed searches on healthcare quality and costs

http://www.nlm.nih.gov/hsrinfo/datasites.html


Inter-University Consortium for Political and Social Research (ICPSR)

•World’s largest archive of social science data•Searchable•Many sub-archives relevant to HSR

–Health and Medical Care Archive–National Archive of Computerized Data on Aging

http://www.icpsr.umich.edu/icpsrweb/ICPSR/partners/archives.jsp


Conclusions

• Secondary data has lots of advantages– Relatively quick, tremendous power, high-profile work

• Approach with a high level of detail and care– Conceptual background and RQ– Validity and use of measures

• Explore range of options available – but also take advantage of resources at hand

division of geriatrics using secondary data analysis for outcomes research epi 211 april 2011...

Documents