Clinician Driven Retrospective

Outcomes Research:Evaluation and Improvement of Research Data Rigor and Access in the Department of Urology

Gregory W. Hruby, MA

Research Project Manager

Department of Urology

Outline A little perspective

Pros and cons of outcomes research

Classic outcomes contributions

Database design: Relational versus flat

Historical perspective

Data flow: Early and contemporary models

Perspective revisited

Urology outcomes research work-flow

Improvements

Conclusions

Putting Things in Perspective

Columbia University publications in 2010 (PubMed)

• 7399 publications∙ 300 clinical trial studies∙ 800 review articles ∙ 6299 retrospective outcomes studies

Roughly 85% of Columbia’s contribution to the literature is represented by retrospective outcomes work

Pros of Outcomes Research No patient intervention

Prospective data collection

Real world scenarios: Databases reflect “real world” clinical practice

Un-biased clinical decision-making

Hypothesis generation

Much less expensive and time-consuming than a randomized, double-blinded, placebo-controlled clinical trial!

Cons of Outcomes Research

Reliability of recorded data

Retrospective study designs

Recall bias

No control groups or randomization

Possibility of “chance associations”

Difficult to establish cause and effect

Can you apply conclusions in real world clinical settings

Classical Outcomes Contributions Apgar Scale (Curr Res Anesth Analg. 1953)

• 10 point scale assessing newborn health

Goldman Criteria (N Engl J Med. 1977)

• Perioperative cardiac risk evaluation for non-cardiac surgery

D’Amico Risk Groups (JCO 2002)

• Clinical stage, gleason sum, and PSA three risk groups for biochemical recurrence

Kattan Nomogram (J Natl Cancer Inst. 1998)

• Clinical stage, gleason Sum, PSA and age predicts risk of biochemical failure

Database DesignFundamental Theorem of Biomedical Informatics

Friedman CP, Wyatt JC, Evaluation Methods in Biomedical Informatics, 2nd ed

+ >

A person working in partnership with an information resource is “better” than that same

person unassisted

Andrews, EB, Eaton, S. Additional Considerations in Longitudinal Database Research. Value in Health, Vol. 6, No. 2, 2003.

Historical Database Perspective

Mid 1980s: Computerized spreadsheets Late 1980s: Claims databases found to be

useful in researching drug safety and patient outcomes

Early 1990s-present: Electronic databases created expressly for patient records and research

Clinical Database Problems

Clinical data is immense and extremely disorganized

How do we organize all of this information into a useful format?

Key Data Elements

Patient Data• Demographics • Encounters (exams, clinical stage and surveys)• Procedures (OR details, procedures and pathology)• Therapies (medical, radiation + QOL Tx)• Diagnostics (imaging and lab tests)• Outcomes (status, toxicities)

Justify blank fields (unknown, not available, not interpretable, not obtainable)

A comprehensive clinical research database is a large monolith of medical data

As an organized whole, the clinical database acts as a unified and powerful force

The Goal

Flat Table Design The good

• Not abstract• Easy to start, simple

And the bad• Proliferating fields (giant spreadsheets)

• Current excel limit: 1,048,576 rows by 16,384 columns

• Snapshot data collection• Deceptively simple learning curve• Can not scale with growth• You can only go so far before it’s too complex to

maintain

Relational Database

The good• Scalable • Longitudinal data collection• Three dimensional

And the bad• Difficult to implement• Abstract• Requires highly skilled users

What is a relational database? All data is stored and accessed via relations

• ie patient data tables linked via a patient identifier or the MRN

Multiple “simple tables” store specific grouped information and are linked by patient identifiers

Patient Demographics

OR Details

Procedures

Pathology

Clinical Stage Encounters Labs

Flat and Relational SolutionsD

ata

Mai

nte

nan

ce

Number of Patients

1000 Patients

Flat Table

Relational Database

Flat and Relational SolutionsE

rro

rs a

nd

Co

st

Number of Patients

1000 Patients

Flat Table

Relational Database

The Urology Research Database

1963

Flat Electronic Data Collection

2001 2008

Paper Research Data Collection CAISIS: A Relational Database

Data Flow: Old Model

Patient Chart

Flat Datasheet

SURGICAL SCHEDULEOR AND PATH NOTESLAB RESULTS, ETC…

Product of The Old Model

The problem: Bladder cancer• Most common site of cancer in urinary system• 4th leading cause of cancer death in men• In 2010: 70,500 new cases and 15,000 deaths1

• ♂ > ♀ 4:1• Median age at Dx: 73 years old• 70-80% disease is superficial at presentation

Treatment options for HGT1 (superficial) bladder cancer• Conservative – Multiple intravesical chemotherapy• Aggressive – Immediate radical cystectomy

Which treatment modality is the more efficacious option?

1. SEER Database. http://seer.cancer.gov/csr/1975_2007/, November 2009 SEER data submission, posted to the SEER web site, 2010

Product of The Old ModelThe increasing use of intravesical therapies for stage T1 bladder

cancer coincides with decreasing survival after cystectomy

This study examines the patterns of use of Intravesical therapy (IVT) in high-grade T1 bladder cancer and the subsequent impact on survival for patients ultimately proceeding to radical cystectomy (RC)

Lambert Erica H; Pierorazio Phillip M; Olsson Carl A; Benson Mitchell C; McKiernan James M; Poon Steven. The increasing use of intravesical therapies for stage T1 bladder cancer coincides with decreasing survival after cystectomy. BJU international 2007;100(1):33-6. Urology, Columbia University Medical Center, New York, NY, USA.

Our experience indicates that patients having RC for T1 high-grade TCC after 1998 were more likely to receive IVT.

These same patients had a worsening disease-free survival. We postulate that the decrease in survival might be related to an increased use of IVT.

Data Flow: Current Model

CDW

IDX

CAISIS CROWN

DemographicsClinic/Surgical ScheduleICD-9 Codes CPT Codes

Radiology, OR, and Path reports, Admit/Discharge, Medical Therapy,Labs

Patient ChartClinical TrialsQOL\Intake FormsExternal Health Data

Product of The New ModelImmediate radical cystectomy versus conservative management for high grade cT1

bladder cancer: Is there a survival difference?

Accepted for presentation, Gregory W. Hruby. Immediate radical cystectomy versus conservative management for high grade cT1 Bladder Cancer: Is there a survival difference?. AUA 2011, Washington, DC

This study examines the patterns of use of Intravesical therapy (IVT) in HGT1 bladder cancer patients treated with conservative management or immediate radical therapy.

Indeed, our experience indicates an increase use of IVT in the current era for patients with T1 high-grade TCC

However, a bladder preservation strategy in the face of HGT1 did not compromise disease specific survival. This observation is at least in part due to better patient section for each of these respective therapies

Keeping things in perspective

Recall, that roughly 85% of Columbia’s contribution to the literature is represented by Retrospective Outcomes Research

To mitigate mis-representation of patient outcomes, a comprehensive interactive research data repository is needed

PROJECT INCEPTION

FEASIBILITY ASSESSMENT

MISSING DATACHART REVIEW

DATASET GENERATION

STATISTICALANALYSIS

RESULTS DISCUSSION

MANUSCRIPT PREPARATION,

SUBMISSION, AND REVISION

Urology Outcomes Work FlowIdea generation and documentationDoes the necessary information exist?• Identify missing key elements • Initiate chart review• Update patient research record

Data Mining and preparation Statistical analysis and interpretationDo the results make sense and what would make this study more complete?

CAISIS User Demo

https://rbwcaisisw.res.cumc.columbia.edu

Patient Data (5351283)

Project Management

Specimen Management

Wish List

How can this outcomes research model be improved?

• AllScripts integration

• Interdepartmental collaboration

Wish List

CDW

IDX

CAISIS CROWN

Helios InterfaceDemographics• DOB• Race• Sex• Contact Info• Death

Encounters• Clinical Trials• Clincal Stage• Status• QOL/Intake• Medical Therapy • ICD-9

Procedures• OR Notes• Pathology• Procedure Detail• CPT

Diagnostics• Labs• Radiology

Wish List

Urology

Wish List

UrologyRadiation

Oncology

Wish List

UrologyRadiation

Oncology

Medical

Oncology

Wish List

Urology

Surgery

Radiation

Oncology

Medical

Oncology

Wish List

Urology

Surgery

Radiation

Oncology

Medical

Oncology

Pathology

Wish List

Urology

Surgery

Radiation

Oncology

Medical

Oncology

Pathology

Cancer Center

Wish List: Tumor Bank

Freezer works database Macromolecular bank database

CORE initiative (Originally proposed in 2008)• Global expansion to new platform

• CaBig compatible (Silver Lever Compliant)• Lymphoma/leukemia• GU Tumors• Pancreas • Brain• Breast dataset migration

Conclusions The Urology department has benefited

immensely from the use of a centralized automated clinical outcomes research tool• Increased HPI Safety• Increased data integrity• Increased research capacities

This model contains ubiquitous logic and can be applied across multiple departments

As a leader in Biomedical Informatics, Columbia needs to offer better solutions to our clinical scientists, rather than leaving them to their own devices

Thank You

• Acknowledgements

Dr. James McKiernan

Dr. Mitchell Benson

Stephen Johnson