EDM ForumEDM Forum Community

Webinars Events

10-31-2012

Clinical Informatics, CER, and PCOR: BuildingBlocks for Meaningful Use of Big Data in HealthCareMark FrisseVanderbilt University

Adam WilcoxColumbia University

Dean SittigThe University of Texas Health Science Center at Houston

Michael KahnUniversity of Colorado, Denver

Marianne Hamilton LopezAcademyHealth

Follow this and additional works at: http://repository.academyhealth.org/webinars

Part of the Health Information Technology Commons, and the Health Services ResearchCommons

This Video/Media is brought to you for free and open access by the Events at EDM Forum Community. It has been accepted for inclusion in Webinarsby an authorized administrator of EDM Forum Community.

Recommended CitationFrisse, Mark; Wilcox, Adam; Sittig, Dean; Kahn, Michael; and Hamilton Lopez, Marianne, "Clinical Informatics, CER, and PCOR:Building Blocks for Meaningful Use of Big Data in Health Care" (2012). Webinars. Paper 6.http://repository.academyhealth.org/webinars/6

Clinical Informatics, CER, and PCOR:

Building Blocks for Meaningful Use of

Big Data in Health Care

Mark Frisse, MD, MBA, Vanderbilt University

Adam Wilcox, PhD, Columbia University

Dean Sittig, PhD, The University of Texas Health

Science Center at Houston

Michael Kahn, MD, PhD, University of Colorado,

Denver

Marianne Hamilton Lopez, MPA, AcademyHealth

October 31, 2012

Welcome

Erin Holve, Ph.D., M.P.H.,

M.P.P.

– Director of Research &

Education, AcademyHealth

– Principal Investigator of the

EDM Forum

AcademyHealth: Improving

Health & Health Care AcademyHealth is a leading national organization serving the fields of health

services and policy research and the professionals who produce and use

this important work.

Together with our members, we offer programs and services that support the

development and use of rigorous, relevant and timely evidence to:

1. Increase the quality, accessibility and value

of health care,

2. Reduce disparities, and

3. Improve health.

A trusted broker of information, AcademyHealth

brings stakeholders together to address the current

and future needs of an evolving health system,

inform health policy, and translate evidence into action.

New Resource from the

EDM Forum

Patient Engagement Framework

– Outlines key opportunities for engagement

– Available for download to your iPad through the iTunes app store

Go to iTunes Preview

Learning Objectives:

Trick or Treat?? Compare six different clinical

informatics technology platforms

used for research (e.g. i2b2 and

CER Hub);

Discuss the tradeoffs inherent to

data collection across a variety of

strategies, including mobile devices;

Explore the current peer-reviewed

literature at the intersection of clinical

informatics and CER;

Describe the desirable

characteristics of data models that

can be used for CER.

Today’s Faculty Mark Frisse, MD, MS, MBA,

Vanderbilt University

Michael Kahn, MD, PhD,

University of Colorado,

Denver

Marianne Hamilton Lopez,

MPA, AcademyHealth

Adam Wilcox, PhD,

Columbia University

Dean Sittig, PhD, The

University of Texas

Health Science Center

at Houston

Building the Informatics

Infrastructure for Comparative

Effectiveness Research (CER):

A Review of the Literature

Marianne Hamilton Lopez, MPA

Senior Manager

AcademyHealth

Understanding the Literature:

Why Now?

Technological advances in clinical informatics

have made large amounts of data accessible

and potentially useful for research.

An influx of new research is likely to result in

new scholarship.

Charting the progress of this emerging

scientific endeavor promises:

– Unique and interesting challenges, and

– New opportunities for discovery.

Literature Review Aims Develop and implement a systematic

search strategy for identifying relevant articles at the intersection of CER & CI

Develop codes to classify the literature

Perform an in-depth literature synthesis/analysis

Review themes in the selected body of work

Identify current gaps

Identifying Articles at the

Intersection of CER & CI

Clinical Informatics

Comparative Effectiveness

Research

A Curated Approach to

Identifying Relevant Articles

PubMed Searches: – MeSH search: clinical

informatics and CER (68)

– KW search: “Learning Healthcare System” (7)

Reference Searches – PROSPECT, DRN, &

Enhanced Registries (1,500)

– HIT for Actionable Knowledge Annotated Bibliography (40)

– 2010 AMIA Symposium (2)

Review of Projects (818): – DARTNet

– DEcIDE

– HMORN

– i2b2

– OMOP

– PhysioMIMI and VISAGE

– RedCAP

– Sentinel/Mini-Sentinel

– SHARP Program

– TRIAD (OSU CTSA)

– VINCI and VA informatics

– iDASH

Exclusions upon Citations Review by

biomedical informatician

N = 2,035

Search Strategy 2

Reference Searches

N = 1,542

Search Strategy 3

Review of Projects

N = 818

Combined Total Citations

N = 2,435

Abstracts Reviewed

N = 400

Exclusions upon Abstracts Review

N = 253

Full-Text Articles Reviewed

N = 147

Articles Meeting Criteria

N = 132

Refining the Set

Search Strategy 1

PubMed Searches

N = 75

Exclusions upon Full-Text Articles Review

N = 15

Coding the Primary Folder of Articles on CER & CI

Context

Platforms, Projects

Clinical Informatics

Platforms

Clinical Informatics

Projects

Natural Language

Processing

Data use and quality

Research Networks

Standardized data collection

Identifiers and De-

Identification

Metadata

Patient Involvement

IRBs

Governance

Library of Phenotypes

The Learning Healthcare

system and CER

Single Point Access

Cloud Computing

Security

Cohort Identification

Three Types of Articles Identified

as Major Areas of Focus

1. Historical context or frameworks for

using clinical informatics for research

2. Platforms, projects, and networks

3. Issues, challenges and applications of

natural language processing (NLP)

Two Cross-Cutting Themes:

1. Standardization: Differences in ontologies,

informatics platforms, and data entry practices

contribute to the complexity of collecting and

analyzing multi-site electronic data for research.

2. Governance: CER based on ECD presents unique

data governance concerns related to:

– Transfer and storage,

– De-identification, and

– Access of ECD.

Three Gaps Identified in the

Literature

1. Cohort Identification

2. Single Point Access

3. Cloud Computing

State of the Literature

Cross-cutting themes reflect a nascent, but

rich, discussion

Efforts continue to expand to develop CI

platforms, models, and tools to support new

infrastructure and studies

Breadth of perspectives in this growing

community of scientists are engaged in

expanding the current paradigm of

effectiveness research

Available Resources and Looking Ahead

www.edm-forum.org/ Products from the literature

review:

A set of PubMed search terms

A list of cross-cutting codes

An abstraction form

An annotated bibliography

A glossary of relevant terms

Next Steps:

Updated peer-reviewed search

Grey literature review

Provide Input:

Submit comments

Sign-up for updates

Join the Discussion Sign up at edmforum@academyhealth.org

W E L C O M E

Event or Meeting Title

Informatics platforms enable distributed

comparative effectiveness research using

multi-institutional heterogeneous clinical data

Dean F. Sittig, PhD

Compare and contrast 6 large

informatics platforms for CER 20

Today: most data manipulations performed using non-coordinated applications with disjointed institutional control

New informatics platform designs provide access to electronic clinical data and the governance required for inter-institutional CER

“platform” is a suite of interconnected, coordinated applications, together with the operational environment that hosts those applications

Focus on specific CER projects that provide implementations of informatics platforms and highlight design requirements and solutions

CER requires comprehensive data on

many patients 22

Enormous amounts, large variety of data types

from different sources to create complete medical

history

Need in- & out-patient EHRs (free text); billing,

laboratory, pharmacy, and radiology

Document patients actually receive care ordered;

pharmacy dispensing and patient-reported data

Data is nearly always incomplete; methods must

be appropriate for measuring health status and

care events

CER requires data on populations from

multiple organizations 23

Identify small differences, bias, subgroup

analyses, generalizability, evaluation of

demographic & geographic variation, rare events

Include data from multiple organizations, non-

traditional data sources,

long-term care facilities,

home and public health agencies,

ascertain patients’ socio-economic status

CER requires data on populations from

multiple organizations 24

Need to merge data from the same patient who

has received healthcare services and had clinical

data captured at multiple institutions

Requires a community-wide MPI - identifies

patients based on multiple demographic data

(e.g., first name, last name, date of birth, gender,

social security or telephone numbers)

Only health information exchanges for patient

care have tackled this extraordinarily difficult

problem; will be a critical success factor.

CER requires data extraction, modeling,

aggregation and analysis 25

Design and development of “mapping” apps is

big challenges in multi-institutional research

Difficult for researchers to appreciate local

idiosyncratic data issues without active

engagement of local experts

Informaticians working to create powerful, user-

friendly tools for data extraction, manipulation,

and analysis

Developing tools to process clinically-rich free-text

notes documenting patient care

CER must conform to local IRB rules and

local and federal legislation 26

Social, legal, ethical, and political challenges in

CER must not be underestimated

“organizations are understandably reluctant to

move data beyond their own boundaries absent a

clear and specific need to do so, and patients will

be less likely to consent to allow this to happen.”

One design is to retain physical control of raw

data while providing for their aggregation as

limited data sets to answer specific questions

CER must conform to local IRB rules and

local and federal legislation 27

Other governance solutions:

Standardizing data models across the project

Limiting access to authorized personnel while

facilitating remote access

Restricting types of queries that can be executed and

masking patient-specific, identifiable data

Logging all data transactions and access activities

As rules evolve CER platforms and governance

processes must evolve accordingly

Summary and Conclusion 28

CER transform healthcare identifying therapies,

procedures, preventive tests, and healthcare

processes most effective based on cost, quality,

and safety

State-of-the-art informatics platforms are

necessary to carry out this type of research

6 generic steps in CER: data identification,

extraction, modeling, aggregation, analysis, and

dissemination

Funding was provided by a contract from AcademyHealth. Additional support was provided by AHRQ 1R01HS019912-01 (Scalable PArtnering

Network for CER: Across Lifespan, Conditions, and Settings), AHRQ 1R01HS019908 (Scalable Architecture for Federated Translational Inquiries

Network), and NIH/NCRR Colorado CTSI Grant Number UL1 RR025780 (Colorado Clinical and Translational Sciences Institute).

Data Model Considerations for Clinical

Effectiveness Researchers

Michael G. Kahn1,3,4, Deborah Batson4, Lisa Schilling2

1Department of Pediatrics, University of Colorado, Denver 2Department of Medicine, University of Colorado, Denver

3Colorado Clinical and Translational Sciences Institute 4Department of Clinical Informatics, Children’s Hospital Colorado

Electronic Data Methods (EDM) Forum Clinical Informatics Webinar

31 October2012

Michael.Kahn@ucdenver.edu

Disclosures

Presentation based on EDM Forum commissioned paper:

30 Medical Care 50(9) 2012 S60-S67 DOI: 10.1097/MLR.0b013e318259bff4.

What is a data model & why should I care?

• A data model determines: – What data elements can be stored

– What relationships between data

can be represented

– Technical stuff: data type,

allowed ranges, required versus

optional (missingness)

• You should care because it

determines: – How easy can data be recorded

– How easy can data be extracted

– Contributes to data quality

31

Visit- versus Patient-centric data models

32

Query Complexity: “For each patient, how many

medications where filled over a period of time?”

33

Four-table join

Three-table join

Query Complexity: “Average number of

prescriptions written per visit?”

34

Two-table join

Three-table join +

Date comparisons

Key questions for a data model

• From Jeff Brown regarding FDA Sentinel Initiative*:

1. What does the system need to do?

2. What data are needed to meet

system needs?

3. Where will the data be stored?

4. Where will the data be analyzed?

5. Is a common data model needed,

and if so, what will the model look like?

35 *Brown JS, Lane K, Moore K, Platt R. Defining and evaluating possible database models to implement the FDA Sentinel initiative. U.S. Food and Drug Administration;

May 2009 2009.

Additional Data Model Requirements for SAFTINet

• Create patient-level analytic data sets

• Calculate ages to the year for adults, and to smaller units

of measurement for children

• Calculate prescribed drug intervals (drug exposures)

• Link data across disparate data sources

• Use standardized terminologies to take advantage of

conceptual hierarchies and relationships

• Identify a patient as being part of a defined cohort

• Support limited data sets compliant with HIPAA 36

37

Potential data models considered by SAFTINet

Name Developing entity Initial Purpose

Observational

Medical Outcomes

Project (OMOP)

Foundation of the NIH Comparative Drug Outcomes Studies

Virtual Data

Warehouse (VDW)

HMO Research Network Distributed data warehouse to allow

comparative studies across collaborating

sites: HMORN, CRN, Oregon CTRI

i2b2 Partners Healthcare Informatics framework for clinical and

biological data integration

OpenMRS Regenstrief Institute Open source enterprise medical record

system platform

OpenEHR OpenEHR Foundation Semantically-enabled open source health

computing platform

38

Summary Findings

• None of the existing publicly-available data

models met all requirements

• License-free, flexibility, active community and

willingness to collaborate were key features for

SAFTINet

• Each project has different requirements and

priorities.

• There is no best model for all potential CER uses

39

Funding was provided by a contract from AcademyHealth. Additional support was provided by AHRQ 1R01HS019912-01 (Scalable PArtnering

Network for CER: Across Lifespan, Conditions, and Settings), AHRQ 1R01HS019908 (Scalable Architecture for Federated Translational Inquiries

Network), and NIH/NCRR Colorado CTSI Grant Number UL1 RR025780 (Colorado Clinical and Translational Sciences Institute).

Data Model Considerations for Clinical

Effectiveness Researchers

Questions? Michael.Kahn@ucdenver.edu

Adam Wilcox, PhD

Columbia University

Necessary

Need data to be stored electronically ◦ Transcription is expensive

Could collect electronically …

BUT,

Tools for electronic data collection have been cumbersome ◦ Some studies say they are useful

◦ Still do not outweigh the ease of use of paper

HOWEVER,

Recent developments in consumer electronic devices

How will this influence electronic primary data collection?

Looked at 5 case studies, drawn from the EDM Forum projects

Performed semi-structured interview about primary data collection tasks

Questions about ◦ Workflow

◦ Connectivity

◦ Security

◦ Data integration

Assessed each method in terms of ◦ Ease of use

◦ Development experience needed

◦ Instrument and distribution costs

◦ Instrument flexibility

◦ Speed of data entry

◦ Accuracy

◦ Data loss potential

◦ Need for technical support

◦ Hardware/software requirements

Integrating data from clinical and research centers at 4 academic institutions ◦ Supporting cross-institution analysis

Web-based forms ◦ Occasionally collect with paper, then enter after

Advantages ◦ Data collection validation

◦ Rapid quality assurance

Expanding a state-wide health information exchange network ◦ Indiana Network for Patient Care

Scannable forms and a barcode scanner tool (caTrack) ◦ Data collection for biological samples

◦ Information collected in structured paper forms that are scanned, and linked with a barcode

Advantages ◦ Avoid human error

Patient research registry linking electronic health record data in different delivery sites ◦ Quality improvement using both EHR and registry

data

Qualitative interviews with some questionnaires ◦ Paper

Advantages ◦ Flexible

◦ Portable

Distributed health data network supporting CER and quality improvement for safety-net populations

Patient surveys connected with the patient encounter ◦ Structured forms within the EHR

◦ Similar to web-based form, but links to patient data

Advantages ◦ Data automatically connected to patient

Integrating patient data from multiple sites of care with community surveys

Tablet computers ◦ Some with direct internet connection, others with

data storage on the device and uploading to cloud

Advantages ◦ Portable

◦ Low levels of user training and technical support

◦ Integration of components

Camera

GPS

COMET Indiana PROSPECT Pediatric Enhanced

Registry

SAFTINet WICER

Purpose

Case report forms

during clinical visits

Collecting information on tissue samples that will be linked to

clinical data

Qualitative interviews with patients and family

members

Administering validated instrument as part of a

patient assessment

Survey about health related issues for individuals within a

community

Data collection site Clinical research site; direct patient entery could be done at

clinic or home

Clinical research site Clinical site, research center, or

patient’s home

Clinical site Patient’s home

Data collection

workflow

Collected and entered by research coordinator; pursuing direct

patient data entry

Collected and entered by research coordinator, both for mobile devices and scannable

forms

Collected and entered by research coordinator during qualitative

interviews

Collected and entered by clinical research coordinator as part of

clinical visit

Collected and entered by clinical research coordinator

during interview

Data entry approach Web-based forms

during interview

Scannable paper forms (with attached

barcodes)

Paper forms and

audio recordings

Form template within EHR (similar to web-

based form)

Tablet computer

form template

Category Paper forms (Pediatric Enhanced Registry)

Scannable forms (Indiana PROSPECT)

Web-based forms (COMET)

Form template in EHR (SAFTINet)

Tablet computer (WICER)

Ease of use + + - +

Experience required of designer

+ + -

End user training

+ + - Cost

+ - Flexibility

- + + Speed of entry

- - Accuracy/error rate

- + +

Potential for data loss

- + Need for technical support

+ +

Equipment/software requirements

+ -

Huge breadth in data collection methods used ◦ Not variation off the same themes

◦ Specific to the project needs

Trade-offs of different technologies ◦ Project setting and goals most significant in

defining what technology is used

Need: ◦ Decision tree algorithm

◦ Best practices for each approach

Continue the Discussion!

www.edm-forum.org

Medical Care supplement

Issue Briefs: – Meaningful Engagement

– ARRA Infrastructure Investments

CER Project Profiles

Descriptions of eHealth data initiatives for research & QI

Event archives

Wiki glossary

Upcoming Webinars – December 18th: Tackling

Practical Methodological Challenges of Using Electronic Data for CER & PCOR

Sign up at edmforum@academyhealth.org

Thank You

Please take a moment to fill out the

brief evaluation which will appear in your browser.