The Promise of Big Data and what it means to PSOsBrad Winters, Ph.D., M.D.The Armstrong Institute for Patient Safety and Quality

Objectives

• Know what Big Data is.• Be able to give examples of how Big Data can

serve patient safety and quality endeavors.• Know how Big Data approaches and analytics

can improve PSO collaboration, scope and efficiency.

• Be able to describe the basics of the AHRQ Quality and Safety Review System.

• Understand how EHRs and other developing healthcare IT systems can further PSO goals.

What is Big Data?

• Data sets that are too large orcomplex for traditional softwareprocessing applications

• But what it really refers to, in termsof its usefulness and where thepromises lies is– The patterns and information

that emerge from the data– Medicine has always relied on

observation and recognition ofpatterns in information

– Because of this, “Big Data” carries huge potential for transforming how we practice medicine and how we make it safer and improve quality

Some Potential Pitfalls of Big Data

• Clinical Information is complex– Large data sets can result in differences that are determined to be

statistically different (i.e., ”significant”) but not clinically different or relevant

– Computers don’t yet have the ability to make this comparison• Thus there will always (?) need to be human assessment

– Humans need to frame the question– Humans need to determine the results’ relevancy

• Data Security and appropriate use• During collection• Storage • Analysis• Reporting

Sepsis

• 12th leading cause of mortality in the U.S.• Often recognized late in its course• Despite this, we have driven mortality down from about 40% to 25%

across the sepsis spectrum– Based largely on better treatment but limited impact of earlier

recognition.• Early recognition and intervention is considered essential to driving

down morbidity and mortality further• The problem is that sepsis is a heterogeneous syndrome• It defies a single definition making it difficult to create a reliable

criteria for diagnosis in the traditional way• But can we, using “big data analytics”, find the underlying patterns

and thread that will reliably predict who is at risk or already starting to develop the condition?

• “Sepsis Sniffers”=mining data

“Sniffing” for Sepsis

– ICUs and Emergency Depts. generate monstrous amounts of data (patient under a microscope) through continuous monitoring and testing

– Physiological monitors (Heart rate, respiratory rate, temperature, blood pressure, oxygenation, etc. etc.)

– Laboratory data (general labs, multiple potential biomarkers)

– Physical exam data• Can we leverage this big data for detecting

patterns and generating clinically useful information?

Yes, we can!

• The Problem: Clinicians identify sepsis throughstatic observation of continuous data withoutaccounting for the dynamic interactions betweendata signals.

• The Approach: Network-based data analysis to separate septicfrom non-septic patients

• Account for interactions in continuous and intermittent data• Look for features that distinguish between sepsis and non-sepsis• While raw data could not distinguish (this is how we do it now!)• Data analyzed using a computer algorithm (median values of

something called an eigenvalue) was able to distinguish the sepsisvs. the non-septic state

• These values also may reflect disease progression (who is gettingworse) Santaniello et al. Conf Proc IEEE Eng Med

Biol Soc. 2014;2014:3825-6.

“Sniffing” for Sepsis on the General Ward: the challenge of data sensitivity and specificity

• Not everyone is in the ICU, most patients are on general wards

• Can we transfer what we learn in the ICU to the general ward?– Staff to patient ratios are low– Data is is more scarce, not continuous– Fidelity of this data and its communication is known to be

poor• Brandt et al. (Am J Med Qual 2015, 30: 559-65) created an electronic algorithm

for sepsis “sniffing” on general wards– 100% sensitive (excellent at ruling out sepsis) compared to

clinicians– 62% specific (not so good at ruling in sepsis) compared to

clinicians– Still relies on intermittent, low fidelity data

Big Data is not Big Data unless the Data is Big!

• The General Hospital ward– While it is a trove of data– It is a desert landscape

compared to the ICU• Can we enrich this data

stream and marry it to these kinds of algorithms to improve specificity for all kinds of threats to the patient?

Failure to Rescue (FTR)

http://www.newsnet5.com/longform/dead-in-bed-a-deadly-hospital-secret

Our main response to FTR?

• The Rapid Response Systems • Effective

– 40% reduction in arrests– Mortality reduction 12-18%

• We have made little progress in addressing the failure of what is known as the Afferent Limb (the process of signal definition and detection and notification)

• Can big data and big data analytics move this forward?• I say it can but;• We need to change little “big data” into big “Big Data”

without disrupting workflow or generating alarm/alert fatigue

Surveillance Monitoring

Taenzar et al.Anesthesiology 2011

Surveillance Monitoring

• Can we monitor all patients easily, with high fidelity?

• Can it go beyond “physiological deterioration”?– Falls, Pressure ulcers– Other harms

• Why this would fail– $$$$$$$$$ this is a red herring

– this is the real problem

Results of a Hopkins Surveillance Pilot Project

422 pts, 1266 pt/days, 4118 total alarms, No statistical differences in

gender or age

True w/intervention=1590

(38.6%)

Total True=2668(64.8%)

False=1249(30.3%)

Unclear=202(4.9%)

𝑃𝑃𝑃𝑃𝑃𝑃 =𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎

𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 + 𝑜𝑜𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎

Total PPV was 68%

Complications statistically reducedNo FTR deaths during study period1 death in historic control period

Results of a Hopkins Surveillance Pilot Project

422 pts, 1266 pt/days, 4118 total alarms, No statistical differences in

gender or age

True w/intervention=1590

(38.6%)

Total True =2668(64.8)%

False=1249(30.3%)

Unclear=202(4.9%)

𝑃𝑃𝑃𝑃𝑃𝑃 =𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎

𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 + 𝑜𝑜𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎

Total PPV was 68%

Complications statistically reducedNo FTR deaths during study period1 death in historic control period

Is This Good Enough?

Staff Satisfaction Survey: Selected Results

Surveillance Monitoring and Big Data

• Where are the Big Data analytics for harm and disease detection?– Currently there aren’t any– Big Data Analytics here are still focused on controlling the

alarm and alert fatigue issue and preventing nurse and physician workflow disruptions.

– However, while many argue that these need to be controlled first, Big Data Approaches can be applied to the process of controlling alarm/alert fatigue.

– Like sepsis, all other harms (including alert and alarm fatigue) are complex and discrete elements don’t change in isolation.

– Big Data approaches can make these connections and not only improve harm prediction, but also control the false alert/alarm problem.

All Patient Harms: The AHRQ Quality and Safety Review System (QSRS)

• Patient Safety Reporting systems• Numerous Patient Safety Surveillance systems

have developed over the last 2 decades.• Some are very specific

– MEDMARX (US Pharmacopeia)• Specifically tracks medications related errors and

harm• Some are very generic

– The Veterans Administration Patient Safety Reporting system

• Some require membership– University Health Consortium Patient Safety Net

• Some are local homegrown, some at the state level and others national or international

Big Data and All Harms

• Many such databases are used for Big Data analytics already (e.g., MedMarx) and have led to a body of literature correlating harms’ causes, predictors, etc.

• But there is so much un-tapped opportunity out there

• Some surveillance systems interface but many function in silos

• Integrated holistic databases would be ideal.Can we do this?

Why the QSRS?

• What if you could directly abstract medical records for harms?

• Generate population and individual event reports for harms analysis?

• Populate a national database for Big Data analytics looking for patterns and predictors of harm?– You could more specifically identify individual

patient harms and link them to events in the record.

– This could then augment the AHRQ PSIs and other tools to help guide hospitals in patient safety and quality improvements.

What is the QSRS?

• The QSRS is a logical query algorithm that guides medical coders through a medical record abstraction.

• Designed to– Identify adverse events – Identify certain quality indicators (e.g., fall

assessment at admission)– Collects demographics data for analysis

The QSRS

• Efficiency– Uses a set of Entry Questions that determine

whether harms are even possible for the admission– For example: asks whether the patient had a

central line placed at any time• If answer is “No” then a CLABSI could not occur

– Entire blocks of subsequent questions are suppressed so for this example all subsequent CLABSI related questions are not posed.

– Depending on the patient, nearly all questions could be suppressed or nearly all questions could be posed.

The QSRS

• POA modifiers built into the query for accurate determination of hospital event attribution

• Uses the Common Formats

• Reports can be generated at several levels:– Population level– Event level– Patient level (Case summary)

Population Reports

Case Summary Report

The QSRS

• QSRS is still under development, Ongoing refinement

• “QSRS 1”– First Pilot assessment outside of a Federally

maintained database and first wholly based on EHRs (previous assessments used EHR printouts)

• “QSRS 2”– 2 Health Systems

• Medstar-Washington, D.C./Baltimore system• Johns Hopkins (2 hospitals) partnered with Allegheny

Health in PA, and Queen’s Medical Center in Hawaii.

“QSRS 1”

• System is efficient• Average data abstraction time ≅30 minutes

• Inter-rater reliability is good

• Usability ratings were good (≥73% on most items in survey)

Record set Mean (SD) (minutes) Median (IQR) (minutes)

All sites 33.55 (26.19) 26.72 (25.79)

Facility Percent Agreement

BMC 78.21%

JHH 74.30%

Sibley 81.59%

Where can the QSRS go?

• Outcomes and Process measures• EHRs are a giant repository of “big data”• So far, they have been little more than hand-writing

tools (eliminating errors due to bad handwriting)• But they can be so much more:

– Automated/semi-automated data abstraction/extraction from EHRs to reduce measurement burden

– Natural Language Processing• will require more standardization of EHRs

Leveraging the EHR and QSRS: the Holy Grail

Big Data???(not so much)

VS

This is Big Data

The future of the QSRSand Big Data

• The QSRS can be the Big Data Analytic for Patient Safety and Quality

• Provide for Standardization (Common Formats, common language and reports)

• Support PSO expansion, sharing and integration– “stop re-inventing the wheel”

• Expand scope and understanding

The future of the QSRSand Big Data• PSO learning for rare events (individual

PSOs may never even see some of theseevents) since Big Data throws a wide net.

• National repository for comparators• Drive the standardization of Patient Safety

and Quality elements we want in the EHRs would be ideal (right now each EHR platform is highly proprietary and therefore different).– Where are the assessments?– Where are the harms documented?– Relevant labs, orders, medications etc., etc.– Improve efficiency and accuracy of data abstraction

and analytics

To improve we need to know wherewe are and where we need to go

Thank you