case study: rapid analytics response to the covid-19 crisis

1

Jeff SheinSenior Director, Data and Analytics, NYU Langone Health

Eduardo Iturra te , MD, MSW Medical Director for Enterprise Data and Analytics, NYU Langone Health

DISCLAIMER: The views and opinions expressed in this presentation are solely those of the author/presenter and do not necessarily represent any policy or position of HIMSS.

Case Study: Rapid Analytics Response to the COVID-19 CrisisSes s ion # 3, Augus t 10 , 2021

2#HIMSS21

Welcome

Senior Director, Data and AnalyticsNYU Langone Health

Jeff SheinMedical Director for Enterprise Data and Analytics

NYU Langone Health

Eduardo Iturra te , MD, MSW

#HIMSS21

Conflict of Interest

Jeff Shein and Eduardo Iturrate, MD, MSW have no real or apparent

conflicts of interest to report.

3

#HIMSS21

Agenda

• COVID-19 Surge

• Meeting the need for Data and Analytics to manage the Surge

• Not the usual Data and Analytics Workflow

• Data Challenges

• Architecture

• Dashboards

• Deidentified Dataset

4

#HIMSS21

Learning Objectives

• Describe the infrastructure and processes in place for rapidly developing new

metrics and dashboards for clinical and executive leadership

• Explain the process to develop a de-identified COVID-19 data repository

• Demonstrate the impact of the dashboards and de-identified data repository

5

is one of the nation’s premier academic medical centers. Our trifold mission to serve, teach, and discover is achieved daily through an integrated academic culture devoted to excellence in patient care, education, and research.

#HIMSS21 10

Stage 7 Stage 6

HIMSS Davies Award Recipient

HIMSS EMRAM/OEMRA

M Stage 7 Enterprise

Staff Recognized for Excellence by

ComputerWorld

Most Wired Hospital

HIMSS AMAM Stage 6

Enterprise

#HIMSS21 11

COVID-19 Pandemic in NYC1st Confirmed Cases Across NYULH:3/5/2020: NYU Langone Hospital – Long Island3/11/2020: NYU Langone Hospital – Brooklyn3/13/2020: Tisch Hospital & Kimmel Pavilion

#HIMSS21 12

March 13th

#HIMSS21 13

April 9th

#HIMSS21 14

• Hospital Operations leveraged Epic Reporting Workbench reports for real time data

• Numbers compiled and emailed to clinical leadership

• Reports executed twice daily

Tracking Initial Cases

• Intubations• ECMO in use• Patients in house• Discharges• Patients downgraded• Lab results

15

COVID-19 Leadership ScorecardWhat started out as “just a few reports”…

• Strategy and Planning + Hospital Operations• Data from RWB reports stored in Excel and converted to PDF• Used past reports for history• Manually executed, complied and

distributed twice daily• Reports run for each of 3 campuses• Emergency situation – bypassed RMS

(Reporting Metrics Subcommittee) process

#HIMSS21

Covid-19 Leaders hip Das hboard Project

16

Requirements

Convert existing scorecard into automated, interactive dashboard

Near real-time updates

Filters for time and campus with drill down to case level detail

Email snapshot reports twice daily

Challenges

No documented metric definitions

Unclear business contacts for project team

Real time data cannot use existing data warehouse ETL processes

Covid-19 workflows and criteria were new and often changing

Manual data

Time

#HIMSS21

Report ing & Metrics Subcommit tee (RMS)• Ensure the consistency of enterprise reporting metrics across NYULH

• Responsible for the management of the governance review of issues and proposals for enterprise metric changes, additions or deletions

17

#HIMSS21

Metric Development / RMS Proces s

18

#HIMSS21

• Metric Name• Definition• Cohort/Inclusions• Cohort/Exclusions• Associated Reference Documents• Abbreviations, Acronyms & Synonyms• Data Source - Numerator• Data Source - Denominator• Source System• Format• Business Owner• Business Contact• Technical Contact• Formula - Numerator• Formula - Denominator

NYULH Bus ines s Glos s ary

19

Metric Data Elements

• Calculation• Target• Target Source• Metric Threshold• Benchmark• Benchmark Source• Optimal Direction• Audience• Metric Drill Downs• Frequency of Reporting• Analytic Center Dashboard Location• Other Metric Publications• Related Business Terms

20

We leverage a centralized data governance approach to maintain standardization and consistency across the enterprise:

• Every dashboard metric has a published definition with a business owner

• Each metric can have only a single definition

• 800 organizational metrics and 200+ business term definitions, reviewed by business owners

• Data management through Collibra

• Business definitions linked to dashboards and available to all through portal

#HIMSS21

Data Challenges

21

• What counts as a COVID-19 patient?• COVID-19 ED patients - Confirmed vs PUI• How to count COVID-19 mortalities?• How to measure Ventilators & Intubations (Epic Flowsheets vs Orders vs LDAs (Epic’s

documentation for tubes)• Changes in bed base• Number of vents and ECMO machines available kept manually

#HIMSS21

COVID-19 Confirmed Update

22

Current State New Criteria Added

COVID-19 Confirmed –COVID Illness

•Positive COVID-19 lab result during encounter or 0-14 days prior to encounter date

Or•COVID-19 billing code (B97.29 before 4/1 or

U07.1 after 4/1)Or•COVID-19 Flag on Bed AssignmentOr•COVID-19 Flag at Discharge

Include all current state criteria andProblem List or Clinical Impression includes:“Due to Covid” or“Acute respiratory failure with hypoxia” or“Acute and chronic respiratory failure with hypoxia” or“Acute on chronic respiratory failure with hypoxia and hypercapnia”

COVID-19 Confirmed –Non-Acute COVID

•Positive COVID-19 lab result during encounter or 0-14 days prior to encounter date

Or•COVID-19 billing code (B97.29 before 4/1 or

U07.1 after 4/1)Or•COVID-19 Flag on Bed AssignmentOr•COVID-19 Flag at Discharge

None, Non-Acute Covid-19 Confirmed replaces current state Covid-19 Confirmed

#HIMSS21

COVID-19 High Level Architecture

23

• Feed real-time reports into EDW• Combine Epic data with manual data in

warehouse• Listener checks for files and updates• Notifications and alerts based on hourly

schedule• EDW updates trigger dashboard refreshes• Daily subscriptions

Epic Hyperspace

Epic Reporting Workbench reports

EDW

Failure – Send alertListener checks for files

ETL

Failure – Send alert

Event-based extract refreshesEpic Clarity

Executive Covid Dashboard

Email Leadership Summary

Manual Data

#HIMSS21

COVID-19 Executive Summary

24

• Trend metrics by day, week, month• Filter for campus• Drill down to patient detail• Running total or % change

#HIMSS21

COVID-19 Executive Summary

25

Scorecard View Executive Leadership Recap

COVID-19 Near Real-Time EDW Architecture

26

Limitations:

1.Limitation of RWB capability

2.Cannot change or recalculate history

3.Performance

4.Stand-alone data model not

connected to the rest of our EDW

#HIMSS21

NYU Langone COVID-19 Analytics 2.0 – In Progress

27

• Rebuild back end to simplify architecture and integrate into historical Data Warehouse

• Regroup with clinical leadership to update or retire metrics

• Track COVID-19 patients long term in both inpatient and outpatient settings

• Pull historical data from Epic Clarity and use Reporting Workbench for current data only

• Add “COVID-19” to dashboard Clinical Cohort filters to existing dashboards

#HIMSS21

NYU Langone COVID-19 Analytics

28

#HIMSS21

COVID-19 Operational Analytics

29

OR Cancelled Cases

Lab Testing

Patient Vaccination Appointments

#HIMSS21

COVID-19 Employee Metrics

30

Employee Testing Daily Symptom Check

Staff Vaccinations

#HIMSS21

COVID-19 De-identified Data Repository

31

#HIMSS21

What is the COVID-19 De-identified dataset?

• Data pulled from NYULH EHR for records starting January 1, 2020

• Dataset is updated daily

• De-identification:

• Unique identifiers (MRN, Encounter IDs) stripped, replaced by system IDs

• Dates shifted for each patient (but chronology preserved within patients)

• Variables aggregated – age is categorized into buckets

#HIMSS21

Data Use Agreement

• Dataset is not shared outside of NYULH

• Does not require IRB review for use for research

• If you see something in the dataset that you believe may be identifying,

immediately inform the dataset administrators

• Cannot explicitly attempt to re-identify patients

#HIMSS21

Clinical Data in the Dataset

34

Core Tables Derived Tables

patient blood_product_transfusionencounter encounter_inpatient_covid_relateddiagnosis encounter_ctpa_peprocedure_order encounter_cxr_opacitylab_result encounter_dopplers_dvtmedication encounter_with_dialysismar encounter_with_ecmosurgical_procedure encounter_with_ent_consultobservation encounter_with_oxygen_supportvent_observation encounter_with_pressorslda encounter_with_tracheostomyambulatory_observation encounter_with_ventilatorsocial_history encounter_with_ventilator_start_stopinpatient_adt_event patient_charlson_comorbidities

patient_elixhauser_comorbiditiespatient_sogiprone_observationrrt_observation_tablewound_skin_observation

#HIMSS21

covid_deident: COVID-19 De-identified database. Users have ‘read’ access to the data only

covid_deident_wksp: Workspace where users can create their own tables for staging purpose (e.g. split up a large query into several intermediate tables) or to share with others.

NYULH COVID-19 Data Challenge

36

Testing hypotheses, Visualizing and Mining Data

An invitation to all clinicians, clinical researchers, data scientists, biostatisticians and students to propose research questions that can be addressed using the COVID-19 Deidentified Dataset, or to devise novel data visualizations and data science techniques that can be applied to glean insights from the Health System's experience combatting COVID-19.

Goal:

To serve as a catalyst to promote innovative use of clinical data to help understand the health system’s experience with COVID

Impact• 150 participants took part in the challenge and submitted 15

projects

• 2 projects were named as winners and received project

support to submit manuscripts to peer-reviewed journals

• 100+ ongoing users of the dataset

37

#HIMSS21

Q&A

38

#HIMSS21

Thank You!

39

Jeff SheinSenior Director, Data and Analytics, NYU Langone HealthJeff.shein@nyulangone.orglinkedin.com/in/jeff-shein-b226696

Eduardo Iturra te , MD, MSW Medical Director for Enterprise Data and Analytics, NYU Langone HealthEduardo.iturrate@nyulangone.orglinkedin.com/in/eduardo-iturrate