case study: rapid analytics response to the covid-19 crisis
TRANSCRIPT
1
Jeff SheinSenior Director, Data and Analytics, NYU Langone Health
Eduardo Iturra te , MD, MSW Medical Director for Enterprise Data and Analytics, NYU Langone Health
DISCLAIMER: The views and opinions expressed in this presentation are solely those of the author/presenter and do not necessarily represent any policy or position of HIMSS.
Case Study: Rapid Analytics Response to the COVID-19 CrisisSes s ion # 3, Augus t 10 , 2021
2#HIMSS21
Welcome
Senior Director, Data and AnalyticsNYU Langone Health
Jeff SheinMedical Director for Enterprise Data and Analytics
NYU Langone Health
Eduardo Iturra te , MD, MSW
#HIMSS21
Conflict of Interest
Jeff Shein and Eduardo Iturrate, MD, MSW have no real or apparent
conflicts of interest to report.
3
#HIMSS21
Agenda
• COVID-19 Surge
• Meeting the need for Data and Analytics to manage the Surge
• Not the usual Data and Analytics Workflow
• Data Challenges
• Architecture
• Dashboards
• Deidentified Dataset
4
#HIMSS21
Learning Objectives
• Describe the infrastructure and processes in place for rapidly developing new
metrics and dashboards for clinical and executive leadership
• Explain the process to develop a de-identified COVID-19 data repository
• Demonstrate the impact of the dashboards and de-identified data repository
5
is one of the nation’s premier academic medical centers. Our trifold mission to serve, teach, and discover is achieved daily through an integrated academic culture devoted to excellence in patient care, education, and research.
#HIMSS21 10
Stage 7 Stage 6
HIMSS Davies Award Recipient
HIMSS EMRAM/OEMRA
M Stage 7 Enterprise
Staff Recognized for Excellence by
ComputerWorld
Most Wired Hospital
HIMSS AMAM Stage 6
Enterprise
#HIMSS21 11
COVID-19 Pandemic in NYC1st Confirmed Cases Across NYULH:3/5/2020: NYU Langone Hospital – Long Island3/11/2020: NYU Langone Hospital – Brooklyn3/13/2020: Tisch Hospital & Kimmel Pavilion
#HIMSS21 12
March 13th
#HIMSS21 13
April 9th
#HIMSS21 14
• Hospital Operations leveraged Epic Reporting Workbench reports for real time data
• Numbers compiled and emailed to clinical leadership
• Reports executed twice daily
Tracking Initial Cases
• Intubations• ECMO in use• Patients in house• Discharges• Patients downgraded• Lab results
15
COVID-19 Leadership ScorecardWhat started out as “just a few reports”…
• Strategy and Planning + Hospital Operations• Data from RWB reports stored in Excel and converted to PDF• Used past reports for history• Manually executed, complied and
distributed twice daily• Reports run for each of 3 campuses• Emergency situation – bypassed RMS
(Reporting Metrics Subcommittee) process
#HIMSS21
Covid-19 Leaders hip Das hboard Project
16
Requirements
Convert existing scorecard into automated, interactive dashboard
Near real-time updates
Filters for time and campus with drill down to case level detail
Email snapshot reports twice daily
Challenges
No documented metric definitions
Unclear business contacts for project team
Real time data cannot use existing data warehouse ETL processes
Covid-19 workflows and criteria were new and often changing
Manual data
Time
#HIMSS21
Report ing & Metrics Subcommit tee (RMS)• Ensure the consistency of enterprise reporting metrics across NYULH
• Responsible for the management of the governance review of issues and proposals for enterprise metric changes, additions or deletions
17
#HIMSS21
Metric Development / RMS Proces s
18
#HIMSS21
• Metric Name• Definition• Cohort/Inclusions• Cohort/Exclusions• Associated Reference Documents• Abbreviations, Acronyms & Synonyms• Data Source - Numerator• Data Source - Denominator• Source System• Format• Business Owner• Business Contact• Technical Contact• Formula - Numerator• Formula - Denominator
NYULH Bus ines s Glos s ary
19
Metric Data Elements
• Calculation• Target• Target Source• Metric Threshold• Benchmark• Benchmark Source• Optimal Direction• Audience• Metric Drill Downs• Frequency of Reporting• Analytic Center Dashboard Location• Other Metric Publications• Related Business Terms
20
We leverage a centralized data governance approach to maintain standardization and consistency across the enterprise:
• Every dashboard metric has a published definition with a business owner
• Each metric can have only a single definition
• 800 organizational metrics and 200+ business term definitions, reviewed by business owners
• Data management through Collibra
• Business definitions linked to dashboards and available to all through portal
#HIMSS21
Data Challenges
21
• What counts as a COVID-19 patient?• COVID-19 ED patients - Confirmed vs PUI• How to count COVID-19 mortalities?• How to measure Ventilators & Intubations (Epic Flowsheets vs Orders vs LDAs (Epic’s
documentation for tubes)• Changes in bed base• Number of vents and ECMO machines available kept manually
#HIMSS21
COVID-19 Confirmed Update
22
Current State New Criteria Added
COVID-19 Confirmed –COVID Illness
•Positive COVID-19 lab result during encounter or 0-14 days prior to encounter date
Or•COVID-19 billing code (B97.29 before 4/1 or
U07.1 after 4/1)Or•COVID-19 Flag on Bed AssignmentOr•COVID-19 Flag at Discharge
Include all current state criteria andProblem List or Clinical Impression includes:“Due to Covid” or“Acute respiratory failure with hypoxia” or“Acute and chronic respiratory failure with hypoxia” or“Acute on chronic respiratory failure with hypoxia and hypercapnia”
COVID-19 Confirmed –Non-Acute COVID
•Positive COVID-19 lab result during encounter or 0-14 days prior to encounter date
Or•COVID-19 billing code (B97.29 before 4/1 or
U07.1 after 4/1)Or•COVID-19 Flag on Bed AssignmentOr•COVID-19 Flag at Discharge
None, Non-Acute Covid-19 Confirmed replaces current state Covid-19 Confirmed
#HIMSS21
COVID-19 High Level Architecture
23
• Feed real-time reports into EDW• Combine Epic data with manual data in
warehouse• Listener checks for files and updates• Notifications and alerts based on hourly
schedule• EDW updates trigger dashboard refreshes• Daily subscriptions
Epic Hyperspace
Epic Reporting Workbench reports
EDW
Failure – Send alertListener checks for files
ETL
Failure – Send alert
Event-based extract refreshesEpic Clarity
Executive Covid Dashboard
Email Leadership Summary
Manual Data
#HIMSS21
COVID-19 Executive Summary
24
• Trend metrics by day, week, month• Filter for campus• Drill down to patient detail• Running total or % change
#HIMSS21
COVID-19 Executive Summary
25
Scorecard View Executive Leadership Recap
COVID-19 Near Real-Time EDW Architecture
26
Limitations:
1.Limitation of RWB capability
2.Cannot change or recalculate history
3.Performance
4.Stand-alone data model not
connected to the rest of our EDW
#HIMSS21
NYU Langone COVID-19 Analytics 2.0 – In Progress
27
• Rebuild back end to simplify architecture and integrate into historical Data Warehouse
• Regroup with clinical leadership to update or retire metrics
• Track COVID-19 patients long term in both inpatient and outpatient settings
• Pull historical data from Epic Clarity and use Reporting Workbench for current data only
• Add “COVID-19” to dashboard Clinical Cohort filters to existing dashboards
#HIMSS21
NYU Langone COVID-19 Analytics
28
#HIMSS21
COVID-19 Operational Analytics
29
OR Cancelled Cases
Lab Testing
Patient Vaccination Appointments
#HIMSS21
COVID-19 Employee Metrics
30
Employee Testing Daily Symptom Check
Staff Vaccinations
#HIMSS21
COVID-19 De-identified Data Repository
31
#HIMSS21
What is the COVID-19 De-identified dataset?
• Data pulled from NYULH EHR for records starting January 1, 2020
• Dataset is updated daily
• De-identification:
• Unique identifiers (MRN, Encounter IDs) stripped, replaced by system IDs
• Dates shifted for each patient (but chronology preserved within patients)
• Variables aggregated – age is categorized into buckets
#HIMSS21
Data Use Agreement
• Dataset is not shared outside of NYULH
• Does not require IRB review for use for research
• If you see something in the dataset that you believe may be identifying,
immediately inform the dataset administrators
• Cannot explicitly attempt to re-identify patients
#HIMSS21
Clinical Data in the Dataset
34
Core Tables Derived Tables
patient blood_product_transfusionencounter encounter_inpatient_covid_relateddiagnosis encounter_ctpa_peprocedure_order encounter_cxr_opacitylab_result encounter_dopplers_dvtmedication encounter_with_dialysismar encounter_with_ecmosurgical_procedure encounter_with_ent_consultobservation encounter_with_oxygen_supportvent_observation encounter_with_pressorslda encounter_with_tracheostomyambulatory_observation encounter_with_ventilatorsocial_history encounter_with_ventilator_start_stopinpatient_adt_event patient_charlson_comorbidities
patient_elixhauser_comorbiditiespatient_sogiprone_observationrrt_observation_tablewound_skin_observation
#HIMSS21
covid_deident: COVID-19 De-identified database. Users have ‘read’ access to the data only
covid_deident_wksp: Workspace where users can create their own tables for staging purpose (e.g. split up a large query into several intermediate tables) or to share with others.
NYULH COVID-19 Data Challenge
36
Testing hypotheses, Visualizing and Mining Data
An invitation to all clinicians, clinical researchers, data scientists, biostatisticians and students to propose research questions that can be addressed using the COVID-19 Deidentified Dataset, or to devise novel data visualizations and data science techniques that can be applied to glean insights from the Health System's experience combatting COVID-19.
Goal:
To serve as a catalyst to promote innovative use of clinical data to help understand the health system’s experience with COVID
Impact• 150 participants took part in the challenge and submitted 15
projects
• 2 projects were named as winners and received project
support to submit manuscripts to peer-reviewed journals
• 100+ ongoing users of the dataset
37
#HIMSS21
Q&A
38
#HIMSS21
Thank You!
39
Jeff SheinSenior Director, Data and Analytics, NYU Langone [email protected]/in/jeff-shein-b226696
Eduardo Iturra te , MD, MSW Medical Director for Enterprise Data and Analytics, NYU Langone [email protected]/in/eduardo-iturrate