Committee on Credible Practice of Modeling & Simulation in Healthcarehttps://simtk.org/home/cpms

COMMON PRACTICE GUIDELINES: A SIGNIFICANT GAP IN COMPUTATIONAL MODELING AND SIMULATION IN HEALTHCARE

https://simtk.org/home/cpms

Can’t hear us? Select Audio -> Integrated VoIP -

> Join Conference

Presenter: Jeff Webb

Moderator: Joy Ku

ABOUT CPMS

& Multiscale Modeling (MSM) Consortium

Committee on Credible Practice of Modeling & Simulation in Healthcare

was established under

2

Can’t hear us? Select Audio -> Integrated VoIP -

> Join Conference

WEBINAR GOALS

3

• To provide examples of how others have translated their modeling and simulation research into clinical use

• To engage the community in discussions on best practices to modeling and simulation

Can’t hear us? Select Audio -> Integrated VoIP -

> Join Conference

WEBINAR LOGISTICS

4

Questions will be addressed at the end of the presentation via the text-based Q&A panel

For more help, consult the “Participating in WebEx Webinars” guide available on the Events page:

https://simtk.org/plugins/moinmoin/cpms/EventsCan’t hear us? Select

Audio -> Integrated VoIP -> Join Conference

Committee on Credible Practice of Modeling & Simulation in Healthcarehttps://simtk.org/home/cpms

COMMON PRACTICE GUIDELINES: A SIGNIFICANT GAP IN COMPUTATIONAL MODELING AND SIMULATION IN HEALTHCARE

https://simtk.org/home/cpms

Can’t hear us? Select Audio -> Integrated VoIP -

> Join Conference

Jeff Webb, Senior EngineerApplied Research Associates, Inc. (ARA)

BioGears Framework for Multiscale Physiology: Lessons Learned

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

6

• Program Background• Users, Integrators, and Collaborators• Implementation and Deployment• Keys to Success• Conclusion

Agenda

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

7

PROGRAM BACKGROUND

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

8

• Organization: Applied Research Associates, Inc. (ARA)

• Telemedicine & Advanced Technology Research Center (TATRC) Award #: W81XWH-13-2-0068

• Principal Investigator: Mr. Jeff Webb

• Amount: $6,959,593

• Period of Performance: Sept 2013 – Sept 2018

• Disclaimer: This work is supported by the US Army Medical Research and Materiel Command. The views, opinions and/or findings contained in this report are those of the author(s) and should not be construed as an official Department of the Army position, policy, or decision unless so designated by other documentation.

Project Information

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

9

High Level Objectives• Create a publicly available physiology research platform that

enables accurate and consistent simulated physiology across training applications

• Lower the barrier to create medical training content• Engage the community to develop and extend physiology models• Meet the training needs of the military• Expand the body of knowledge regarding the use of simulated

physiology for medical education

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

10

• Powerful backbone for applications• Powers immersive training content

and other M&S tools• Extensible standardized data model

with leverageable tools• Patient variability focused on

homeostasis with conditions, insults, and interventions for pathophysiology

What BioGears IS What BioGears IS NOT• Not a game or fancy application• Not (currently) a predictive model• Not (currently) designed to

directly ingest patient records

BioGears Conceptual System Architecture

BioGears Powered Applications

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

11

The Team

Past Contributors:Rachel Clipp, PhDJerry HeneghanYeshitila Gebremichael, PhDZack SwarmPat RusslerBeth SmithPaul RutledgeFederico MenozziAlex SomersKatie CarterCassidy LimerUNC Eshelman School of PharmacyDr. Brett Talbot

Government Side:JPC-1 Director: Jan Harris, PhD, RNJPC-1 Portfolio Manager: Kevin Kunkler, MDTATRC Grants Officer Representative: Harvey MageeTATRC Research Scientist: Geoff Miller, MS

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

12

Timeline / Milestones2013

2014

2016

2017

Milestone: Project Kick Off (Sept 15, 2013)Creation of Common Data ModelRe-architecture of Physiology EngineBase System DevelopmentBase System ValidationMilestone: Mini Build Release and Website Launch (Oct 2014)Base System Additions & ImprovementsSystem/Feature Development & ValidationCommunity Outreach BioGears Conference PlanningMilestone: Beta Build Release and Users’ Group Conference (Fall 2015)System/Feature Development & ValidationUser Community SupportContinued Community OutreachPublications & Conference Presentations

(9/2017 – 9/2018) Website Maintenance Only

Current

2018

2015

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

13

USERS, INTEGRATORS, AND COLLABORATORS

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

14

• USAMRMC: Advanced Modular Manikin (4 teams for phase 1; 1 for phase 2) – Patient simulation hardware and software

• RDECOM: Combat Medic – UnReal serious game (early version of BioGears)

• TATRC: HumanSim Sedation and Airway – clinician anesthesia training (pre-BioGears physiology)

• PEOSTRI: Medical Simulation Training Architecture RIF - requires open framework with direct reference to BioGears

• DHP: Warrior Health Avatar SBIR references BioGears directly

• ARL: Advanced Virtual Patient Applications RFP references BioGears directly

• Others specify leveraging existing open source software where possible

Government Programs

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

15

Other Known IntegratorsCategory Clinical Simulation

System Integrators Hardware Integrators Researchers/ Educaters Contributors

Advantages Offered

• Patient variability and management

• Existing library of conditions, insults, and interventions

• VR & AR

• Cross-platform• Low SWaP• Manikin driver• Task trainer extensions

• Understanding physiology and mechanisms

• Teaching fluid mechanics

• Predictive analytics

• Advancing the state-of-the-art

• Publishable• Notoriety

Examples

3,400+ downloads, looking for first external upload

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

16

IMPLEMENTATION AND DEPLOYMENT

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

17

• Dr. Ty N. Smith, M.D began working on a human physiology computer model in 1968 using fluid circuit models

• Contemporary of Guyton• Numerous research papers in PubMed

• 1990’s to 2007 Dr. Smith and Ken Starko (Aerospace Industry – flight simulation) developed “Body.dll”

• Body Purchased by Virtual Heroes / ARA in mid 2000’s from Advanced Simulation Corporation

• Body was advanced and used in ARA products• HumanSim serious game product line

• Notably Sedation and Airway, Anesthesia, PDAATS• Very Anesthesiology centric

• Combat Medic• 2013 ARA physiology engine open-sourced for use

as the starting point for BioGears• BioGears continues with the lumped parameter

circuit modeling approach

Design Approach and Brief History

http://patf-biokyb.lf1.cuni.cz/wiki/dokumenty/guyton

Schematic of Guyton's 1972 Blood Pressure Model

A Simple 3-Element Windkessel Model

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

18

Physiology Fidelity Approaches Considered

Graph # Approach Description Typical Simulation

Size Typical Applications Ease of Impl.

1 State-Based Cause and effect using state machines and discrete-event systems Whole Body Proctor Driven

Manikins +++

2 Statistical Fitting of data Whole Body Epidemiology modeling ++3 0-D Physics Based Lumped component model Whole Body Biomedical curriculum +4 1-D CFD Larger geometries with laminar flow Regions of Interest Aortic flow analysis -

5 3-D CFD Specific geometries where flow direction and stresses are important Regions of Interest Aneurysm or stenosis

analysis ---

3) Lumped element modeling:• Simplifies spatially distributed physical

systems into a topology of discrete entities• Reduces state space to a finite dimension

– PDEs of the continuous (infinite-dimensional) time and space model into ODEs with a finite number of parameters

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

19

Translating Needs to Specifications

• Key decisions:• Mechanistic where possible/appropriate, phenomenological otherwise• Precision and Accuracy – Rigor and Reproducibility

• Correlation does not mean causation• Correlation for validation purposes• Causation for implementation

• Design challenges – how to balance fidelity• Equal parts biology, math, science, engineering, and black arts

Modeling Approach:• Middle-out

Body Systems Organs Tissues Cells Proteins Genes

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

20

• How we choose (with an example system):• We always start with a literature search• Some we inherited general approach (e.g., cardiovascular)• Some we extended from existing (e.g., renal)• Some are derived from an external source (e.g., drugs/PK-PD)• Some are a combination of external sources (e.g., environment & acid-base balance)• Some are developed from scratch (e.g., energy)• Some are re-implemented fresh (e.g., anesthesia machine)• Some are mainly just warehouses of information (e.g., blood chemistry)• All to meet stated contractual requirements

• We are both a consumer and creator of models• Typical criteria for leveraging:

• Typically peer reviewed• Validated with, or derived from empirical data• Sound physics – conservation• Mechanistic/feedback based

• Creating to fill gaps:• Often developed from math/relationships presented in text books or published papers• We employ a methodical design review approach

• Parent models are often only as good as their worst child model (garbage in is garbage out, even when scaled) – high complexity

Model Selection

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

21

• Guidelines for credible practice of modeling & simulation in healthcare

• Did not intentionally follow rules during design and implementation, but our approach fits within this context

• See biogearsengine.com

CPMS Ten “Not So” Simple Rules

https://simtk.org/plugins/moinmoin/cpms/Ten%20Simple%20Rules%20of%20Credible%20Practice

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

22

KEYS TO SUCCESS

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

23

• Common Data Model (CDM): Well-defined, intuitive, interchangeable format to standardize interfaces• Standardized inputs, outputs, units,

and naming conventions to aid model additions and external model integrators

• Application Programming Interface (API): Easy integration and interaction in any programming language• Data organized logically by Anatomy

so that users are able to easily find and pull relevant data

• Software Development Kit (SDK): Application examples and stand-alone execution• Tutorials, How-to’s, scenario

examples

Solid Software Architecture Foundation

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

24

Modularity and ExtensibilitySupports multi-scale,

mixed-time-step modeling

https://simtk.org/home/simvascular

Increased Fidelity

Cardiovascular System

Renal System

Capillary

Increased FidelityIncreased Time-StepBoundary Conditions

Conceptual

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

25

Credible ApproachLayer/Level Tools/Utilities

Data Model:• Inputs/outputs• Unit conversion• Mapping of generic types• Compartment hierarchy• All target platforms (GCC compliance –

Windows, Mac, Linux)

• Software documentation• Data probes

Solvers:• Circuit solver (fluid, thermal, and

electrical)• Substance transporter (liquid and gas)• Blood gas distribution (saturation, acid-

base balance)• Compartment balancing equations

• Unit tests

Mechanisms:• Feedback• Chemoreceptors/Baroreceptors• Patient parameters

• Experimental data• Comparison plots

Systems:• Homeostasis• Actions• Conditions

• Scenario driver (scenarios for each)• Methodology reports• Validation tables

Whole Body Physiology:• Patient variability• Combined effects

• Software Development Kit (SDK)• Showcase scenarios (4 total)• Subject Matter Experts

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

26

Verification and Validation1. Verification: Unit tests ensure correct

implementation and sound physics principles for all tools

2. System Level Validation: All major systems (cardiovascular, respiratory, blood chemistry, etc.) are validated for clinical output level data

3. Compartment Level Calibration: Individual organs (kidney, liver, etc.) or functional units (trachea, alveoli, etc.) are validated wherever possible

4. Scenario Calibration & Validation: Every insult, intervention, and assessment includes a matrix with validation data for whole body combined effects from multiple systems

5. Combined Scenario Validation: All four showcases and several other scenarios validated for combined effects – heavily leveraged SME consultants Bryan Bergeron MD, Nicholas Moss PhD, and Stephen Mangum PharmD

1,886 output parameters validated!138 unit tests for verification not shownVerification and methodology waveform validation not shown – see methodology reports

Resting Physiology

System Data TypeValidation Measures in Deviation Category

Total< 10% 10 – 30% > 30%

CardiovascularSystem 18 4 2 24Compartment 54 4 6 64

RespiratorySystem 10 0 2 12Compartment 31 2 3 36

Renal

System 28 7 15 50Compartment 25 12 8 45Substance Params 15 6 9 30Assessments 6 0 0 6

Energy System 9 0 0 9

Blood ChemistrySystem 37 2 5 44Assessments 16 0 0 16

Patient (5) Patient Params 85 0 0 85Total 334 (79%) 37 (9%) 50 (12%) 421

Insults, Interventions, and Conditions Scenarios

Group Number of Scenarios

Validation Measures in Deviation CategoryTotal

< 10% 10 – 30% > 30%Cardiovascular 15 205 17 14 236Respiratory 27 147 35 142 324Renal 3 15 5 4 24Nervous 1 0 3 0 3Drugs 12 58 2 0 60Energy & Environment 5 45 16 19 80Anesthesia Machine 12 276 28 38 342Inhaler 5 39 3 0 42Combined 5 124 1 0 125Showcases 4 163 29 37 229Total 89 1072 (73%) 139 (9%) 254 (17%) 1465

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

27

• Many publications, presentations, seminars, etc.• Our forums include an authentication system so that

people can log-in, post questions and/or discussion topics, and get alerts

• Extremely beneficial to use standard terms for both inputs (e.g., physicochemical properties) and outputs (e.g., assessments, vitals, compartments)

• What users care about:• Most clinicians do not necessarily care what goes on under the

hood, as long as the results are correct and meets the training objectives

• Most researchers/educators care immensely what models are employed and how

• Interactions are/were very helpful to shift focus and reorder tasks

• Determined future needs:• Serialization (saved states for instantaneous loading)• Vitals override• “Try before you buy” graphical user interface• Faster, less functional version• New acute injuries and other conditions

Community Outreach

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

28

CONCLUSION

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

29

• Use the website: information on downloading, running, and integrating

• BioGears is designed for model extensions and improvements to be integrated by the user base over time

• Let us know how we can help

• We would love to work with CPMS as they define a certification process

How to Use BioGears

Cop

yrig

ht 2

015.

All

right

s re

serv

ed. A

pplie

d R

esea

rch

Ass

ocia

tes,

Inc.

30

• Use the software for any and all applications (please let us know)

• Report problems• Post and respond to Forums (biogearsengine.com/forums)• Submit code

• Currently just email us (biogearsengine.com/workwithus)• Moving to a public repository – GitHub/BitBucket hosted

How to Contribute

HOW TO ASK QUESTIONS

31

2. Type your question in the box at the bottom of the Q&A Panel.

Questions must be asked of “All Panelists”

1. Click on the Q&A button to open the Q&A panel.

CONTINUE THE CONVERSATION

32

To learn more about CPMS, visithttps://simtk.org/projects/cpms

Participate in the discussion forum

Please fill out the survey that will appear after the webinar ends