Medical 3D (m3D) Lab

Korak Sarkar, MD Department of Neurology

Innovation Ochsner Ochsner Health System

New Orleans, LA March 15th, 2019

Korak Sarkar Department of Neurology

No Relevant Disclosures

Objectives

•To learn about additive manufacturing and immersive technologies •To learn about their healthcare and

neuroscience applications •To learn how you can be begin to apply these

technologies to medical education, research, and clinical care

What is Additive Manufacturing?

various processes for making a three-dimensional object of almost any shape from a

3D model or other electronic data source primarily through additive processes in which successive layers of material are laid down

under computer control. In contrast to subtractive manufacturing

SCULPTURES

ADDITIVE MANUFACTURING

Fusion Deposition Molding Stereolithography

Polyjet Selective Laser Sintering

OUR STORY

Feasibility and application of Additive Manufacturing in the Neurosciences. Sarkar K, Rudman E, Kalyvas J.

Ochsner 13th Annual Research Day, May 2016.

COMPONENTS OF M3D LABAcquisition

Transfer Post processing (Segmentation)

Printing

Cardiothoracic

Transplant

Neurosurgery

Orthopedics

Imaging Orders Radiology

Clinical imaging needs

Image acquisition

Ochsner Medical 3D(Om3d) lab

Image segmentation and STL generation

3D Printer*

Om3d VR

Om3d AR

3D rendering process

3D Print

VR

AR

*3D printers used include: Form 2 SLA, Projet 660, ProJet MJP 2500, Prusa i3,

3D Models

Figure 1. Clinical workflow for creating 3D printed, Virtual and/or Augmented Reality(VR/AR) models. Clinicians order scans for their patients per standard of care, not for the purpose of 3D modeling. Imaging is acquired in the Department of Radiology and saved in DICOM format. The clinician consults the medical 3D group if they wish to create a 3D model of their patient’s anatomy. The DICOM files are obtained from Radiology and processed to generate STL files. The STL file can be used to created printed, VR, or AR models. Om3d refers to Ochsner’s medical 3d lab.

Om3d lab Workflow

DICOM

Atlanto-Axial Instability

Is 3D printing the Future?

. Yoo SJ, Spray T, Austin EH, 3rd, et al. Hands-on surgical training of congenital heart surgery using 3-dimensional print models. The Journal of thoracic and cardiovascular surgery 2017;153(6):1530-40.

FDA Final Guidance on m3DP

Growth of m3DP labs

Labs around the world

• Cleveland Clinic • The Johns Hopkins University • Boston’s Children’s Experience • Mayo Clinic • Walter Reed National Military Medical Center • Northwestern University • Hospital for Special Surgery • University of Miami • Ottawa Hospital

Video Game Development

What are immersive technologies?

• aka mixed reality(XR), refers to technologies that augment and/or replace the physical world with a digital or simulated world, thereby creating a sense of immersion, e.g. Virtual and Augmented Reality

What are immersive technologies? Virtual and Augmented Reality

Cardiothoracic

Transplant

Neurosurgery

Orthopedics

Imaging Orders Radiology

Clinical imaging needs

Image acquisition

Ochsner Medical 3D(Om3d) lab

Image segmentation and STL generation

3D Printer*

Om3d VR

Om3d AR

3D rendering process

3D Print

VR

AR

*3D printers used include: Form 2 SLA, Projet 660, ProJet MJP 2500, Prusa i3,

3D Models

Figure 1. Clinical workflow for creating 3D printed, Virtual and/or Augmented Reality(VR/AR) models. Clinicians order scans for their patients per standard of care, not for the purpose of 3D modeling. Imaging is acquired in the Department of Radiology and saved in DICOM format. The clinician consults the medical 3D group if they wish to create a 3D model of their patient’s anatomy. The DICOM files are obtained from Radiology and processed to generate STL files. The STL file can be used to created printed, VR, or AR models. Om3d refers to Ochsner’s medical 3d lab.

Om3d lab Workflow

DICOM

Neuroscience applications

Vascular

Epilepsy

Neuro-Oncology

Spine: 3DP

Spine: VR and AR in the OR

Lumbar Puncture TrainingImplementation of Augmented Reality In Medical

Student Lumbar Puncture Simulation TrainingJeffrey D. Coote1, Bhumit Desai1, Jack Mcgee2, Charles Kantrow, MD1,

Gerald Denton, MD1, Korak Sarkar, MD3

1. The University of Queensland-Ochsner Clinical School, New Orleans, LA 2. Ochsner m3D Lab, New Orleans, LA 3. Department of Neurology, Ochsner Health System, New Orleans, LA

Introduction

Objectives

Methods Results

References

Augmented reality (AR) is a mixed reality format in which most of the environment is real with supplementation of virtual objects, aka holograms. AR’s unique ability to provide realistic haptic feedback and real-time performance evaluation has resulted in a surge in popularity worldwide, with applications ranging from video games to industrial training.1 It has only recently become technically reasonable to utilize AR in simulation-based medical education and procedural training.2-4 AR is being studied in training for procedures like laparoscopic surgeries, lumbar punctures, facet join injections, and pedicle screw placements.5-7

A lumbar puncture (spinal tap) is a medical procedure that requires the insertion of a needle into the lower back between two vertebrae to remove a sample of cerebrospinal fluid. This is the fluid that surrounds the brain and spinal cord and can be analyzed for a variety of diseases, like meningitis, multiple sclerosis, and certain kinds of intracranial hemorrhages.

We have established feasibility of creating a Microsoft Hololens based AR application that can project and register a spine hologram on to a LP trainer in the Ochsner Simulation Center. An IRB protocol has been submitted and pending approval. A fellow has been funded to execute the study. Study recruitment is expected to begin before the end of the year.

1. Lyon, Jeff. "Augmented reality goes bedside." Jama 317.2 (2017): 127-127. 2. Barsom, E. Z., M. Graafland, and M. P. Schijven. "Systematic review on the effectiveness of augmented reality applications in medical training." Surgical endoscopy 30.10 (2016): 4174-4183. 3. Herron, Jennifer. "Augmented reality in medical education and training." Journal of Electronic Resources in Medical Libraries13.2 (2016): 51-55. 4. Zhu, Egui, et al. "Augmented reality in healthcare education: an integrative review." PeerJ 2 (2014): e469. 5. Agten, Christoph A., et al. "Augmented Reality–Guided L u m b a r F a c e t J o i n t I n j e c t i o n s . " I n v e s t i g a t i v e radiology (2018). 6. Pfandler, Michael, et al. "Virtual reality-based simulators for spine surgery: a systematic review." The Spine Journal 17.9 (2017): 1352-1363. 7. White, Marjorie L., et al. "Transfer of simulated lumbar puncture training to the clinical setting." Pediatric emergency care 28.10 (2012): 1009-1012.

Aim 1: To determine the feasibility of implementing augmented reality (AR) in medical student training for lumbar puncture.

• Is the AR interface easy to navigate? • Does AR guidance alter the time required to

perform a lumbar puncture? • Can an augmented reality model be co-

registered to physical landmarks?

We have developed a Microsoft Hololens AR application for use in the Ochsner Clinical Simulation and Patient Safety Center on a lumbar puncture/epidural trainer. Third and fourth year medical student volunteers will take a pretest and then be given training on lumbar puncture procedures and an orientation to AR equipment. After this, students will be randomized to the intervention group (using AR to perform an LP) or a control group (usual LP without AR). Data will be collected on the number of sticks required for a successful lumbar puncture, location of sticks in relation to the ideal tap location, and time required for students performing AR-guided lumbar punctures and those performing lumbar punctures using the current standard of care. Additionally a qualitative analysis consisting of open-ended questions will be used to compare student experience using the AR interface versus those using the current standard of care.

Aim 2: To examine the efficacy of AR-guided lumbar punctures as compared to the current standard of care.

• How does AR guidance affect the student experience in lumbar puncture training?

• How does AR guidance affect student confidence in their ability to perform a lumbar puncture?

• How does AR guidance affect student ability to perform a lumbar puncture?

ConclusionsWe have developed the technical abilities and applications for AR guided LP training. The goal of this study is to determine the feasibility of utilizing AR technology in medical simulation as well to investigate its efficacy in improving procedural training. This initial study can serve as the basis to further explore current limitations in translating and increasing adoption of AR in healthcare, like user design and image co-registration.

Experiential VR in HealthCare

Next Steps: Neuroscience m3d Clinic • •Co-Lab

Implementation •Co-Immersion •Quantify Efficacy

Medical Virtualist

Follow us @OchsnerM3D

Publications• Seal JB, Wright M, Serrano MS, Bergeron J, Vasquez R, Gimenez J, Loss G, Sarkar K. 3-D Printed Anatomical Model of

Hepatoblastoma in Pediatric Patient. 2017 Studies in Pediatric Liver Transplant(SPLIT) Meeting. Submitted

• Planning Complex Spine Surgeries Using 3D Printed and Virtual Reality Models: A Case Series. Nguyen T, Coote J, Tholen K, Celestre P, Sarkar K,. Ochsner Neuroscience Research Symposium, March 2018

• Virtual Reality Based on 3D Modeling of Intramedullary Subependymoma Metastasis and Associated Neural Tracts. Stewart C, Nguyen T, Kalyvas J, Sarkar K. Ochsner Neuroscience Research Symposium, March 2018

• 3-D Printed Patient Models in Complex Pediatric Spinal Surgery Jeffrey D. Coote; Theresa Nguyen, MBBS; Kaitlyn Tholen; Caleb Stewart; Elizabeth Verter, MBBS; Paul Celestre, MD; Korak Sarkar, MD Ochsner m3D Lab, New Orleans, LA. Submitted Ochsner Journal

• Use of Virtual Reality to Illustrate an Unusual Solution to Repair a Hepatic Aneurysm. Hayek G, Mihindu E, Seal J, Stewart C, Sarkar K, Sternberg WC. Society for Vascular Surgery 2018 Annual Meeting

• "Employing Virtual and Augmented Reality for Intracranial Aneurysm Treatment and Resident Education" by Caleb Stewart, Casey Spinelli, Henry Sanicola, Frank Berry, Roger Smith, Edison Valle-Giler, Richard Milani and Korak Sarkar, submitted Journal of Neurosurgery. Submitted 9/1/18

• Predicting aortic endograft spatial position and shape pre-operatively using 3D and virtual reality technologies Davis Moon, MD, Caleb Stewart, BS, John McGee, BS, WC Sternbergh III, Richard Milani, MD, Korak Sarkar, MD, Hernan A. Bazan, MD. Southern Association of Vascular Surgeons, Submitted 9/4/18

• Image Processing Workflow for Virtual and Augmented Reality Platforms in Transplant Surgery . John Seal, Caleb Stewart, Jack McGee, Theresa Nguyen, Dennis Sonnier, Richard Milani,

George Loss, Korak Sarkar Winter conference ASTS 2018. submitted

• THE ABCS OF AR AND VR. By Daniel Ortiz, MD; Vivek Kalia, MD, MPH; Theresa Nguyen, BS; James Vogler, MD; José Morey, MD; Neil Lall, MD; Jan Fritz, MD; Falgun Chokshi, MD, MS; and Korak Sarkar, MD. Radiology Today, submitted September 2018 (article link)

• Preplanning Resection of a Glioma with DTI Tractography and Virtual Reality. Caleb Stewart BS, Henry Sanicola BS, Stephen Fletcher MS, Marcus Ware MD, Korak Sarkar MD. Ochsner Neuroscience Philanthropy symposium.

Delta = Growth Area: non-anatomical 3DP applications

Anatomical prints

Total Prints