big data visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/bbd-week-2018...biomedical big data...

45
Alexandros Sigaras | @AlexSigaras Research Associate in Computational Biomedicine Weill Cornell Medicine Sophia Roshal | @RoshalSophia Software Engineer Intern Weill Cornell Medicine Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual Reality & Augmented Reality

Upload: others

Post on 28-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Alexandros Sigaras | @AlexSigarasResearch Associate in Computational BiomedicineWeill Cornell Medicine

Sophia Roshal | @RoshalSophiaSoftware Engineer InternWeill Cornell Medicine

Biomedical Big Data Week We’ll Cornell Medicine

6.20.2018

Big Data VisualizationMixed Reality, Virtual Reality & Augmented Reality

Page 2: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Source: EMC & IDC“The Digital Universe. Driving Data Growth in Healthcare”

https://www.emc.com/analyst-report/digital-universe-healthcare-vertical-report-ar.pdf

Healthcare & Big DataToday

Page 3: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Healthcare & Big Data2025 Predictions

Astronomy Twitter Youtube Genomics

Acquisition 25zetta-bytes/year

0.5–15billion

tweets/year

500–900 million

hours/year

1zetta-

bases/year

Storage 1EB/year

1–17PB/year

1–2EB/year

2–40EB/year

Source: NIH“Big Data: Astronomical or Genomical Zachary Stephens et al.”

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4494865/

Page 4: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Source: NIHhttps://www.genome.gov/sequencingcosts/

Cost per Genome

Page 5: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 6: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

“My hope is that this becomes the foundation,the architecture,

whereby in 10 years fromnow we can look back

and say that we haverevolutionized

medicine.”

Page 7: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Precision Medicine InitiativeBudget

$130MF Y 1 6 E N A C T E D F Y 1 7 P R E S I D E N T ’ S

R E Q U E S T

$230M

Page 8: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Precision Medicine InitiativeRegional Medical Centers across USA

• Able to enroll diverse patient populations

• Strong electronic health record capacity

• Geographic spread

• Capacity to enroll 35,000 a year

Northwestern UniversityInstitute for Public Health and MedicineEvanston, IL

Universityof PittsburghClinical and Translational

Science Institute

Pittsburgh, PA

Universityof ArizonaCollege of Medicine

& Public Health

Tucson, AZ

Columbia/Cornell/NYP/Harlem HospitalInstitute for Genomic

Medicine

New York, NY

Universityof CaliforniaSan DiegoDepartment of Biomedical InformaticsLa Jolla, CA

Massachusetts General HospitalGenetics & Genomics Unit

Boston, MA

Henry FordHealth SystemPublic Health Sciences

Detroit, MI

GeisingerHealth SystemGenomic Medicine Institute

Danville, PA

Page 9: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Precision Medicine InitiativeNYC

Page 10: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Cancer MoonshotState of the Union | January 2016

Page 11: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

21st Century Cures Act SignedDecember 13th 2016

https://www.whitehouse.gov/CancerMoonshot

Page 12: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Mission

Page 13: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Weill Cornell MedicineMission

Care

TeachDiscover

Page 14: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

The New style of tailored treatment

http://meyercancer.weill.cornell.edu/news/2015-01-22/new-style-tailored-treatment

Irene Price | 76y Female

• 2009 | Diagnosed with Bladder Cancer; initially followed traditional treatment showing signs of success

• Relapse | High-grade urothelial carcinoma in bladder and several lymph nodes

• Cystectomy (Bladder removal), sample sent for Genetic Analysis (WES)

• Underwent Radiation Therapy, Genetic Analysis Results showed HER2 Amplification (mutation common in breast cancer)

• Relapse | Signs of metastasis to the liver• Chemotherapy targeted treatment | 8 cycles with

Taxol and Herceptin• Disease free since Nov 2014

Medical History

Page 15: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

NGS Test Workflow

15

Orders Test Receives Report

Patient

Referring Physician

Page 16: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

2000 Cores2PB Storage

Institute for Computational Biomedicine

Today

Page 17: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 18: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 19: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 21: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 22: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

https://youtu.be/_Fly4U9eFRY

Page 23: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 24: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

*credits: Lucas Films

Star WarsEpisode V – The Empire Strikes Back (1980)

Page 25: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

*credits: Microsoft Researchhttps://www.microsoft.com/en-us/research/project/holoportation-3/

Video: https://youtu.be/7d59O6cfaM0

Microsoft HololensHoloportation

Page 26: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

*credits: Case Western Reserve University

Microsoft HololensHealthcare

Page 27: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

*credits: Case Western Reserve University

Microsoft HololensHealthcare

Page 28: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Reality-VirtualityContinuum

Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). "Augmented Reality: A class of displays on the reality-virtuality continuum"

PhysicalReality (PR)

VirtualReality (VR)

AugmentedReality (AR)

AugmentedVirtuality (AV)

Mixed Reality (MR)

Page 29: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Gaze VoiceGesture

Ways to Interact

Page 30: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 31: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Visualizing Protein Structures

Page 32: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 33: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Visualizing Networks

Source: http://www.creativeapplications.net/linux/gephi-mac-windows-linux/

Page 34: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Visualizing Cancer Drug Networks

Data: Kaitlyn Gayvert

Page 35: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Bringing Networksinto Reality

Page 36: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

https://youtu.be/Yu3Vxy6lQwg

Page 37: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Tumor BoardUsing Hololens across the world

Page 38: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual
Page 39: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

https://youtu.be/O99CMi2ahD0

Page 40: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Demos

Page 41: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

Minority Report

Release Date: June 21 2002

Page 42: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

John Underkoffler

https://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture

Page 43: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

HealthcareLooking into the future

Page 44: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

https://youtu.be/ZhAzqaV-lQk

Page 45: Big Data Visualizationbigdata.citytech.cuny.edu/bd_media/2018/06/BBD-Week-2018...Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual

45

eipm.weill.cornell.edu