big data visualization - city university of new...

45
Alexandros Sigaras | @AlexSigaras Research Associate in Computational Biomedicine Weill Cornell Medicine Sophia Roshal | @RoshalSophia Software Engineer Intern Weill Cornell Medicine Biomedical Big Data Week We’ll Cornell Medicine 6.20.2018 Big Data Visualization Mixed Reality, Virtual Reality & Augmented Reality

Upload: others

Post on 26-May-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Alexandros Sigaras | @AlexSigarasResearch Associate in Computational BiomedicineWeill Cornell Medicine

Sophia Roshal | @RoshalSophiaSoftware Engineer InternWeill Cornell Medicine

Biomedical Big Data Week We’ll Cornell Medicine

6.20.2018

Big Data VisualizationMixed Reality, Virtual Reality & Augmented Reality

Page 2: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Source: EMC & IDC“The Digital Universe. Driving Data Growth in Healthcare”

https://www.emc.com/analyst-report/digital-universe-healthcare-vertical-report-ar.pdf

Healthcare & Big DataToday

Page 3: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Healthcare & Big Data2025 Predictions

Astronomy Twitter Youtube Genomics

Acquisition 25zetta-bytes/year

0.5–15billion

tweets/year

500–900 million

hours/year

1zetta-

bases/year

Storage 1EB/year

1–17PB/year

1–2EB/year

2–40EB/year

Source: NIH“Big Data: Astronomical or Genomical Zachary Stephens et al.”

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4494865/

Page 4: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Source: NIHhttps://www.genome.gov/sequencingcosts/

Cost per Genome

Page 5: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 6: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

“My hope is that this becomes the foundation,the architecture,

whereby in 10 years fromnow we can look back

and say that we haverevolutionized

medicine.”

Page 7: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Precision Medicine InitiativeBudget

$130MF Y 1 6 E N A C T E D F Y 1 7 P R E S I D E N T ’ S

R E Q U E S T

$230M

Page 8: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Precision Medicine InitiativeRegional Medical Centers across USA

• Able to enroll diverse patient populations

• Strong electronic health record capacity

• Geographic spread

• Capacity to enroll 35,000 a year

Northwestern UniversityInstitute for Public Health and MedicineEvanston, IL

Universityof PittsburghClinical and Translational

Science Institute

Pittsburgh, PA

Universityof ArizonaCollege of Medicine

& Public Health

Tucson, AZ

Columbia/Cornell/NYP/Harlem HospitalInstitute for Genomic

Medicine

New York, NY

Universityof CaliforniaSan DiegoDepartment of Biomedical InformaticsLa Jolla, CA

Massachusetts General HospitalGenetics & Genomics Unit

Boston, MA

Henry FordHealth SystemPublic Health Sciences

Detroit, MI

GeisingerHealth SystemGenomic Medicine Institute

Danville, PA

Page 9: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Precision Medicine InitiativeNYC

Page 10: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Cancer MoonshotState of the Union | January 2016

Page 11: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

21st Century Cures Act SignedDecember 13th 2016

https://www.whitehouse.gov/CancerMoonshot

Page 12: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Mission

Page 13: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Weill Cornell MedicineMission

Care

TeachDiscover

Page 14: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

The New style of tailored treatment

http://meyercancer.weill.cornell.edu/news/2015-01-22/new-style-tailored-treatment

Irene Price | 76y Female

• 2009 | Diagnosed with Bladder Cancer; initially followed traditional treatment showing signs of success

• Relapse | High-grade urothelial carcinoma in bladder and several lymph nodes

• Cystectomy (Bladder removal), sample sent for Genetic Analysis (WES)

• Underwent Radiation Therapy, Genetic Analysis Results showed HER2 Amplification (mutation common in breast cancer)

• Relapse | Signs of metastasis to the liver• Chemotherapy targeted treatment | 8 cycles with

Taxol and Herceptin• Disease free since Nov 2014

Medical History

Page 15: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

NGS Test Workflow

15

Orders Test Receives Report

Patient

Referring Physician

Page 16: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

2000 Cores2PB Storage

Institute for Computational Biomedicine

Today

Page 17: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 18: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 19: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 21: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 22: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

https://youtu.be/_Fly4U9eFRY

Page 23: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 24: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

*credits: Lucas Films

Star WarsEpisode V – The Empire Strikes Back (1980)

Page 25: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

*credits: Microsoft Researchhttps://www.microsoft.com/en-us/research/project/holoportation-3/

Video: https://youtu.be/7d59O6cfaM0

Microsoft HololensHoloportation

Page 26: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

*credits: Case Western Reserve University

Microsoft HololensHealthcare

Page 27: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

*credits: Case Western Reserve University

Microsoft HololensHealthcare

Page 28: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Reality-VirtualityContinuum

Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). "Augmented Reality: A class of displays on the reality-virtuality continuum"

PhysicalReality (PR)

VirtualReality (VR)

AugmentedReality (AR)

AugmentedVirtuality (AV)

Mixed Reality (MR)

Page 29: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Gaze VoiceGesture

Ways to Interact

Page 30: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 31: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Visualizing Protein Structures

Page 32: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 33: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Visualizing Networks

Source: http://www.creativeapplications.net/linux/gephi-mac-windows-linux/

Page 34: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Visualizing Cancer Drug Networks

Data: Kaitlyn Gayvert

Page 35: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Bringing Networksinto Reality

Page 36: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

https://youtu.be/Yu3Vxy6lQwg

Page 37: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Tumor BoardUsing Hololens across the world

Page 38: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality
Page 39: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

https://youtu.be/O99CMi2ahD0

Page 40: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Demos

Page 41: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

Minority Report

Release Date: June 21 2002

Page 42: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

John Underkoffler

https://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture

Page 43: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

HealthcareLooking into the future

Page 44: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

https://youtu.be/ZhAzqaV-lQk

Page 45: Big Data Visualization - City University of New Yorkbigdata.citytech.cuny.edu/wp-content/uploads/2018/... · Big Data Visualization. Mixed Reality, Virtual Reality & Augmented Reality

45

eipm.weill.cornell.edu