epad: a p latformfor motivation for developing epad s ‐b c i...
TRANSCRIPT
Copyright Daniel Rubin 2016 1
EPAD: A PLATFORM FORSTANDARDS‐BASED COLLABORATIVE
IMAGE MANAGEMENT ANDANALYSIS
Daniel Rubin MD, MSMete Akdogan, PhD
Cavit Altindag Ozge YurtseverEmel Alkim, PhD
Department of Biomedical Data Science, Radiology, and Medicine (Biomedical Informatics)
Stanford University
Motivation for developing ePAD
Fragmentation of resources for imaging research◦ Image archives◦ Image viewing/annotation tools◦ Image analysis tools
Lack of tools to integrate image‐ and non‐image data ◦ Data related to images (“annotations”) recorded
separately from images◦ Image labels /ROIs recorded in disparate formats
Opportunity to create a platform ◦ Image archive, project management◦ User‐friendly image viewing/annotation ◦ Image analysis, pipelines cross‐experiment analysis◦ Modular, extensible (plugins)
Copyright © Daniel Rubin 2019
The electronic Physician Annotation Device (ePAD) ePAD is a web based system for managing, viewing,
annotating, and analyzing images Built originally to manage human image data,
recently extended to support animal studies Organizes images in projects◦ Cohorts in a lab◦ Can serve as an image registry for multiple labs
Image viewing/annotation◦ Akin to Osirix but using a web browser with no software
installation◦ Tools to automate annotation (segmentation)◦ Adopts standards for interoperability
Incorporates image processing plugins to integrate analyses as part of image management worfklow
CentOSVirtual Machine
ePAD Viewer & Semantic Annotator
ePADWeb Client
AIM XML (MongoDB)
ePADWeb
Services
Client‐side Plugins
MongoDB
ePAD system architectureServer‐side Plugins
Image processing(segmentation,
quantitation)
Analysis and applications
ePAD GUI AIM‐compliant annotation; supports AIM templates Plugins for quantifying lesion features
Template
ROI Valu
es
Rubin, Willrett, O'Connor, Hage, Kurtz, Moreira, Translational Oncology 7(1):23-35, 2014http://epad.stanford.edu
Quantitative image features
Annotations linked to images
Qualitative image features
Annotation and Image Markup (AIM)
• Emerging standard for semantic annotation of images– Imaging observations
– Anatomy
– ROIs
– Etc
• Makes the image contents explicit, computer‐accessible
• Developed by National Cancer Imaging Program at NCI
Rubin DL, et. al: Medical Imaging on the Semantic Web: Annotation and Image Markup, AAAI 2008.
Copyright Daniel Rubin 2016 2
AIM captures both semantic and quantitative features
Finding: massLocation: Lung, LULLength: 2.3cmWidth: 1.2cmMargins: spiculatedCavitary: YCalcified: NSpatial relationships:Abuts pleural surfaceinvades aortaTexture: {T1, T2, T3,…}Shape: {S1, S2, S3,…}
Controlled terminology:
CAVITARY MASS
Copyright © Daniel Rubin 2015
ePAD Template Builder
Copyright © Daniel Rubin 2019
ePAD in MIPS: Cohort/study management
Copyright © Daniel Rubin 2019
Projects
Studies
Series
ePAD in MIPS: Descriptive ROIs (imported from Osirix)
Annotation Tools: Smart Paintbrush Annotation Tools: ROI interpolation
Copyright Daniel Rubin 2016 3
Uploading Images/Annotations
Images
◦ Upload a zip in ePAD web viewer
◦ DICOM push
◦ Drop in server folder
Annotations
◦ Can import annotations from Osirix
◦ Can import from DICOM‐SR
◦ Can import AIM
Can do custom imports via ePAD API
Copyright © Daniel Rubin 2019
Exporting Annotations
ePAD Plugins
Image pre‐processing
◦ Lesion segmentation
Pre‐defined radiomics features (standard and novel quantitative image biomarkers)
◦ Linear dimension, volume, SUV, statistics
◦ JJvector > 400 features
◦ QIFE > 500 features through QIFP (QIIR QRR015)
Integration with QIFP for training machine learning applications
Copyright © Daniel Rubin 2019
Example plugin: Automated segmentation
LesionSeg 2D: • Single slice• Input: A seed polygon• Output: Lesion contour (as coordinates
in AIM/DICOM-SR)
LesionSeg 3D: • Input: A seed longaxis or polygon and
start and end slice numbers• Output: Lesion volume (as DICOM
Segmentation Object-DSO)
ePAD enables collecting radiomics features in radiology workflow
Copyright © Daniel Rubin 2018
You can extract many different kinds of features to correlate with outcomes
Example plugin: ePAD plugin to compute novel imaging biomarker ADLA: The Attenuation Distribution across the Long Axis
Automatically computed from linear annotation of cancer lesion
Copyright Daniel Rubin 2016 4
Computing treatment response directly from images in ePAD
Application: Assessment of treatment effectiveness• Automated waterfall plot (summary of best response in all patients in a cohort)
• User selects the imaging biomarker to use for assessing response (e.g., “RECIST” vs. “ADLA”)
• Generated by querying image annotations
Example study using ePAD: Predicting recurrence in lung cancer
Depeursinge, et. al, and Rubin, Med Phys 42(4):2054-63, 2015
Rieszwavelets
for texture
AIM enables AI development by mining image data
Copyright © Daniel Rubin 2018
e.g., discover better imaging biomarkers of treatment response..
Evaluation ‐ Comparison of Waterfall Plots on Evaluation Dataset ePAD compares different imaging biomarkers by
showing differences in waterfall plots generated by each imaging biomarker
Using linear measure (RECIST)
Using SD of pixel intensity
`Image
feature matrix
`
Survival,response to therapy,
etc.AI/ML modelAI/ML model
Imageviewing and annotation
Imagefeature extraction
ePAD is integrated into QIFP
http://qifp/stanford.edu
Copyright Daniel Rubin 2016 5
http://qifp.stanford.edu
From ePAD into radiomics pipelines ePAD can Interface to Other Resources
Kheops
XNAT
NCI TCIA
NCI Imaging Data Commone
ACR AI‐Lab
Copyright © Daniel Rubin 2019
Future of ePAD: Federated AI Learning
There are barriers to data sharing
Instead of bringing the data to the algorithm, bring the algorithm to the data
ePAD located in different labs can work collaboratively without sharing data
J Am Med Inform Assoc 25(8):945-954, 2018
Conclusion
ePAD fills gaps in current tools for collection annotated image data for radiomics and AI
• Freely available, web based, standards‐based
• Extensible plugins for pre‐processing images
• Automated computation of radiomics image features as part of routine image viewing workflow
• Integration with QIFP for developing AI models
• Geared to fostering an ecosystem of collaborative acquisition of image annotations and image analysis
Acknowledgements
Funding support
NCI QIN grants U01CA142555,1U01CA190214, 1U01CA187947
Copyright © Daniel Rubin 2019
Thank you.
Contact info:[email protected]