Open Source Imaging ToolkitsOpen Source Imaging Toolkits
Rick AvilaRick Avila
Director of Medical ApplicationsDirector of Medical ApplicationsKitware, Inc.Kitware, Inc.
[email protected]@kitware.com
April 27, 2006 KitwareKitware
KitwareKitware
The Software Gap The Software Gap
Digital healthcare networks Digital healthcare networks
are rolling out across the are rolling out across the
planet…planet…
Digital acquisition devices are Digital acquisition devices are
exceeding observer ability…exceeding observer ability…
Comprehensive disease Comprehensive disease
management tools are needed management tools are needed
within a PACS workflow…within a PACS workflow…
BrainBrainLungLung
ColonColonCardiacCardiac
CT Lung Scan Slice ThicknessCT Lung Scan Slice Thickness
0.00
2.00
4.00
6.00
8.00
10.00
12.00
1996 1998 2001 2002 2004
KitwareKitware
Improving Global Development EfficiencyImproving Global Development Efficiency
ClinicalStudy Data
Publication
Closed systems yield slow advancementClosed systems yield slow advancement
BaselineData
Need rapid acquire / develop / validate / review cyclesNeed rapid acquire / develop / validate / review cycles
Validation/Analysis
Env/Algs/Apps
Open ScienceOpen Science
1.1. Open DataOpen Data
2.2. Open SourceOpen Source
3.3. Open PublicationOpen Publication
YearsYears
KitwareKitware
Leveraging Global Resources Leveraging Global Resources
ExpertsExperts• ClinicalClinical
• ComputingComputing
Imaging ToolkitsImaging Toolkits• Software ArchitectureSoftware Architecture
• AlgorithmsAlgorithms
• DocumentationDocumentation
• Validation TestsValidation Tests
Development ToolsDevelopment Tools• Build UtilitiesBuild Utilities
• Quality DashboardQuality Dashboard
Computing ResourcesComputing Resources• Hardware & SoftwareHardware & Software
DatasetsDatasets• All Diseases/ModalitiesAll Diseases/Modalities
KitwareKitware
Open Source Imaging ToolkitsOpen Source Imaging Toolkits
Visualization Toolkit (VTK)Visualization Toolkit (VTK) Insight Toolkit (ITK) Insight Toolkit (ITK)
Rendering and DisplayRendering and Display Image Analysis Image Analysis
High Quality, State-of-the-art Toolkits for Research and Product DevelopmentHigh Quality, State-of-the-art Toolkits for Research and Product Development
KitwareKitware
The Visualization Toolkit The Visualization Toolkit (www.vtk.org)(www.vtk.org)
– C++ class library for visualizationC++ class library for visualization
– Designed using object-oriented principlesDesigned using object-oriented principles
– Portable across Unix, Linux, Windows, MacOSXPortable across Unix, Linux, Windows, MacOSX
– Automated Java, TCL, Python bindingsAutomated Java, TCL, Python bindings
– Automated nightly testingAutomated nightly testing
– Supports 3D/2D graphics, visualization, image Supports 3D/2D graphics, visualization, image
processing, and volume renderingprocessing, and volume rendering
– BSD-style licenseBSD-style license
– C++ class library for visualizationC++ class library for visualization
– Designed using object-oriented principlesDesigned using object-oriented principles
– Portable across Unix, Linux, Windows, MacOSXPortable across Unix, Linux, Windows, MacOSX
– Automated Java, TCL, Python bindingsAutomated Java, TCL, Python bindings
– Automated nightly testingAutomated nightly testing
– Supports 3D/2D graphics, visualization, image Supports 3D/2D graphics, visualization, image
processing, and volume renderingprocessing, and volume rendering
– BSD-style licenseBSD-style license
KitwareKitware
VTKVTK
KitwareKitware
VTK by the NumbersVTK by the Numbers
CommunityCommunity6565 core developerscore developers26552655 person mailing listperson mailing list
Source CodeSource Code1,2981,298 C++ classesC++ classes737,359737,359 lines of C++ codelines of C++ code
Daily Testing SnapshotDaily Testing Snapshot7272 machine configurations (hw machine configurations (hw
+ sw)+ sw)7474 build and test sessions/daybuild and test sessions/day897897 regression tests/buildregression tests/build
VTK C++ FilesVTK C++ Files
16751822
2013
2443
0
500
1000
1500
2000
2500
3000
2003 2004 2005 2006
KitwareKitware
t0
t 8
t
I(t0,) = Q(t) e dt (1)
t = t0
t 8 - a(t’) + sc(t’)dt’t’ = t0
t
’ I(t, ’) d ’ (2)Q(t) = q(t) + sc(
4
’ (In Scattering)
(Emission)
(Absorption)
(Out Scattering)
t’
Volume RenderingVolume Rendering
KitwareKitware
Volume Rendering ArchitectureVolume Rendering Architecture
RendererProp
Collection
Volume
VolumeMapper
VolumeProperty
ImageData
......
Property
Input Mapper
VolumeRayCastMapper
VolumeRayCastFunction
GradientEstimator
GradientEncoder
GradientShader
KitwareKitware
Volume Rendering MethodsVolume Rendering Methods
1) Ray Casting1) Ray Casting
• Floating PointFloating Point
• Composite w/shadeComposite w/shade
• Gradient Magnitude OpacityGradient Magnitude Opacity
• MIPMIP
• IsoSurfaceIsoSurface
• Fixed PointFixed Point
2) Shear Warp2) Shear Warp
3) Texture Mapping3) Texture Mapping
• 2D2D
• 3D3D
4) VolumePro Hardware (past)4) VolumePro Hardware (past)
……with support for LOD control, multi-threading, …with support for LOD control, multi-threading, …
KitwareKitware
VTK Dashboard using DARTVTK Dashboard using DART
KitwareKitware
CMake CMake (www.cmake.org)(www.cmake.org)
Recently AdoptedRecently Adopted
By Linux KDEBy Linux KDE
KitwareKitware
ITK: The Insight Toolkit ITK: The Insight Toolkit (www.itk.org)(www.itk.org)
• Toolkit started in 2000Toolkit started in 2000• Open source segmentation & registration algorithmsOpen source segmentation & registration algorithms
$13M directly invested to date$13M directly invested to date
KitwareKitware
ITK DevelopersITK Developers
KitwareKitware
• Image ProcessingImage Processing
• SegmentationSegmentation
• RegistrationRegistration
• No Graphical User Interface (GUI)No Graphical User Interface (GUI)
• No VisualizationNo Visualization
ITKITK
KitwareKitware
ITK by the NumbersITK by the Numbers
CommunityCommunity5858 core developerscore developers11381138 person mailing listperson mailing list
Source CodeSource Code11751175 C++ classesC++ classes458,770458,770 lines of C++ codelines of C++ code
Daily Testing SnapshotDaily Testing Snapshot8484 machine configurations (hw + sw)machine configurations (hw + sw)107107 build and test sessions/daybuild and test sessions/day11951195 regression tests/buildregression tests/build
KitwareKitware
ImageRegistrationFramework
MultiResolution
RegistrationFramework
PDEBased
Registration
FEMBased
Registration
Components
Registration in ITKRegistration in ITK
FixedImage
MovingImage
Metric
Transform
Interpolator
Optimizer
Registration Method
Fixed Image Moving Image Registered Moving Image
KitwareKitware
Code
InputData
JournalRepository
WebSite
ResultsData
Author
BuildMachines
PDF doc
Insight JournalInsight Journal
KitwareKitware
Use of VTK / ITKUse of VTK / ITK
• ResearchResearch– Algorithm Development & ComparisonAlgorithm Development & Comparison– PrototypesPrototypes– Higher-Level Application ToolkitsHigher-Level Application Toolkits
• ProductsProducts– ScannersScanners– Analysis WorkstationsAnalysis Workstations– PACSPACS– Interventional DevicesInterventional Devices– Research InterfacesResearch Interfaces
KitwareKitware
Open Source ImpactOpen Source Impact
Hans Johnson Lab, University of IowaHans Johnson Lab, University of Iowa
– Lab uses ITK and VTKLab uses ITK and VTK
– Student working on atlas based segmentation methodStudent working on atlas based segmentation method
– Danielsson signed distance filter is a bottleneckDanielsson signed distance filter is a bottleneck
Insight Journal 2/20/2006Insight Journal 2/20/2006
– 9:26am:9:26am: IJ mail arrives about EDT filter (PAMI IJ mail arrives about EDT filter (PAMI
‘03)‘03)
– 12:30am:12:30am: Print and read documentationPrint and read documentation
– 1:00pm:1:00pm: Embed and run EDT codeEmbed and run EDT code
– 1:03pm:1:03pm: !#%?! It runs in only 3 minutes? !#%?! It runs in only 3 minutes?
(10x)(10x)
– 1:10pm:1:10pm: Compare results & do dance of joyCompare results & do dance of joy
– 4:00pm:4:00pm: Submit Insight Journal reviewSubmit Insight Journal review
KitwareKitware
NAMIC: SLICERNAMIC: SLICER
KitwareKitware
Mac: OSIRIXMac: OSIRIX
Over 8000 Users ~250 Downloads/DayOver 8000 Users ~250 Downloads/Day
KitwareKitware
BrainLab Vector Vision LinkBrainLab Vector Vision Link
Provides a direct Provides a direct interface with the interface with the VectorVision IGS VectorVision IGS navigation system to navigation system to design and implement design and implement new features and new features and functionalityfunctionality
KitwareKitware
Product Use: MeVisLabProduct Use: MeVisLab
KitwareKitware
VolView Clinical (Work in Progress)VolView Clinical (Work in Progress)
Not FDA Approved for Clinical UseNot FDA Approved for Clinical Use
KitwareKitware
Open Source Toolkit GrowthOpen Source Toolkit Growth
7,359,0007,359,000 hits/month on public.kitware.com hits/month on public.kitware.com
KitwareKitware
SummarySummary
Open Source Imaging ToolkitsOpen Source Imaging Toolkits
– Efficiently leverage global skills and resourcesEfficiently leverage global skills and resources
– Represents a large and growing resourceRepresents a large and growing resource
– Accelerates healthcare research and product Accelerates healthcare research and product developmentdevelopment
– Will drive down the cost of healthcareWill drive down the cost of healthcare
KitwareKitware
Thank YouThank You
KitwareKitware
Doxygen Manual PagesDoxygen Manual Pages
KitwareKitware
Insight Software Consortium (ISC)Insight Software Consortium (ISC)
A non-profit that provides open-science environments A non-profit that provides open-science environments for developers and users involved in medical image for developers and users involved in medical image
analysis researchanalysis research
Users’ and Developers’ Point of ViewUsers’ and Developers’ Point of View ISC ProvidesISC ProvidesDoes the software work?Does the software work? DashboardsDashboards
Time and interaction to download and install?Time and interaction to download and install? PackagingPackaging
Documentation online and available for download?Documentation online and available for download? Documentation StandardDocumentation Standard
Consistent code quality?Consistent code quality? Coding StandardCoding Standard
Active support for individual packages, e.g., email lists?Active support for individual packages, e.g., email lists? Support StandardSupport Standard
How can I learn more about the software?How can I learn more about the software? ISC hosts tutorials: ISC hosts tutorials: SPIE…SPIE…
How can I contribute my research and dataHow can I contribute my research and data The Insight JournalThe Insight Journal
Open-source changes too quickly! Open-source changes too quickly! Backward compatible policyBackward compatible policy
What open-science sites are available?What open-science sites are available? The ISC website, The IJThe ISC website, The IJ
KitwareKitware
• 37 Submissions37 Submissions
• 90 Reviews 90 Reviews
• by 31 Reviewersby 31 Reviewers
• 261 Registered Users261 Registered Users
• Anonymous reviews allowed, but none were Anonymous reviews allowed, but none were
submittedsubmitted
• Reviewers scored Reviewers scored
– SensitivitySensitivity
– HelpfulnessHelpfulness
Workshop ResponseWorkshop Response
KitwareKitware
Open Data: NIH RIDER DatabaseOpen Data: NIH RIDER Database
KitwareKitware
vtkVolumeMappervtkVolumeMapper
Code is ContinuouslyCode is Continuously
Explored and Tested Explored and Tested
by a Large Number ofby a Large Number of
Users and Experts Users and Experts
KitwareKitware
Image Guided Surgery ToolkitImage Guided Surgery Toolkit
KitwareKitware