virtual bronchoscopic approach for combining 3d ct and
TRANSCRIPT
![Page 1: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/1.jpg)
Virtual Bronchoscopic approach for combining 3D CT and
endoscopic video
Anthony J.Anthony J. SherbondySherbondy,,11 AtillaAtilla P.P. KiralyKiraly, , 11 Allen L. Austin,Allen L. Austin,11
James P. James P. HelfertyHelferty, , 11 ShuShu--Yen Wan,Yen Wan,11 Janice ZJanice Z TurlingtonTurlington,,11 Tao Tao Yang,Yang,11 ChaoChao Zhang,Zhang,1 1 Eric A. Hoffman,Eric A. Hoffman,22 Geoffrey McLennan,Geoffrey McLennan,22
and and WilliamWilliam E. HigginsE. Higgins1,21,2
11Penn State University, University Park, PA 16802Penn State University, University Park, PA 1680222University of Iowa, Iowa City, IA 52246University of Iowa, Iowa City, IA 52246
SPIE Med. Imaging 2000SPIE Med. Imaging 2000, San Diego, CA, 12 February 2000, San Diego, CA, 12 February 2000
![Page 2: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/2.jpg)
Virtual Endoscopy:
New field: 1994.New field: 1994.Virtual Virtual bronchoscopy bronchoscopy (VB) (VB) ---- focus on chestfocus on chest
VB Needs: better reporting, path planning, VB Needs: better reporting, path planning, bridge to live bridge to live bronchoscopybronchoscopy
CTCT--only reporting: Summers98, Vining99only reporting: Summers98, Vining99
CTCT--BronchoscopyBronchoscopy linkage: Bricault98linkage: Bricault98
![Page 3: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/3.jpg)
Our Work:
Complete pulmonary assessment
3D CT Assessment BronchoscopySuite of graphics tools to augment visionaugment vision
Case StudyMultimedia report
Bronchoscopic guidance
![Page 4: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/4.jpg)
Remainder of Presentation:
I. TwoI. Two--stage CTstage CT--toto--VideobronchoscopyVideobronchoscopy paradigmparadigm
II. CTII. CT--only examples: Humansonly examples: Humans
III. CTIII. CT--video progress: video progress: bronchoscopybronchoscopy training devicetraining device
![Page 5: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/5.jpg)
Two-Stage CT-Video Paradigm
Stage 1 (CT Assessment)Stage 1 (CT Assessment)1. Create new Case Study.1. Create new Case Study.2. Invoke graphics tools.2. Invoke graphics tools.3. Identify key sites.3. Identify key sites.4. Compute guidance data.4. Compute guidance data.5. Build complete Case Study.5. Build complete Case Study.
Roadmap to bronchoscopy
![Page 6: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/6.jpg)
Stage 2 (Bronchoscopy)Stage 2 (Bronchoscopy)
1. Load Case Study.1. Load Case Study.
2. Invoke graphics tools.2. Invoke graphics tools.
3. Do virtual3. Do virtual--guided guided bronchoscopybronchoscopy..
4. Perform biopsy.4. Perform biopsy.
![Page 7: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/7.jpg)
Case Study:
•• Multimedia report Multimedia report 3D CT assessment 3D CT assessment
•• Supplemental plan Supplemental plan guideguide bronchoscopybronchoscopy
Build with Graphics/Processing Tools
![Page 8: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/8.jpg)
Elements of Case Study:
1. Data Sources1. Data Sources3D CT 3D CT ImageImageBronchoscopic Bronchoscopic VideoVideo
2. Data Abstractions2. Data AbstractionsRoot SiteRoot SiteKey SitesKey SitesPathsPathsTreeTree
3. Reporting Abstractions3. Reporting AbstractionsSnapshotsSnapshotsPlotsPlotsMoviesMoviesCase NotesCase NotesMeasurementsMeasurements
![Page 9: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/9.jpg)
Graphics Tools - 1
Case Study Manager3D Surface Tool
Virtualscope Projection Tool (Coronal)
![Page 10: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/10.jpg)
Graphics Tools - 2
Slicer Tool (MPR View, Coronal) Sliding Thin Slab Tool (Transverse)
Cross Section Tool (Horizontal)Plot Tool
![Page 11: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/11.jpg)
II. CT-only Examples
Example 1: Healthy FemaleExample 1: Healthy Female
3D EBCT image at 90% TLC
115 slices; 512x512 per slice
3.0mm slice thickness; 0.684mm [x-y] resol.
Emphasize automatic tree generation
![Page 12: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/12.jpg)
Stage 1 (CT Assessment)1. Create new Case Study.1. Create new Case Study.2. Invoke graphics tools.2. Invoke graphics tools.3. Identify Key Sites.3. Identify Key Sites.4. Compute guidance 4. Compute guidance
data.data.5. Build complete Case5. Build complete Case
Study.Study.
Case Study registry.Case Study registry.
![Page 13: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/13.jpg)
Stage 1 (CT Assessment)1. Create new 1. Create new Case Study.Case Study.2. Invoke Graphics tools.2. Invoke Graphics tools.3. Identify Key Sites.3. Identify Key Sites.4. Compute guidance 4. Compute guidance
data.data.5. Build complete 5. Build complete
Case Study.Case Study.
Set up data forSet up data forKey sites and airwayKey sites and airwaytree calculation.tree calculation.
![Page 14: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/14.jpg)
Stage 1 (CT Assessment)1. Create new 1. Create new Case Study.Case Study.2. Invoke graphics tools.2. Invoke graphics tools.3. Identify Key Sites.3. Identify Key Sites.4. Compute guidance4. Compute guidance
data.data.5. Build complete 5. Build complete
Case Study.Case Study.
Invoke automatedInvoke automatedrun to compute run to compute airway tree and pathsairway tree and pathsto defined key sites.to defined key sites.
![Page 15: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/15.jpg)
Stage 1 (CT Assessment)1. Create new 1. Create new Case Study.Case Study.2. Invoke graphics tools.2. Invoke graphics tools.3. Identify Key Sites.3. Identify Key Sites.4. Compute guidance4. Compute guidance
data.data.5. Build complete5. Build complete
Case Study.Case Study.
Rendered airwayRendered airwaytree, with extractedtree, with extractedpaths throughpaths throughairways.airways.
Coronal weighted-sumprojection showing extracted airway tree.
![Page 16: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/16.jpg)
Composite Case Study -- Part 1
![Page 17: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/17.jpg)
Composite Case Study - part 2
![Page 18: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/18.jpg)
Complete Case Study – Snapshots are saved.
Coronal weighted-sum projection with extracted tree.
Oblique cross-section.
Coronal Slicerview.
Sagittal Slicer view.
![Page 19: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/19.jpg)
Complete Case Study - part 2
Rendered airway tree with extracted airway paths.
Transverse Sliding Thin Slab (STS) view. Transverse slice image.
![Page 20: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/20.jpg)
Viewable Movie Sequences saved with final Case Study
Virtualscope Tranverse Slicer
Oblique Cross-Section Sagittal Slicer Coronal Slicer
![Page 21: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/21.jpg)
Other Navigation Tools
![Page 22: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/22.jpg)
![Page 23: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/23.jpg)
Example 2: Pathology CaseExample 2: Pathology Case
Patient suffering from tracheomalacia
EBCT scan; 133 slices; 512x512 voxels/slice
1.5mm slice thickness
0.586mm axial-plane resolution
Illustrates utility of a Key Site
![Page 24: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/24.jpg)
Ex #2: Coronal Weighted-Sum Projection showing computed path
Site #20 approachingtracheal collapse
Looking back toward tracheal collapsefrom site #99
Site #56 withintracheal collapse
Site #86 leavingtrachea
Site #99 near carina
Virtual endoscopic renderings shown for selected sites.
Preselected Key Siteused to initiate path.
![Page 25: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/25.jpg)
Example 2: Plot of Airway Cross-Sectional Area along Path
Oblique Cross-Section
at site #18, near
tracheal collapse.
Oblique Cross-Section
at site #48, within
tracheal collapse.
Plot clearly shows drop in cross-section where blockage occurs.
![Page 26: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/26.jpg)
Coronal Slicer snapshot clearly shows pathology.
Renderings of Airway tree clearly show pathology.
Example 2: Captured Snapshots of Pathology
![Page 27: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/27.jpg)
Example 2: Movie Sequences saved with Case Study
Virtualscope
Orthogonal Cross-Section Sagittal Slicer
Vertically Oriented Cross-Section
Transverse STS-Max
Coronal Slicer
![Page 28: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/28.jpg)
![Page 29: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/29.jpg)
1. Overview
2. Mutual information algorithm
3. Test results: bronchoscopy training device
III. Complete CT-Video: progress
Virtually guided Virtually guided bronchoscopybronchoscopy
![Page 30: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/30.jpg)
Application to TBNA (needle biopsy)
• TBNA -- blind procedure for sampling tissue
• Use VB-generated path:»bronchoscopist sees more, maintains orientation
• Matched video with rendered 3D CT scan
»identify target areas for biopsy
![Page 31: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/31.jpg)
CT-Video Matching: mutual-information algorithm
• Match rendered endoluminal CT view to video.
• Normalized Mutual Information Criteria - Studholme, IEEE TMI, Jan 1999
• Rendered Images with Graphical Accelerator- Hata, Lect. Notes in Comp. Sci., vol. 1131
• Steepest decent optimization.
![Page 32: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/32.jpg)
Barrel-Distortion Correction of Bronchoscopic Video
After correctionBefore correction(video frame of a test pattern)
Necessary for proper registration of video to rendered CT
*See Zhang, ICIP2000
![Page 33: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/33.jpg)
Registration of
Rendered 3D CT & Bronchoscopic Video
1. Use bronchoscopy training device.
2. Collect high-resolution EBCT scan.
3. “Perform” bronchoscopy on device
collect video
![Page 34: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/34.jpg)
Bronchoscopy Training Device
3D CT image rendered
![Page 35: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/35.jpg)
Initial Point:
Registration near Carina
• Initial point chosen in virtual 3D-CT world.
• Bronchoscope moved in “live” world to point.
• Optimal viewpoint calculated using mutual information.
![Page 36: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/36.jpg)
Registration in Left
Bronchus
Bronch video frame
Matching rendered CT view
![Page 37: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/37.jpg)
Future Work
Design specific VBDesign specific VB--based protocols: based protocols: lymphlymph--node location,node location, stentstent designdesign
Combine CTCombine CT--based analysis with video based analysis with video during live during live bronchoscopic bronchoscopic procedures.procedures.
![Page 38: Virtual Bronchoscopic approach for combining 3D CT and](https://reader031.vdocument.in/reader031/viewer/2022012516/619015bc60aa285614140697/html5/thumbnails/38.jpg)
Other SPIE Talks
5:30 tonight 5:30 tonight ---- California RoomCalifornia Room“Place of Virtual “Place of Virtual Bronchoscopy Bronchoscopy in Clinical in Clinical Practice: Barriers and Solutions”Practice: Barriers and Solutions”
1:20 today 1:20 today ---- Image DisplayImage Display conferenceconference“New Techniques for Fast Sliding Thin“New Techniques for Fast Sliding Thin--Slab Slab Volume Visualization” by J. Volume Visualization” by J. TurlingtonTurlington