ece6780 a3 pillcam project demonstration

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 1/42

PillCam Foundation and Image Retrieval

Project Report

Georgia Institute of Technology 

Medical Imaging 

Spring 2007 

Point of Contact

Todd Stokes

Instructor 

Dr. May Wang

David

McCann

Stephen

Conover 

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 2/42

Overview

Problem

Solution

Methodology Implementation

Results

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 3/42

What is PillCam?

Wireless Capsule

Endoscopy (WCE)

Non-invasive

Low Complication Rate

Out Patient

Potentially End-to-End Good diagnosis and

first look

http://www.nch.org/about/tech.shtml

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 4/42

Problem

Diagnosis Takes a Long Time

Where are we? What are we looking at?

Hours required to review video

10 ± 6Time for physician to write report

10 ± 3Time for nurse to begin download

30 ± 7Time for nurse to prepare patients

100 ± 14Time to read video by physician

Time (min) ± SDTask

* Bossa, et al. "Detection of Abnormal Lesions Recorded by Capsule Endoscopy: A Prospective Study

Comparing Endoscopist's and Nurse's Accuracy." Digestive and Liver Disease 38 (Aug 2006): 599-602.

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 5/42

Solution

Provide a tool to retrieve “similar” images

from a database

Allow reviewer to find related frames for 

comparison/reference

Provide general 3D model as an

alternative to frames

Reduce review time through search

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 6/42

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 7/42

Methodology Overview

Metrics Symmetry

Curvature

Color 

3D Modeling

Ranking

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 8/42

Methodology

Symmetry

Finding the “Center” of the Image

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 9/42

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 10/42

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 11/42

Nominal Results

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 12/42

Methodology

Curvature

Extracting the Shape of the Image

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 13/42

Curvature Overview

Why curvature? Good metric for features

and lumen/wall views.

A B C D

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 14/42

Process

1. Threshold

2. Extract Curvature

3. Histogram4. Compare

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 15/42

Thresholding

Global average of intensity

Simple but effective

Sample results:( )

( )ij

ij

 I  Avg T 

 BG R I 

=

++=3

1

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 16/42

Curvature Extraction

Curvature Equations (Osher and Sethien)

Binary Image Gaussian Kappa 1/Kappa

kappaC 

NormGrad NormGrad w kappa

normGrad normGrad w 

NormGrad 

Y Y 

 X  X ij 

ij 

Y  X 

1

elsewhere0

)max(*75.0where1

22

22

=

   ∇

∇+   

   ∇

∇=

>

=

∇+∇=

φ φ 

φ φ 

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 17/42

Base Test Case

0 50 100 150 2000

50

100

150

Curvature

      C     o    u     n      t

Curvature Histogram

50 100 150 200 250

50

100

150

200

250

 50 100 150 200 250

50

100

150

200

250

30

40

50

60

70

80

90

100

Circle

Radius of 50 

Extracted 

Curvature

Histogram of 

Curvature is Centered 

 Around 50! 

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 18/42

Process Development and Refinement I

Initial draft in MATLAB m-code

Ported to and refined in Java

Java Application for Testing:

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 19/42

Process Development and Refinement II

Bad

Better 

Best

Dilated 

& Masked 

Masked 

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 20/42

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 21/42

Comparison

Sum of the squares difference in

histogramed values 0:10:1000:

Must be normalized afterwards:

( )∑= −= N 

n

 B A n Binn Bin1

2][][ScoreComparison

( ) )(Score

Min(Score)-ScoreScoreComparison  Normalized

ScoreMinMax −=

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 22/42

Methodology

Color 

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 23/42

Overview

HSI Histogram

Sum Power Compare

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 24/42

Methodology

Final Scoring ∑=

=Metrics

k 

ii sw#

1

 ScoreFinal

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 25/42

Methodology

3D Modeling

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 26/42

Overview

3D OBJ Model interpolated from MRI slices

Basic lighting and flat color texturing

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 27/42

Smoothed Surface Normals

The surface normal at

a given vertex is the

average of the

surface normals of thepolygons meeting at

the vertex.

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 28/42

Methodology

Summary

Final Rationale: How does all this work together? 

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 29/42

Implementation

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 30/42

Implementation Overview

Completely Java

OpenGL and JOGL

30 files 3,000 SLOC

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 31/42

Class Diagram - Model

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 32/42

Modules

Curvature

Symmetry

Color Sum of 

Squares

Distance

Score AllPresent

Results

Single

Image Load

3D ModelNavigation

Tracks

Rendering

N or m al   i   z  e

Source

Selection

Metric

Extraction

Comparison Results

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 33/42

Features

Easy to use graphical user interface Navigate manually through model with AWSD and

mouse Record camera positions to a track

Save and load camera tracks Quickly scan through loaded camera track Add images and directories of images to database Compare database images to 3D model or image file

Quick link back to results Save/Load feature database functionality

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 34/42

Weaknesses

Thresholding Simple

Simple Ranking

Need higher fidelity model

Slow feature extraction

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 35/42

Results

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 36/42

Interface

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 37/42

Results

VideoFrame

Independent

Image

Model

Frame

Bad Frame

Bad Results

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 38/42

Results I: Search for Image from Video

1 2

3 4

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 39/42

Results II: Search for Image from Independent Frame

1 2

3 4

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 40/42

Results III: Search for Image from 3D Model

1 2

3 4

5

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 41/42

Results IV: Bad Results

1 2

3 4

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 42/42

Summary

Metrics perform well

Need deformable model

Review time is improved

Questions?