ece6780 a3 pillcam project demonstration

8/15/2019 ECE6780 A3 PillCam Project Demonstration

http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 1/42

PillCam Foundation and Image Retrieval

Project Report

Georgia Institute of Technology

Medical Imaging

Spring 2007

Point of Contact

Todd Stokes

Instructor

Dr. May Wang

David

McCann

Stephen

Conover



Overview

Problem

Solution

Methodology Implementation

Results



What is PillCam?

Wireless Capsule

Endoscopy (WCE)

Non-invasive

Low Complication Rate

Out Patient

Potentially End-to-End Good diagnosis and

first look

http://www.nch.org/about/tech.shtml



Problem

Diagnosis Takes a Long Time

Where are we? What are we looking at?

Hours required to review video

10 ± 6Time for physician to write report

10 ± 3Time for nurse to begin download

30 ± 7Time for nurse to prepare patients

100 ± 14Time to read video by physician

Time (min) ± SDTask

* Bossa, et al. "Detection of Abnormal Lesions Recorded by Capsule Endoscopy: A Prospective Study

Comparing Endoscopist's and Nurse's Accuracy." Digestive and Liver Disease 38 (Aug 2006): 599-602.



Solution

Provide a tool to retrieve “similar” images

from a database

Allow reviewer to find related frames for

comparison/reference

Provide general 3D model as an

alternative to frames

Reduce review time through search



Methodology Overview

Metrics Symmetry

Curvature

Color

3D Modeling

Ranking



Methodology

Symmetry

Finding the “Center” of the Image



Nominal Results



Methodology

Curvature

Extracting the Shape of the Image



Curvature Overview

Why curvature? Good metric for features

and lumen/wall views.

A B C D



Process

1. Threshold

2. Extract Curvature

3. Histogram4. Compare



Thresholding

Global average of intensity

Simple but effective

Sample results:( )

( )ij

ij

I Avg T

BG R I

=

++=3

1



Curvature Extraction

Curvature Equations (Osher and Sethien)

Binary Image Gaussian Kappa 1/Kappa

kappaC

NormGrad NormGrad w kappa

normGrad normGrad w

NormGrad

Y Y

X X ij

ij

Y X

1

elsewhere0

)max(*75.0where1

22

22

=

∇

∇+

∇

∇=

>

=

∇+∇=

φ φ

φ φ



Base Test Case

0 50 100 150 2000

50

100

150

Curvature

C o u n t

Curvature Histogram

50 100 150 200 250

50

100

150

200

250

50 100 150 200 250

50

100

150

200

250

30

40

50

60

70

80

90

100

Circle

Radius of 50

Extracted

Curvature

Histogram of

Curvature is Centered

Around 50!



Process Development and Refinement I

Initial draft in MATLAB m-code

Ported to and refined in Java

Java Application for Testing:



Process Development and Refinement II

Bad

Better

Best

Dilated

& Masked

Masked



Comparison

Sum of the squares difference in

histogramed values 0:10:1000:

Must be normalized afterwards:

( )∑= −= N

n

B A n Binn Bin1

2][][ScoreComparison

( ) )(Score

Min(Score)-ScoreScoreComparison Normalized

ScoreMinMax −=



Methodology

Color



Overview

HSI Histogram

Sum Power Compare



Methodology

Final Scoring ∑=

=Metrics

k

ii sw#

1

ScoreFinal



Methodology

3D Modeling



Overview

3D OBJ Model interpolated from MRI slices

Basic lighting and flat color texturing



Smoothed Surface Normals

The surface normal at

a given vertex is the

average of the

surface normals of thepolygons meeting at

the vertex.



Methodology

Summary

Final Rationale: How does all this work together?



Implementation



Implementation Overview

Completely Java

OpenGL and JOGL

30 files 3,000 SLOC



Class Diagram - Model



Modules

Curvature

Symmetry

Color Sum of

Squares

Distance

Score AllPresent

Results

Single

Image Load

3D ModelNavigation

Tracks

Rendering

N or m al i z e

Source

Selection

Metric

Extraction

Comparison Results



Features

Easy to use graphical user interface Navigate manually through model with AWSD and

mouse Record camera positions to a track

Save and load camera tracks Quickly scan through loaded camera track Add images and directories of images to database Compare database images to 3D model or image file

Quick link back to results Save/Load feature database functionality



Weaknesses

Thresholding Simple

Simple Ranking

Need higher fidelity model

Slow feature extraction



Results



Interface



Results

VideoFrame

Independent

Image

Model

Frame

Bad Frame

Bad Results



Results I: Search for Image from Video

1 2

3 4



Results II: Search for Image from Independent Frame

1 2

3 4



Results III: Search for Image from 3D Model

1 2

3 4

5



Results IV: Bad Results

1 2

3 4



Summary

Metrics perform well

Need deformable model

Review time is improved

Questions?

ece6780 a3 pillcam project demonstration

Documents