SURFACE REGISTRATION USING TEXTURED POINT CLOUDS AND MUTUAL

INFORMATION

By

Tuhin Kumar Sinha

Thesis

Submitted to the Faculty of the

Graduate School of Vanderbilt University

in partial fulfillment of the requirements

for the degree of

MASTER OF SCIENCE

in

Biomedical Engineering

December, 2002

Nashville, Tennessee

Approved by:

Michael I. Miga

Robert L. Galloway

ACKNOWLEDGEMENTS

I would like to thank my advisors for their guidance while completing this thesis. I would

specifically like to thank Dr. Galloway for his support before and during my career as a graduate

student. I would also like to specifically thank Dr. Miga for his diligence and patience in working

with me.

I would like to acknowledge the support of the many lab members that I have known in SNARL

and BML. Specifically, Andy Bass, Mark Bray, David Cash, Steve Gebhart, Steven Hartmann, Jim

Stefansic, and Chad Washington.

Finally, I would like to thank my family and friends for their support during this process.

Specifically, Ashnil Chopra, Chad Grout, Agnella Izzo, Bryan Martin, Neal Massand, Dana Pow-

ers, Sandy Rathor, my parents and my better half, Tanoy Sinha, my brother.

ii

TABLE OF CONTENTS

Page

ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Chapter

I. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

I.1 Image-guided Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1I.2 Model-updated image guided procedures . . . . . . . . . . . . . . . . . . . . . . . 3I.3 Surface Registration and Mutual Information . . . . . . . . . . . . . . . . . . . . . 6

II. METHODS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

II.1 Textured Surface Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11II.2 Surface Registration Using Mutual Information . . . . . . . . . . . . . . . . . . . 14

III. EXPERIMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

IV. RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

V. CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

VI. FUTURE WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

iii

LIST OF TABLES

Table Page

II.1 Table of data used to generate the K-D Tree in Figure II.7. . . . . . . . . . . . . . . . 18

IV.1 Results for simulated inter-modal registration of the brain phantom using Surface MI. . 31

iv

LIST OF FIGURES

Figure Page

I.1 Model-updated IGP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

II.1 Laser scanner used to acquire textured point clouds. . . . . . . . . . . . . . . . . . . . 12

II.2 Graphical Law of Sines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

II.3 Textured point cloud from Laser-range scanner . . . . . . . . . . . . . . . . . . . . . 14

II.4 Surface Projection Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

II.5 Graphical Bilinear Interpolation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

II.6 Effect of bilinear interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

II.7 An example K-D tree generated from Table II.1. Reproduced from Bentley [1]. . . . . 18

III.1 Sample textured point cloud generated using a laser range scanner. . . . . . . . . . . . 23

III.2 Sample textured point cloud generated using surface projection on a Gadolinium en-hanced MR volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

III.3 Use of a clipping plane to select a region of interest in the surface projection. . . . . . 26

IV.1 Sample registration results for the ball phantom. . . . . . . . . . . . . . . . . . . . . . 28

IV.2 Sample Registration Results for the brain phantom. . . . . . . . . . . . . . . . . . . . 30

IV.3 Sample Registration results for simulated multi-modality registrations. . . . . . . . . . 32

IV.4 Distribution of target registration errors for each set of experiments. . . . . . . . . . . 33

VI.1 Example dataset taken with the laser range scanner in the operating room. . . . . . . . 37

v

CHAPTER I

INTRODUCTION

An emerging area of research within image-guided procedures (IGP) is deformation tracking

and model-updated IGP. Research in this area is aimed at augmenting current IGP for greater

intra-operative efficacy and accuracy. As a comment on current surgical protocol, Galloway states

that “a surgeon puts up his pre-operative images on a light box, gathers the information he needs,

then turns his back and does the surgery.” [2] There are two concerns with this approach: (1) the

surgeon is relying on his or her memory for target planning during surgery, and (2) the surgeon is

relying on out-dated patient data for target planning. IGP can be used to address the first concern,

that is, an interactive system can be used to quantitatively involve the pre-operative image sets

during surgery. Thus, the physician quantitatively locates positions in the image in real-time, which

relieves dependence on memory-based therapy targeting. The second concern is the focus of much

research in extending IGP systems. More specifically, this area of research is developing strategies

to update the shapes defined by pre-operative images during surgery to reflect conditions in the

operating room. The work of this thesis takes initial steps at incorporating the data taken in the

operating room into IGP for neurosurgery. A unique surface registration algorithm is introduced

that allows patient-to-image registration of the data generated by a laser scanner. This registration

provides the ability to easily and accurately measure changes in the brain’s surface during surgery

for model-updated IGP.

I.1 Image-guided Procedures

IGP are defined as procedures that use pre-operative datasets (3-D images) quantitatively to guide

the surgical process[3]. The central component of any IGP concerns how accurately one can reg-

ister the patient during surgery to his/her patient-specific three dimensional datasets. Registration

establishes the relationship (i.e. mathematical mapping) between the intra-operative coordinate

system (physical-space) and the pre-operative image coordinate system (image-space). This pro-

1

cess allows the physician to map intra-surgical positions to the most appropriate position in image-

space, i.e. stereotaxy.1

Several methods have been developed to provided accurate stereotaxy. The first attempts to reg-

ister physical-space to image-space were taken by Horsley and Clarke [4] applied to non-human

subjects. Their method employed a rigid frame affixed to anatomical landmarks. The position of

a probe attached to the frame was then precisely resolved relative to the anatomical points. Subse-

quent stereotactic frames expanded on this theme and applied their techniques to human subjects

[5]. Crucial to the method of stereotaxy using frames was the rigidity of the frame to the patient’s

anatomy. That is, positions of the patients anatomy relative to the frame are fixed in both image-

space and physical-space. Researchers realized that this rigidity could be maintained without using

cumbersome frames, thus leading to the development of frameless stereotaxy. The most common

method of frameless stereotaxy involves rigid markers in both image-space and physical-space,

localization of the markers in each space, and a point-based registration of the markers. Rigidity

here implies that the distances between markers are preserved between spaces. Initially intrinsic

(anatomic) markers, such as bony structures on the face, were used for frameless stereotaxy [6].

However, difficulty in accurate localization limited their widespread adoption for IGP. A group

at Vanderbilt University thoroughly researched the localization errors associated with markers in

different imaging modalities and physical space, their work has lead to the development of highly

accurate extrinsic markers (Acustar R

) for use in IGP [7]. For stereotaxy, the centroids of the

Vanderbilt markers are localized in image-space and physical space. In image-space, localization

is performed either manually, or automatically using standard image processing techniques. In

physical-space, the markers are localized using either optical, electromagnetic or acoustical meth-

ods, with optical methods being the most accepted [6]. Once the markers have been localized in

each space, the mapping between them is found as a solution to the Orthogonal Procrustes’ Prob-

lem (OPP). This process is often called a point-based registration [8]. The mathematical formula-

tion of this problem and its solution are discussed later. This protocol for frameless stereotaxy (i.e.

1From the Latin: stereo = three dimensions, tactus = to touch.

2

using extrinsic markers and point-based registration) has become the de facto method of stereotaxy

for IGP.

However, rigid extrinsic markers and point-based registrations are limited in the information

they provide. In the case of neurosurgery, they are attached the to the skull, not the brain. Thus, the

registration provided is not relative to the organ of interest. Furthermore, the registration provided

using point-based systems are affected by the number and placement of markers in each modality.

Too few markers lead to inaccurate registrations and poorly placed markers result in ineffective

registrations [9]. Finally, point-based registrations are not retro-active. That is, soft-tissue defor-

mations, e.g. brain deformations in neurosurgery, are not resolved by this registration protocol, the

implications of which will be discussed in the next section. Nonetheless, point-based registrations

and extrinsic markers for IGP provide improved therapy planning and intra-operative targeting,

and are essential to current IGP.

I.2 Model-updated image guided procedures

Recent studies have reported the need for organ shift compensation strategies to further improve

IGP navigation [10][11]. Kelly et al. initially reported brain deformation (shift) during stereotactic

procedures [12]. The shifting of the brain during stereotactic procedures occurs due to a variety

factors including swelling, pharmacological agents, physiology, and the therapy itself. Nimsky

et al. have quantified deformations in the brain to be on the order of millimeters for deep tissue

and centimeters for surface tissue, thus, showing the quantitative effect of shift is significant given

that submillimetric accuracies are desired for surgery [11]. Reconciling the differences between

the intra-operative reality and the pre-operative images is referred to as image-updated IGP [13].

Some research groups, i.e. Nabavi et al. and Nimsky et al., compensate for organ shift by using

intra-operative imaging devices [14]. Within these frameworks the intra-operative data acquisition

and image-update happen concurrently. Some groups have researched intra-operative CT as an

intra-operative image acquisition, but logistical concerns, including dose, have limited its adoption

for intra-surgical use [15]. Registered intra-operative ultrasound has also been developed for IGP

[16]. However, poor image quality and degradation of tissue contrast over the course of a surgical

3

Pre-operativeImage Acquisition

// ImageProcessing

// ModelInitialize

// Image GuidedSurgery System

// Physician

ModelUpdate

88r

rr

rr

rr

rr

rr

Intra-operativeData Acquisition

oo

Figure I.1: Model-updated image guided surgery data pipeline.

procedure limit its effectiveness. Nimsky et al. have reported successful implementation of image-

updated IGP using intra-operative magnetic resonance imaging (iMR) and show millimetric target

accuracy results for this method in the brain [17]. They go on to report that out of 145 patients

subjected to IGP in the brain, only 14 required image-updating during surgery. This observation,

along with the high cost and difficult implementation of iMR, question the overall efficacy of iMR

as a tool for image-updated IGP.

More available methods of image-updated IGP have been proposed recently [18]. These meth-

ods rely on physical or statistical models to update pre-operative data in the operating room. A

schematic representation of the model-updated IGP pipeline is shown in Figure I.2. The intra-

operative data collected provides constraints for the update model. For example, in a physical

computational model of the brain, surface characteristics provide boundary conditions [19]. For

statistical models, surface movements provide similar constraints [20]. This thesis focuses on the

link between intra-operative surface data acquisition and model-updating for the IGP data pipeline.

Specifically, the aim is to observe cortical surface characteristics during surgery and reflect those

observations in a computational model. Thus, the intra-operative data collected is used in conjunc-

tion with pre-operative data for the IGP. This is in contrast to the methods described by Nimsky

and Nabavi and does not produce the ground truth results observed by them. However, the imple-

mentation details for a computational approach are somewhat less complicated. Patient transport

during surgery is not needed, as described by Nimsky, and constraining the range of movement

of the surgeon is also not necessary as in the case of the “double-donut” intra-operative magnetic

resonance unit used by Nabavi. Furthermore, advancements in numerical modeling of the brain

will hopefully reduce the residual error between the two methods.

4

A necessity in any model-updated IGP is characterization of intra-surgical conditions, many

methods have been explored to provide an accurate and efficient characterization. Intra-operative

data has been acquired during surgery using: A-mode ultrasonography [21] and tracked palpata-

tion [22] for surface characterization, tracked endoscopy for characterization of internal structures

[23], B-mode ultrasound for deep tissue visualization [24], laser spectroscopy for quantification

of deep tissue properties [25][26], and electrical stimulation for localization of eloquent areas

of the brain [27]. Thus, there is a wealth of intra-operative data that can be acquired and used

within a model-updated image-guided procedure. There is a subtlety to model-updated IGP that is

worth recognizing; the model-updating pipeline can accommodate not only anatomical changes,

but physiological changes as well. In fact an ultimate goal of model-updated IGP is reflecting both

anatomical and physiological changes in the operating theater [17]. All of the aforementioned

methods of intra-operative data acquisition provide raw data for the model-updating pipeline.

This thesis uses a laser range scanner (LRS) for novel characterization of the cortical surface

for model-updated IGP. In addition to laser triangulation of the surface, the scanner also reports

texture map coordinates [28] for each triangulated point. The texture coordinates allow scalar

values from a CCD image acquired during the scan to be associated with each point in the cloud.

The hypothesis here is that the textured point cloud captured during surgery will contain both

cortical sulcal and vessel patterns that can be used to register the patient to their pre-operative

images. Using features from the cortical surface for registration does have precedent. Nakajima et

al. demonstrated a target registration error of 1.3 1.4 mm using projected cortical vessels from

video images for registration [29]. More recently, Nimsky et al. reported a deformable surface

approach to quantify surface shifts using a variation on the iterative closest point (ICP) algorithm

[11]. Also, some preliminary work using a scanning based system for cortical surface registration

has been reported but a systematic evaluation has not been performed to date [30]. The novelty of

the approach reported here is that both vessel information and three-dimensional geometry is used

as the basis of alignment. Furthermore, the scanner provides a highly accurate method for tracking

the brain surface that will also serve as a valuable source of input data to a model based brain shift

5

compensation strategy.

I.3 Surface Registration and Mutual Information

As an initial step in the registration approach, an implementation has been developed using an

iterative closest point (ICP) [31] framework with mutual information (MI) [32]. The iterative

closest point algorithm (ICP), first postulated by Besl and McKay[31] is a well understood and

documented strategy for surface alignment. Building on the point-based registration technique

described earlier, the heart of the algorithm lies in the solution of multiple (OPP)[8]. The general

form for the (OPP) is given in Equation I.1.

AT B

E (I.1)

where A and B are given and an orthogonal T (T T T I)2 is desired to minimize the trace,

minEET min

AT B AT B T (I.2)

Mathematically, the solution of the problem transforms matrix A into B via T , with the trace rep-

resenting the residual error in alignment. For the canonical basis set (x,y,z) used in this paper, the

4x4 matrix T can be separated further into a translation and rotation component (see Equation I.3).

T R

D (I.3)

where R is an orthogonal rotation matrix, and D is the translation matrix. Arun found a solution

for the rotational component of T using singular value decomposition methods (SVD)[33] that is

used in this thesis. The translation component to the transformation (T ) is found trivially and in-

dependently of the rotation component. Arun’s method is expressed systematically in the methods

section. Nonetheless, this conceptual solution is easily applied to medical image registration. For

fiducial registrations, A and B represent the two sets of corresponding fiducials and T represents

2In this thesis, uppercase E represents a matrix and lowercase e represents a vector.

6

the transformation of coordinates in physical space to imaging space (or vice versa). For ICP,

each iteration of the algorithm creates point set correspondence using a closest distance metric

between points in the two clouds. The transformation relating the corresponding point sets found

as a solution to a Procrustes’ problem and the process is repeated. The algorithm ends when a

predetermined distance metric has been optimized, usually the mean closest Euclidean distance

(mean L2 norm)3. A limitation of ICP, when applied to cortical surface registration, lies in the

geometry of the cortex. The hemispherical shape of the cortical surface creates many orientations

that are valid solutions for ICP. Thus, in a least-squared distance sense, the points in two point

clouds might register well with ICP without correct orientation.

Information theory as applied to medical image registration can be used to overcome this limi-

tation for cortical surfaces. A way to measure the mathematical information content within a signal

was originally proposed by Claude E. Shannon in his landmark paper “A Mathematical Theory of

Communication”4[34]. Referred to as the Shannon entropy (see Equation I.4 for the entropy of

a continuous signal and I.5 for the entropy of a discrete signal), this measure used the transition

probabilities of random variables in a signal to ascertain the information content within the signal.

Hx K

∞

∞px log p

x dx (I.4)

Hx K ∑

ipxi log

pxi (I.5)

where px is the probability density function of the signal x and K is any scalar value5. As a

simple example, the entropy of a signal whose transitions are precisely known (that is, the signal

can be analytically characterized) has an entropy of 0. For a signal whose transition probabilities

are distributed equally over the range of the signal the entropy is equal to 1. Although the entropies

in I.4 and I.5 are written as functions, the random variable in parentheses is used as a label. Mathe-

3Other distance metrics can also be used in ICP, e.g. L1, L∞ for real coordinates or “cityblock” distances for integercoordinates, and the RMS distance maybe substituted for the arithmetic mean.

4The paper was later renamed “The Mathematical Theory of Communication”.5K is usually used to put the entropy in terms of physical quantities.

7

matical entropies are always scalar, and thus Hx should not be interpreted as a function of x. This

labelling convention follows that of Shannon and is used throughout this thesis. Novel uses for en-

tropy were proposed by Shannon including the information carrying capacity of communication

channels and information generation rates of communication channels.

Hawkes and Hill were the first to apply entropy and information theory to medical imaging

registration [35]. However, Viola and Wells, and Collignon independently introduced an extension

to Shannon’s initial work that allowed an accurate and robust method for registering a surface

[32][36][37]. Called Mutual Information (MI), MI approximates the similarity in information

between two random variables using both the marginal and joint entropies of the random variables.

A modification to the standard measure of MI, called Normalized MI (NMI), was proposed by

Studholme [38]. Registration of inter-modality medical imaging volumes showed that NMI proved

more robust than conventional MI. For this reason, NMI values were used in this paper. NMI can

be written concisely as follows,

NMIx y

H

x H

y

Hx y (I.6)

for a discrete random variable x. As written I.6, the registration becomes a problem in optimizing

NMI for a global extremum. The specific implementation of the registration process used in this

thesis is discussed in the methods section.

Although ICP and MI have been used extensively [39][40], previously published registration

frameworks do not entirely apply to the unique data provided by the scanner or this particular

registration approach. The data acquired by our scanner will allow for a one to one correspon-

dence between contour point and image intensity, i.e. the need to create an image from the

three-dimensional surface is not necessary. However, intensity correspondence between a three-

dimensional MR surface and an intra-operatively acquired laser-scanned cortical surface is some-

what more elusive. ICP may provide an adequate correspondence but this is an area that is actively

being investigated in the approach. The most similar work relating to this registration framework

is that by Johnson and Kang [41] in which these investigators used an objective function for reg-

8

istration based on a combined Euclidean distance and color difference metric. Used primarily in

a landscape alignment application, this technique would not be amenable to the alignment process

here, since the intensity distribution between scanner and MR image data is fundamentally very dif-

ferent. No registration algorithm has been developed that will register textured three-dimensional

surfaces from two different imaging modalities within the context of cortical surface registration

as presented in this thesis.

9

CHAPTER II

METHODS

Algorithms for textured surface generation and registration in physical space and imaging space

are explored in this chapter. For model-updated IGP textured surfaces in the physical world are

registered to the pre-operative images taken before surgery. These registered datasets allow for the

application of physical models to the pre-operative datasets for improving guidance during surgery.

The following sections outline how each type of surface is generated and the registration algorithm

used in the experiments.

Before outlining the algorithmic methods used in this thesis, an introduction to the Visual-

ization toolkit (VTK) is useful. VTK, or The Visualization Toolkit (Kitware Inc.) provided the

data object framework used in this research. The research and literature on computer graph-

ics and visualization is extensive, however, the research goals are relatively few and easy to

understand. One of the most compelling topics is the efficient and accurate display of multi-

dimensional objects on a two-dimensional screen [28]. To simplify this task the open graphics

library (OpenGL, www.opengl.org) was created, which provides a standardized computer graphics

rendering pipeline for multi-dimensional data visualization. Abstracting a layer above OpenGL

(the Open Graphics Library, Silicon Graphics Inc.), VTK provides a C++ class based approach

to handling multi-dimensional datasets for computation and visualization. VTK allows an easily

extensible interface for multi-dimensional data objects. All objects are manipulated in the VTK

framework before being passed to OpenGL for rendering. Data objects in VTK can be split into

two categories. The first, handles the geometry and topology of datasets. Briefly, in VTK the

geometry of a dataset reflects the 3 dimensional location of points and vertices; and the topology

reflects the relationship between (i.e. connections) points and/or vertices. The second category of

data represents a list of attributes of the geometry dataset, including: scalar values, vector values,

texture map coordinates, normals, tensors, etc. Furthermore, VTK defines a framework for process

objects that can be used to generate or filter VTK datasets and their attributes. Finally, VTK’s in-

10

put/ouput process object design allows for building on previously designed algorithms and linking

them together to create very intricate algorithms. This feature, along with its extensibility, existing

code-base, and mature API determined the use VTK for data management and manipulation in this

thesis.

II.1 Textured Surface Generation

A laser scanner system (RealScan 3D, 3D Digital Corporation, Danbury, CT) capable of capturing

three-dimensional topography as well surface texture mapping to sub-millimeter accuracy was

utilized (see Figure II.1) to generate phantom textured data from a ball. Laser range scanning, or

laser triangulation, builds on the Law of Sines (see Figure II.2 and Equation II.1). The distance

between transmit and receive points, the separation distance d, was known in the scanner. Also

the emission angle, b, and reflection angle, a were known. From these quantities and Equation

II.1, the range distance r was calculated. For the laser scanner used in this thesis, the transmit

signal was a laser pulse that was diffracted and guided using a pivoting mirror. The receiver

was a CCD grid that detected the reflection of the laser light from the surface of interest. The

scanning field consists of 500 horizontal by 494 vertical points per scanline and was accomplished

in approximately 5 seconds. Extensive calibration and characterization has been performed by

Cash et al. and has demonstrated the fidelity at which surface data can be acquired [42]. They

have shown repeatability in scanning a Teflon ball (diameter: 25.40 mm) on the order of 0.05 mm.

Tracking experiments in that study also show sub-millimetric errors while tracking the ball during

translations of 15 cm to 40 cm.

r sinb dsina

(II.1)

Some laser scanners also allow for the scalar encoding of range data. Typically laser scanners

use an RGB laser (i.e. three different wavelengths coupled together) for both range data and

color information. The color data is extracted per range point by using a diffraction prism on the

receive end [43]. The laser scanner used for this thesis implements a different approach for scalar

encoding. Scalar values were recorded as RGB data in a bitmap via CCD at the time of scanning.

11

Figure II.1: Laser scanner used to acquire textured point clouds.

Figure II.2: Graphical Law of Sines

12

In this thesis the RGB data was converted into luminance (Y ) values using a linear transformation

(see Equation II.2).

Y R 0 299

G 0 587

B 0 114 (II.2)

A single color channel was used in the registration process to reduce the complexity of the cur-

rent algorithm. Scalar encoding of the point cloud was done via the texture map coordinates

(u v 0 1 ) reported from the scanner. The registration between scanner coordinates and texture

coordinates was calibrated at the manufacturer. Accuracy of the registration has not been verified,

but is being planned as future research. In most graphics applications, the process of assigning to

scalar values to the point clouds for visualization (texture mapping) occurs in hardware. However,

for the purposes of this thesis, the texture mapping process was performed manually. The texture

coordinates were scaled to the size of the captured bitmap1, floored to the closest integral value and

the resulting texture was recorded as VTK data attributes. Figure II.3 shows the process of gen-

erating textured point clouds from the data collected by the scanner. Manually texturing the point

cloud reduced unnecessary computations during the registration process. For the point clouds from

the laser scanner, the geometry was stored directly and the topology was defined as each point by

itself. Dataset attributes for the point clouds included texture map coordinates and scalar values.

For the bitmaps, a regularly spaced (structured) grid with scalar values representing the bitmap

was used. A VTK source class (process object) was written to parse and read the binary data file

from the scanner and output the point cloud and bitmap as two separate data outputs. A VTK filter

class was used to generate luminance values from the RGB data bitmap. Finally, another VTK

filter class was developed to manually texture the point cloud given the point cloud and bitmap as

inputs.

To generate textured surfaces from MR volumes, a surface projection algorithm was used (Ana-

lyze AVW - Biomedical Imaging Resource). The MR volumes were gadolinium enhanced coronal

slices of the brain (1.015625 mm resolution in the slice plane and 1.5 mm out of plane) that were

segmented by hand to preserve the vessel structure. Rays were cast down the axial axis of the

1640x480 pixels for the scanner used in this paper.

13

Figure II.3: Texture point cloud generation from the laser scanner. Left to Right: CCD Bitmapcaptured at the time of scanning, point cloud captured at the time of scanning, textured point cloudafter texture mapping process.

volume. Only voxels with intensity values greater than five (out of 255) were recorded as surface

points. The texture at a given point was calculated by averaging the first five voxels following the

surface voxel (see Figure II.4). Finally, the surface was windowed empirically between 0 and 145

to provide high contrast between vasculature and tissue in the point cloud. The resulting dataset

and its scalar attributes were used for registration purposes. A high resolution surface was created

using bilinear interpolation (see Equation II.3 and Figure II.5) and super-sampling the surface in

and out of the image plane. The surfaces used in this thesis were super-sampled 2x in the saggital

axis and 4x in the coronal axis. The effect of super-sampling can be see in Figure II.6. The edge

effects seen with increasing super-sampling and their influence on the registration is discussed in

the results section.

Ix

r y

q 1 r 1 q I

x y q

1 r I

x y

1

r1 q I

x

1 y rq Ix

1 y

1 (II.3)

II.2 Surface Registration Using Mutual Information

The registration framework of this thesis, called SurfaceMI, involved two primary steps in its exe-

cution. The first step involved acquisition and preparation of the registration surfaces. The scanner

14

AveragedVoxels

Ray

Figure II.4: Surface Projection Algorithm.

q

r

(x,y+1)

(x,y) (x+1,y)

(x+1,y+1)

(x+r,y+q)

Figure II.5: Graphical Bilinear Interpolation.

15

Figure II.6: Effect of bilinear interpolation on surface projection. From left to right: no interpola-tion, 2X interpolation in the coronal axis, 4X interpolation in the coronal axis and 2X interpolationin saggital axis.

was placed approximately 30-60 cm from the surface of interest using a tripod. The extents of the

scanner’s fan beam were established and the laser stripe was passed over the surface in approxi-

mately 5 seconds. Data collected via the scanner was post-processed off-line to remove features

from the scan that were not used for registration. The MR surfaces were acquired according to the

surface projection algorithm outlined earlier.

The final step in the approach was to perform surface registration using a two-stage process. To

reiterate, the goal of the algorithm was to find T (a 4x4 matrix) such that the trace in Equation I.2

was minimized. In the first stage, an iterative closest point (ICP) algorithm was performed initially

to align the point clouds of interest (i.e. laser-scanned surface and/or MR surface). The Procrustes

transformation in each iteration of the ICP was calculated using an SVD method, outlined by Arun

[33]. The SVD method utilizes cross-covariances between corresponding fiducials to reduced the

trace of the registration. The algorithm is as follows for two sets of corresponding points (the

source points, which are registered to the target points):

1. Calculate Dt , the distance of each target point to the centroid of the target points

2. Calculate D f , the distance of each source point to the centroid of the source points

3. Calculate H DtDTf

16

4. Perform an SVD on H, resulting in U , D, and V matrices2

5. The rotation matrix R in T is given by UV T

6. Finally the translation is given by calculating the difference in centroids

To implement the ICP algorithm, the SVD solution to the Procrustes problem was used iteratively

(i.e. each rotation matrix calculated was concatenated to the previous matrix) until the mean closest

distance fell below the resolution of the surface (empirically determined to be 0 1 mm).

The standard (exhaustive) search algorithm used to determine corresponding fiducials for the

ICP was replaced with an optimized K-D search tree [44]. Proposed originally by Bentley, K-

D trees are optimized binary search trees for multi(K)-dimensional spaces. The search tree was

generated by partitioning the input dataset at each level along the median of the dimension with

the greatest range. The tree was terminated when there are no more points to be partitioned at a

given sub-tree. An example dataset and resulting K-D tree can be see in Figure II.7. Searching for

nearest neighbors is done by traversing the the tree in a manner similar to standard binary trees.

Given a query value, determine whether the value is less than, greater than, or equal to the value

at any given sub-tree. The conditional was evaluated at the root node for every sub-tree for only

the dimension being partitioned at that node. The search moves down the tree in the direction of

the conditional that is satisfied. However, for K-D trees the size of the partitions in k-dimensional

space must be tracked as the search progresses. Furthermore, when appropriate, traversing the

complimentary branch for a given sub-tree must occur in order to find the true nearest neighbor.

Optimized searching algorithms are outlined in [44][1][45]. The performance gained with K-

D trees compensates for its complex implementation. For a search composed of N points in k

dimensional space the search time goes from ON2 in an exhaustive search to O

N logk N in a K-

D tree search. This thesis used a C++ implementation of K-D tree creation and searching by David

Mount called ANN [45]. ANN is extensible to any integral dimension k; this might be of benefit

in future work, as the scanner returns 6 dimensional data by nature. Using the optimized nearest

2U and V are unitary matrices and D is a diagonal matrix containing the singular values of H, where H U 1DV

17

D

A

B

G

F

C

E

(0,0)

(100,100)

(100,0)

(0,100)A

B C

D E F

G

Figure II.7: An example K-D tree generated from Table II.1. Reproduced from Bentley [1].

Table II.1: Table of data used to generate the K-D Tree in Figure II.7.Point Location

A (50,50)B (10,70)C (80,85)D (25,20)E (40,85)F (70,85)G (10,60)

neighbor search and the SVD implementation in VTK allowed accurate and efficient calculation

of ICP transformations.

The ICP correspondence provides the necessary relationship to proceed with the second stage,

a constrained intensity-based registration, provided by NMI. As reported earlier NMI is used to

register random variables based on a mathematical information criteria. For the purposes of this

thesis, the information criteria was calculated using histogram methods. The intensity at each point

in a given surface cloud was binned between 0 and 255. Binning provides a quick and efficient

way of approximating the probability density function (PDF) for the intensities in the cloud. Joint

PDF’s were calculated by associating intensities in two different clouds according to the nearest

neighbors. Again, K-D trees were used to accomplish this task quickly. In order to penalize non-

18

overlapping surfaces, closest points that were not within the resolution of the surfaces were binned

to a 0 intensity value in the joint pdf 3. Marginal and joint Shannon entropies were calculated using

non-zero bins in the respective histograms and (I.5). Finally, the calculated value of NMI (I.6) was

passed into the optimization pipeline.

The optimization chosen for this was Powell’s method taken from [46]. Powell’s optimiza-

tion is a direction set method for optimization in multi-dimensions. The optimization determines

conjugate directions for line minimizations at each iteration. Conjugate directions implies “non-

interfering” lines for optimization. That is, optimizing in any one of the directions will not com-

promise the optimization in any other direction. An efficient method for determining conjugate

directions is to use the first derivatives of the objective function (called “Conjugate Gradient De-

scent” optimization). Ideally, for this method to work, functional forms of the objective and its

derivatives must be known. In some cases, if the functional forms are not known, numerical ap-

proximations can be substituted. Experiences during the development of the registration algorithm

proved that numerical approximations of the derivatives could not be employed in the optimiza-

tion. Local minima in the objective functions space and the shape of the space itself prohibited

gradient descent methods from converging to the global minimum. Although the shape of the ob-

jective space has not been characterized, empirical evidence indicates that it is ellipsoidal with a

high major axis to minor axis ratio. Thus, the gradient in the direction of one of the basis vectors

might be orders of magnitude larger than the other gradients. This property of the space easily

confounds direct gradient descent methods, as one direction dominates the optimization process.

Furthermore, using numerical approximations for the derivatives does not provide truly conjugate

directions for optimization. Thus, the conjugate methods of optimization succumb to local minima.

Powell’s method of optimization compensates for the shape of the objective space by calculating

a better approximation to the conjugate directions for line minimization at each iteration. After

the conjugate directions were found in Powell’s method, a line minimization was done in each di-

rection. The line minimization was done using Brent’s method of inverse parabolic interpolation,

3The criteria for overlap was that the closest point must be less than 1 millimeter

19

which is quadratically convergent. For a bracketed minimum, the minimum was found by fitting

a parabola through three points within the bracket. The objective function was then evaluated and

re-bracketed at the minimum of the fitted parabola. This process was repeated until the minimum

was found for the given direction, and was repeated for each conjugate direction.

To further help the optimization process, a geometry constraint was imposed on the calculated

transformation. The constraint requires the alignment transformation to operate in spherical co-

ordinates with a fixed radius. This constraint was hypothesized based on empirical observations

of the surface of the brain. To implement the constraint, a sphere was fitted geometrically in a

least-squares sense to the target surface [47]. Given the set of points X , the fitting was provided by

minimizing the objective function in Equation (II.4).

minσ2 min

X F

a T

X Fa (II.4)

where Fa is the parametric equation of a sphere. The parameters

a are found using Gauss-

Newton iteration method: ak 1

ak λ∆

a (II.5)

with λ step-size. ∆a is found using an SVD decomposition of the Jacobian matrix:

Jk ∂F∂ak

UDV T (II.6)

where U , D, and V T are the result of the SVD (described earlier), leading to,

∆a V DUT

X Fak (II.7)

The iterations were stopped when ∆a was below a threshold value of 0.01. For a complete descrip-

tion of the fitting process refer to Ahn [47].

Given the radius of the fitted sphere, all transformations of the source surface were constrained

to move along the surface of the sphere. By enforcing this restriction on the transformation, the

20

degrees of geometric freedom were reduced from six to three, i.e. elevation φ, azimuthal θ, and

roll ψ at the fitted radius r.

21

CHAPTER III

EXPERIMENTS

A series of experiments were developed to gauge the limits of the algorithm. Additionally, these

initial trials also gauged the computational costs of the algorithm. The first series of experiments

tested the accuracy of the algorithm under intra-modal conditions. Having successfully developed

the algorithm using the first series of experiments, the second series experiments tested the regis-

tration algorithm under simulated multi-modal conditions. Results to both series of experiments

are discussed in the results section.

A series of intra-modal experiments were used to verify the accuracy and robustness of the

newly devised surface registration algorithm. The first intra-modal experiment in this series was to

register two surfaces of a ball (Franklin Sports INC., Stoughton, MA, 02072) from the range scan-

ner. The ball was used to simulate the hemispherical geometry of the exposed brain surface seen

intra-surgically. The surfaces used in this paper for registration experiments occupied a solid angle

of Ω 1 2π steradians1 and contained 67257 points (see Figure III.1). A known spherical trans-

formation about the fitted radius was applied to the target surface to generate the floating surface.

The limits for the elevation and azimuthal angles were sampled were between 13 degrees and the

roll angle was sampled between 25 degrees. Five hundred uniformly distributed combinations of

φ, θ, and ψ were tested for registration accuracy.

Having tested the intra-modal registration accuracy of the laser range scan data, a second set of

experiments tested registration accuracy of the point clouds generated from the surface projections

of the MR volume. The target surface generated via surface projection and clipping had a solid

angle of approximately Ω 38533π steradians and contained 48429 points (see Figure III.2 and

III.3). The misregistration protocol from the ball experiments was applied to the brain surface.

The range for the parameters φ, θ, and ψ were the same as those for the ball but the transformation

was applied about the fitted center and radius of the brain. Again, the parameters for the known

1The solid angle of a unit sphere Ω 4π steradians.

22

Figure III.1: Sample textured point cloud generated using a laser range scanner.

23

transformations were sampled for 500 trials.

The third set of experiments outlines the efficacy of the developed algorithm in registering

surfaces across modalities. Inter-modality surfaces were simulated by inverting the texture of

the point cloud generated via surface projection. Texture values were inverted according to the

following equation,

I x y z 255 I

x y z (III.1)

Five hundred trials were run using the target surface defined earlier and different ROIs from the

inverted surface projection as the floating surfaces. The ROIs were generated by varying the normal

of the clipping plane used to create the target surface between 0.1 cm in the sagittal and coronal

axis while holding the axial value at 1 cm. To create the misregistration between the float and target

surface, each surface was re-centered about it’s geometric centroid. An initial approximation to

the correct registration was provided by applying an ICP transformation to the floating surface.

24

Figure III.2: Sample textured point cloud generated using surface projection on a Gadoliniumenhanced MR volume.

25

Figure III.3: Use of a clipping plane to select a region of interest in the surface projection.

26

CHAPTER IV

RESULTS

Since the same scan was used for both target and floating surfaces in the registration experi-

ments, the one-to-one correspondence in points was known. This allowed calculation of the root-

mean-squared target registration error (RMS TRE), as defined by Fitzpatrick and Mandava [48],

between point clouds after registration. The equation for RMS TRE (given in IV.1) is as follows,

RMS T RE 1

N

N

∑j

y j T

x j 2 (IV.1)

where y j is a point on the target cloud and Tx j is the corresponding point on the floating cloud

transformed in the coordinate space of the target cloud. The one-to-one correspondence also meant

that the NMI for a fully optimized registration was known. Thus, the registration results from the

500 trials are split into two categories. The first set, over the entire 500 trials, yielded an RMS TRE

of 11.38 28.75 mm. The second category represents the 345 trials where the registration attained

an ideal value for NMI, i.e. the maximum value for NMI given a set of marginal entropies1. In the

successful registration set the RMS TRE was 0.20 0.05 mm. There was no significant difference

between the means for φ, θ, and ψ, between the two categories. This result implies that the range for

successful registration is within the sampling range of φ, θ, and ψ. An indication of the robustness

of the algorithm is the 70% of registration trials that registered to better than 1 millimeter accuracy.

Furthermore, the RMS TRE results of the experiment on the ball show that SurfaceMI and Powell’s

optimization register the surfaces very accurately (i.e. below the tolerances of the scanner) when

successful. A sample result of the registration algorithm on the ball can be seen in Figure IV.1.

Note that the sampling range used in this paper is anticipated to be much greater than what will

be observed during surgery. Inter-operatively, the expected range of misregistration should be

1In this case, since both marginal entropies are the same and the joint probability of a correctly aligned surface isequal to the marginal entropy, the ideal value of NMI is 2 .

27

Figure IV.1: Sample registration results for the ball phantom with spherical constraint. Top row,from left to right: two misregistered surfaces, two surfaces after registration. Bottom row, left toright: off-axis view of misregistered surfaces, off-axis view of registered surfaces. Initial misreg-istration of 15 degrees in φ, θ, and ψ

.

28

no greater than 5 degrees in each parameter (in this range the ball phantom results are 100%

successful).

All trials using the intra-modal imaging phantom resulted in an ideal value of NMI. The RMS

TRE for these trials were 0.14 0.04 mm. The difference in the results of the brain and the ball

are likely due to the differences in the topologic distribution of the intensity information. Most of

the intensity information of the ball is contained in the central area of the surface. In some cases,

when the initial mis-registration of the ball causes sufficient non-overlap of the central area, the

algorithm can not register the surfaces correctly. For the brain, the intensity pattern of the vessel

structure occupies most of the surface. Thus, even though the brain’s surface occupies a smaller

solid angle than that of the ball, the distribution of the intensity pattern allows for more severe

misregistration to be aligned. A sample result of the registration results on the brain surface can be

seen in Figure IV.2.

In the simulated inter-modal registration experiments, statistical analysis of the results show

that SurfaceMI did produced a more accurate registration of the surfaces over all trials (t-test:

p 0.05). The second category in Table IV.1 shows descriptive statistics for trials in which the

RMS TRE of the registration was less than 2 standard deviations plus the mean RMS TRE calcu-

lated earlier using known transformations. There were only 137 trials which registered the target

and floating surfaces to this tolerance. The variance of the successfully registered trials is signifi-

cantly different from the entire trial population. The extent of the variance is reduced by an order

of magnitude for the successful registrations. Analysis of the surfaces that failed to register showed

that the spherical constraint prevented accurate surface registration. In general, the algorithm failed

to register surfaces clipped from or containing the periphery of the surface projection. The periph-

ery of the surface projection contained a much higher surface curvature when compared to the

target surface. This discrepancy in surface curvatures between target and floating surfaces caused

the sub-optimal registrations. For cortical resection and retraction, the visible surface of the brain

will likely be from a single region, this inherent region constraint will be incorporated into future

iterations of the algorithm. A sample of the simulated inter-modality registration experiment can

29

Figure IV.2: Sample Registration Results for the brain phantom with spherical constraint. Top row,left to right: two misregistered surfaces, two surfaces after registration. Bottom row, left to right:off-axis view of misregistered surfaces, off-axis view of registered surfaces. Initial misregistrationof -15 degrees in φ, θ, and ψ.

30

Table IV.1: Results for simulated inter-modal registration of the brain phantom using Surface MI.All Trials Successful Registration Trials

Number of Trials 500 137ICP RMS TRE (mm) 5.25 3.50 2.69 1.81MI RMS TRE (mm) 3.36 7.17 0.21 0.06

be seen in Figure IV.3. The results are shown with an intra-modal registration of the same surfaces

for clarity.

Figure IV.4 shows the results of each set of experiments in histogram fashion. As the figure

shows, the results for both intra-modality experiments show high success rates for TRE’s under

0.5 millimeter. The results in the intra-modal brain experiments showed a lower percentage below

0.5 millimeter and, but also a a larger percentage between 0.5 and 3.5 millimeters when compared

to the intra-modal experiments. The results depicted in the histogram supplement the conclusions

drawn from analyzing the RMS TRE’s within each set of experiments. That is, the concentrated in-

tensity pattern in the sphere phantom prevented proper alignment. However, the distribution of the

sphere phantom correlated well with the brain phantom, in that the successful and non-successful

registrations were mutually exclusive with regard to TRE. Thus, in very few occasions when the

registration failed did it produce RMS TRE below 3.5 mm. The inter-modality distribution shows

that there were more instances, as compared to the intra-modal trials, where the algorithm did not

reach ideal NMI, but still produced RMS TRE values below 3.5 mm. This is in contrast to the intra-

modal trials and shows that even when the algorithm fails to reach a highly accurate alignment it

may achieve a “good” ( 3 5mm) alignment.

The results here have shown that SurfaceMI registers textured surfaces well. The topological

distribution of the texture effects the ability for SurfaceMI to register. However, good topographi-

cal texture distributions are expected intra-operatively. Intra-modal registrations demonstrate low

error residuals when successfully registered, but generated very high residuals when unsuccess-

ful. Simulated inter-modal registrations exhibited lower residuals for unsuccessful registrations,

implying that SurfaceMI can register reasonably well even when unsuccessful.

31

Figure IV.3: Sample Registration results for simulated multi-modality registrations of the brainphantom. Top row, from left to right: two intra-modal surfaces registered via ICP and after regis-tration via SurfaceMI. Bottom row, from left to right: two simulated multi-modal surfaces regis-tered via ICP and registered via SurfaceMI. Clipping plane parameters: saggital = 0.02 cm, coronal= 0.1 cm, and axial = 1 cm.

32

0.0-0.5 0.5-1.0 1.0-1.5 1.5-2.0 2.0-2.5 2.5-3.0 3.0-3.5 >3.5RMS Target Registration Error (mm)

0

100

200

300

400

500

600

Cou

nt

Sphere PhantomBrain PhantomBrain/Inverted Brain Phantom

Figure IV.4: Distribution of target registration errors for each set of experiments.

33

CHAPTER V

CONCLUSIONS

The results of this thesis show that the ICP and MI framework is a useful tool for cortical surface

registration. Results of both intra- and inter-modality surface registration show sub-millimetric

accuracies using a phantom. This thesis outlines preliminary steps taken with the laser range

scanner and the algorithm. Future work, to be discussed in the following chapter, will hopefully

prove the value of both the laser range scanner and unique surface registration algorithm in model-

updated image guided-surgery.

The use of a laser range scanner in the operating room is a novel method of intra-surgical

surface characterization. As seen in Figure VI.1, this method has been shown to be a viable non-

contact method of capturing intra-operative data. The surfaces collected by the scanner are accurate

with regard to the physical world and provide, not only geometric data, but also rich surface texture

information. The wealth of information contained within the scans will provide critical information

for computationally updated IGP.

With respect to incorporating the feature-rich laser range scan data, a novel registration algo-

rithm has been developed. The algorithm, which is built on a framework sensitive to intensity

information (i.e. mutual information), capitalizes on both the intensity and geometry contained

in each scan. To provide data similar to that acquired by the laser range scanner, a method to

extract textured point clouds from the pre-operative data has been developed. The surface extrac-

tion method has shown that gadolinium enhanced MR images provide good texture and geometry

information for the registration algorithm.

The results of the techniques discussed in the thesis show viability for clinical application. The

intra-modality experiments demonstrate the successful modification and application of existing

registration techniques to the specific problem of registering textured point clouds. The simulated

inter-modal experiments show that the algorithm can effectively extended to multi-modal datasets,

i.e. a patient-specific range scan to his or her pre-operative image volume.

34

The import of the ideas and methods outlined in this thesis are a critical component in the de-

velopment of model-updated IGP. Registering intra-operative data to pre-operative data will allow

for boundary conditions in the computational models used for model-updated IGP. The computa-

tional models will then provide the physician with intra-surgically relevant images which can be

used for more accurate therapy delivery.

35

CHAPTER VI

FUTURE WORK

There are many research avenues to pursue as a result of the work presented in this thesis.

The immediate goal of future work is to verify in vivo accuracy of the registration algorithm.

Preliminary designs for true inter-modal registration experiments include using watermelons with

CT/MR contrast-enhanced vasculature inlaid on the surface. The vasculature will be imaged using

the scanner as well as CT and MR imaging devices. The methods described in this thesis will

then be applied for registration. Error residuals similar to those reported in this thesis will be

reported for the inter-modal registrations. These phantoms will also be validated using a gold

standard for registration that is being developed by Cash et al. Furthermore, the phantoms will

allow for comparisons with existing registration techniques, e.g. Nakajima’s method for point

based registrations using vessel bifurcations and ICP methods using extracted vessel contours.

These experiments will examine the potential registration accuracy in an inter-modal setting using

the laser range scan data.

In vivo proof of concept and accuracy studies are currently under way. In these studies, con-

senting patients primarily under going tumor resection and similar surgeries will be monitored

during the course of therapy. These patients will be operated on using image-guidance with the

scanner also being tracked by the image-guidance system. Before the skull cap is removed, expos-

ing the area of therapy, a laser scan will be taken. Another scan will be taken after removal of the

dura. Subsequent scans will be taken periodically during the course of surgery. This experimental

setup will map the laser range scanner space to physical-space, and consequently image-space.

This mapping will allow for the registration results using the scanner to be checked against results

provided by the image-guidance system. Furthermore, this experimental protocol will test the abil-

ity to of the scanner to track the intra-operative deformations. After surgery, all of the scans will

be registered and verified for accuracy. An example dataset from surgery is given in Figure VI.1.

This dataset clearly shows that the cortical surface can be resolved with the current laser scanner

36

Figure VI.1: Example dataset taken with the laser range scanner in the operating room. Left, aCCD image of the surgical area. Right, a tessellated point cloud with texture mapped points on theright.

and thus can be used for in vivo validation.

The algorithm itself is also under research. Future iterations of the algorithm will use non-

canonical basis sets for registration. Basis sets, like B-spline sets or Fourier sets will be used to

allow for non-affine transformations from physical to image space. Arun’s solution to the Pro-

crustes’ problem does allow for higher dimensions in the basis sets for registration. More pliant

shape constraints on the registration algorithm are also planned for review. The ultimate goal would

include finding a general form for the registration algorithm (that is computationally tractable) for

any two surfaces over any number of degrees of freedom.

With regard to Mutual Information, different estimations of mathematical entropy are under

investigation. Viola and Well’s original work presented a method for probability density estima-

tion called the Parzen Window [36]. This method uses a moving average of Gaussian distributions

provided by the random variable [49]. Although this method is not anticipated to be less computa-

tionally intensive, it will allow for calculation of NMI in higher dimensions more easily. That is,

37

future NMI calculations might look like the following:

NMI Hrx gx bx H

ry gy by

Hrx gx bx ry gy by (VI.1)

Also, the use of non-standard entropy measures, e.g. Renyi and Kullback entropies, is also under

investigation [50][51]. The objective function space associated with these non-standard entropy

measure might yield more tractable optimizations.

Another area of active research is more effective optimization of the Mutual Information cri-

teria. The simple line set method used in this thesis will be replaced with more advanced meta-

heuristic search strategies. One method under investigation is the Tabu search [52]. This search

strategy allows one to use more simple search methods in a heuristic fashion. For example, in the

specific case of optimizing the NMI in two textured point clouds, the Tabu search will use Pow-

ell’s method for line minimizations within the search space. The Tabu search will then guide the

optimization within the search space by expanding search areas out of regions of local minima,

e.g. using Tabu criteria [52]. Initial experiences with this advanced search strategy show it to be

an effective alternative to a purely numerical optimization.

Finally, this thesis and the work included are stepping stones in the path to accurate model-

updated image guided surgery. Novel work has to be completed in the area of the physical model,

the update procedure, and the image-guidance system for the complete model-updating application

to be successful. These topics are to be looked at in the future as a part the development of model-

updated image guided procedures.

38

REFERENCES

[1] J. L. Bentley. Multidimensional binary search trees used for associative searching. Commu-nications of the ACM, 19, 1979.

[2] Robert L. Galloway. Personal correspondence, January 2002. Vanderbilt University Biomed-ical Engineering Graduate Student Research Day.

[3] R. L. Galloway. The process and development of image-guided procedures. Annual Reviewof Biomedical Engineering, 3:83–108, 2001.

[4] V. Horsley and R. H. Clarke. The structure and function of the cerebellum examined by anew method. Brain, 31:45–124, 1908.

[5] R. L. Galloway and Robert J. Maciunas. Stereotactic neurosurgery. Critical Reviews inBiomedical Engineering, 18(3):207–233, 1990.

[6] R. L. Galloway. The process and development of image-guided procedures. Annual Reviewof Biomedical Engineering, 3:83–108, 2001.

[7] C. R. Maurer, J. M. Fitzpatrick, M. Y. Wang, R. L. Galloway, R. J. Maciunas, and G. S. Allen.Registration of head volume images using implantable fiducial markers. Ieee Transactionson Medical Imaging, 16(4):447–462, 1997.

[8] Peter H. Schoenmann. A generalized solution of the orthogonal procrustes problem. Psy-chometrika, 31:1–10, 1966.

[9] J. B. West, J. M. Fitzpatrick, S. A. Toms, C. R. Maurer, and R. J. Maciunas. Fiducial pointplacement and the accuracy of point-based, rigid body registration. Neurosurgery, 48(4):810–816, 2001. English Article APR NEUROSURGERY.

[10] D. L. G. Hill, C. R. Maurer, R. J. Maciunas, J. A. Barwise, J. M. Fitzpatrick, and M. Y. Wang.Measurement of intraoperative brain surface deformation under a craniotomy. Neurosurgery,43(3):514–526, 1998.

[11] C. Nimsky, O. Ganslandt, S. Cerny, P. Hastreiter, G. Greiner, and R. Fahlbusch. Quantifi-cation of, visualization of, and compensation for brain shift using intraoperative magneticresonance imaging. Neurosurgery, 47(5), 2000.

[12] P. J. Kelly. Computer-assisted stereotaxic laser resection of intraaxial brain neoplasms. Neu-rosurgery, 64(3):427–439, 1986.

[13] Michael I. Miga, Keith D. Paulsen, F. E. Kennedy, V. M. Tronier, D. W. Roberts, A. Hartov,L. A. Platenik, and K. E. Lunn. Model-updated image-guided neurosurgery: preliminaryanalysis using intraoperative mr. LNCS: Medical Image Computing and Computer-AssistedIntervention, 1935:115–124, 2000.

39

[14] A. Nabavi, P. M. Black, D. T. Gering, C. F. Westin, V. Mehta, R. S. Pergolizzi, M. Ferrant,S. K. Warfield, N. Hata, R. B. Schwartz, W. M. Wells, R. Kikinis, and F. A. Jolesz. Se-rial intraoperative magnetic resonance imaging of brain shift. Neurosurgery, 48(4):787–797,2001.

[15] L. D. Lunsford. Magnetic resonance imaging stereotactic thalamotomy: report of a case withcomparison to computed tomography. Neurosurgery, 23(3):363–7., 1988.

[16] R. D. Bucholz, D. D. Yeh, J. Trobaugh, L. L. McDurmont, C. D. Sturm, C. Baumann, J. M.Henderson, A. Levy, and P. Kessman. The correction of stereotactic inaccuracy caused bybrain shift using an intraoperative ultrasound device. In Cvrmed-Mrcas’97, volume 1205 ofLECTURE NOTES IN COMPUTER SCIENCE, pages 459–466. 1997.

[17] C. Nimsky, O. Ganslandt, P. Hastreiter, and R. Fahlbusch. Intraoperative compensation forbrain shift. Neurosurgery, 56(6):357–364, 2001.

[18] D. W. Roberts, M. I. Miga, A. Hartov, S. Eisner, J. M. Lemery, F. E. Kennedy, and K. D.Paulsen. Intraoperatively updated neuroimaging using brain modeling and sparse data, 1999.

[19] M. I. Miga, D. W. Roberts, F. E. Kennedy, L. A. Platenik, A. Hartov, K. E. Lunn, and K. D.Paulsen. Modeling of retraction and resection for intraoperative updating of images. Neuro-surgery, 49(1):75–84, 2001.

[20] E. Fontenla, C. A. Pelizzari, J. C. Roeske, and G. T. Y. Chen. Using serial imaging data tomodel variabilities in organ position and shape during radiotherapy. Physics in Medicine andBiology, 46(9):2317–2336, 2001.

[21] C. R. Maurer, R. P. Gaston, D. L. G. Hill, M. J. Gleeson, M. G. Taylor, M. R. Fenlon, P. J.Edwards, and D. J. Hawkes. Acoustick: A tracked a-mode ultrasonography system for reg-istration in image-guided surgery. LNCS: Medical Image Computing and Computer-AssistedIntervention, 1679:953–962, 1999.

[22] A. J. Herline, J. L. Herring, J. D. Stefansic, W. C. Chapman, R. L. Galloway, and B. M.Dawant. Surface registration for use in interactive image-guided liver surgery. LNCS: Med.Image Computing and Computer-Assisted Intervention, 1697:892–899, 1999.

[23] A. Herline, J. D. Stefansic, J. Debelak, R. L. Galloway, and W. C. Chapman. Techni-cal advances toward interactive image-guided laparoscopic surgery. Surgical Endoscopy-Ultrasound and Interventional Techniques, 14(7):675–679, 2000.

[24] N. Hata, T. Dohi, H. Iseki, and K. Takakura. Development of a frameless and armless stereo-tactic neuronavigation system with ultrasonographic registration. Neurosurgery, 41(3):608–613, 1997.

[25] W. C. Lin and et al. Brain tumor demarcation using optical cpectroscopy; an in vitro study.Journal of Biomedical Optics, 5(2):214–220, 2000.

40

[26] W. C. Lin and et al. In vivo brain tumor demarcation using optical spectroscopy. Journal ofBiomedical Optics, 73(4):396–402, 2001.

[27] Steven L. Hartmann. Intraoperative identification and display of cortical brain function. PhDthesis, Vanderbilt University, 2002.

[28] James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes. Computer Graph-ics: Priciples and Practice. Addison-Wesley Publishing Company, second edition, 1990.

[29] S. Nakajima, H. Atsumi H, R. Kikinis, T. M. Moriarty, D. C. Metcalf, F. A. Jolesz, and P. M.Black. Use of cortical surface vessel registration for image-guided neurosurgery. Neuro-surgery, 40(6), 1997.

[30] Michel A. Audette, K. Siddiqi, and T. M. Peters. Level-set surface segmentation and fast cor-tical range image tracking for computing intrasurgical deformations. LNCS: Medical ImageComputing and Computer-Assisted Intervention, 1679, 1999.

[31] P. J. Besl and N. D. McKay. A method for registration of 3-d shapes. IEEE Transactions onPattern Analysis and Machine Intelligence, 14(2), February 1992.

[32] William M. Wells, Paul Viola, Hideki Atsumi, Shin Nakajima, and Ron Kikinis. Multi-modal volume registration by maximization of mutual information. Medical Image Analysis,1(1):35–51, 1996.

[33] K. S. Arun, T. S. Huang, and S. D. Blostein. Least-squares fitting of 2 3-d point sets. IEEETransactions of Patter Analysis and Machine Intelligence, 9(5):699–700, 1987.

[34] C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal,27:379–423 and 623–656, 1948.

[35] D. L. Hill, C. Studholme, and D. J. Hawkes. Voxel similarity measures for automated imageregistration. pages 205–216. SPIE: In Proceedings of the Third Conference of Visualizationin Biomedical Computing, 1994.

[36] Paul Viola and William M. Wells. Alignment of maximization of mutual information. Inter-national Journal of Computer Vision, 24(2):137–154, 1997.

[37] A. Collignon, F. Maes, D. Delaere, D. Vandermeulen, P. Suetens, and G. Marchal. Auto-mated Multimodality Image Registration Based on Information Theory. Kluwer AcademicPublishers, Dordrecht, 1995.

[38] Colin Studholme, D. L. G. Hill, and D. J. Hawkes. An overlap invariant entropy measure of3d medical image alignment. Pattern Recognition, 32(1):71–86, 1999.

[39] Frederik Maes, Andre Collignon, Dirk Vandermeulen, Guy Marchal, and Paul Suetens. Mul-timodality image registration by maximization of mutual information. IEEE Transactions onMedical Imaging, 16(2):187–198, April 1997.

41

[40] M. A. Audette, F. P. Ferrie, and T. M. Peters. An algorithmic overview of surface registrationtechniques for med. imag.. Medical Image Analysis, 4:201–217, 2000.

[41] A. E. Johnson and S. B. Kang. Registration and integration of textured 3d data. Image andVision Computing, 17(2):135–147, 1999.

[42] David M. Cash, Tuhin K. Sinha, Will C. Chapman, Robert L. Galloway, and Michael I. Miga.Fast accurate surface acquisition using a laser scanner for image-guided surgery. SPIE: Med.Imag. 2002, 2002.

[43] M. Rioux. Visual information technology. website: www.vit.iit.nrc.ca.

[44] J. H. Friedman, J. L. Bentley, and R. A. Finkel. An algorithm for finding best matches inlogarithmic expected time. ACM Transactions on Mathematical Software, 3(3):209–226,1977.

[45] D. Mount. Ann programming manual. website: www.cs.umd.edu/mount/ANN, 1998.

[46] William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. NumericalRecipies in C : The Art of Scientific Computing. Cambridge University Press, second edition,1993.

[47] S.J. Ahn, W. Rauh, and H.-J. Warnecke. Least-squares orthogonal distances fitting of circle,sphere, ellipse, hyperbola, and parabola. Pattern Recognition, 34(12):2283–2303, 2001.

[48] V. R. Mandava, J. M. Fitzpatrick, C. R. Maurer, and et al. Registration of multimodal volumehead images via attached markers. In Medical Imaging IV: Image Processing, volume 1652,pages 271–282. SPIE, 1992.

[49] Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern Classification. Wiley-Interscience, 2 edition, 2001.

[50] Alfred Renyi. Probability Theory. North Holland Publication Co., 1970.

[51] Solomon Kullback. Information Theory and Statistics. Wiley, 1959.

[52] F. Glover. Tabu search: A tutorial. Interfaces, 20(4):74–94, 1990.

42