vessel-based non-rigid registration of mr/ct and 3d ultrasound for navigation in liver surgery

Computer A&d Surgery 8:228-240 (2003)

Biomedical PaDer

Vessel-Based Non-Rigid Registration of MR/CT and 3D Ultrasound for Navigation in Liver Surgery

Thomas Lange, Sebastian Eulenstein, Michael Hiinerbein and Peter-Michael Schlag Department of Surgery and Surgical Oncology, Chariti - Universitary Medicine, Berlin, Gemzany

ABSTRACT Objective: Computer assisted planning of liver surgery based on preoperative computed to-

mography (CT) or magnetic resonance imaging (MRI) data can be an important aid to operability decisions and visualization of individual patients' 3D anatomy. A navigation system based on intraoperative 3D ultrasound may help the surgeon to precisely localize vessels, vascular territories or tumors. The preoperative planning must be transferred to the intraoperative ultrasound data and thus to the patient on the operating table. Due to deformations of the liver between planning and surgery, a fast non-rigid registration method is needed.

Materials and Methods: A feature-based non-rigid registration approach based on the centerlines of the portal veins has been developed. The combination of an iterative closest point (ICP) approach and Multilevel B-Spline transformations offers a fast registration method. The vessels are segmented and their centerlines extracted from preoperative CT/MRI and intraoperative 3D Power- doppler ultrasound data. Anatomical corresponding points on the centerlines of both modalities are determined in each iteration of the ICP algorithm. The search for corresponding points is restricted to a given search radius and the direction of the vessels is incorporated.

Results: The algorithm has been evaluated on two transcutaneous and one intraoperative clinical ultrasound data set from three different patients. Only a very few vessel segments were not assigned correctly compared to manual assignments. Using non-rigid transformations improved the root mean square target registration error of the vessels by approximately 3-5 mm.

Conclusions: The proposed registration method is fast enough for clinical application in liver surgery. Initial accuracy results are promising and must be further evaluated, particularly in the operating room. Comp Aid surg 8:228-240 (2003). 02003 CAS Journal, LLC

Key wordc image-guided surgery, intraoperative navigation, liver, 3D ultrasound, non-rigid image registration

INTRODUCTION Several systems have been developed for liver surgery planning in the last couple of years.'-3 These systems are applied for planning living-

related liver transplantations (LRLT) and oncological liver resections for individual patient a n a t ~ m i e s . ~ . ~ In LRLT, a healthy volunteer do-

This paper is based on a presentation at the Second Annual Meeting of the German Society for Computer- and Robot-Assisted Surgery (CURAC), held in Erlangen, Germany, in November 2003.

Address correspondencelreprint requests to: Thomas Lange, CharitC, Universitatsmedizin Berlin, Campus Berlin-Buch, Robert- Rossle-Klinik im Helios-Klinikum Berlin, Lindenberger Weg 80, 13125 Berlin, Germany. Telephone: 030 9417 1629; E-mail: [email protected]

02003 CAS Journal, LLC

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

Lunge et al.: Non-Rigid Registration of W C T and 3D US 229

nates a portion of his liver to a recipient. It must be ensured that the volumes of both the liver remaining in the donor and the portion transplanted into the recipient are sufficiently large to sustain their vital functions. Knowledge of anatomical variations in the vascular system is of great im- portance in this context. The 3D anatomy of the vessels and the volumes of the transplanted and remaining liver parenchyma are computed and vi- sualized by the planning systems. In oncological liver surgery, the aim is to completely resect one or several lesions with a security margin and to resect as little healthy parenchyma as possible. In most cases, however, healthy parenchyma must also be resected if its blood supply and drainage would be disrupted by the surgery. The purpose of the planning systems is to compute anatomical resection proposals based on the vascular territories.

The planning systems aid the surgeons in deciding whether surgery is suitable for a specific patient and, if so, where to resect. Until now, the results of preoperative planning (3D anatomical model, definition of resection volumes, etc.) have had to be mentally transferred to the patient in the operating room (OR) by the surgeon. There is no navigation system to help the surgeon precisely localize the tumor, major vessels and vascular territories during liver surgery. Requirement analysis for such a navigation system has been under- taken by Hassenpflug et a1.6 Several commercial 3D navigation systems for image-guided surgery are available for bony structures, e.g., in neurosurgery or orthopaedics, but there are only a few systems - mostly research systems - that deal with navigation in soft tissue, as is the case in liver surgery. The reason for this is that bony structures do not change their shape between preoperative acquisition of image data such as magnetic resonance imaging (MRI) or computed tomography (CT) scans and positioning of the patient for surgery. Thus, it is possible to register preoperative images rigidly to the physical patient space. In liver surgery, the liver usually deforms significantly between preoperative imaging and positioning of the patient in the OR. Nevertheless, most published attempts to register preoperative liver data to the intraoperative patient space apply rigid registration.

Two approaches for registration of preoperative CT scans of the liver to physical patient space have been reported in the literature. Her- line et al.7 digitized the liver surface by moving an optically tracked probe over the visible portions

of a liver phantom and rigidly registering it to the liver surface extracted from CT scans by the It- erative Closest Point (ICP) algorithm.8 A contact- less variant of this approach has been implemented by Cash et al. by using a laser range scanner instead of a tracked probe.' They performed phantom studies and have applied their system to one clinical case so far. A mean residual error of 1.72 +/- 1.43 mm for the clinical case and target registration errors (TRE)" ranging between 0.6 mm and 7.74 mm for different tumors embedded in a deformable phantom were reported. Never- theless, it is not obvious how accurately tumors lying deep in the liver are registered in clinical cases.

Three-dimensional ultrasound offers the possibility of imaging deep-lying structures of the liver intraoperatively. Ultrasound (US) is an intraoperative imaging modality that is cheap, compared to intraoperative MRI, and radiation-free, unlike fluoroscopy. In addition, US can easily be integrated into the OR. Two different 3D ultrasound technologies exist. The first is called freehand 3D ultrasound." A position sensor of a lo- calker system is clamped to a 2D US transducer and the transducer is manually swept over the volume of interest while the position and orientation of the imaging planes are recorded by the localizer system. After scanning, the 2D image planes are composed to a 3D image volume. In the second approach, called swept-volume 3D ultrasound, the 2D transducer is swept by a motor contained in a specific 3D ultrasound transducer. The authors are aware of only one commercial system using a motor: the VOLUSON system developed by Kretztechnik and now distributed by General Electric.

In combination with a tracking system, ultrasound offers the possibility of directly navigat- ing surgical instruments in relation to intraoperative image data. In neurosurgery, 2D ultrasound has been integrated into navigation systems to detect brain shift. To our knowledge, only one 3D ultrasound navigation system is available: SonoWand from MISON.12 This system was developed for neurosurgery and is based on freehand 3D US. Three-dimensional US is also well suited for liver surgery navigation, because vessels have a good contrast, particularly in Powerdop- pler ultrasound. However, tumor borders are not clearly visible in all cases. In addition, 3D US alone is insufficient to localize the preoperatively planned resection volume precisely. For these

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

230 Lunge et al.: Non-Rigid Registration of MWCT and 3D US

purposes, it is necessary to register preoperative data to the intraoperative 3D US images.

Intensity-based, feature-based and hybrid approaches have been investigated to register CT or MR data to 3D US. In almost all cases, freehand 3D US and rigid transformations have been used. Different similarity measures have been applied in intensity-based methods. Roche et al. incorporated gradient information to mimic US image properties and registered US and MR images of the head.13 A highly successful similarity measure for multi-modality rigid registration is Nor- malized Mutual Information (NMI).I4 NMI has been applied to MR and PD-US of the carotid artery.I5 Penney et al. report that registration based on NMI has not been successful on their sparse 3D freehand US of the liver.16 Instead, they converted intensity values to vessel probabil- i t y values and used the normalized cross- correlation metric.

A very fast hybrid registration approach was described by Aylward et al.17 They developed a model-to-image metric that compares extracted vessel models of one modality with the original image of another modality. The behavior of their metric was shown on one data set for the case where vessels are extracted from and registered to the same 3D US data of the liver,” In this case, no deformations were present. They recently reported an extension of their method to piecewise rigid and elastic transformations, and showed a result on one pre- and post-surgery brain data pair.”

A voxel-based method that actually uses features for registration has been investigated by Porter et a1.20 Vessel voxels are identified in MR and 3D US data of the liver and the distance between those voxel sets is minimized. Voirin et al. presented a feature-based approach using the liver surface.2’ They manually segmented liver contours from CT and US and registered them using the ICP algorithm. Another surface-based method was described by King et al.” They used a statistical surface shape model to align US images and quantlfy liver motion due to respiration. Manually extracted surface points and points on vessel centerlines were used by Penney et al. for a stochastically modified ICP registration of 3D US and MR data.u

Our registration approach is also based on vessel centerlines. Thus, we will review some similar work that has been applied neither to ultrasound data nor to liver data. Tschirren et al. computed anatomically corresponding branch points

of human airway tree data via association graph matching.24 Edges were added to the association graph according to geometrical (distance, vessel segment length, vessel segment directions) and topological (distance, inheritance relationship) dif- ferences of branches of the two trees. Tolerances were introduced for all these measures to improve robustness against false and missing branches. A generalization of the ICP technique considering multiple correspondences for each point was presented by Stewart et al.= Points of vessel centerlines of 2D retina and 3D neuronal images were registered. For correspondence determination, they restricted the search to a given radius around each model point and took into account closest points of nearby reference vessel segments, as well as vessel orientation and width. They also used low-degree-of-freedom quadratic transformations.

So far, no non-rigid registration method has been presented that registers vessels from MWCT data to 3D US data of the liver. To build a fast and intraoperatively applicable non-rigid registration algorithm, we combined the ICP algorithm and Multilevel B-Splines, as described by Xie et a1.26 In contrast to the work of Xie et al., the correspondence determination is not based on surface similarity but on vessel centerline points in both modalities, making use of similar ideas to Stewart et a1.=

METHODS An overview of the entire navigation system is given in Figure 1. The aim is to navigate surgical instruments in relation to preoperative image data or 3D models of the liver. Intraoperatively, a 3D ultrasound scan is performed and the position T,, of a passive position tracker attached to the ultrasound probe is measured by a Polaris tracking system. The probe calibration Tcl, which de- scribes the position of the US volume in relation to the tracker position, is computed using a commercial 3D US phantom (CIRS, Model 51). An improved version of the precalibrated pointer calibration approach described in an earlier re- port27 is used, which will be published elsewhere. The instrument can be navigated in relation to the US volume via a tracker attached to the surgical instrument and the instrument calibration T,. The instrument calibration can be computed by the procedure reported by Detmer et a1.28 All transformations mentioned above are rigid transformations. To navigate in relation to the preoperative image data in a correct manner, an additional

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

Lange et al.: Non-Rigid Registration of W C T and 3 0 US 231

Polaris camera /”,

Preoperative data

Track& 2

Fig. 1. Schematic overview of the ultrasound-based navigation system.

non-rigid transformation T,, between preoperative and intraoperative data must be computed.

Overview of Registration Procedure We have developed a fast feature-based non-rigid registration algorithm that is based on the centerlines of liver vessels. This approach becomes fea- sible due to the high contrast in 3D PD US and contrast-enhanced C T or MR images (Fig. 2a and b). So far we have used only the portal veins, but in the future we intend to incorporate the hepatic veins into our system.

Before surgery, the vessels are segmented from preoperative CT or MR by a region-growing algorithm and manual post-processing. This step is performed with great care to assure the segmentation of as many vessels as possible. Interactive improvements of the segmentation in the preoperative phase are not time-critical. Next, the centerlines of the vessels are automatically extracted by the TEASAR algorithm introduced by Sat0 et al. (see Fig. 2c and d).29 The centerline voxel representation is converted to a set of lines consisting of discrete points. The intraoperative pre- processing starts with a reformatting of the 3D PD US data to Cartesian coordinates. Due to the swept curved-array US transducer, the original imaging geometry is very special. After this reformatting, the centerlines of the vessels are extracted from the US volume as they were extracted from the preoperative data. The first step of the registration procedure is a coarse rigid reg-

istration of the centerlines via 3 4 manually selected paired landmarks near the main branching of the portal vein. The second step consists of an automatic ICP-like rigid registration starting with the manual pre-registration until no further improvements can be achieved. In the third step, non-rigid transformations modeled by Multilevel B-Splines are incorporated into the ICP-like registration. In the following sections we will give a short review of the ICP algorithm and explain how it can be expanded for non-rigid registration of vessel centerlines.

Iterative Closest Point (ICP) Algorithm

The ICP algorithm seeks a rigid transformation that minimizes the root mean square (RMS) distance between corresponding points of two surfaces. The algorithm consists of two steps. First, for each point on the model surface the closest point on the reference surface is determined. Next, a rigid transformation is computed and applied to the model surface that maps these two point sets. This process is iterated until a stopping criterion is met. The algorithm converges to a local minimum of the RMS cost function. Only if the start position is “close enough” to the optimal position and the geometry of the surfaces is “simple” will the local minimum also be the global minimum. However, even the global minimum might not be the transformation that leads to the correct correspondences, since closest points do not have to be anatomically corresponding points.

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

232 Lange et al.: Non-Rigid Registration of W C T and 3D US

Fig. 2. (a) Contrast-enhanced CT image of the liver. (b) Corresponding slice of a 3D Powerdoppler US. (c) Transparent surface representation of portal veins extracted from CT data and computed vessel centerlines. (d) Vessel representation as in (c), but extracted from 3D Powerdoppler US.

Because of the tree-like structure of the vessels and incomplete imaging of some vessels in one of the modalities, this problem becomes a great con- cern for the registration of vessel centerlines. In the next section we describe the algorithm for computing correspondences based on the vessel centerline representation. We then explain the details of computing non-rigid transformations of the vessel centerlines using Multilevel B-Splines.

Correspondence Computation for Vessel Centerlines The original ICP algorithm computes for each point on the preoperative vessel centerlines (model) the closest point on the intraoperative

vessel centerlines (reference). Yet not all vessels in the model exist in the reference and vice versa. Due to the restricted field of view of the US scan, most - but not all - vessels imaged in preoperative CTMR are also imaged in PD US. Consequently, a vessel existing in the model but not in the reference will be assigned a wrong nearby vessel in the reference using the original ICP algorithm. A simple way to avoid some of these incorrect assignments is to restrict the search for the closest point to a given search radius R. This, however, prevents corresponding vessel segments that are farther apart than the radius R from being matched correctly. Moreover, points lying inside the search radius R might nevertheless not corre-

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

Lange et al.: Non-Rigid Registration of MWCT and 3D US 233

spond in a correct way to one other. For example, some vessel branch of the reference might cross the regarded vessel branch of the model. Near the crossing, the model and reference vessels are close to each other, yet their directions are different, as illustrated in Figure 3a. Simple exclusion of all closest point pairs for which the angle between the vessel directions at these points exceeds a given threshold would lead to a significant loss of information.

Therefore, in our approach, we search for the closest vessel segment with a similar direction. A vessel segment is to be defined as a part of the centerline between two branching points. For each point M on the model those vessel segments Si in the reference are sought which have a closest point C, to M that is inside the search radius R. All the potential corresponding segments Si are sorted by increasing distance of M to Ci. Starting with S,, the closest segment Sc is determined for which the angular difference of the vessel direction at M and Ci is below a given threshold. An illustration of this correspondence method is shown in Figure 3b and c. The direction at a vessel point is computed by the difference vector of the two neighbored points on the vessel. To increase robustness, we averaged the directions of 5 neigh- boring points. If no corresponding segments can be found that fulfil distance and angle difference constraints, no correspondence is introduced for this model point M.

Additionally, all point correspondences are excluded that point to an endpoint of the reference, because this would result in a shift and deformation of the vessel along the vessel direction

t 3

if the end segments have different lengths (see Figure 3d). An example of resulting correspondences for vessel centerlines from MR and 3D Powerdoppler US is shown in Figure 4.

Multilevel B-Spline Approximation of Non-Rigid Transformations Having found corresponding centerline points in both modalities, a deformation transformation that maps points of the model to their corresponding points on the reference has to be determined. There are many approaches to modeling non-rigid deformations of soft tissue for image registra- t i ~ n . ~ ' They can be divided into geometrical and physical methods. Physical methods try to model the biomechanical properties of soft tissue and compute the deformations of the tissue numeri- cally (see, for example, the work of Ferrant et al.31). Geometrical methods often try to mimic physical deformations by using sufficiently smooth deformation maps. Prominent examples of such deformation maps are polynomial functions like Thin-Plate Splines3' and Multilevel B- Splines.33 In contrast to Rohlfing et al.34 and Riickert et al.,35 who used B-Splines for intensity- based registration, we use B-Spline transformations for a feature-based approach, similar to the approach of Xie et a1.26 The advantages of the feature-based approach over the intensity-based approach are the ability to explicitly determine the interpolating or approximating transformations from the deformation vectors of the corresponding points and not having to perform a costly optimization in a high-dimensional parameter space.

I: Fig. 3. Correspondence determination from model to reference. (a) Closest point correspondences. (b) Vessel segments intersected by the search radius R and closest points C, and C, on these segments. The direction of the vessel is similar in C,, but is not similar in C,, as compared to the direction in M. (c) Resulting correspondences using nearby segments with similar direction, as explained in (b). (d) Removal of correspondences to endpoints of reference vessels.

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

234 Lange et aL.: Non-Rigid Registration of MRlCT and 3 0 US

Fig. 4. Example of correspondences between vessel centerlines from MR and 3D Powerdoppler US.

B-Splines are defined by a uniform control grid that is overlaid on the domain where the B- Spline is to be computed. Via the control grid spacing, it is possible to control the smoothness of the resulting deformations: finer grids lead to less smooth deformations. Computations of Multi- level B-Splines start with a coarse grid that is suc- cessively refined until a given minimal grid spacing is reached. The number of degrees of freedom increases with the refinement level. We decided to use approximating instead of interpolating Multilevel B-Splines because the correspondences are not error-free and the resulting B-Splines are smoother.

To improve the robustness of the registration process, we start with a coarse minimal grid spacing and refine it if no further improvements can be reached until we obtain suitable deformation results. In each iteration, the B-Spline transformation between the starting position of the model centerline points and the current corresponding reference points is determined on the basis of the transformed model points of the last iteration. This implies that only one B-Spline transformation is computed after gradually updat- ing the deformation vectors, as opposed to several B-Spline transformations applied one after the

other. Two termination criteria are enforced: a) the relative variation of the RMS distance between corresponding points at two consecutive iterations is below a given threshold; and b) a maxi- mum number of iterations is reached.

The algorithm can be summarized as fol- lows:

Let Ml, . . . , MN be points on model centerlines, Rl, . , . , R, points on reference centrelines, and Tl, , . . , T N transformed model points.

1. Set coarsest possible B-Spline control grid

2. Determine correspondences (T,,, Rnl),

3. Compute B-Spline approximating the dif- ferences (R,, - Mnl), . . . , (RnK - MnK) between reference and model.

4. Apply B-Spline transformation to original model points Ml,. , , , MN to get new Tl,. . . , TN.

5. Check termination criterion A and B. If none of them is satisfied, go to step 2 and continue.

6. Has the minimum B-Spline control grid spacing been reached? If not, refine control grid spacing by a factor of 2 and go to step 2.

and Mls . . . . MN to . . . I TN.

. . . > (TnK, RnK) , K I N

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

Lange et al.: Non-Rigid Registration of W C T and 3D US 235

EXPERIMENTS AND RESULTS

Evaluation of non-rigid registration algorithms is a non-trivial task, since the ground truth of the registration is difficult to access for clinical data. One possibility is to use a so-called bronze standard.36 Such a bronze standard can, for example, be determined manually by experts. Further pos- sibilities are the use of phantom experiments or numerical simulations. However, phantoms mostly lack sufficiently realistic imaging and deformation properties, while simulations may not reflect the real situation either. We will elaborate on these issues with regard to non-rigid registration in the discussion section. All experiments were performed on three individual patient data sets, which are described in detail in the next section. The main focus of our investigations is ex- pressed by the following questions, each of which is addressed in one of the following sections: a) Are the vessel segments correctly assigned? b) How reproducible are the results for different starting positions and correspondence parameters? c) How much can the registration be improved by non-rigid registration as compared to rigid registration?

Data Acquisition, Pre-Processing and Performance

All data were acquired during breath hold. Con- trast agents were used in preoperative CT or MR data. Patients 1 and 3 got preoperative T1- weighted Flash 3D VIBE MR sequences with 2.5 mm slice thickness acquired on a 1.5 T Siemens Symphony scanner. For Patient 2, a single-slice spiral CT with 2 mm reconstructed slice thickness (5 mm collimation, pitch 1.5) was acquired on a General Electric scanner. Three-dimensional B- mode and Powerdoppler ultrasound was simulta- neously acquired transcutaneously for patients 1 and 3 and intraoperatively for patient 2. A Kretz- technik VOLOSON 730D with a 3.5-MHz ab- dominal 3D probe was used. The original resolu- tion of the Powerdoppler scans was 0.2 mm in the scan line direction, 0.5-0.7" in the scan plane, and 0.9" between consecutive scan planes. A scan depth of 12-16 cm, an angle of 66-70" inside the scan planes, and an angle of 75" between first and last scan plane were obtained.

The original data are resampled isotropi- cally to 1 mm Cartesian coordinates. Afterwards, the portal vessels are segmented interactively by

region growing in less than 10 min, and the centerlines are automatically computed in a few sec- onds. The skeletonization algorithm produces between 1200 and 2000 centerline points for the MR or CT data and 330-520 points for the PD-US data. Manual selection of 3 4 landmarks near the portal vein trunk for pre-registration lasts 1-2 min. The automatic procedures without interactive segmentation of PD-US data and manual pre- registration are possible in 1-2 min. The whole registration process lasts less than 15 min and can be significantly accelerated by an improved segmentation step.

Correctness (Accuracy) The algorithm produced reasonable results in all three examined cases, as can be seen for patient 1 in Figure 5 and patient 2 in Figure 6. We used a search radius of 10 mm and a direction difference of maximal 30" for the correspondence determination. The minimal control grid spacing has been set to 15 mm, the stop criteria to minimal mm relative RMS change and maximal 300 iterations. We observed that it is very difficulty to assign certain corresponding points precisely in the original image data or on the extracted centerlines. Even branching points of the extracted vessel centerlines are not in all cases uniquely assignable. Particularly for main vessels with high diameter, the branching points showed significant shifts produced by the skeletonization algorithm. In a very few cases, additional connecting edges were found in CT.M data that were not identified in US data because the vessels appeared thicker in Powerdoppler US at these locations (Fig. 7). For the same reason, in one case even two nearby branching points had been swapped.

We therefore decided against manually as- signing single points in both modalities, but to ascertain which vessel segment in one modality belongs to which vessel segment in the other modality. Then we compared this manual segment assignment to the assignment found by the registration algorithm. The number of correctly and incorrectly assigned US segments is shown in Table 1. We further classified correct and incorrect assignments. The correct assignments were classified as missing segment, additional segment, and mixed-up segments. Missing segments are segments that had been imaged in the US data but not in the CT or MR data. All missing segments had been correctly classified. The reason for additional segments and mixed-up segments was explained above. Nevertheless, these cases have led

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

236 Lange et al.: Non-Rigid Registration of MWCT and 3 0 US

Fig. 5. Results for patient 1. The upper row shows the vessel centerlines and the lower row the vessel surfaces. In (a), the four manually selected pairs of landmarks can be seen. (a) and (d) show the start position after registration based on the four landmarks. The results after rigid registration are illustrated in (b) and (e), and those after non-rigid registration in (c) and (9.

to correct assignments. We only observed two wrong assignments. In both cases (one is shown in Figure 6), a branch had been assigned ambiguously to two neighbored branches with a very similar direction. This might be circumvented in some cases by introducing additional properties like the vessel radius. Alternatively, such branches could be detected by the algorithm and removed from the correspondence-finding process. The residual RMS distances of the correspondences computed in the last iteration of the algorithm were 1.8 mm for patient 1, 1.5 mm for patient 2, and 1.9 mm for patient 3.

Reproducibility To assess the reproducibility of the results, the algorithm was run for different starting positions of the model centerlines. This simulates different manual pre-registrations. Uniformly distributed noise was added to the pre-registration position obtained with the manually selected landmarks.

Fifty start positions were investigated for each data set in the range of +/- 5 mm for each of the three translational parameters, and in the range of +/- 5 degrees for each of the three Euler angles. The RMS target registration error (TRE)" of all points on the model centerlines between the dis- turbed and undisturbed starting position was used. After automatic rigid registration, almost all start positions resulted in the same position within 0.3 mm RMS TRE. Four failures (8%) were observed for patient 1, leading to 4.3,7.7,5.4 and 8.3 mm RMS TRE; two failures (4%) for patient 2 with 1.7 and 1.8 RMS TRE; and no failure for patient 3. It was also investigated how the result of the non-rigid registration is altered for these failure cases. RMS TRE values of 4.4,6.2,5.0, and 6.5 rnm were computed for patient 1, and values of 1.2 and 1.3 mm were computed for patient 2. The incorrect rigid registration could only be slightly improved by the non-rigid registration.

The influence of the search radius and maxi-

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

Lunge et al.: Non-Rigid Registration of W C T and 3D US 237

Fig. 6. registration from different views. In (b), the ambiguously assigned branch is labeled with an exclamation mark.

Results for patient 2. (a) Vessel surfaces after rigid registration. (b), (c) and (d) Vessel surfaces after non-rigid

ma1 direction difference in the correspondence determination step on the results has not been systematically investigated so far. However, our experience showed that the direction difference angle is not very sensitive within a reasonable range. The search radius is more sensitive: if it is too large, there will be many incorrect double correspondences; if it is too small, valuable correspondences will not be used. Nevertheless, it is possible to use the same parameter settings for all three data sets.

Quantification of Local Deformations in Relation to Rigid Registration

tions are used. The Rh4S TRE has been computed between rigidly registered model centerline points before and after non-rigid registration. RMS TRE values of 5.6,3.4 and 5.7 mm resulted for patients 1, 2 and 3, respectively.

DISCUSSION AND CONCLUSION We have presented an algorithm that can register liver vessels from preoperative data to intraoperative 3D Powerdoppler US data using non-rigid transformations. In contrast to intensity-based methods using B-Splines, no optimization in a high-dimensional parameter space is necessary. Thus, our method is fast. Promising initial results -

One important question is how the registration results can be improved if non-rigid transforma-

have been obtained on two transcutaneous and one intraoperative US patient data sets. Only a

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

238 Lange et al.: Non-Rigid Registration of W C T and 3 0 US

Fig. 7. An additional vessel segment in the MR data (marked by circles) that is missing in the US data because the vessel diameters appear larger in Powerdoppler US.

very few vessel segments have been assigned incorrectly, and the results are insensitive to the chosen start position. Using non-rigid transformations reduces the RMS TRE of the vessel centerlines by approximately 3-5 mm. Further accuracy studies need to be performed on more data, particularly intraoperative data. Penney et al. reported an RMS TRE for the entire rigidly registered liver in the range of 2.3 to 5.5 111111, as compared to manually constructed bronze stan- dards.16 We have ascertained that it is very difficulty to assign specific corresponding points precisely, but only limited alternatives exist when using realistic clinical data. One alternative we wish to work on is to simulate B-Spline deformations on clinical data and check whether the algorithm is able to retrieve the simulated transformations. Although this would allow an estimation of the algorithm’s robustness, it is restricted to B-Spline deformations. Up to now, we have only evaluated the registration accuracy of the vessel centerlines involved in the registration process. We plan to extend these evaluations to the vessel surface, to

vessels like hepatic veins that are not used for registration (see Figure 8), and to parts of the liver surface imaged in the US data.

A disadvantage of the presented approach is that a good intraoperative segmentation of the vessels is needed. Powerdoppler US provides a high contrast for vessel structures, and we will investigate how robustly the algorithm works for automatically segmented vessels. An alternative method is to register a model of the vessels segmented from preoperative data directly into the ultrasound data without segmenting them, as in the work of Aylward et a1.l’ We wish to extend our B-Spline-based method to this possibility in future work. Other extensions under consider- ation are the introduction of additional geometric properties such as vessel radius and topological properties, as we know that the vessels have a tree-like structure. Registration using graph matching might be one possibility to avoid the manual pre-registration step, but this step is not time-critical.

From our point of view, the intraoperative use of a laser range scanner for registration is only a supplemental method for methods using deep- lying structures. Depending on the location of the tumor, the exact knowledge of the liver surface deformation alone might not be sufficient to determine deformations at greater depth. One possible approach could be to use physical modeling of liver tissue and finite element computations to determine deformations inside the liver based on deformations of the surface. On the other hand, vessel deformations might not be sufficient to compute deformations near the surface. Thus, information acquired by a laser range scanner is possibly a suitable supplement to US information, and can at least be used for evaluation.

The fast, robust and accurate non-rigid registration of preoperative and intraoperative liver data is an important prerequisite for using preoperative data in a liver surgery navigation system. We have now implemented initial versions of all necessary components of such a navigation system and will investigate its usability in clinical cases.

Table 1. Number of Correctly and Incorrectly Assigned Vessel Segments Compared to Manual Assignment

MWCT us Correct Incorrect Missing Additional Mix-uD Patient 1 52 24 24 0 5 1 1 Patient 2 101 31 30 1 2 3 0 Patient 3 118 32 31 1 2 0 0

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

Lznge et a[.: Non-Rigid Registration of MlUCT and 30 US 239

Fig. 8. One slice of 3D Powerdoppler US, together with intersections of non-rigidly transformed portal veins and hepatic veins (HV) . The hepatic veins were not used for determination of the transform.

ACKNOWLEDGMENTS This work was supported in part by the DFG (Deutsche Forschungsgemeinschaft) grant 201879 “Navigationssystem basierend auf pra- und intra- operativen Patientendaten als Unterstutzung hochpraziser chirurgischer Eingriffe”. We wish to thank Steffen Prohaska from the Zuse Institute Berlin for providing an implementation of the TEASAR algorithm and Christian Hege from the Zuse Institute Berlin for providing the AmiraTM visualization and modeling software. In addition, we thank Hans Lamecker from the Zuse Institute Berlin for proof reading and very helpful com- ments.

REFERENCES Selle D, Preim B, Schenk A, Peitgen HO. Analysis of vasculature for liver surgery planning. IEEE Trans Med Imag 2002;21(11):1344-1357. Soler L, Delingette H, Malandain G, Montagnat J, Ayache N, Koehl C, Dourthe 0, Malassagne B, Smith M, Mutter D, Marescaux J. Fully automatic anatomical, pathological and functional segmentation from CT scans for hepatic surgery. Comput Aided Surg

Meinzer HP, Thorn M, Cardenas C. Computerized planning of liver surgery - an overview. Computers & Graphics 2002;26(4):569-576. Bourquain H, Schenk A, Link F, Preim B, Peitgen HO. HepaVision2 - a software assistant for preoperative planning in LRLT and oncologic liver surgery. In:

2001 ;6(3):131-142.

Lemke HU, Vannier M W , Inamura K, Farman AG, Doi K, Reiber JHC, editors: Computer Assisted Ra- diology and Surgery. Proceedings of the 16‘h Interna- tional Congress and Symposium (CARS 2002), Paris, June 2002. Berlin: Springer, 2002. p 341-346.

5. Preim B, Bourquain H, SeUe D, Peitgen HO, Oldha-

6.

7.

8

9

10.

11.

12.

13.

14.

15.

fer KJ. Resection proposals for oncologic liver surgery based on vascular territories. In: Lemke HU, Vannier MW, Inamura K, Farman AG, Doi K, Reiber JHC, editors: Computer Assisted Radiology and Surgery. Proceedings of the 16* International Congress and Symposium (CARS 2002), Paris, June 2002. Berlin: Springer, 2002. p 353-358. Hassenpflug P, Vetter M, Cardenas C, Thorn M, Mei- nzer HP. Navigation in liver surgery - results of a requirement analysis. In: Lemke HU, Vannier M W , Inamura K, Farman AG, Doi K, editors: Computer Assisted Radiology and Surgery. Proceedings of the 15‘h International Congress and Exhibition (CARS 2001), Berlin, June 2001. Amsterdam: Elsevier, 2001. p 1162. Herline AJ, Herring JL, Stefansic JD, Chapman WC, Galloway RL, Dawant BM. Surface registration for use in interactive, image-guided liver surgery. Com- put Aided Surg 2000;5:11-17. Besl PJ, McKay ND. A method for registration of 3-D shapes. IEEE Trans Pattern Anal Machine Intell

Cash DM, Sinha TK, Chapman WC, Terawah H, Dawant BM, Galloway RL, Miga MI. Incorporation of a laser range scanner into image-guided liver surgery: Surface acquisition, registration, and tracking. Med Phys 2003;30(7): 1671-1 682. Fitzpatrick JM, West JB, Maurer CR. Predicting error in rigid-body point-based registration. IEEE Trans Med h a g 1998;17(5):694-702. Prager RW, Rohling RN, Gee AH, Berman L. Rapid calibration for 3-D freehand ultrasound. Ultrasound Med Biol 1998;24(6):855-869. Lindseth F, Lango T, Bang J, Hernes TAN. Accuracy evaluation of a 3D ultrasound-based neuronavigation system. Comput Aided Surg 2002;7(4):197-222. Roche A, Pennec X, Rudolph M, Auer DP, Malan- dain G, Ourselin S, Auer LM, Ayache N. Generalized correlation ratio for rigid registration of 3D ultrasound with MR images. In: Delp s, DiGioia AM, Jaramaz B, editors: Proceedings of Third Interna- tional Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2000), Pittsburgh, PA, October 2000. Lecture Notes in Com- puter Science Vol1935. Berlin: Springer, 2000. p 567- 517. Studholme C, Hawkes DJ, Hill DLG. An overlap in- variant entropy measure of 3D medical image align- ment. Pattern Recognition 1999;32:71-86. Slomka PJ, Mandel J, Downey D, Fenster A. Evalu- ation of voxel-based registration of 3-D power Dopp- ler ultrasound and 3-D magnetic resonance angio-

1992;14(2):239-256.

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

240 Lange et al.: Non-Rigid Registration of W C T and 3 0 US

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

graphic images of carotid arteries. Ultrasound Med Biol 2001;945-955. Penney GP, Blackall JM, Hamady MS, Sabharwal T, Adam A, Hawkes DJ. Registration of freehand 3D ultrasound and magnetic resonance liver images. Med Image Anal 2004;8(1):81-94. Aylward SR, Jomier J, Weeks S, Bullitt E. Registra- tion and analysis of vascular images. Int J Computer Vision 2003;55(2/3):123-138. Aylward SR, Weeks S, Bullit E. Analysis of the parameter space of a metric for registering 3D vascular images. In: Niessen W, Viergever M, editors: Pro- ceedings of Fourth International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2001), Utrecht, The Nether- lands, October 2001. Lecture Notes in Computer Sci- ence Vol2208. Berlin: Springer, 2001. p 932-939. Jomier J. Vessel registration for pre-post surgery. Proceedings of Mth Annual Radiology Research Symposium, Chapel Hill, NC, April 2003. p 34. Porter BC, Rubens DJ, Strand JG, Smith J, Totter- man s, Parker KJ. Three-dimensional registration and fusion of ultrasound and MRI using major vessels as fiducial markers. IEEE Trans Med h a g 2001; 20(4):354-359. Voirin D, Payan Y, Amavizca M, Letoublon C, Troc- caz J. Computer-aided hepatic tumor ablation: re- quirements and preliminary results. CR Biologies 2002;325:309-319. King AP, Blackall J, Penney G, Hawkes D. Tracking liver motion using 2D ultrasound and a surface based statistical shape model. In: Proceedings of JEEE Workshop on Mathematical Methods in Biomedical Image (MMBIA). Kauai, Hawaii, December 2001. Penney GP, Blackall JM, Hayashi D, Sabhanval T, Adam A, Hawks DJ. Overview of an ultrasound to CT or MR registration system for use in thermal ablation of liver metastases. In: Proceedings of Medical Image Understanding and Analysis (MIUA), Bir- mingham, England, July 2001. Tschirren J, Palagyi K, Reinhardt JM, Hoffman EA, Sonka M. Segmentation, skeletonization, and branch- point matching - a fully automated quantitative evaluation of human intrathoratic airway trees. In: Dohi T, Kikinis R, editors: Proceedings of Fifth In- ternational Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2002), Tokyo, Japan, September 2002. Lecture Notes in Computer Science Vol2488. Berlin: Springer, 2002.

Stewart CV, Tsai CL, Perera A. A view-based ap- p 12-19.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

proach to registration: theory and application to vascular image registration. Proceedings of International Conference on Information Processing in Medical Imaging (IPMI), Ambleside, UK, July 2003. Xie Z, Farin GE. Deformation with hierachical B- splines. In: Lyche T, Schumaker LL, editors: Proceed- ings of International Conference on Mathematical Methods in Computer Aided Geometric Design, Oslo, Norway, 2000. Vanderbilt University Press,

Lange T, Eulenstein S. Calibration of swept-volume 3D ultrasound. In: Proceedings of Medical Image Un- derstanding and Analysis (MIUA), Portsmouth, En- gland, 2002. p 29-32. Detmer PR, Bashein G, Hodges T. 3D Ultrasonic image feature localization based on magnetic scan- head tracking. Ultrasound Med Biol1994;20:923-936. Sato M, Bitter I, Bende M, Kaufmann A, Nakajima M. TEASAR: Tree-structure extraction algorithm for accurate and robust skeletons. In: Barsky BA, Shi- nagawa Y, Wang W, editors: Proceedings of the 8Ih Pacific Graphics Conference on Computer Graphics and Application (Pacific Graphics OO), LQS Alamitos,

Dawant BM. Non-rigid registration of medical images: Purpose and methods, a short survey. In: Pro- ceedings of IEEE International Symposium on Bio- medical Imaging 2002. p 465-468. Ferrant M, Nabavi A, Macq B, Kikinis R, Warfield S. Registration of 3D interoperative MR images of the brain using a finite element biornechanical model. IEEE Trans Med Imag 2001;20(12):1384-1397. Bookstein FL. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans Pat- tern Anal Machine Intell 1989;11(6):567-585. Lee S, Wolberg G, Shin SY. Scattered data interpo- lation with Multilevel B-Splines. IEEE Trans Visual Comput Graphics 1997;3(3):228-244. Rohlfing T, Maurer CR, Bluemke DA, Jacobs MA. Volume-preserving nonrigid registration of MR breast images using free-form deformation with an incompressibility constraint. IEEE Trans Med Imag 2003;22(6):730-741. Riickert S, Sonoda LI, Hayes C, Hill DLG, Leach MO, Hawkes DJ. Non-rigid registration using free- form deformations: Application to breast MR images. IEEE Trans Med Imag 1999;18(8):712-721. Jannin P, Fitzpatrick JM, Hawkes DJ, Pennec X, Sha- hidi R, Vannier MW. Validation of medical image processing in image-guided therapy. IEEE Trans Med Imag 2002;21(12):1445-1449.

2001. p 545-554.

CA, 2000. p 281-289.

Com

pute

r A

ided

Sur

gery

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f L

aval

on

07/1

1/14

For

pers

onal

use

onl

y.

vessel-based non-rigid registration of mr/ct and 3d ultrasound for navigation in liver surgery

Documents