A NEW INTERACTIVE METHOD FOR CORONARY ARTERIES SEGMENTATION BASEDON TUBULAR ANISOTROPY

Fethallah Benmansour, Laurent D. Cohen

CEREMADE, UMR CNRS 7534, Universite Paris Dauphine,Place du Marechal De Lattre De Tassigny, 75775 PARIS CEDEX 16 - FRANCE

benmansour, cohen@ceremade.dauphine.fr

ABSTRACTIn this paper we present a new interactive method for tubu-lar structure extraction. The main application and motivationfor this work is vessel tracking in 3D medical images. Thebasic tools are minimal paths solved using the fast marchingalgorithm. This leads to interactive tools for the physicianby clicking on a small number of points in order to obtaina minimal path between two points or a set of paths in thecase of a tree structure. Our method is based on a variant ofthe minimal path method that models the vessel as a center-line and surface by adding one dimension for the local radiusaround the centerline. The crucial step of our method is thedefinition of the local metrics to minimize (based on the lo-cal orientation using a Riemannian Metric). This approachis made available for the physician using an Object OrientedLanguage (OOL) interface. We show results on two CT car-diac images for coronary arteries segmentation.

Index Terms— Image segmentation, Image enhance-ment, Medical diagnosis

1. INTRODUCTIONCoronary artery disease (CD) is the leading cause of deathin the United States. In symptomatic patients, diagnosis ofthe presence and severity of coronary artery disease is criti-cal for determining appropriate clinical management. Indirectevaluation of coronary stenosis, such as through stress testing,has limited diagnostic ability as compared with direct conven-tional coronary angiography. Conventional coronary angiog-raphy reveals the extent, location, and severity of coronaryobstructive lesions, which are potent predictors of outcome,and identifies high-risk patients who may benefit from revas-cularization. Thus, invasive coronary angiography, despitethe associated risks, remains the standard for the diagnosis ofobstructive coronary artery disease. Multidetector computedtomographic (MDCT) angiography has been proposed as anoninvasive test to determine the presence of CD. AlthoughMDCT angiographic data acquisition is straightforward, ef-fective visualization and communication of the complex mul-tifocal manifestations of CD remain a major challenge. Foraccurate image interpretation, precise, fast and semiautomaticimage processing tools are mandatory.

This need has attracted the attention of the computer vi-sion community yielding to multidisciplinary collaborations.Moreover, segmentation of tubular structures like blood ves-sels or coronary arteries is a very active research domain sincethe early 90’s. Indeed, various methods such as vascular im-age enhancement methods [1–3], or others were proposed,see [4] for a complete survey.

Siddiqi and Vasilevskiy proposed in [5] a vessel segmen-tation method based on flux maximizing flows. On the sameresearch line, Bouix et al proposed in [6] a method for auto-matic centerline extraction once the boundaries of the vesselare found. Nonetheless, their approach suffers from huge timeconsuming due to the used level set formulation and to theused iterative algorithm. Moreover, the inward flux they max-imize is not concave at all with respect to the evolving shapeand may be stuck on local maxima yielding to bad result. Co-hen and Kimmel introduced in [7] the minimal path methodthat captures the global minimum curve between two pointsgiven by the user. This leads to the global minimum of anactive contour energy. Using the Fast Marching method [8],the minimal path problem can be solved efficiently. The maindrawback of the minimal path method when it is applied tovessel segmentation is that it provides only a trajectory (or aset of trajectories), and does not give any information aboutthe vessel boundary and local width. In [9], Deschamps andCohen used the image itself as a propagation potential whilethey applied the Fast Marching method to an excellent con-trast air-filled colon on CT scanner and considered the propa-gating front as an estimate of the vessel boundary. However,classical segmentation problems do not provide such an ex-cellent contrast yielding the propagating front to flow overthe boundaries of longer and thinner objects. Again, in [9],authors proposed to extract the centerline using the estimatedvessel boundaries.

Our method is based on a variant of the classical, purelyspatial, minimal path technique by incorporating an extranon-spatial dimension into the search space. This approachwas first proposed by Li et al [10]. Each point of the 4D path(after adding the extra dimension for the 3D image) consistsof three spatial coordinates plus a fourth coordinate which de-

scribes the vessel thickness at that corresponding point. Thuseach 4D point represents a sphere in 3D space, and the ves-sel is obtained by taking the envelope of these spheres as wemove along the 4D curve. The main drawback of their methodis that they considered only isotropic media that does not takeinto account the orientation of the vessels. Our first contribu-tion is to take into account the vessel orientation by defininga suited anisotropic metric that makes the propagation fasteralong the center lines and for the adequate radius. The secondcontribution is to build an anisotropic metric based on theOptimally Oriented Flux (OOF) descriptor. The OOF wasfirst presented by Law et al in [11] for vessel enhancement.Its main advantage is that the disturbance introduced by theclosely located nearby structures is avoided. But they did notexploit the orientations given by the OOF. We propose to doso by using the OOF’s scalar functions and its orientation.That makes the propagation faster along the vessel’s centerline and for exact associated scale. This means that the pathlocation, orientation and scale have to be coherent with thelocal geometry of the image extracted by the OOF.

In section 2, we give some background on minimal pathmethod and Anisotropic Fast Marching. In section 3 the Op-timally Oriented Flux descriptor is presented as well as themetric construction. In section 4, we will show segmentedcoronary arteries using our method. Finally, conclusions andperspectives follow in section 5.

2. BACKGROUND ON MINIMAL PATH METHODA minimal path, first introduced in the isotropic (P does notdepend on the orientation of the path) case [7], is a pathwayminimizing the energy functional :

E(γ) =∫

γ

P(γ(s), γ′(s)

)ds (1)

where, P(γ, γ′) =√

γ′TM(γ)γ′ describes an infinitesimaldistance along a pathway γ relative to a metric tensor M(symmetric definite positive). M is defined from some fea-tures extraction in the image. A curve connecting p1 to p2

that globally minimizes the above energy (1) is a minimal pathbetween p1 and p2, noted Cp1,p2 . The solution of this mini-mization problem is obtained through the computation of theminimal action map U : Ω → R+ associated to p1 on the do-main Ω which can be a 2D, 3D or 4D domain (in the case of3D tubular structures, Ω is a 4D domain). The minimal actionis the minimal energy integrated along a path between p1 andany point x of the domain Ω:

∀ x ∈ Ω, U1(x) = minγ∈Ap1,x

∫γ

P(γ(s), γ′(s)

)ds

, (2)

where Ap1,x is the set of smooth paths linking x to p1. Thevalues of U1 may be regarded as the arrival times of a frontpropagating from the source p1 with oriented velocity relatedto the metric tensor M−1. U1 satisfies the Eikonal equation

‖∇U1(x)‖M−1(x) = 1 for x ∈ Ω, and U1(p1) = 0, (3)

where ‖v‖M =√

vT Mv. The flow lines of U1 satisfy theEuler-Lagrange equation of functional (1). Thus, the minimalpath Cp1,p2 can be retrieved with a simple gradient descenton U1 from p2 to p1 (see Fig. 1). Proof of (3) can be foundin [8].

p1

p2

p1

p2

0 100 200 300

Fig. 1. Minimal path examples on an isotropic case on theleft image. On the middle, visualization by small ellipsesof eigenvalues of a metric constant on each half side of theimage, and the minimal action map associated to the sourcepoint p1 with the minimal path Cp1,p2 on the right.

On figure 1, we show some examples of the minimal pathmethod on an isotropic case and an anisotropic one. On thefirst image of figure 1 the metric is isotropic and the potentialin the grey region is twice as low as the white one. Isolevelsets of the minimal action map associated to the source pointp1 and the minimal path Cp1,p2 are displayed. The secondimage represents a metric M. We took two constant metricsin each half side of the image with different orientations. Onthe last image, the minimal action map U1 associated to themetric M and to the source point p1 is shown. The minimalpath Cp1,p2 is found through a simple gradient descent. Theanisotropic Eikonal equation is solved using an adapted ver-sion of the Fast Marching algorithm. Descriptions and detailson the isotropic and anisotropic Fast Marching can be foundin [7, 8].

3. OPTIMALLY ORIENTED FLUX : ANANISOTROPY DESCRIPTOR

We are interested in the construction of a metric that extractsfrom the image the geometric information leading to recon-struction of vessels. This means that we wish to find an esti-mate for the local orientation and scale and a criterion on thelocal geometry to distinguish the presence of vessels from thebackground.

At the position x on an image I , the amount of the imagegradient projected along the axis v flowing out from a 3Dsphere Sr of radius r is measured as in [11],

f(x,v; r) =∫

∂Sr

((∇(G ∗ I)(x + h) · v)v) · h|h|

da, (4)

where G is a Gaussian function with a scale factor equal tothe size of the image voxel, h is the position vector along ∂Sr

and da is the infinitesimal area on ∂Sr. To detect vessels hav-ing higher intensity than the background region, one wouldbe interested in finding the vessel direction which minimizesf(x,v; r),

arg minv

f(x,v; r). (5)

Using the divergence theorem, it can be shown thatf(x,v; r) can be calculated using a simple convolution,

f(x,v; r) = vT(∂i,jG) ∗ I ∗ 1Sr

v, (6)

where (∂i,jG) is the Hessian matrix of function G and 1Sris

the indicator function inside the sphere Sr. By differentiatingthe above equation with respect to v, the solution of equa-tion (5) is in turn acquired as solving a generalized eigen-value decomposition problem. Solving the aforementionedgeneralized eigen decomposition problem gives three eigen-values, λ1(·) ≤ λ2(·) ≤ λ3(·) and three eigenvectors vi(·),i.e. λi(x; r) = f(x,vi(x; r); r) for i = 1, 2, 3. The twoeigenvectors associated to the first eigenvalues (λ1, λ2) rep-resent directions that are orthogonal to the vessel. v3 repre-sents the direction along the vessel. To handle the vessels hav-ing various radii, a multi-scale approach should be employedalong with the OOF method. In [11], Law and Chung haveproposed to normalize the OOF’s eigenvalues by the spheresurface area (4πr2) when the OOF method is incorporated ina multi-scale approach for 3D image volumes.

Li and Yezzi [10] proposed a new variant of the classi-cal, purely spatial, minimal path technique by incorporatingan extra non-spatial dimension into the search space. Thecrucial step of this method is to build an adequate metric thatdrives the propagation. Li and Yezzi [10] proposed differentisotropic potentials. The main drawback, as they mention, isthat these potentials are very parameter dependent and theydo not exploit the vessel orientation. Our main contribution isto improve Li and Yezzi method by adding to it an anisotropicformulation, and the anisotropic metric is constructed by ex-tension of the OOF descriptor presented by Law et al [11].

The 4D minimal path is found by minimizing the follow-ing energy: ∫

γ

√γ′(s)TM(γ(s))γ′(s)

ds,

where M is the 4D anisotropic metric we want to construct.It not natural to consider anisotropy on the fourth dimension,i.e the radii dimension. Thus one can decompose by block themetric M as follows :

M(x, r) =(M(x, r) 0

0 Pradii(x, r)

)(7)

where M(x, r) is a 3 × 3 symmetric definite positive matrixgiving the spatial anisotropy and Pradii(x, r) is the radii po-tential (also strictly positive).

Since the result given by the anisotropic minimal pathmethod is very dependent on the metric, results inherit ad-vantages and drawbacks of the constructed metric, thus weshould be very carful with its construction. First, let us fixconditions on the desired metric. The spatial metric M hasto be well oriented along the vessel centerline. And the radiipotential Pradii has to be small for the adequate scale for any

point of the image. Pradii corresponds to the inverse speed forthe radii dimension. Since M is symmetric definite positive,we can decompose it as follows:

M(.) =3∑

i=1

mi(.)ui(.)ui(.)T ,

where 0 < m1 ≤ m2 ≤ m3 are the eigenvalues and ui arethe associated eigenvectors. The velocity of the propagatingfront along direction ui is equal to 1/

√mi. We used the OOF

descriptor to construct the metric as follows:

M(.) =

3∑i=1

exp(

α

∑j 6=i λj(.)

2

)vi(.)vi(.)T ,

Pradii(.) = β exp(α

P3i=1 λi(.)

3

).

(8)The constants α and β are controlled by two intuitive pa-

rameters, which are the maximal spatial anisotropy ratio andradii speed ratio respectivelly.

4. RESULTSOur method is minimally interactive. First, the user has todecide if the desired vessels are darker or brighter than thebackground. So, we can consider different criteria on signsof the eigenvalues. Then the scale range [rmin, rmax], whichcorresponds to the range of radii of the vessel one wants toextract, is given by the user. Finally few points are requiredas source points or end points of the Fast Marching algorithm.We used the metric described in the previous section to findthe minimal anisotropic path (as described in section 2) be-tween two or more selected points (see figure 2). For anyselected point, the associated radius is equal to the minimalradius rmin given by the user. One figure 2, left anterior de-scending (LAD) arteries and right coronary arteries (RCA)are segmented.

One can see that the obtained radii on the principal coro-nary branches are larger than those of the secondary. Thus,our approach is robust to scale changing and bifurcations.Nevertheless, our current implementation requires huge mem-ory allocations due to the 4D and anisotropic aspects. To over-come this issue, we added a pre-processing interactive tool toselect a subvolume containing the desired vessels (see figure2). Moreover, we are working on a new implementation ofthe tubular anisotropy approach to make the memory alloca-tion dynamic and hence to benefit from the front propagationaspect of the fast marching algorithm. Besides the reductionof the computation time (which has been actually achieved),we will save on memory allocation and will have a new ver-sion of our algorithm that extract the whole coronary arteriesusing a regular PC.

5. CONCLUSIONSIn this paper we have proposed a new method for 3D tubu-lar structure extraction. Our method exploit the orientations

Fig. 2. First two images : RCA segmentation using the tubu-lar anisotropy approach shown on the whole image and onthe selected sub-volume. Second two images : LAD segmen-tation shown on the whole image and on the selected sub-volume. Only few points are required (the extremities of thepaths). The tubular anisotropy method provides the centerlineas well as vessels boundaries.

of the vessels by using the optimally oriented flux to con-struct a multi-resolution anisotropic metric that describes thevessels orientation and scales. Combining this metric withanisotropic minimal path technique, we are able to find a com-plete description of the tubular structure, i.e the center lineand the boundary. To summarize, our method is minimallyinteractive, robust to scale variations and to bifurcations. Wedeveloped a user friendly interface for coronary arteries seg-mentation and we are working on its medical validation.AcknowledgementsWe would like to thank Professor Anthony J. Yezzi, Max Wai-Kong Law and Professor Philippe Charles Douek for fruit-ful discussions. Also, Eduardo Davila for his precious helpfor the implementation of the interface. This work was par-tially supported by ANR grant SURF -NT05-2 45825. andthe France-Israel project?

6. REFERENCES[1] Y. Sato, S. Nakajima, N. Shiraga, H. Atsumi1,

S. Yoshida, T. Koller, G. Gerig, and R. Kikinis, “Three-dimensional multi-scale line filter for segmentation andvisualization of curvilinear structures in medical im-ages,” MedIA, vol. 2, no. 2, pp. 143–168, 1998.

[2] K. Krissian, “Flux-based anisotropic diffusion appliedto enhancement of 3D angiogram,” TMI, vol. 21, no. 11,pp. 1440–1442, 2002.

[3] Ro F. Frangi, Wiro J. Niessen, Koen L. Vincken, andMax A. Viergever, “Multiscale vessel enhancement fil-tering,” 1998, vol. 1496, pp. 130–137, Springer-Verlag.

[4] Cemil Kirbas and Francis K. H. Quek, “A review of ves-sel extraction techniques and algorithms,” ACM Com-puting Surveys, vol. 36, pp. 81–121, 2004.

[5] K. Siddiqi and A. Vasilevskiy, “3d flux maximizingflows,” in EMMCVPR02, 2002, p. 636 ff.

[6] Sylvain Bouix, Kaleem Siddiqi, and Allen Tannenbaum,“Flux driven automatic centerline extraction,” MedicalImage Analysis, vol. 9, no. 3, pp. 209–221, 2005.

[7] L. D. Cohen and R. Kimmel, “Global minimum for ac-tive contour models: a minimal path approach,” Int JComput Vision, vol. 24, pp. 57–78, 1997.

[8] J. A. Sethian and A. Vladimirsky, “Fast methods forthe eikonal and related hamilton- jacobi equations onunstructured meshes,” Proceedings of the NationalAcademy of Sciences, vol. 97, no. 11, pp. 5699–5703,May 2000.

[9] T. Deschamps and L.D. Cohen, “Fast extraction of min-imal paths in 3D images and applications to virtual en-doscopy,” MIA, vol. 5, no. 4, Dec. 2001.

[10] H. Li and A. Yezzi, “Vessels as 4D curves: Global min-imal 4D paths to extract 3D tubular surfaces and center-lines,” IEEE TMI, vol. 26, no. 9, pp. 1213–1223, 2007.

[11] Max W. K. Law and Albert C. S. Chung, “Three dimen-sional curvilinear structure detection using optimallyoriented flux,” ECCV, vol. 4, pp. 368–382, 2008.