newton-style optimization for emission tomographic...

Journal of Electronic Imaging 9(3), 269–282 (July 2000).

Newton-style optimization for emissiontomographic estimation

Jean-Baptiste ThibaultGeneral Electric Medical Systems

Global Technology Operations, W646Waukesha, Wisconsin 53188

E-mail: [email protected]

Ken SauerUniversity of Notre Dame

Department of Electrical EngineeringNotre Dame, Indiana 46556

E-mail: [email protected]

Charles BoumanPurdue University

School of Electrical EngineeringWest Lafayette, Indiana 47907-0501E-mail: [email protected]

bleingly

ro-

vit

L

ved

ld

vexge

ives

he

ba-tot-ical

this

lyti--ula-

ost

be-thethee-rstdtiveofn

999

aticolo

Abstract. Emission computed tomography is widely applied inmedical diagnostic imaging, especially to determine physiologicalfunction. The available set of measurements is, however, often in-complete and corrupted, and the quality of image reconstruction isenhanced by the computation of a statistically optimal estimate.Most formulations of the estimation problem use the Poisson modelto measure fidelity to data. The intuitive appeal and operational sim-plicity of quadratic approximations to the Poisson log likelihoodmake them an attractive alternative, but they imply a potential lossof reconstruction quality which has not often been studied. This pa-per presents quantitative comparisons between the two models andshows that a judiciously chosen quadratic, as part of a short seriesof Newton-style steps, yields reconstructions nearly indistinguish-able from those under the exact Poisson model. © 2000 SPIE andIS&T. [S1017-9909(00)00502-X]

1 Introduction

Statistical methods of reconstruction are widely applicain emission tomography and other photon-limited imagproblems. Unlike the relatively rigid, deterministicalbased methods such as filtered back projection~FBP!, theycan be used without modification to data with missing pjections or low signal-to-noise ratios~SNRs!. The Poissonprocesses in emission and transmission tomography inthe application of maximum-likelihood~ML ! estimation.However, due to the typical limits in fidelity of data, M

Paper M-004 received Sep. 30, 1999; revised manuscript received Nov. 15, 1accepted for publication Dec. 15, 1999.This paper is a revision of a paper presented at the Conference on MathemModeling, Bayesian Estimation, and Inverse Problems, June 1999, Denver, Crado. The paper presented there appears~unrefereed! in Proc. SPIEVol. 3816.1017-9909/2000/$15.00 © 2000 SPIE and IS&T.

e

estimates are usually unstable, and have been improupon by methods such as regularization, or maximumaposteriori probability ~MAP! estimation.1 With the choiceof convex potential functions for Markov random fie~MRF! style a priori image models, both ML and MAPreconstructions may be formulated as large scale conoptimization problems. Many approaches to this challenhave been proposed, among which popular alternathave been variants of expectation maximization~EM!,2 anapproach derived from indirect optimization through tintroduction of the notion of an unobservablecompletedataset whose conditional expectations form the algorithmicsis. EM and related methods are called on primarily duethe Poisson likelihood function, while earlier work on leassquares solutions resorted to a variety of more classnumerical methods;3–6 the motivation for formulating suchproblems in least squares seems often to have beenavailability of simple optimization tools. Linearity of theresulting estimator also opens the problem to more anacal scrutiny. Some previous work7–9 has considered the differences between the Poisson and least-squares formtions in terms of image qualities in final estimates, but mcurrent work follows the Poisson model.

We study here the visual and quantitative differencetween MAP emission tomographic estimates underPoisson model and under quadratic approximations tolog likelihood. The quadratics are derived from Taylor sries expansions of the Poisson log likelihood. As a figoal, we revisit the issue of the viability of a single, fixeapproximation based on an expansion before any iteraoptimization takes place. Although the diagnostic costany loss of quality is not clear from our work, one ca

;

al-

Journal of Electronic Imaging / July 2000 / Vol. 9(3) / 269

ayrese-malo

ofwe

thto

e tdfuratirbe

theeaarndbleisa-yeingearaghe

mctltt oo-fo

eelyateillthene

elAPtic

tionicterb-

n

r

and

torat-to-

hbe

n to

yighLlygeren

w,-ap-

nlygli-

edrith

fses aal

for-ont ofss.s-e

Thibault, Sauer, and Bouman

observe that the simplest of quadratics does not alwyield a result indistinguishable from the Poisson. We thefore propose a generalization in the form of a shortquence of global quadratic approximations whose minican be made convergent to the minimum of the Poissonlikelihood. In its limiting case, this technique is a formNewton iterations in the dimension of the image, andtherefore call the method global Newton~GN!. However,as experimental results show, after only a few updates,approximation yields final image estimates very closethose of the Poisson model. Thus we expect in practicbe able to freeze the approximation after, at most, a hanof updates. Between the refinements of the global quadlikelihood model, any form of optimization appropriate fothe highly coupled equations for the image pixels mayused. This view of, and computational approach toMAP reconstruction does not necessarily promise grcomputational savings, since optimization techniquesalready in place which converge in few iterations, awhich have per-iteration costs similar to those possiwithin GN. Rather, we hope to show that the likelihoodin practice sufficiently close to good quadratic approximtions that choices of numerical methods to solve the Basian tomographic inversion problem may be made viewthe optimization as one of solving a large system of linequations. Should a user have available software packfor quadratic problems, this may allow its application to temission problem with negligible loss of quality.

While Newton’s method would require that the probleformed by each successive approximation be solved exabefore updating the global quadratic, we suggest thasingle update of all image pixels suffices to capture mosthe per-iteration gain if a relatively fast-converging algrithm is applied. Various enhanced gradient methods,example, may be well suited to the problem.10,11Because itis itself based onlocal quadratic approximations to thPoisson log likelihood and has demonstrated relativrapid convergence, we apply a form of iterative coordindescent~ICD! to the pixel optimization phase. Thus we wdiscuss two cases of quadratic approximation below;first is global and forms the basis of GN; the second is odimensional, iteratively solving functions of single pixvalues. The latter is used both to compute the exact Mestimates below and to optimize under the global quadraof GN.

2 A Global Newton Algorithm for the MAPReconstruction Problem

2.1 Formulation of the MAP Objective

Statistical image reconstruction requires the evaluaand/or optimization of functionals viewing the probabilistlink between observations and the unknown paramethrough the log-likelihood function. For the emission prolem, X is theN-dimensional vector of emission rates,Y isthe M -dimensional vector of projection data~Poisson dis-tributed photon counts!. According to the standard emissiotomographic model, the observed photon countsyi for pro-jection i follow a Poisson distribution with parameteAi* x, whereAi* is the i th row of the projection matrixA.

270 / Journal of Electronic Imaging / July 2000 / Vol. 9(3)

s-

g

e

olc

te

-

es

yaf

r

s

s

Using the convention thatpi5Ai* x, the log likelihood maybe expressed as the sum of strictly convex continuousdifferentiable functions

logP~Y5yux!5(i

~yi log pi2pi2 log~yi ! !!. ~1!

A variety of system nonidealities, such as varying detecsensitivity, scatter under linearized approximations, andtenuation and random coincidences in positron emissionmography~PET! can easily be included in this model witminor modifications. The first three above may typicallyincorporated into the transform matrixA, while pre-estimated random coincidence rates are a simple additiothe mean.

Maximum-likelihood ~ML ! estimation methods are bconsensus poor for most emission problems, since hspatial frequency noise tends to dominate Mreconstructions.12 These high frequencies converge slowunder EM methods of optimization, well after basic imastructure is visible. Some ML estimators are therefo‘‘regularized’’ by a uniform initial image estimate and aearly termination of the optimization.13 We formulate thereconstruction instead from the Bayesian point of viewith an explicit a priori image model stabilizing the estimator, and optimization methods which are designed toproach the unique maximum of thea posterioriprobabilitydensity as rapidly as possible. Iterations are terminated owhen the estimate has stopped evolving more than negible amounts visually or quantitatively.

We will use throughout this paper the generalizGaussian MRF~GGMRF!14 as prior model to illustrate oumethods. The GGMRF model has a density function wthe form

gx~x!51

zexpH 2 1qsq ($ j ,k%PC bj ,kuxj2xkuqJ ,

whereC is the set of all neighboring pixel pairs,bj ,k is thecoefficient linking pixelsj andk, s represents the scale othe prior image, and 1

ture

s isbeteth

re-ist

ofactivthndx-vityertotoople

,

thv-omay

theem

wdd

heg-nc-toodRFgh

geatationth

e-o-oto

thisxelep

s.

ob-isPxi-

sua-Nma-the

isasicnc-en

ntoheriesionw-his

ion

isa-

..

e-alr-

Newton-style optimization

2.2 A Global Newton Method for Poisson LogLikelihood

Direct optimization of Eq.~2! can be simplified somewhaby taking advantage of the approximately quadratic natof the global log-likelihood function.15 Dealing directlywith the true Poisson during numerical optimization stepthen unnecessary, and efficient optimization canachieved with a variety of well known methods to computhe descent steps. This is not necessarily motivated bygoal of reducing the number of numerical operationsquired for the reconstructions; optimization methods exwhich converge to the MAP solution rapidly in termsiteration counts regardless of whether we solve the exPoisson or a quadratic approximation. These alternamethods can be applied with nearly the same cost toPoisson likelihood as to the quadratics. But the unbouness of the Poisson log likelihood at the origin, for eample, may prove a nuisance, and enforcement of positiconstraints is more easily understood and analyzed undwell-behaved, convex quadratic cost functional. The aumatic observation of positivity in pixel values seemshave been an important motivation in the widespread adtion of the EM algorithm for emission tomography. WhiEM has been accelerated,16 generalized to the MAPproblem,17 reformulated as a faster sequential algorithm18

and speeded~at the cost of reliable convergence! in theordered subsets EM approach,19 its derivation and evendefinition are less well understood by most users thanprinciples of optimizing a quadratic or, equivalently, soling systems of linear equations. It is our hope that freedto cogitate on this optimization as a quadratic problem mlead to greater insight into, and greater exploitation of,unique characteristics of the tomographic inverse probl

Because edge-preservinga priori models may havehighly nonquadratic cost functionals, all the work beloincludes the exact log priors. Most MRF image models arelatively little to the per-iteration computational cost in ttomographic problem, which is dominated by the lolikelihood component. They also serve as stabilizing futionals for the MAP estimator, and will therefore tendmute, not amplify, the error due to use of the likelihoapproximation. In the case of the popular Gaussian Mof course, the problem remains entirely quadratic. Thouthe edge-preserving priors have quantitative advantathe linearity of the reconstruction as a function of the din the Gaussian case still may facilitate easier interpretaof errors than the nonlinear case, and we expect thatGaussian will remain a common choice as prior.

We first compute the leading two terms of a Taylor sries expansion of the log-likelihood function in the singram domain, where independence of the Poisson phcounts simplifies analysis. Using Eq.~1!, the gradient andthe diagonal Hessian evaluated atp5 p̂ have entries

] logP~Y5yux!]pi

Up5 p̂

5211yip̂i

~3!

]2 logP~Y5yux!]~pi !

2 Up5 p̂

52yi

~ p̂i !2 . ~4!

e

tee-

a-

-

e

.

,

s,

e

n

Different approaches may be used to take advantage ofapproximation. We may evaluate it once before any piupdate atp̂5y, using the raw measurement data, and kethis approximation fixed during the convergence proces15

In this case, an approximation to the log likelihood is

logP~Y5yux!'2 (i PS1

1

2yi~yi2Ai* x!

2

1 (i PS0

Ai* x1c~y!, ~5!

where S05$ i ,yi50%, S15S2S0 . Since this approxima-tion converges to the exact log likelihood as( i yi

21/2,15 weexpect the error in the approximation to be far less prlematic in high-count data than in low. The FBP imageperhaps the most practical, low-cost starting point for MAestimation. Therefore one may also profit from an appromation at the pointp̂5AxFBP.

A potentially more broadly applicable method involveupdating the point of expansion after solving the given qdratic problem, in the manner of Newton’s method indimensions. Newton’s search takes a quadratic approxition to the objective function at each step to computenext element of the sequential series

xk115xk2@¹2f ~xk!#21•¹ f ~xk!. ~6!

The computationally impractical inversion of the Hessianoften approximated through methods suchpreconditioning.20 Convergence is also locally quadratbut generally not assured for nonconvex objective futions, and improvements on the original idea have bedeveloped to make Newton’s method globally robust.21,22

Applying a similar methodology to the MAP estimatioproblem in Eq.~2!, we may use the previous estimatesglobally approximate the log likelihood as quadratic for tnext image iteration. The parameters of the Taylor seexpansion are then evaluated from the forward projectof the current image estimate. Due to its relation to Neton’s method in the dimension of the image, we refer to talgorithm as global Newton~GN!. Let p̂k be the forwardprojection of the image estimate obtained after optimizaton the approximate objective. The expansion pointp̂k iskept fixed for all pixel updates in a stage of GN, thenupdated to form the next quadratic likelihood approximtion. It is computed from the initial image~typically theFBP reconstruction! for the first iteration. Evaluating Eqs~3! and~4! at p5 p̂k, we obtain a new approximation to Eq~1!

logP~Y5yux!'(i 51

M S yip̂ik 21D ~yi2Ai* x!1(

i 51

M

2yi

2~ p̂ik!2

~yi2Ai* x!21c~y!. ~7!

To best exploit the nearly quadratic form of the log liklihood, we seek to update only to a point where the finGN reconstructed image will show no substantial diffe


ra-theor

r inonsaroluant toallveionateo-

tom-up-

noofsi-

nch

nglso

er-

-geizetlymon-dsiae-

nnd

di-

li-

e

hns

r a

xel

-Ith byof

e.is-

xi-ua-

eh

li-

he’s

m-n

. A–tct,ta-hod


ence from the exact MAP estimate, terminating the itetions of GN early in the sequence. In Newton’s method,members of this series of quadratic problems each in thepose a computational load similar to the direct estimatoEq. ~2!. Therefore, the complexity of a precise realizatiof GN appears to be the product of the number of necesupdates of the global quadratic and the cost of each stion. As discussed below, however, a single iteration ofefficient method on a global quadratic appears sufficienwarrant the next step in GN, eliminating this additioncost. Though our implementation does not exactly soeach global quadratic before computing a new expansthe updates rapidly bring the minimum of the approximobjective close to the exact MAP solution point. The prjection estimatep̂k may then be fixed, and the solutionthe final problem form may be refined to essentially coplete convergence if desired. The necessary number ofdates ofp̂k may be determined by off-line training appropriate to the SNRs typical of the given setting.

2.3 Computation of Estimates via ICD

Under formulations such as Eqs.~2! or ~7! large-scale con-vex optimization must be solved. Such problems aredifficult, but the speed of convergence and simplicityadaptation to the addition of the regularizing term and potivity constraints in Eq.~2! are important considerations ipractice. A variety of techniques may be applied for eaGN step, including all classic quadratic~constrained! opti-mization methods.~As mentioned above, edge-preserviprior models may add a nonquadratic term and may aaffect convergence behavior, but typically add minor piteration computational cost.! In light of all the above fac-tors, we find types of iterative coordinate descent~ICD!well suited to the problem. The ICD algorithm is implemented by sequentially updating each pixel of the imaWith each update, the current pixel is chosen to minimthe MAP cost function. The ICD method can be efficienapplied to the log-likelihood expressions resulting frophoton-limited imaging systems, is demonstrated to cverge very rapidly~in our experiments typically 5–10 iterations! when initialized with the FBP reconstruction, aneasily incorporates convex constraints and non-Gausprior distributions. While use of approximate second drivative ~Hessian! information in this optimization has beeintroduced in the form of preconditioners for gradient aconjugate gradient algorithms,10,11,16 ICD’s greedy updateuses the exact local second derivative directly in onemension.

Using the exact expression of the emission log likehood in the MAP estimator, the ICD update of thej th pixelis

xjn115arg min

l>0H (

i 51

M

@Ai j l2yi log~Ai j ~l2xjn!1Ai* x

n!#

11

qsq (kPNjbj ,kul2xk

nuqJ , ~8!whereNj is the set of pixels neighboringxj . In this casexnand xn11 differ at a single pixel, so a full update of th


y

y-

,

-

t

.

-

n

image requires that Eq.~8! be applied sequentially at eacpixel. Following each pixel update, the forward projectioAxn are corrected for the differencexj

n112xjn . Since n

indexes single pixel updates applied to optimization undefixed indexk for GN approximations,$xk% could be writtenas a subsequence of$xn%.

Rather than solving the exact equation for each piresulting from Eq.~2!, the ICD/Newton–Raphson~ICD/NR! algorithm15 exploits the approximately quadratic nature of the log likelihood to reduce computation time.uses a technique similar to the Newton–Raphson searclocally applying a second order Taylor series expansionthe log likelihood as a function of the single pixel valuWe retain, however, the exact expression for the prior dtribution, because the prior term is often not well appromated by a quadratic function. Should this function be qdratic, we find the exact solution in a single step. Wemphasize that thislocal quadratic approximation for eacpixel update is a separate process from theglobal approxi-mation to the log likelihood of the GN approach. Letu1andu2 be the first and second derivatives of the log likehood evaluated for the current pixel valuexj

n . Using theNewton–Raphson-type update, the new pixel value is

xjn115arg min

l>0H u1~l2xjn!1 u22 ~l2xjn!2

11

qsq (kPNjbj 2kul2xk

nuqJ . ~9!This equation may be solved by analytically calculating tderivative and then numerically computing the derivativeroot. With pi

n as the current forward projections, the paraeters for the update equations for emission data are the

u15(i 51

M

Ai j S 12 yipinD , ~10!u25(

i 51

M

yi S Ai jpin D2

. ~11!

We simply choose a half-interval search to solve Eq.~9!since the function being rooted is monotone decreasingfull iteration consists of applying a single NewtonRaphson update to each pixel inx. We have observed thain all cases, the convergence of ICD/NR is stable. In fawe have shown that a small modification in the compution of u2 guarantees the global convergence of the metwith any strictly convex prior.23

Using an approximation such as GN, with Eq.~7! inplace of the exact Poisson log likelihood in Eq.~2!, thecomputation of the update of thej th pixel is realized as inEq. ~9! with new expressions foru1 andu2

u15(i 51

M

Ai j S 11 yip̂ik S pin

p̂ik 22D D , ~12!

at-

qs.tohisgeeivere

andts.nd

i-ua-ua-in

ionxi-

tengesor-erxi

cageedn-

ndotednceonran

pe-

adtison

il-aptof

for

nuecua

atetly

veor-ias

onlps

theheod

n-prior

allyial-actsenns,the

by

rof


u25(i 51

M

yi S Ai jp̂ik D2

. ~13!

We label this form of coordinate descent ICD/GN. Upding this quadratic approximation at each pixel~replacingp̂i

k

with pin! would reduce to the exact solution case of E

~10! and ~11!, which means that ICD/GN reducesICD/NR in this case. In the present ICD/GN, we keep tapproximation fixed for a whole iteration through the imabefore updating it. Powell20 proved the global convergencof an algorithm of this type when the Hessian is positdefinite everywhere in an open convex set. This corsponds geometrically to strict convexity as in our case,we have observed stable convergence in all experimen

Computation time is approximated by the multiplies adivides required to computeu1 andu2 . Both in ICD/NR ofSec. 2.3, and in Eqs.~12! and ~13!, noting M0 the numberof nonzero projections, this results in 4M0N operations perfull image update with the appropriate storage ofyi /( p̂i

k)2

and pn. Therefore, ICD/GN and ICD/NR require approxmately the same time for one iteration, including reevaltion of the expansion point. If we terminate these evaltions,u2 is constant. In addition, both are also equivalentterms of the number of indexings through the projectmatrix A, each requiring two, and the use of the appromation is not computationally more expensive.

The optimal scan pattern for the sequential pixel updaof the greedy ICD algorithm is not obvious, since nothiin the information carried by the sinogram dictates the border in which the image pixels should be visited. Theder of pixel updates affects convergence speed, and thfore may warrant study to determine the best method. Lecographic scans which iterate in horizontal, then verticoordinates every other iteration have advantaanalytically,24 but in our experiments here, simple repeathorizontal scans performed slightly better. Improved covergence speed also seems to be achieved using a rascan pattern.25 Unless otherwise noted, all results presenhere are from random patterns which visit each pixel oper scan. We present below comparisons in objective cvergence between a simple lexicographic pattern and adom pattern.

3 Experimental Performances

Three different sets of data have been used to test theformance of MAP-type estimation with the ICD/GN algorithm. We first realized simulations with a synthetic hephantom in a 2003200 mm field with a total photon coun'3.03106. The original, with the FBP reconstruction,shown in Fig. 1. In addition, real medical single photemission computed tomography~SPECT! data of a humanthorax from T99 sestamibi heart perfusion was used tolustrate a medical application. Reconstructions coverproximately 3203256 mm, with a total photon coun'1.53105 in this case. Finally, we considered a sectionthe Derenzo phantom from PET data to study the permance of the algorithm in a low SNR case ('8.83104

counts!. The Derenzo estimates were corrected for atteation and detector sensitivity by means of a map of corrtion factors included in the Bayesian model. The atten

-

s

t

e--ls

m

--

r-

-

-

---

tion image itself was a MAP transmission image estimfor the phantom. The projection matrix was subsequenmodified to incorporate the corrections as multiplicatifactors. We also included a simple additive model to crect for random coincidences by adding an estimated bof the emission counts. In addition to correcting photstatistics for random coincidences, this additive factor hestabilize the convergence of the objective by movingunbounded points of the likelihood function outside tconstraint region. We chose an eight-point neighborhosystem with normalized weights for the GGMRF, and ivestigated results for both Gaussian and non-Gaussianmodels. ML parameter estimation26 provided the values ofs in the first two cases, while the last was chosen manufor best visual appearance. All reconstructions were initized with the FBP image, and all estimates labeled ‘‘exMAP’’ are computed via ICD/NR with random pixel scanfor at least 100 iterations. Independently of the chosnumber of updates of the global quadratic approximatioICD/GN ran 30 and 40 iterations for the PET data, andhead and heart data, respectively.

In each case, we have terminated the GN updatesfixing the quadratic approximation at a point we labelp̄.Following arrival at p̄5 p̂k, where p̂k corresponds to theforward projection of the image estimate obtained aftekiterations of the ICD/GN algorithm, the expansion point

Fig. 1 Head phantom used for reconstruction comparisons. (a)Original phantom, (b) FBP reconstruction.


ingte

tio(

tion

pe

ap-sion

int


the quadratic approximation is kept fixed for the remainnumber of iterations. Referring to the two cases presenin Sec. 2.2, the parameters of the quadratic approximamay be computed directly from the measurement datap̄5y) and kept fixed, as in Eq.~5!, or from the forwardprojection of the FBP image and updated at each itera( p̄5 p̂k5Axk), as in Eq.~7!, until the decision is made tostop the updates. In Figs. 2–7, we can see small but

Fig. 2 Reconstructions of head phantom with Gaussian prior. (a)Exact MAP image, Gaussian prior and s50.58; (b) Estimate withquadratic log-likelihood approximation fixed at p̄5y; (c) ICD/GN re-sult for quadratic approximation fixed after two iterations (p̄5p̂2

5Ax2).


dn

r-

ceptible differences between the exact image and itsproximation for p̄5y @~a!, ~b!#, whereas those differenceare not visible after updating the quadratic approximatonly a small number of times@~a!, ~c!#. It is interesting todetermine the smallest number of evaluations of the po

Fig. 3 MAP reconstruction with GGMRF, Gaussian prior (q52.0)and s50.58. (a) Error between original phantom and MAP recon-struction; (b) difference between MAP and estimate using quadraticfixed at p̄5y; (c) difference between MAP and estimate using qua-dratic fixed at p̄5p̂25Ax2.

ceN.lv-ra-

-

ET


of expansion that is required to yield negligible differenbetween the images computed with ICD/NR and ICD/GKeeping the approximation fixed thereafter allows for soing the simple quadratic problem for the remaining itetions. A set of reconstructions with changingp̄5 p̂k showedthat after only two iterations~three evaluations of the expansion point, i.e.,p̄5Ax2!, the resulting final imagesshow essentially no difference from the exact estimate@~a!,~c!, ~e!#. For low SNR cases as in our SPECT and P

Fig. 4 Reconstructions of a head phantom with non-GaussianGGMRF prior (q51.1), s50.25. (a) Exact MAP image; (b) estimatewith fixed quadratic approximation evaluated at p̄5y; (c) ICD/GNresult with fixed approximation after two iterations (p̄5p̂25Ax2).

Fig. 5 MAP reconstruction with GGMRF, q51.1, s50.25. (a) Errorbetween original phantom and MAP reconstruction; (b) differencebetween MAP and estimate using quadratic at p̄5y; (c) differencebetween MAP and estimate from quadratic using p̄5p̂25Ax2.



Fig. 6 Reconstructions of SPECT data with Gaussian prior and s50.028. (a) Exact MAP image; (b) estimate using quadratic at p̄5y; (c)ICD/GN result for two updates of the expansion point (p̄5p̂25Ax2); (d)(e) differences between (a) and (b),(c), respectively. Data courtesy ofT. S. Pan and M. King, University of Massachusetts.

enid

st

naesan

er-an-ratetednat-ete

data, the algorithm uses a minimum pixel value expontially decreasing to zero with iterations, in order to avofrequently hitting the positivity constraint during the firiterations.

Table 1 shows error measures relative to the origiimage, to provide context for the magnitude of differencbetween the MAP and ICD/GN estimates of the head ph


-

l

-

tom appearing in Table 2. The magnitudes of the diffences displayed in Table 2 for different numbers of expsion updates show that the results are sufficiently accuafter one or two updates of the expansion first compufrom the FBP image for ICD/GN to be very similar to aexact MAP reconstruction. It also indicates that with upding the expansion once per iteration during the compl


Fig. 7 Reconstructions of SPECT data with non-Gaussian GGMRF prior, q51.1 and s50.017. (a) Exact MAP image; (b) estimate usingquadratic at p̄5y; (c) ICD/GN result for two updates of the expansion point (p̄5p̂25Ax2); (d), (e) differences between (a) and (b), (c),respectively. Data courtesy of T. S. Pan and M. King, University of Massachusetts.

asdyfresidtodthes

c-heta-ofa-

veti-

reconstruction process, the ICD/GN image appears toymptotically converge to the exact estimate. Detailed stushows that the largest differences occur about the highquency regions of the image, at the boundaries and outthe object, and especially in the regions where the phocounts are low. This relates to the studies in Refs. 15 anshowing that the larger the photon counts, the betterquadratic approximation to the log likelihood. This do

-

-e

n8e

not, however, visibly affect the quality of the reconstrution. In fact, differences computed only in the region of tobject drop by a factor of 10 compared with the compution on the whole image. In the low SNR PET exampleFig. 8, we see similarly that after a total of three evalutions of the Taylor series for the log likelihood, we achiea result nearly indistinguishable from the true MAP esmate computed by ICD/NR.


ngthe

sie

theth

he-N

the

adeforod

ed.-

hate-hi

ra-le

oveed ahethemee

er-is

ve

e

-terall/

somby

ten-an-ntffersnzoeritedifi-ostx it-can.e11

detheAPix-tiveic

tot isThetheve,to

feroflly

e athe

on-p-,


Interestingly, the quantitative results indicate that fixithe quadratic expansion from the forward projection ofFBP image is not necessarily better than using the rawnogram withp̄5y. The high frequency noise present in thoriginal sinogram is spread over the entire image byFBP reconstruction, and may therefore be spread overentire sinogram resulting from the forward projection of tFBP image. In contrast, the vectory remains as the standard for the estimate’s fidelity to the data throughout Gupdates. Therefore, it is plausible that the minimum ofresulting GN quadratic approximation atp̄5 p̂05AxFBPmay be farther from the MAP estimate than the one mat p̄5y. The use of the FBP image as starting pointICD/GN has several motivations. The FBP image is a gofirst estimate of image densities which is quickly computIn addition, the ICD algorithm is very efficient in converging the high frequencies in the image, while somewslower for low frequencies; the FBP image offers low frquency components already near their optimal value. T

Table 1 Error in estimate relative to the original head phantom.

Reconstructed image MSD MAD MAX

Head phantom q52 s50.584

p̄5y 0.0442 0.1076 2.2722

p̄5p̂05AxFBP 0.0456 0.1085 2.4000

p̄5p̂15Ax1 0.0434 0.1073 2.2793

p̄5p̂25Ax2 0.0428 0.1070 2.1711

p̄5p̂405Ax40 0.0428 0.1070 2.1702

ICD/NR exact MAP estimate 0.0428 0.1070 2.1702

Table 2 Difference measurements between MAP and ICD/GN im-ages for varying numbers of evaluations of the global quadratic ap-proximation.

Expansion point MSD MAD MAX

Head phantom q52 s50.584

p̄5y 4.66331024 8.41331023 5.16031021

p̄5p̂05AxFBP 8.39331024 6.0031023 5.16031021

p̄5p̂15Ax1 1.32531024 2.20831024 2.32731021

p̄5p̂25Ax2 1.42831027 6.68531025 1.4531022

SPECT data q52 s50.0283

p̄5y 3.96931025 2.57231023 5.8831022

p̄5p̂05AxFBP 2.16531025 1.81331023 4.1731022

p̄5p̂15Ax1 3.77231026 6.35831024 2.4331022

p̄5p̂25Ax2 5.31931027 2.16531024 1.1831022

Derenzo PET data q52 s50.10

p̄5y 4.46831024 6.14531023 2.73631021

p̄5p̂05AxFBP 9.84031025 3.10531023 1.51031021

p̄5p̂15Ax1 2.49031025 1.36931023 1.02431021

p̄5p̂25Ax2 5.61931026 4.91931024 5.8731022


-

e

s

makes the FBP image a good starting point for most itetive estimation algorithms. However, it is quite conceivabthat in some instances of GN, one might still usex0

5xFBP with p̂05y in place ofAxFBP.

The convergence plots of Fig. 9 correspond to the abresults. They were obtained for Gaussian priors and usregular scan pattern in horizontal coordinates first. Tplots are similar in the case of non-Gaussian priors. Forhead and the Derenzo phantoms, ICD/GN exhibits the sabehavior as ICD/NR, indicating experimentally that thglobal quadratic approximations do not affect the convgence properties of the ICD algorithm. Whatever choicemade for fixing the quadratic approximation, the objectifunction converges quickly~typically about five iterations!.However, it appears in the SPECT data plot that if wevaluate the quadratic approximation atp̄5y and keep itfixed thereafter~‘‘0’’ plots !, the limiting a posterioriprob-ability is significantly different from the exact MAP objective. Updating the quadratic approximation only once afits first evaluation from the FBP image leaves only smdifferences in the objective values of ICD/GN and ICDNR.

With the exception of the plots in Fig. 9, all the figurepresented thus far result from ICD pixel updates in randorder. The patterns analyzed in Ref. 24 were regular,lines or by columns, but recent results have shown a potial for improvement in convergence speed with these rdomly ordered scans.25 Figure 10 presents the improvemein convergence speed that a random update pattern oover a regular lexicographic scan in the case of the Derephantom. A simple linear congruential random numbgenerator guarantees that each pixel in the image is visonce and only once per iteration. The gain here is signcant, since after only two iterations the objective has almconverged when the random scan is used, whereas sierations are necessary when using the lexicographic sIn addition, it is interesting to study the influence of thscan pattern on the ICD/GN results. Figures 10 andcompare the image estimates forp̄5 p̂2 when using thelexicographic or random update pattern. The magnituplots show that the random pattern eliminates some ofdifferences between the GN estimate and the exact Mimage. With a sequential update of the pixels, the first pels in the image tend to be overcorrected since the objecfunction is greedily optimized at each step. A lexicographscan, such as in Figs. 10~b! and 11~a!, therefore createsmore low frequency artifacts in the image and tendsconcentrate those pixels on whichever side of the objecupdated first, whereas a random pattern scatters them.differences in the above figures appear in fact to besuperposition of both the overcorrection discussed aboand the low frequency components which take longerconverge with ICD. Thus the random pattern may offaster convergence of both the global approximationsGN, and the sweeps of the image under ICD. A potentiainteresting variation on the random update would involvnonuniform spatial distribution of updates dependent oncurrent image estimate.

These results suggest that any algorithm which cverges rapidly for the first few iterations might quickly suply a final global quadratic likelihood approximation


Fig. 8 Reconstructions of PET data with Gaussian prior. (a) Original Derenzo phantom; (b) MAP reconstruction with s50.1; (c) ICD/GN resultfor p̄5y; (d) ICD/GN result for quadratic expansion fixed after two iterations (p̄5p̂25Ax(2)); (e), (f) differences between MAP image of (b) andresults in (c) and (d), respectively. Data courtesy of G. Hutchins, Indiana University.

hmd,ng

.g.er-

a--ea-an

which could then be attacked by any numerical algoritwell suited to quadratic objectives. An initially quite rapithough nonconvergent, form of EM which cycles amosubsets of data, known as ordered subsets EM~OSEM!,19

might be used for the initial estimates, and another, econjugate gradient or ICD, might be applied to convgence of the resulting fixed quadratic problem.25

,

4 Conclusion

In Bayesian tomography, the substitution of an approximtion to the log-likelihood function allows simpler optimization of the MAP objective without computation under thPoisson model. After only a few updates of the global qudratic, a coordinate descent method rapidly converges to


sti-to

ingnyre-ratic


estimate nearly indistinguishable from the exact MAP emate. For emission tomographic problems of medium

Fig. 9 Convergence of objective vs iterations for (a) the head phan-tom, (b) the SPECT data, and (c) the PET data, for various numbersof evaluations of the expansion point. Convergence plots obtainedfrom ordered pixel updates in horizontal coordinates first in ICD. ‘‘0’’indicates p̄5y, ‘‘1’’ indicates p̄5p̂05AxFBP , ‘‘2’’ indicates p̄5p̂

1

5Ax1, etc.


high signal-to-noise ratios, our results suggest that viewthe log likelihood as quadratic may be adequate for avisual interpretations, and would open the problem tosearchers and practitioners more accustomed to quad

Fig. 10 Difference in convergence behavior between lexicographicscan in horizontal coordinates first, and randomly ordered pixel up-dates in ICD, for the Derenzo phantom with Gaussian prior. (a) Con-vergence of log a posteriori probability in both cases. Residual dif-ference between true MAP estimate and result from quadraticapproximation fixed after two iterations (p̄5p̂25Ax2) under (b)regular scan pattern; (c) random scan pattern.

on-tlea-erapr-nd

da

tion

r

om

a-

e-

r

ruc-

M

o-

ra-,

re-re-

tsion

es

ing

del

to-

or

r

EMg.

ing

blytruc-

ive

n of,’’

on


constrained optimization than photon-limited image recstruction. Even in low-count problems, it appears that litappreciable quality would be sacrificed in optimizing a qudratic log-likelihood approximation fixed very early in thprocess. These claims all assume the use of relativelyidly converging techniques for the first few iterations. Futher research will evaluate differences between ICD aOSEM for the initial stages.

Acknowledgment

This work was supported by the National Science Fountion under Grant No. CCR97-07763.

References

1. S. Geman and D. McClure, ‘‘Bayesian image analysis: An applicato single photon emission tomography,’’ inProc. Statist. Comput.Sect. Amer. Stat. Assoc., pp. 12–18, Washington, DC~1985!.

Fig. 11 Residual difference between true MAP estimate andICD/GN result from quadratic approximation fixed after two itera-tions (p̄5p̂25Ax2) under (a) lexicographic scan pattern; (b) randomscan pattern. (Head phantom with Gaussian prior, q52.0, s50.58.)

-

-

2. L. Shepp and Y. Vardi, ‘‘Maximum likelihood reconstruction foemission tomography,’’IEEE Trans. Med. ImagingMI-1 , 113–122~1982!.

3. R. Gordon and G. Herman, ‘‘Three-dimensional reconstruction frprojections: A review of algorithms,’’ inInternational Review of Cy-tology, G. Bourne and J. Danielli, Eds., Vol. 38, pp. 111–151, Acdemic, New York~1974!.

4. G. T. Herman and A. Lent, ‘‘A computer implementation of a Baysian analysis of image reconstruction,’’Inf. Control. 31, 364–384~1976!.

5. E. Artzy, T. Elfving, and G. T. Herman, ‘‘Quadratic optimization foimage reconstruction,’’Comput. Graph. Image Process.11, 242–261~1979!.

6. S. L. Wood and M. Morf, ‘‘A fast implementation of a minimumvariance estimator for computerized tomography image reconsttion,’’ IEEE Trans. Biomed. Eng.BME-28, 56–68~1981!.

7. B. Tsui, E. Frey, and G. Gullberg, ‘‘Comparison between ML-Eand WLS-CG algorithms for SPECT image reconstruction,’’IEEETrans. Nucl. Sci.38, 1766–1772~1991!.

8. J. Fessler, ‘‘Hybrid Poisson/polynomial objective functions for tmographic image reconstruction from transmission scans,’’IEEETrans. Image Process.4, 1439–1450~1995!.

9. P. Koulibaly,Regularisation et Corrections Physiques en Tomogphie d’Emission, PhD thesis, University of Nice-Sophia AntipolisFrance, Oct. 1996.

10. N. Clinthorne, T. Pan, P.-C. Chiao, W. Rogers, and J. Stamos, ‘‘Pconditioning methods for improved convergence rates in iterativeconstruction,’’IEEE Trans. Med. Imaging12, 78–83~1993!.

11. E. Ü. Mumcuoǧlu, R. Leahy, S. R. Cherry, and Z. Zhou, ‘‘Fasgradient-based methods for Bayesian reconstruction of transmisand emission pet images,’’IEEE Trans. Med. Imaging13, 687–701~1994!.

12. D. Snyder and M. Miller, ‘‘The use of sieves to stabilize imagproduced with the EM algorithm for emission tomography,’’IEEETrans. Nucl. Sci.NS-32, 3864–3871~1985!.

13. J. Llacer and E. Veklerov, ‘‘Feasible images and practical stopprules for iterative algorithms in emission tomography,’’IEEE Trans.Med. Imaging8, 186–193~1989!.

14. C. A. Bouman and K. Sauer, ‘‘A generalized Gaussian image mofor edge-preserving map estimation,’’IEEE Trans. Image Process.2,296–310~1993!.

15. C. A. Bouman and K. Sauer, ‘‘A unified approach to statisticalmography using coordinate descent optimization,’’IEEE Trans. Im-age Process.5, 480–492~1996!.

16. L. Kaufman, ‘‘Implementing and accelerating the EM algorithm fpositron emission tomography,’’IEEE Trans. Med. ImagingMI-6 ~1!,37–51~1987!.

17. A. D. Pierro, ‘‘A modified expectation maximization algorithm fopenalized likelihood estimation in emission tomography,’’IEEETrans. Med. Imaging14~1!, 132–137~1995!.

18. J. Fessler and A. Hero, ‘‘Complete data spaces and generalizedalgorithms,’’ in Proc. IEEE Int’l Conf. on Acoust., Speech and SiProc., Vol. IV, pp. 1–4, Minneapolis, Minnesota~April 1993!.

19. H. Hudson and R. Larkin, ‘‘Accelerated image reconstruction usordered subsets of projection data,’’IEEE Trans. Med. Imaging13,601–609~1994!.

20. J. Dennis and R. Schnabel,Numerical Methods for UnconstrainedOptimization and Nonlinear Equations, Prentice–Hall, EnglewoodCliffs, NJ ~1983!.

21. D. Luenberger,Introduction to Linear and Nonlinear Programming,Addison–Wesley, Reading, MA~1973!.

22. S. Jacoby, J. Kowalik, and J. Pizzo,Iterative Methods for NonlinearOptimization Problems, Prentice–Hall, Englewood Cliffs, NJ~1972!.

23. S. Saquib, J. Zheng, C. A. Bouman, and K. D. Sauer, ‘‘Provaconvergent coordinate descent in statistical tomographic reconstion,’’ in Proc. IEEE Int’l Conf. on Image Proc., Vol. II, pp. 741–744, Lausanne, Switzerland~September 1996!.

24. K. Sauer and C. A. Bouman, ‘‘A local update strategy for iteratreconstruction from projections,’’IEEE Trans. Signal Process.41~1993!.

25. J. Bowsher, M. Smith, J. Peter, and R. Jaszczak, ‘‘A comparisoOSEM and ICD for iterative reconstruction of SPECT brain imagesJ. Nucl. Med.39, 79P~1998!.

26. C. A. Bouman and K. Sauer, ‘‘Maximum likelihood scale estimatifor a class of Markov random fields,’’ inProc. IEEE Int’l Conf. onAcoust., Speech and Sig. Proc., Vol. 5, pp. 537–540, Adelaide, SouthAustralia ~April 1994!.



Charles A. Bouman received a B.S.E.E.degree from the University of Pennsylvaniain 1981, and a MS degree in electrical en-gineering from the University of Californiaat Berkeley in 1982. From 1982 to 1985, hewas a staff member in the Analog DeviceTechnology Group at the MassachusettsInstitute of Technology, Lincoln Laboratory.In 1987 and 1989, he received MA andPh.D. degrees in electrical engineeringfrom Princeton University under the sup-

port of an IBM graduate fellowship. In 1989, he joined the faculty ofPurdue University where he currently holds the position of AssociateProfessor in the School of Electrical and Computer Engineering.Professor Bouman’s research interests include statistical imagemodeling and analysis, multiscale processing, and the display andprinting of images. He is particularly interested in the applications ofstatistical signal processing techniques to problems such as fastimage search and inspection, tomographic reconstruction, anddocument segmentation. Professor Bouman has 20 full journal pub-lications, over 50 conference publications, and two awarded pat-ents. He has performed research for numerous government andindustrial organizations including National Science Foundation, U.S.Army, Hewlett-Packard, NEC Corporation, Apple Computer, Xerox,and Eastman Kodak. From 1991–1993, he was also an NEC Fac-ulty Fellow. Professor Bouman is a member of IEEE, SPIE, andIS&T professional societies. He has been both chapter chair andvice chair of the IEEE Central Indiana Signal Processing Chapter,and an associate editor of the IEEE Transactions of Image Process-ing. He is currently a member of the IEEE Image and Multidimen-sional Signal Processing Technical Committee.


Ken Sauer was born in Decatur, Indiana.He received the BSEE in 1984 and theMSEE in 1985 from Purdue University,West Lafayette, IN, and the Ph.D. fromPrinceton University in 1989 as an AT&TFoundation fellow. He is currently an Asso-ciate Professor of Electrical Engineering atthe University of Notre Dame. ProfessorSauer is involved in research of statisticalmethods for tomographic image estimationand other nondestructive evaluation prob-

lems, numerical optimization and stochastic image modeling.

Jean-Baptiste Thibault was born inRouen, France, in 1975. He graduatedfrom Ecole Superieure d’Electricite, Gif-sur-Yvette, France, and completed the MSdegree in electrical engineering from theUniversity of Notre Dame, Indiana, in 1998.He is currently working in Global Technol-ogy Operations at General Electric MedicalSystems. His research interests includenumerical methods, Bayesian estimation,and applications to statistical image recon-struction.

newton-style optimization for emission tomographic...

Documents