Applied Mathematical Sciences Volume 155

Editors S.S. Antman lE. Marsden L. Sirovich

Advisors 1.K. Hale P. Holmes 1. Keener 1. Keller B.l. Matkowsky A. Mielke C.S. Peskin K.R.S. Sreenivasan

Springer New York Berlin Heidelberg Hong Kong London Milan Paris Tokyo

Applied Mathematical Sciences

I. John: Partial Differential Equations. 4th ed. 2. Sirovich: Techniques of Asymptotic Analysis. 3. Hale: Theory of Functional Differential Equations.

2nd ed. 4. Percus: Combinatorial Methods. 5. von MiseslFriedrichs: Fluid Dynamics. 6. Freiberger/Grenander: A Short Course in

Computational Probability and Statistics. 7. Pipkin: Lectures on Viscoelasticity Theory. 8. Giacaglia: Perturbation Methods in Non-linear

Systems. 9. Friedrichs: Spectral Theory of Operators in Hilbert

Space. 10. Stroud: Numerical Quadrature and Solution of

Ordinary Differential Equations. II. Wolovich: Linear Multivariable Systems. 12. Berkovitz: Optimal Control Theory. 13. Bluman/Cole: Similarity Methods for Differential

Equations. 14. Yoshizawa: Stability Theory and the Existence of

Periodic Solution and Almost Periodic Solutions. 15. Braun: Differential Equations and Their

Applications. 3rd ed. 16. Lefschetz: Applications of Algebraic Topology. 17. CollatzIWetterling: Optimization Problems. 18. Grenander: Pattern Synthesis: Lectures in Pattern

Theory. Vol. I. 19. MarsdenIMcCracken: Hopf Bifurcation and Its

Applications. 20. Driver: Ordinary and Delay Differential

Equations. 2l.CourantiFriedrichs: Supersonic Flow and Shock

Waves. 22. RouchelHabetslLaloy: Stability Theory by

Liapunov's Direct Method. 23. Lamperti: Stochastic Processes: A Survey of the

Mathematical Theory. 24. Grenander: Pattern Analysis: Lectures in Pattern

Theory. Vol. ll. 25. Davies: Integral Transforms and Their

Applications. 2nd ed. 26. Kushner/Clark: Stochastic Approximation

Methods for Constrained and Unconstrained Systems.

27. de Boor: A Practical Guide to Splines: Revised Edition.

28. Keilson: Markov Chain Models-Rarity and Exponentiality.

29. de Veubeke: A Course in Elasticity. 30. Sniatycki: Geometric Quantization and Quantum

Mechauics. 31. Reid: Sturmian Theory for Ordinary Differential

Equations. 32. MeisIMarkowitz: Numerical Solution of Partial

Differential Equations.

33. Grenander: Regular Structures: Lectures in Pattern Theory. Vol. III.

34. Kevorkian/Cole: Perturbation Methods in Applied Mathematics.

35. Carr: Applications of Centre Manifold Theory. 36. Bengtsson/GhiVKiilJen: Dynamic Meteorology:

Data Assimilation Methods. 37. Saperstone: Semidynamical Systems in Infinite

Dimensional Spaces. 38. LichtenbergILieberman: Regular and Chaotic

Dynamics. 2nd ed. 39. PiccinilStampacchialVidossich: Ordinary

Differential Equations in Rn. 40. Naylor/Sell: Linear Operator Theory in

Engineering and Science. 41. Sparrow: The Lorenz Equations: Bifurcations.

Chaos. and Strange Attractors. 42. Guckenheimer/Holmes: Nonlinear Oscillations.

Dynamical Systems. and Bifurcations of Vector Fields.

43. Ockendon/Taylor: Inviscid Fluid Flows. 44. Pazy: Semigroups of Linear Operators and

Applications to Partial Differential Equations. 45. Glashoff/Gustafson: Linear Operations and

Approximation: An Introduction to the Theoretical Analysis and Numerical Treatment of Semi­Infinite Programs.

46. Wilcox: Scattering Theory for Diffraction Gratings.

47. Hale/Magalhiies/Oliva: Dynamics in Infinite Dimensions. 2nd ed.

48. Murray: Asymptotic Analysis. 49. Ladyzhenskoya: The Boundary-Value Problems of

Mathematical Physics. 50. Wilcox: Sound Propagation in Stratified Fluids. 51. Golubitsky/Schaeffer: Bifurcation and Groups in

Bifurcation Theory. Vol. I. 52. Chipot: Variational Inequalities and Flow in

Porous Media. 53. Majda: Compressible Fluid Flow and System of

Conservation Laws in Several Space Variables. 54. Wasow: Linear Turning Point Theory. 55. Yosida: Operational Calculus: A Theory of

Hyperfunctions. 56. Chang/Howes: Nonlinear Singular Perturbation

Phenomena: Theory and Applications. 57. Reinhardt: Analysis of Approximation Methods

for Differential and Integral Equations. 58. Dwoyer/HussainWoigt (eds): Theoretical

Approaches to Turbulence. 59. SandersIVerhulst: Averaging Methods in

Noulinear Dynamical Systems.

(continuedfollowing index)

Bernard Chalmond

Modeling and Inverse Problems in Imaging Analysis

With 11 0 Figures

, Springer

Bernard Chalmond Department of Physics University of Cergy-Pontoise Neuville sur Oise 95031 Cergy-Pontoise Cedex France bernard.chalmond@cmla.ens-cachan.fr

Editors S.S.Antman Department of Mathematics and Institute for Physical Science

and Technology University of Maryland College Park, MD 20742-4015 USA ssa@math.umd.edu

I.E. Marsden Control and Dynamical Systems, 107-81 California Institute of

Technology Pasadena, CA 91125 USA marsden@cds.caltech.edu

L. Sirovich Division of Applied

Mathematics Brown University Providence, RI 02912 USA chico@camelot.mssm.edu

Mathematics Subject Classification (2000): 69U10, 6802, 6207, 68Uxx

Library of Congress Cataloging-in-Publication Data Chalmond, Bernard, 1951-

[Elements de modelisation pour l'analyse d'images. English] Modeling and inverse problems in image analysis / Bernard Chalmond.

p. cm. - (Applied mathematical sciences; 155) Includes bibliographical references and index. ISBN 978-1-4419-3049-1 I. Image processing-Digital techniques-Mathematical models. 2. Image

analysis-Mathematical models. 3. Inverse problems (Differential equations) I. Title II. Applied mathematical sciences (Springer-Verlag New York Inc.) ; v. 155 QA1.A647 [TAI637] 006.4'2--dc21 2002026662

Printed on acid-free DaDer.

ISBN 978-1-4419-3049-1 ISBN 978-0-387-21662-1 (eBook) DOI 10.1007/978-0-387-21662-1

© 2003 Springer-Verlag New York, Inc. Softcover reprint of the hardcover 1st edition 2003

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

9 8 7 6 5 432 I SPIN 10887014

Typesetting: Pages created by the author using a Springer TEX macro package.

www.springer-ny.com

Springer-Verlag New York Berlin Heidelberg A member of BertelsmannSpringer Science+Business Media GmbH

To my family

Foreword

In the last decade of the past century we witnessed an exceptional partic­ipation of mathematicians in the development of digital image processing as a science. These contributions have found a natural place at the low level of processing, with the advent of mathematical morphology, differen­tial equations, Markov random fields, and wavelet theory. They are also reflected in the increasingly important role modeling has played in solving complex problems.

Although modeling is often a hidden stage of the solution, the benefits of correctly modeling an image processing problem are huge. Modeling is the very place where "sensitivity" steals a lead over "geometry", to put it in Pascal's words. Correct modeling introduces information that cannot be expressed by data or deduced by equations, but reflects the subtle depen­dency or causality between the ingredients. Modeling has more to do with pots and pans than recipes and spices, to draw out the culinary metaphor. Bernard Chalmond's work is mainly dedicated to modeling issues. It does not fully cover this field, since it is mostly concerned with two types of model: Bayesian models issued from probability theory and energy-based models derived from physics and mechanics. Within the scope of these models, the book deeply explores the various consequences of the choice of a model; it compares their hypotheses, discusses their merits, explores their validity, and suggests possible fields of application.

Chalmond's work falls into three parts. The first deals with the process­ing of spline functions, interpolation, classification, and auto-associative models. Although splines are usually presented with their mechanical in-

viii Foreword

terpretation, Bernard Chalmond provides the Bayesian interpretation as well. The last chapter of this part, devoted to auto-associative models and pursuit problems, provides an original view of this field. The sec­ond part is concerned with inverse problems considered as Markovian energy optimization. Consistent attention is paid to the choice of poten­tials, and optimization is classically presented either as a deterministic or as a stochastic issue. The chapter on parameter estimation is greatly ap­preciated, since this aspect of Markovian modeling is too often neglected in textbooks. In the last part, Bernard Chalmond takes the time to subtly dissect some image processing problems in order to develop consistent mod­eling. These particular problems are concerned with denoising, deblurring, detecting noisy lines, estimating parameters, etc. They have been specif­ically chosen to cover a wide variety of situations encountered in robot vision or image processing. They can easily be adapted to the reader's own requirements.

This book fulfills a need in the field of computer science research and education. It is not intended for professional mathematicians, but it undoubtedly deals with applied mathematics. Most of the expectations of the topic are fulfilled: precision, exactness, completeness, and excellent references to the original historical works. However, for the sake of read­ability, many demonstrations are omitted. It is not a book on practical image processing, of which so many abound, although all that it teaches is directly concerned with image analysis and image restoration. It is the perfect resource for any advanced scientist concerned with a better un­derstanding of the theoretical models underlying the methods that have efficiently solved numerous issues in robot vision and picture processing.

HENRI MAITRE

Paris, France

Acknowledgments

This book describes a methodology and a practical approach for the effec­tive building of models in image analysis. The practical knowledge applied was obtained through projects carried out in an industrial context, and benefited from the collaboration of

R. Azencott, F. Coldefy, J.M. Dinten, S.C. Girard, B. Lavayssiere, and J.P. Wang,

to all of whom I extend my warmest thanks. Without their participation, this work could not have achieved full maturity.

I am obliged to several groups of students who have participated over the years in the course on modeling and inverse problem methods at the University of Paris (Orsay), the Ecole Normale Superieure (Cachan), and the University of Cergy-Pontoise, in mathematics, physics, and engineer­ing, and have contributed to the evolution of the present material. The English manuscript was prepared by Kari Foster for the publishers, and I am very grateful to her.

Contents

Foreword by Henri Maitre

Acknowledgments

List of Figures

Notation and Symbols

1 Introduction 1.1 About Modeling ..... .

1.1.1 Bayesian Approach 1.1.2 Inverse Problem .. 1.1.3 Energy-Based Formulation 1.1.4 Models.....

1.2 Structure of the Book ...... .

I Spline Models

2 Nonparametric Spline Models 2.1 Definition ...... . 2.2 Optimization ........ .

2.2.1 Bending Spline .. . 2.2.2 Spline Under Tension.

vii

ix

xiii

xix

1 3 3 8

10 11 14

21

23 23 26 26 28

xii Contents

2.2.3 Robustness. . . . . . . . . . 31 2.3 Bayesian Interpretation . . . . . . . 34 2.4 Choice of Regularization Parameter 36 2.5 Approximation Using a Surface 39

2.5.1 L-Spline Surface . . . . . . . 40 2.5.2 Quadratic Energy . . . . . . 43 2.5.3 Finite Element Optimization . 46

3 Parametric Spline Models 51 3.1 Representation on a Basis of B-Splines 51

3.1.1 Approximation Spline ... 53 3.1.2 Construction of B-Splines 54

3.2 Extensions . . . . . . . . . . . 57 3.2.1 Multidimensional Case 57 3.2.2 Heteroscedasticity . . 62

3.3 High-Dimensional Splines ... 67 3.3.1 Revealing Directions 68 3.3.2 Projection Pursuit Regression 70

4 Auto-Associative Models 75 4.1 Analysis of Multidimensional Data 75

4.1.1 A Classical Approach .... 76 4.1.2 Toward an Alternative Approach 80

4.2 Auto-Associative Composite Models. 82 4.2.1 Model and Algorithm . . . . . . . 82 4.2.2 Properties ............. 84

4.3 Projection Pursuit and Spline Smoothing . 86 4.3.1 Projection Index 87 4.3.2 Spline Smoothing 90

4.4 Illustration . . . . . . . . 93

II Markov Models 97

5 Fundamental Aspects 99 5.1 Definitions . . . . . ..... 99

5.1.1 Finite Markov Fields 100 5.1.2 Gibbs Fields . . . . . 101

5.2 Markov-Gibbs Equivalence . 103 5.3 Examples ......... 106

5.3.1 Bending Energy . 106 5.3.2 Bernoulli Energy 107 5.3.3 Gaussian Energy 108

5.4 Consistency Problem .. 109

6 Bayesian Estimation 6.1 Principle ............ . 6.2 Cost Functions ......... .

6.2.1 Cost Function Examples 6.2.2 Calculation Problems.

7 Simulation and Optimization 7.1 Simulation .............. .

7.1.1 Homogeneous Markov Chain. 7.1.2 Metropolis Dynamic ..... 7.1.3 Simulated Gibbs Distribution

7.2 Stochastic Optimization .. 7.3 Probabilistic Aspects .... 7.4 Deterministic Optimization.

7.4.1 ICM Algorithm ... 7.4.2 Relaxation Algorithms

8 Parameter Estimation 8.1 Complete Data .......... .

8.1.1 Maximum Likelihood ... . 8.1.2 Maximum Pseudolikelihood 8.1.3 Logistic Estimation .

8.2 Incomplete Data. . . . . . . . . 8.2.1 Maximum Likelihood .. 8.2.2 Gibbsian EM Algorithm 8.2.3 Bayesian Calibration ..

III Modeling in Action

Contents xiii

113 113 118 119 121

123 124 124 125 127 130 134 138 138 141

147 148 149 150 153 156 157 161 170

175

9 Model-Building 177 9.1 Multiple Spline Approximation .......... 177

9.1.1 Choice of Data and Image Characteristics 179 9.1.2 Definition of the Hidden Field 181 9.1.3 Building an Energy .....

9.2 Markov Modeling Methodology .. 9.2.1 Details for Implementation.

10 Degradation in Imaging 10.1 Denoising .................. .

10.1.1 Models with Explicit Discontinuities 10.1.2 Models with Implicit Discontinuities

10.2 Deblurring.................. 10.2.1 A Particularly Ill-Posed Problem . 10.2.2 Model with Implicit Discontinuities

183 185 185

189 190 190 198 201 202 204

xiv Contents

10.3 Scatter.......... 10.3.1 Direct Problem . 10.3.2 Inverse Problem .

10.4 Sensitivity Functions and Image Fusion . 10.4.1 A Restoration Problem . . . . . .

205 206 211 216 217

10.4.2 Transfer Function Estimation . . 221 10.4.3 Estimation of Stained Transfer Function 224

11 Detection of Filamentary Entities 227 11.1 Valley Detection Principle . . . 228

11.1.1 Definitions . . . . . . . . 228 11.1.2 Bayes-Markov Formulation 230

11.2 Building the Prior Energy . 231 11.2.1 Detection Term . . . 231 11.2.2 Regularization Term 234

11.3 Optimization......... 236 11.4 Extension to the Case of an Image Pair. 239

12 Reconstruction and Projections 243 12.1 Projection Model . . . . . . . 243

12.1.1 Transmission Tomography 243 12.1.2 Emission Tomography .. 246

12.2 Regularized Reconstruction 247 12.2.1 Regularization with Explicit Discontinuities 248 12.2.2 Three-Dimensional Reconstruction 252

12.3 Reconstruction with a Single View 256 12.3.1 Generalized Cylinder . . . . . . . . 256 12.3.2 Training the Deformations . . . . . 259 12.3.3 Reconstruction in the Presence of Occlusion 261

13 Matching 269 13.1 Template and Hidden Outline 270

13.1.1 Rigid Transformations 270 13.1.2 Spline Model of a Template 272

13.2 Elastic Deformations . . . . . . . . 276 13.2.1 Continuous Random Fields. 276 13.2.2 Probabilistic Aspects . . . . 282

References 289

Author Index 301

Subject Index 305

List of Figures

1.1 Non-uniform lighting. 1.2 Field of deformations. 1.3 Face modeling. . ... 1.4 Filamentary entity detection. 1.5 Image sequence analysis .... 1.6 Tomographic reconstruction. 1.7 Hidden field. .....

2 2 3 4 4 5 6

1.8 Image characteristics. 14

2.1 Smoothing function. . 32 2.2 Robust smoothing. . . 33 2.3 Effect of regularization parameter. 36 2.4 Bias/variance dilemma. ...... 37 2.5 Approximation of a field of deformations. 43 2.6 Image deformation. . 43

3.1 B-spline function. . . 52 3.2 Effect of distance between knots on spline approximation. 55 3.3 Tensor product of B-splines. . . . . . . . . . . . . 58 3.4 Smoothing by two-dimensional regression spline. 59 3.5 Suppression of a luminous gradient. . 60

4.1 PCA and AAC of translated curves. 79 4.2 Various approximation situations. . . 81

xvi List of Figures

4.3 Diagram showing travel through regions. . . . . 4.4 Choice of number of knots. .......... . 4.5 Simulation with an incorrect number of knots .. 4.6 PCA and AAC approximation of faces ..

5.1 Bernoulli fields .............. .

7.1 Distance Ll between two probability distributions. 7.2 Surface sampled regularly and irregularly. . ... 7.3 Approximation of an irregularly sampled surface.

8.1 Segmentation of a radiographic image.

9.1 Multiple spline approximation. . ... 9.2 Spots distributed in a sequence of images. 9.3 Gaussian approximation of spots. . 9.4 Configuration of pointers. . . . . . 9.5 Pointers undergoing regularization. 9.6 Extreme local configurations. 9.7 Regularization of 3-D paths ..

89 92 93 95

108

135 143 145

169

178 180 181 182 184 187 188

10.1 Mixed grid. . . . . . . . . . . 191 10.2 Mixed neighborhood system. 192 10.3 Edge configurations on fourth-order cliques. 193 10.4 Denoising an NMR image. . . . . . . . . . . 196 10.5 Examples of photon paths in a material. . . 206 10.6 Collision parameters of a probabilistic transport model. 208 10.7 Simulated radiant image. ........... 211 10.8 Collision parameters of the analytical model. 213 10.9 Restoration-deconvolution. . . . . . . . . 216 10.10 Simulation of an artificial degradation. 218 10.11 Restoration-fusion.. . . . . . . . . . . . 219 10.12 Stained transfer function. . . . . . . . . 223 10.13 X-ray imaging for nondestructive testing. 224

11.1 Estimation of the luminosity gradient. . . 228 11.2 A'Y image after suppression of the luminosity gradient. 229 11.3 Two examples of curved surfaces. . . . . . . . . . . 232 11.4 Estimation of valleys on a single image. ...... 237 11.5 Estimation of valleys with fusion of an image pair. 240

12.1 Transmission tomography apparatus. . . . 245 12.2 Emission tomography apparatus. . . . . . 247 12.3 3-D transmission tomography apparatus. . 253 12.4 Four radiographic views of two inclusions. 255

List of Figures xvii

12.5 3-D tomographic reconstruction. .......... 255 12.6 Generalized cylinder with rectangular section. . . . 257 12.7 Radiographic projection of a generalized cylinder. . 258 12.8 Electron microscope views of soldered tabs. . . . . 264 12.9 Estimation of the weld of a soldered tab from a single view. 265

13.1 Outline and visible contour. . . . . . . . . . . . . . . . .. 271 13.2 Rigid realignment of three images using a template. 272 13.3 Matching of a template with three radiographic projections. 275 13.4 Elastic deformations of a template. 280 13.5 Elastic matching. . . . . . . . . . . . . . . . . . . . . . .. 282

Notation and Symbols

Part I

AAC B(t) B BT Cov(X, Y) em

m ~ D g, dt m

D i,j ai+jg g, at-au>

~ ~mg

E[X] c£ g(t) g(t, u) 9 Ie G(x) = 0 g gtheo, gemp

Hm

auto-associative composite model. B-spline function. matrix of B-splines. matrix of B-splines along the Taxis. covariance. functions that are m times continuously differentiable. mth derivative of g(t). partial derivative of g(t, u). Laplacian. mth-order difference operator. mathematical expectation of the random variable X. space of finite elements e of diameter f. curve. surface. restriction of the function 9 to e. equation of a manifold. two-dimensional grid. generalization error. mth-order Sobolev space.

xx Notation and Symbols

I(a, X) j(a, X) IK(a, X) Id LG(p" V)

L L2 L(()) M', tM \1f O(X, W) P(x) pa

PCA PPR R Sb S('T) SN('T) 0-2

T = (a,X) U(x) U(y I x) Var(X) W X,x X

Y, y Z lA, 1 [A] IAI

A~B

< s,t > (X,Y)

(x{r

index of the AAC of a for the cloud X. index of the PCA of a for the cloud X. Kullback index of a for the cloud X. identity matrix or application. Laplace-Gauss distribution of expectation p, and variance V. linear differential operator. square integrable function space. likelihood of (). transpose matrix. gradient of f. operator for degradation of X by W. probability distribution. projection operator with parameter a. principal component analysis. projection pursuit regression. real numbers. smoothing operator with parameter b. space of cubic splines with knots T. space of natural cubic splines with knots T. variance of a noise, of residuals, etc. coordinate of X on the a-axis. prior energy (or internal energy). fidelity energy y (or external energy). variance or variance-covariance matrix. noise, residual, etc. random variable X and its occurrence x. cloud of points in IRn. symmetry. vector or field of observations. relative integers. indicator function. cardinality of A if A is a set, or absolute value if A is a number. by definition B is written as A. sand t are neighbors. scalar product.

a raised index and an exponent can be present

at the same time, and should be distinguishable according to the context. to indicate that y is a vector. probabilistic conditioning. estimation of (), or the argument of the minimum of a function.

Part II

c G E Es r Hi H=(H1 , ... ,Hq )'

ICM C(()), Pe(x) Ns Ns(x) Pyx

P Pe(x) II 8

S Tn l}(y) Vc(x) Xs yO

{Zn} Z fal

Part III

G, Gi , Ge,i G(t) fl.! F ¢ <p(. ) {¢i} GC 0(81,82), g(81, 82)

Notation and Symbols xxi

clique. set of cliques of a Markov random field. set of states. set of states at site 8.

cost function. local energy difference. vector of local energy differences. iterated conditional mode algorithm. pseudolikelihood of (). set of neighborhood sites to 8.

values in the neighborhood Ns of 8.

probability of transition of y to x of a Markov chain. transition matrix of a Markov chain. pseudolikelihood of () for the field x. discrete probability distribution. site. set of sites. simulated annealing temperature. energy U(x) limited to a subset y of x. potential of the clique c. a field x deprived of the value Xs.

a specific observation of Y. Markov chain. normalization constant of a Gibbs distribution. integer part of a.

set of cliques of a Markov random field. curve. Laplacian of f. radiographic film .. smoothing function. scattered flux. vector basis. generalized cylinder. surface.

xxii Notation and Symbols

G(Sl,S2,C) g, go 9 fs H(C) Hi(., .), Hi,j(.,.) 1-l,llI f, fo, I K(n)

.x, A As NDT Ns(C) N PSF Rs R~ Rx(., .) S, SP, Sb S SNR STF TF Ts T(s, a)

T p , TT' T, VBL X P , xP

X b , x b

Xl, xl

{Zn} Z(t)

stained transfer function at s of level C. projection of aCe. continuous deformation field. indicator of curvature line at s. transfer function for the gray level C. local energy difference. convolution kernel. an image. kernel of n-step transition probability . energy of a photon. parameters of a confidence region. nondestructive testing. set of neighboring sites to s with value C. outline. point spread function. confidence region in s. ray between source C and detector d. Radon transform. set of sites. spatial position. signal-to-noise ratio. stained transfer function. transfer function. degradation by a stain at site s. intersection between an object and a straight line (s, a). rigid transformations. valley bottom line. field in gray levels. edge field or segmentation field. VBL field. Markov chain. Brownian process.