digital image processing lecture 7: geometric transformation march 16, 2005 prof. charlene tsai

Digital Image Processing Lecture 7: Geometric Transformation

March 16, 2005

Digital Image Processing Lecture 7: Geometric Transformation

March 16, 2005

Prof. Charlene TsaiProf. Charlene Tsai

Digital Image Processing Lecture 7 2

ReviewReview

A geometric transform of an image consists of two basic steps Step1: determining the pixel co-ordinate transformation Step2: determining the brightness of the points in the

digital grid.

In the previous lecture, we discussed brightness interpolation and some variations of affine transformation.

A geometric transform of an image consists of two basic steps Step1: determining the pixel co-ordinate transformation Step2: determining the brightness of the points in the

digital grid.

In the previous lecture, we discussed brightness interpolation and some variations of affine transformation.

(x,y)T(x,y)


Affine Transformation (con’d)Affine Transformation (con’d)

An affine transformation is equivalent to the composed effects of translation, rotation and scaling.

The general affine transformation is commonly expressed as below:

An affine transformation is equivalent to the composed effects of translation, rotation and scaling.

The general affine transformation is commonly expressed as below:

0th order coefficients 1st order coefficients

1

1

'

'

y

xAB

y

x


General FormulationGeneral Formulation

A geometric transform is a vector function T that maps the pixel (x,y) to a new position (x’,y’):

Tx(x,y) and Ty(x,y) are usually polynomial equations.

This transform is linear with respect to the coefficients ark and brk.

A geometric transform is a vector function T that maps the pixel (x,y) to a new position (x’,y’):

Tx(x,y) and Ty(x,y) are usually polynomial equations.

This transform is linear with respect to the coefficients ark and brk.

m

r

rm

k

krrky

m

r

rm

k

krrkx yxbyxTyxayxT

0 00 0

, ,

yxTyyxTx yx , , ''


Finding CoefficientsFinding Coefficients

How to find ark and brk, which are often unknown? Finding pairs of corresponding points (x,y),

(x’,y’) in both images,

Determining ark and brk by solving a set of linear equations.

More points than coefficients are usually used to get robustness. Least-squares fitting is often used.

How to find ark and brk, which are often unknown? Finding pairs of corresponding points (x,y),

(x’,y’) in both images,

Determining ark and brk by solving a set of linear equations.

More points than coefficients are usually used to get robustness. Least-squares fitting is often used.


JacobianJacobian

A geometric transform applied to the whole image may change the co-ordinate system, and a Jacobian J provides information about how the co-ordinate system changes

The area of the image is invariant if and only if |J|=1 (|J| is the determinant of J).

What is the Jacobian of an affine transform?

A geometric transform applied to the whole image may change the co-ordinate system, and a Jacobian J provides information about how the co-ordinate system changes

The area of the image is invariant if and only if |J|=1 (|J| is the determinant of J).

What is the Jacobian of an affine transform?

yT

xT

yT

xT

yx

TTyxJ

yy

xx

yx

,

,,


Variation of Affine (2D)Variation of Affine (2D)

Translation: displacement

Euclidean (rigid): translation + rotation

Similarity: Euclidean + scaling

(for isotropic scaling, sx = sy)

Translation: displacement

Euclidean (rigid): translation + rotation

Similarity: Euclidean + scaling

(for isotropic scaling, sx = sy)

0000 , , byxTayxT yx

cossin,

sincos,

00

00

yxbyxT

yxayxT

y

x

)cossin(,

)sincos(,

00

00

yxsbyxT

yxsayxT

yy

xx


Types of transformations (2D)Types of transformations (2D)

Affine: Similarity + shearing

Some illustrations of shearing effects (courtesy of Luis Ibanez)

Affine: Similarity + shearing

Some illustrations of shearing effects (courtesy of Luis Ibanez)

ybxbbyxT

yaxaayxT

y

x

011000

011000

,

,

y

x

bb

aa

b

a

y

x

0110

0110

00

00

'

'


Affine transformAffine transform

Shearing in x-direction Shearing in x-direction

y

xa

y

x

10

1

0

0

'

' 01

y

x

a01y

(x,y)( x + a01y , y )

y’

x

y

x’


Affine transformAffine transform

Shearing in y-direction Shearing in y-direction

y

x

b10x

(x,y)

( x , y + b10x )

y

x

by

x

1

01

0

0

'

'

10

y

x

y’

x’


Invariant PropertiesInvariant Properties

Translation:

Euclidean:

Similarity:

Affine:

Translation:

Euclidean:

Similarity:

Affine:


Types of transformations (2D)Types of transformations (2D)

Bilinear: Affine + warping

Quadratic: Affine + warping

Bilinear: Affine + warping

Quadratic: Affine + warping

xybybxbbyxT

xy ayaxaayxT

y

x

11011000

11011000

,

,

2

022

2011011000

202

22011011000

,

,

ybxbxybybxbbyxT

yaxaxyayaxaayxT

y

x


Least-Squares EstimationLeast-Squares Estimation

Because of noise in the images, if there are more correspondence pairs than minimally required, it is often not possible to find a transformation that satisfies all pairs.

Objective: To minimize the sum of the Euclidean distances of the correspondence set C={pi,qi}, where pi={xi,yi}, and qi is the corresponding point.

Because of noise in the images, if there are more correspondence pairs than minimally required, it is often not possible to find a transformation that satisfies all pairs.

Objective: To minimize the sum of the Euclidean distances of the correspondence set C={pi,qi}, where pi={xi,yi}, and qi is the corresponding point.

i

ii qpTCF2

;;


Least-Squares Estimation - AffineLeast-Squares Estimation - Affine

For affine model, The Euclidean error , where

With the new notation,

To find which minimize we want

For affine model, The Euclidean error , where

With the new notation,

To find which minimize we want

Tbbaaba ],,,,,[ 011001100000

iii qXe

Ti

T

TTi

ipO

OpX

10

01

ii

T

iii qXqXCF ;

CF ;

0F


Least-Squares Estimation - AffineLeast-Squares Estimation - Affine

This leads to

Therefore,

This leads to

Therefore,

022 i

iTii

Ti qXXX

ii

Ti

ii

Ti qXXX

1


Some remarksSome remarks

So far, we only discussed global transformation (applied to entire image)

It is possible to approximate complex geometric transformations (distortion) by partitioning an image into smaller rectangular sub-images.

for each sub-image, a simple geometric transformation, such as the affine, is estimated using pairs of corresponding pixels.

geometric transformation (distortion) is then performed separately in each sub-image.

So far, we only discussed global transformation (applied to entire image)

It is possible to approximate complex geometric transformations (distortion) by partitioning an image into smaller rectangular sub-images.

for each sub-image, a simple geometric transformation, such as the affine, is estimated using pairs of corresponding pixels.

geometric transformation (distortion) is then performed separately in each sub-image.


Real ApplicationReal Application

Registration of retinal images of different modalities Importance?

Registration of retinal images of different modalities Importance?

Color slide Fluorocein Angiogram


MosaicingMosaicing

Mosaic


Fusion of medical informationFusion of medical information


Change DetectionChange Detection


Computer-assisted surgeryComputer-assisted surgery


What is needed?What is needed?

A known transformation, which is less likely

OR

Computing the transformation from features. Feature extraction Features in correspondence Transformation models Objective function

A known transformation, which is less likely

OR

Computing the transformation from features. Feature extraction Features in correspondence Transformation models Objective function


What might be a good feature?What might be a good feature?

Color slide Fluorocein Angiogram


Feature ExtractionFeature Extraction


Features in Correspondence(crossover & branching points)

Features in Correspondence(crossover & branching points)


Features in Correspondence(vessel centerline points)

Features in Correspondence(vessel centerline points)


Transformation ModelsTransformation Models

)();(M pXpq qq θθ

121110987

654321

aaaaaa

aaaaaaθq

Tyxyxyx 1)( 22pXTyx ),(p

12-parameter quadratic transformation

p

q

q


Transformation Model HierarchyTransformation Model Hierarchy

1

)M(p;q y

x

b a t

a -b tθ

y

xs

1

)M(p;q y

x

c d t

a b tθ

y

xa

1

)M(p;q

22

y

x

yx

td b a

tc a -b θ

y

xr

)M(p;q qθ

Similarity

Affine Reduced Quadratic

Quadratic


Objective FunctionObjective Function

C

ii

ii

CE)q,p(

)/)q;p(M(d)(

Transformation parameters

Feature point in image pI

Feature point in image qI

Mapping of features from to based on

pI qI

Set of correspondences {}


ResultResult

digital image processing lecture 7: geometric transformation march 16, 2005 prof. charlene tsai

Documents

general affine transformation

y x a01y

global transformation

known transformation

affine warpingquadratic

variation of affine

pixel x

y yxyxaffine transformshearing