an image registration technique for recovering rotation, scale and translation parameters march 25,...

An image registration technique for recovering rotation, scale and

translation parameters

March 25, 1998

Morgan McGuire

3/25/98 Morgan McGuire 2

Acknowledgements

• Dr. Harold Stone, NEC Research Institute

• Bo Tao, Princeton University

• NEC Research Institute


Problem DomainSatellite, Aerial, and Medical sensors produce series images which need to be aligned for analysis. These images may differ by any transformation (possible noninvertible).

Images courtesy of Positive Systems


New Technique

• Solves subproblem (practical case)

• O(ns(NlogN)/4k+Nk) compared to O(NlogN), O(N3)

• Correlations typically > .75 compared to .03


Structure of the Talk• Differences Between Images

• Fourier RST Theorem

• Degradation in the Finite Case

• New Registration Algorithm– Edge Blurring Filter– Rotation & Scale Signatures

• Experimental Results

• Conclusions


Differences Between Images

• Alignment

• Occlusion

• Noise

• Change


Sub-problem Domain

• Alignment = RSTL• Occlusion < 50%• Noise + Change = Small• Square, finite, discrete

images• Image cropped from

arbitrary infinite texture

n

n

N pixels


RST Transformation

1100

cossin

sincos

1p

p

r

r

y

x

yss

xss

y

x

))cossin(,)sincos((, syxysyxxpyxr


Fourier Rotation, Scale, and Translation Theorem†

ssFseF yxyxpsyxj

yxryx /cossin,/sincos, 2/)(

prr FPFRrF ,Let .DTFT Where

Pixel Domain Fourier Domain

p = rotate(r, ) P = rotate(R, )

p = dilate(r, s) Fp = s2 . dilate(Fr, 1/s)

p = translate(r, x, y) Fp = translate(Fr, x, y)


†For Infinite Images


In practice, we use the DFT

Let X0 = DFT(x0)

X0 and x0 are discrete, with N non-zero coefficients.

Let X = DTFT(x)

k

kXX )2(** sin0

k

kNtxx )(*0

X0 and x0 are sub-sampled tiles (one period spans) of X and x. The Fourier RST theorem holds for X and x... does it also hold for X0 and x0?


Fourier Transform and Rotations


Theorem

Infinite case: Fourier transform commutes with rotation

Folklore: It is true for the finite case


Using Fourier-Mellin Theory

• Magnitude of Fourier Transform exhibits rotation, but not translation

• Registration algorithm:– Correlate Fourier Transform magnitudes

for rotation– Remove rotation, find translation

• Generalizes to find scale factors, rotations, and translation as distinct operations


Folklore is wrongImage

Image

Tile

Tile Rotate

Rotate


The Mathematical Proof

F f x y , ,

Transform, then rotate

The Finite Fourier transform

f x y,( , ) F

j x y N2 ( ) / D f x yyx

, ,

Windowing, sampling,

infinite tiling

D f x y eyx

j x y N , , ( )/2

continuous


The Mathematical ProofRotate, then transform

F f x y , ,

D f x y eyx

, , j x y N2 /

D f x y eyx

, , j x y N2 /

x y

D f x y e j x y N , ( , ) ( )/2


Finite-Transform Pairs


The Artifacts


Fourier Transforms

ts.coefficien nonzero with finite

are they so,~

,~ of periods single are and :

sampled.-sub is ~

. period ~

,~

][~][~

;][~

][~:

][;][:

;:

00

1

0

/21

0

1/21

21

21

N

XxXxDFT

XXNXx

enxkXekXnxDFS

enxXdeXnxDTFT

dtetxXdeXtxCTFT

N

n

NnkjN

kN

Nknj

N

n

njnj

jtjt

Oppenheim & Willsky Signals & Systems; Oppenheim and Schafer, Discrete-Time Signal Processing


Tiling does not Commute with Rotation

Tiled Image Rotated Tiled Image Tiled Rotated Image

…so the Fourier RST Theorem does not hold for DFT transforms.


Correlation Computation

( , )x y

x yN

x y

xN

x yN

y

i i i i

i i i i

1

1 122

22

xy

x y


Prior Art

• Alliney & Morandi (1986) – use projections to register translation-only in

O(n), show aliasing in Fourier T theorem

• Reddy & Chatterji (1996) – use Fourier RST theorem to register in

O(NlogN)

• Stone, Tao & McGuire (1997) – show aliasing in Fourier RST theorem


An Empirical ObservationEven though the Fourier RST Theorem does not hold for finite images, we observe the DFT does have a “signature” that transforms in a method predicted by the Theorem.

Image

DFT Magnitude


Sources of Degradation

• Frequency– Aliasing (from Tiling)

– “+” Artifact

– Sampling Error

• Pixel– Image Window Occlusion

– Image Noise


Algorithm Overview

Norm. Circ. Corr.

r p

G G

FMT

f,d f,d

f,logd

f,logd

J

Maximum Value Detector

Peak Detector

Norm. Corr.

List of scale factors (s)

exp

J

FMT

W W

HH

Coarse (x, y)

FFTFFT

Dilate

Rotate

FFT

Dilate

Rotate

FFT

(Pixel) Correlation

W W W W

r m p h

1. Pre-Process

3. Recover

Scale

Parameter

4. Recover

Rotation

Parameter

5. Recover Translation Parameters

2. FMLP Transform


Problem: “+” Artifact

None Rotation Dilation TranslationTransformation

DFT

Image


Solution: “Edge-Blurring” Filter, G

Image

None

DFT

Disk BlurFilter


Problem:Need Orthogonal Invariants

dxdyeyxrG

RR

yxj

yx

)(,

,sin,cos

In the “log-polar” (log,) domain:

Added NoiseTranslate

AliasingNoise,,logby Translateby Dilate

AliasingNoise,,by Shift Cyclicby Rotate

ss

Domain FMLPDomain Pixel

Fourier-Mellin transform:


Mapping (x,y) to (log,)

y

x

x=8

y=8

log

log

/4

log

/4

x=4

y=4


Sample Image Pair

G(r) G(p)

= 17.0o

s = 0.80

x = 10.0

y = -15.0

N = 65536

k = 2


Nonzero Fourier Coefficients

R P


Solution I: Rotation Signature

2/

0

sin,cosn

r dRJ

1. Selectively weight “edge coefficients” (J filter)

2. Integrate along axis

is Scale and Translation Invariant. Pixel rotation appears as a cyclic shift => use simple 1d O(nlogn) correlation to recover rotation parameter.


Signatures of r and p


Correlations


Solution II: Scale Signature

0

1 sin,coslog dRHSr

1. Integrate along axis (rings)

2. Normalize by (area)

3. Enhance S/N ratio (H filter)

S is Rotation and Translation Invariant. Pixel dilation appears as a translation => use simple 1d O(nlogn) correlation to recover scale parameter.


Raw S Signature


Filtered S Signature


S Correlation


New Registration Algorithm

Norm. Circ. Corr.

r p

G G

FMT

f,d f,d

f,logd

f,logd

J

Maximum Value Detector

Peak Detector

Norm. Corr.

List of scale factors (s)

exp

J

FMT

W W

HH

Coarse (x, y)

FFTFFT

Dilate

Rotate

FFT

Dilate

Rotate

FFT

(Pixel) Correlation

W W W W

r m p h

Compute full-resolution Correlation for small neighborhood of Coarse (x, y) to refine.


Recovered Parameters s (x, y)

Fish Actual 17.00 0.80 (10.00, -15.00)(natural) Recovered 17.25 0.75 (10.17, -14.98)

Peak Corr. 0.82 0.53 0.90

1d RMS = 3.42 2d RMS = 4.84

Image


Disparity Map


Multiresolution for Speed

• Algorithm is O(NlogN) because of FFT’s

• With kth order wavelet, O((NlogN)/4k)

• To refine, search 22k = 4k positions

• Using binary search, k extra trials @ O(N) each

• Total algorithm is O((NlogN)/4k + Nk)


Results & ConfidenceTrial s (x, y)

1 suburban Actual 31.00 1.00 (0.00, 0.00)(aerial) Recovered 31.00 1.00 (0.00, 0.00)

Peak Corr. 0.90 0.48 0.972 suburban Actual 0.00 1.40 (0.00, 0.00)

Recovered 0.25 1.37 (0.00, 0.00)Peak Corr. 0.86 0.31 0.84

3 suburban Actual 0.00 1.00 (10.00, -3.00)Recovered 0.00 1.00 (10.00, -3.00)Peak Corr. 0.91 0.83 1.00

4 fish Actual 24.00 0.70 (7.00, 12.00)(natural) Recovered 25.00 0.72 (7.06, 11.99)

Peak Corr. 0.78 0.45 0.955 mamogram Actual -10.00 1.20 (10.00, 9.00)

(x-ray) Recovered -10.25 1.15 (9.76, 8.75)Peak Corr. 0.94 0.73 0.58

6 essai Actual -15.00 1.50 (-10.00, -10.00)(satellite) Recovered -15.25 1.56 (-11.33, -11.40)

Peak Corr. 0.94 0.40 0.78

Image


Analysis of Results

s (x, y) TotalMean Correlation 0.89 0.53 0.85 0.76

Mean RMS Error (1d) 1.92


Future Directions

• Better scale signature

• Use occlusion masks for FM techniques?

• Combining FM technique with feature based techniques

an image registration technique for recovering rotation, scale and translation parameters march 25,...

Documents