2D Signal Processing

1D signal has one independent variable - f(t)

2D signal has two independent variables - f(x,y)

Concepts of linearity, spectra,filtering, etc, carry over from 1-D. But conceptof causality not relevant as image is a fn of space, not time.

2-D systems are more complex, e.g we can factor 1-D polynomials into aproduct of 1st and 2nd order polynomial, and thus study stability and systemresponse. Stability for a 1-D system can be determined from system poles, butfor 2-D system, poles are surfaces in 4-D space.

2-D algorithms can offer more flexibility implementation, e.g one can processimage data in a non-causal way, in parallel. We now extend 1-D results to 2-Dwhere possible.

65

1

A 2-D analog signal is a function of 2 continuous variables.

A 2-D pulse is defined by :

PA(x, y) = { 1 x, y ∈ A0 x, y /∈ A

The 2-D (Dirac) unit impulse is defined by:

δ(x, y) =∫ ∞

−∞

∫ ∞

−∞f(x, y)δ(x − a, y − b)dxdy = f(a, b)

It can be represented as product of two 1-D impulses:

δ(x, y) = δ(x).δ(y)

Response of a system to 2-D impulse is termed point-spread fn.

66

2

Linear Systems and Convolution

A 2-D system is a mapping: g(x, y) = T [f(x, y)] which is linear if superpo-sition holds, i.e, if

T [af1(x, y) + bf2(x, y)] = aT [f1(x, y)] + bT [f2(x, y)]

and is shift-invariant if

T [f(x − x1, y − y1)] = g(x − x1, y − y1)

when T [f(x, y)] = g(x, y)

The point-spread fn is:

h(x, y) = T [δ(x, y)].Since

f(x, y) =∫ ∞

−∞

∫ ∞

−∞f(m, n)δ(x − m, y − n)dmdn

,

g(x, y) =∫ ∞

−∞

∫ ∞

−∞f(m, n)T [δ(x − m, y − n)]dmdn

=∫ ∞

−∞

∫ ∞

−∞f(m, n)h(x − m, y − n)dmdn

= f ∗ ∗h - 2D convolution

67

3

Seperable 2-D Systems

A system is seperable if h(x, y) = h1(x).h2(y)

Then the 2D convolution reduces to:

g(x, y) =∫ ∫

f(m, n)h1(x − m)h2(y − n)dmdn

=∫

h2(y − n)[∫

f(m, n)h1(x − m)dm]dn

= h2(y) ∗ [h1(x) ∗ f(x, y)]

i.e, two 1-D convolutions.

For a fixed but arbitary value of n, one performs the 1-D convolution overthe 1st dummy variable m:

φ(x, n) =∫

f(m, n)h1(x − m)dm

Then φ(x, n) is convolved with h2(y):

g(x, y) =∫

φ(x, n)h2(y − n)dn

68

4

Fourier Transform

As for 1-D, ”test” signal is complex exponential:

f(x, y) = ej(ω1x+ω2y) = ejω1x.ejω2y

The response is:

g(x, y) =∫ ∫

ej[ω1(x−m)+ω2(y−n)]h(m, n)dmdn

= H(ω1, ω2)ej(ω1x+ω2y),

where

H(ω1, ω2) =∫ ∫

e−j[ω1m+ω2n]h(m, n)dmdn

is the Fourier Tfm of h (x,y). The inverse formula is

h(x, y) = (12π

)2∫ ∫

H(ω1, ω2)ej(ω1x+ω2y)dω1dω2

Since the complex sinusoid is seperable, the output can be got by two 1-Dconvolutions:

1. Θ(ω1, y) =∫

f(x, y)e−jω1xdx

2. F (ω1, ω2) =∫

Θ(w1, y)e−jω2ydy

As for 1-D :

f1(x, y) ∗ ∗f2(x, y) < −−−− > F1(ω1, ω2).F2(ω1, ω2)

as does Parseval’s theorem:∫ ∫|f(m, n)|2 = (

12π

)2∫ ∫

|F (ω1, ω2)|2dω1dω2

69

5

Sampling 2-D signals

Consider band-limited signal,f ( x, y ), where

F (ω1, ω2) = 0 for (ω1, ω2) /∈ R

The 2-D sampling fn is

i(x, y) =∑m

∑n

δ(x − m∆x, y − n∆y)

The Fourier Tfm of a 2-D series of impulses is also a series of impulses:

I(ω1, ω2) =2π

∆x

2π

∆y

∑m

∑n

δ(ω1 − m2π

∆x, ω2 − n

2π

∆y)

The impulse - sampled signal is

fs(x, y) = f(x, y).i(x, y) =∑m

∑n

f(m∆x, n∆y)δ(x − m∆x, y − n∆y)

70

6

The Fourier Tfm of the sampled signal is

Fs(ω1, ω2) = F [f(x, y).i(x, y)] = (12π

)2F (ω1, ω2) ∗ ∗I(ω1, ω2)

=1

∆x∆y

∑m

∑n

F (ω1 − m2π

∆x, ω2 − n

2π

∆y)

i.e, the F.T of the sampled signal is the periodic repitition of F (ω1, ω2) at aspacing of 2π

∆x , 2π∆y

If ∆x and ∆y are small enough that the ”basebands” don’t overlap, we canrecover the signal by a LPF:

F (ω1, ω2) = H(ω1, ω2).Fs(ω1, ω2),

H(ω1, ω2) = { 1 (ω1, ω2) ∈ R0 (ω1, ω2) /∈ R

f(x, y) = h(x, y) ∗ ∗fs(x, y)

=∑m

∑n

f(m∆x, n∆y)h(x − m∆x, y − n∆y)

The impulse response of the LFP is:

h(x, y) =sinΘ1x

Θ1x.sinΘ2y

Θ2y, Θ1 =

π

∆x, Θ2 =

π

∆y

71

7

2D Discrete Signals

Normalise ∆x = ∆y = 1 to give x(n1, n2) which can be represented by array:

Some discrete signals:

1. 2-D impulse:

δ(n1, n2) = δ(n1).δ(n2) = { 1 at n1, n2

0 otherwise

2. 2-D line impulse:

x(n1, n2) = δ(n1, 1)δ(n1).1 = { 1, n1 = 00 otherwise

3. Unit step fn:

u(n1, n2) = { 1, n1 ≥ 0, n2 ≥ 00 otherwise

4. ( Complex ) Exponential :

x(n1, n2) = an1bn2 (= ej(ω1n1+ω2n2) if a, b complex)

72

8

2-D Systems

The system is linear if

T [ax1(n1, n2) + bx2(n1, n2)] = aT [x1(n1, n2)] + bT [x2(n1, n2)]

Its impulse response is

T [δ(n1 − k1, n2 − k2)] = h(n1, k1; n2, k2)

By superposition, system o/p, y(n1, n2) is

T [x(n1, n2)] = T [∑k1

∑k2

x(k1, k2)δ(n1 − k1, n2 − k2)]

=∑k1

∑k2

x(k1, k2)T [δ(n1 − k1, n2 − k2)]

A system is spatially invariant if

T [x(n1, n2)] = y(n1, n2)

impliesT [x(n1 − m1, n2 − m2)] = y(n1 − m1, n2 − m2)

Then, impulse response reduces to

h(n1, k1; n2, k2) = h(n1 − k1, n2 − k2)

73

9

Convolution

For LSI systems, the output becomes

y(n1, n2) =∑k1

∑k2

x(k1, k2)h(n1 − k1, n2 − k2) = x ∗ ∗h

E.g: Convolve 3 x 2 and 2 x 2 arrays x(k1, k2) and h(k1, k2):

Convolution of N1xN2 and M1xM2 arrays yields array of size (N1 + M1 −1)x(N2 + M2 − 1)

2D Convolution simplifies for seperable signal. Then:

y(n1, n2) =∑k1

x1(k1)∑k2

x(k2)h(n1 − k1, n2 − k2)

can be computed in 2 stages:

1. g(n1, n2) =∑k2

x2(k2)h(n1, n2 − k2) = x2 ∗ ∗h

2. y(n1, n2) =∑k1

x1(k1)g(n1 − k1, n2) = x1 ∗ ∗g

Needs NM(N + M − 1) + M(N + M − 1)2 mpy/adds if N1 = N2 = N andM1 = M2 = M cf (N + M − 1)2M2 for non-seperable signals.

74

10

Frequency response and Fourier Transform

LSI system response to complex 2D sinusoid ej(ω1n1+ω2n2)

y(n1, n2) =∑k1

∑k2

ej(ω1n1+ω2n2)h(k1, k2)

= H(ω1, ω2)ej(ω1n1+ω2n2)

H(ω1, ω2) is Fourier series of periodic fn of 2 variables and h is Fourier coeff:

h(n1, n2) =12π

∫ π

−π

∫ π

−π

H(ω1, ω2)ej(ω1n1+ω2n2)dω1dω2

Ify(n1, n2) = x(n1, n2) ∗ ∗h(n1, n2)

thenY (ω1, ω2) = X(ω1, ω2).H(ω1, ω2)

2-D Correlation:

h oo g =∑n1

∑n2

g∗(n1, n2)h(n1 + k1, n2 + k2)FT↔ G∗(ω1, ω2)H(ω1, ω2)

75

11

DFT

2-D DFT pair is extrapolated from 1-D version:

X̃(k1, k2) =N1−1∑n1=0

N2−1∑n2=0

x̃(n1, n2)e−j(2πn1k1/N1+2πn2k2/N2)

x̃(n1, n2) =1

N1N2

N1−1∑k1=0

N2−1∑k2=0

X̃(k1, k2)ej(2πn1k1/N1+2πn2k2/N2)

The DFT for N1 by N2 array x(n1, n2) is computed in 2 stages:

1. Compute 1-D DFT for each row:

T (k1, n2) =N1−1∑n1=0

x(n1, n2)e−j(2πn1k1/N1), n2 = 0, 1.....N2 − 1

This takes (N2.N1/2)log2N1 complex mults using FFT

2. Compute 1-D DFT for each column of T (k1, n2):

X̃(k1, k2) =N2−1∑n2=0

T (k1, n2)e−j(2πn2k2/N2), n1 = 0, 1.....N1 − 1

which takes N1 FFTs each length of N2, i.e another N1N22 log2N2 complex

mults.⇒ Total complex mults: N1N2

2 log2(N1N2)

76

12

Z Transform

Similar to 1-D case with 2 main limitations:

1. No polynomial factorization for 2-D polynomials so concept of pole andtests for stability are more complex

2. The difference eqn for a 2-D IIR filter must be recursively computable.

X(z1, z2) =∑n1

∑n2

x(n1, n2)z−n11 z−n2

2

X(ω1, ω2) = X(z1, z2)|z1=ejω1 , z2=ejω2

Egx(n1, n2) = an1bn2u(n1, n2) = an1u(n1).bn2u(n2)

X(z1, z2) =∑n1=0

(az−11 )n1

∑n2=0

(bz−12 )n2

=1

1 − az−11

.1

1 − bz−12

X(z1, z2) converges for |z1| > |a| and |z2| > |b|

In 1-D, poles of X(z) are points in complex z-plane

In 2-D, poles of X(z1, z2) are surfaces in 4-D complex z1, z2 plane

In above example, pole surfaces are seperable:

z1 = a for any z2

z2 = b for any z1

77

13

2-D IIR Filters

We need Recursive Computability, which permits calculation of o/p sig-nal samples recursively; ie, in terms of adjacent previously calculated o/ps. Eg:

y(n1, n2) = a1y(n1 − 1, n2) + a − 2y(n1 + 1, n2) + x(n1, n2)

y(0, 0) = a1y(−1, 0) + a2y(1, 0) + x(0, 0)

y(1, 0) = a1y(0, 0) + a2y(2, 0) + x(1, 0)

This is impossible as computing y(0,0) needs y(1,0), but y(1,0) needs y(0,0)

[All FIR filters are recursively computable]

In 2D image processing, we can process pixels in any direction, so causalityisn’t an issue

78

14

General diff. Eqn:∑k1

∑k2

a(k1, k2)y(n1 − k1, n2 − k2) =∑r1

∑r2

b(r1, r2)x(n1 − r1, n2 − r2)

The coeff. masks [a] and [b] are given over RA, RB

To compute o/p, normalise a(0,0) = 1 and rewrite:

y(n1, n2) = −∑∑

a(n1−k1, n2−k2)y(k1, k2)+∑ ∑

b(n1−k1, n2−k2)x(k1, k2)

The recursions are performed as follows:

1. At any point (n1, n2), the i/p mask [b] is positioned over i/p array. I/pvalues mulitplied by b values and summed.

2. Simultaeneously, the o/p mask [a] covers the already computed o/p values,weights these values by the a’s and sums them

3. Results of steps 1 and 2 are added to give y(n1, n2)

4. Masks then shifted by one position and repeat steps 1-3.

79

15

Conditions for recursive Computability

IIR system is recursively computable if the o/p mask has wedge support( i.e, its values lie in region bounded by 2 lines from origin at less than 180degrees).

A simpler sufficient condition is that coefs a(k1, k2) have 1st quadrant sup-port:

A recursive eqn also needs initial conditions. The i.c mask for above [a] is:

The i.c.s within the L-shaped region of N1 cols and N2 rows must be specifiedto allow recursive computation of y(n − 1, n− 2) for n1 > 0 and n2 > 0.

For example, to calculate y(0,0) needs i.c.s in dotted top rectangle,theny(0,1) is calculated using y(0,0) and i.c.s in bottom rectangle.

80

16

IIR Eg:

H(z1, z2) =Y (z1, z2)X(z1, z2)

=1 + 2z−1

1

1 − 12z−1

1 + 14z−1

2 + 18z−2

2

y(n1, n2) = x(n1, n2)+2x(n1−1, n2)+12y(n1−1, n2)−1

4y(n1, n2−1)−1

8y(n1, n2−2)

Let

x(n1, n2) = { 1, −1 ≤ n1, n2 ≤ 10 otherwise

y(−1,−1) = x(−1,−1)+2x(−2,−1)+12y(−2,−1)−1

4y(−1,−2)−1

8y(−1,−3) = 1

y(−1, 0) = x(−1, 0) + 2x(−2, 0) +12y(−2, 0) − 1

4y(−1,−1)− 1

8y(−1,−2) =

34

y(0,−1) = x(0,−1) + 2x(−1,−1) +12y(−1,−1)− 1

4y(0,−2) − 1

8y(0,−3) = 3

12

80b

17

Stability of IIR Filters

A bi-variate polynomial can’t, in general, be factored into lower order poly-nomials. Also poles of H(z1, z2) are surfaces in 4-D complex (z1, z2) space sostability tests are not simple extensions of 1-D results

But spatial domain condition for BIBO stability is:∑n1

∑n2

|h(n1, n2)|2 < ∞

- however, it’s difficult to translate to tfm domain.

We need consider only

H(z1, z2) =1

A(z1, z2)

Stability tests for these IIR filters have been developed as De Carlo - Strintzistheorem:

A 2-D IIR filter with 1st quadrant o/p support is stable if and only if:

1. A(z1, z2) �= 0 for |z1| = 1, |z2| = 1

2. A(z1, 1) �= 0 for |z1| ≥ 1

3. A(1, z2) �= 0 for |z2| ≥ 1

81

18

2-D FIR Filters

Simpler than IIR as a(k1, k2) = 0:

y(n1, n2) =∑∑

b(n1 − k1, n2 − k2)x(k1, k2)

⇒ b(n1, n2) = h(n1, n2)

FIR filter has zero phase if h(n1, n2) is symmetrical If h(n1, n2) is real andeven, then H(ω1, ω2) is a real, even fn:

H(ω1, ω2) = H(−ω1,−ω2)

Since spatial causality is not a constraint it is easy to make

h(−n1,−n2) = h(n1, n2)

E.g 1.

h(n1, n2) = δ(n1 − 1, n2) + δ(n1 + 1, n2) + δ(n1, n2 − 1) + δ(n1, n2 + 1)

H(z1, z2) = (z−11 + z1) + (z−1

2 + z2)

H(ω1, ω2) = 2cos ω1 + 2cos ω2

82

19

Seperable 2-D filters can be reduced to two 1-D filters in tandem:

So design of seperable 2-D FIR filter reduces to the design of 1-D FIR filtersby, e.g window method, freq-sampling or Parks-McClellan algorithm.

Eg 2H(z1, z2) = (z−1

1 + 2 + z1)(z−12 + 2 + z2)

= z−11 z−1

2 + 2z−11 + z1z

−12 + 2z−1

2 + 4 + 2z2 + z2z−12 + 2z1 + z1z2

H(ω1, ω2) = (2 + 2cos ω1) + (2 + 2cos ω2) = (4cos .5ω1cos .5ω2)2

83

20

Types of 2-D filter (circularly symmetric)

Non-seperable 2-D FIR Filters can be designed by simple extension of1-D techniques

E.g Windowing Method:

1. Specify H(ω1, ω2)

2. Compute h(n1, n2) by inverse FFT

3. Truncate h(n1, n2) by window w(n1, n2): h(n1, n2) = h(n1, n2)w(n1, n2)

4. Pad h(n1, n2) with 0s and compute H ′(k1, k2) via FFT

84

21

”‘Frequency”’ transformation method does not have a 1-D counter-part but is practical method of designing 2-D filters from 1-D:

1st design 1-D prototype 0-phase FIR Filter of form:

H(ω) =M∑

n=−M

h(n)e−jnω = h(0) +M∑

n=1

2h(n)cos ωn =M∑

n=0

ancos ωn

which can be shown to be =∑M

n=0 bn(cos ω)n

2-D freq response H(ω1, ω2) is obtained by substituting 2-D TransformationT (ω1, ω2) for cos ω:

H(ω1, ω2) = H(ω)|cosω=T (ω1,ω2) =M∑

n=0

bn(T (ω1, ω2))n

Where T (ω1, ω2) is finite-extent, 0-phase, real, even fn, which can be ex-pressed as:

T (ω1, ω2) =∑ ∑

c(n1, n2)cos(ω1n1 + ω2n2)

An eg of T (ω1, ω2) often used in practice is the Mc Clellan Transformation:

T (ω1, ω2) = −.5 + .5cos ω1 + .5cos ω2 + .25cos(ω1 + ω2) + .25cos(ω1 − ω2)

= −.5 + .5cos ω1 + .5cos ω2 + .5cos ω1 cos ω2

Then c(n1, n2) is:

85

22

For McClellan Tfm, set of contours for cos ω = T (ω1, ω2) is:

T (ω1, ω2) maps 1-D frequency response to 2-D Eg 1. BPF:

E.g 2. Apply transformation method to binomial LPF:

H(ω) = 6 + 8cos ω + 2cos2 ω = 4 + 8cos ω + 4cos2 ω

⇒ H(ω1, ω2) = 4 + 8T (ω1, ω2) + 4[T (ω1, ω2)]2

For T (ω1, ω2) = .5cos ω1 + .5cos ω2 + .5cos ω1cos ω2 − .5,

H(ω1, ω2) = 1+2cosω1+2cosω2+4cosω1cosω2+cos2ω1+cos2ω2+2cosω1cos2ω2+2cos2ω1cosω2+cos2ω1cos

2ω2

= (1 + cos ω1)2(1 + cos ω2)2 = 16(cos .5ω1)4(cos .5ω2)4

86

23

Implementation of 2-D FIR Filters

(a) Direct computation of convolution sum:

y(n1, n2) =∑∑

x(n1 − k1, n2 − k2)h(k1, k2)

This needs N muls and N adds for each o/p point, where N = no. of non-0 coeffs of y(n1, n2). E.g a 512 x 512 image filtered by 10 x 10 filter needs26,214,400 mult/adds

As for 1-D FIR 0-phase filters, computation reduced 50% by exploiting sym-metry of h (n1,n2)

(b) Using FFT: Y (ω1, ω2) = H(ω1, ω2)X(ω1, ω2)

For 512 x 512 image with 10 x10 filter this needs inverse DFT ≥ (512+10-1)x(512+10-1).

Use of FFT for this will need less computation (approx N2

2 log2(N2)x2 +N2 ≈ 5, 000, 000 mults)than direct convolution, but needs all x(n1, n2), H(ω1, ω2)and X(ω1, ω2) to be stored.

An alternative is to divide data into blocks and do smaller FFTs. As FFTgives circular convolution, overlap-add or overlap-save method can be used.

87

24

Linear vs Circular Convolution

Eg (1-D)

Linear:

x1(n) = x2(n) = [3, 2, 1], x3(n) = x1(n) ∗ x2(n) =∑m

x1(m)x2(n − m)

x3(0) =∑

x1(m)x2(0 − m) = 9

x3(1) =∑

x1(m)x2(1 − m) = 12

x3(2) = 10, x3(3) = 4, x3(4) = 1, x3(n) = 0, n > 4

Circular:

x3p(n) =N−1∑m=0

x1(m)x2(n − m, (modN)), n = 0...N − 1

x3p(0) =2∑

m=0

x1(m)x2(0 − m, (mod 3)) = 13

x3p(1) =2∑

m=1

x1(m)x2(1 − m, (mod 3)) = 13

x3p(2) =2∑

m=2

x1(m)x2(2 − m, (mod 3)) = 10

x3p(3) = x3p(0), x3p(4) = x3p(1)...repeating

If x1 and x2 are padded with 2 zeros to give x́1 = x́2 = 3, 2, 1, 0, 0, then 1period of x́1 ∗© x́2 = x3 - same as linear convolution.

88

25

2-D Overlap Add

Image x(n1, n2) divided into L1L2 segments xij(n1, n2):

x(n1, n2) =L1∑i=1

L2∑j=1

xij(n1, n2)

x(n1, n2) =L1∑i=1

L2∑j=1

xij(n1, n2) ∗ h(n1, n2) :

Since xij is much smaller than x, x́ij ∗© h́ can be computed by much smallerDFT, IDFT and storage.

89

26