digital image processing image transform (fourier ... · properties some examples fourier...

Table of Content

Digital Image ProcessingImage Transform (Fourier) / Quantization / Quality

O. Le [email protected]

Univ. of Rennes 1http://www.irisa.fr/temics/staff/lemeur/

December 6, 2010

1

mailto:[email protected]

http://www.irisa.fr/temics/staff/lemeur/

Table of Content

1 Introduction

2 Image transformation

3 Fourier transformation

4 Time sampling

5 Discrete Fourier Transform

6 Scalar Quantization

7 Distortion/quality assessment

8 Conclusion

2

IntroductionImage transformationFourier transformation

Time samplingDiscrete Fourier Transform

Scalar QuantizationDistortion/quality assessment

Conclusion

Introduction

1 Introduction



4 Time sampling




8 Conclusion

3




Conclusion

Introduction

What is a signal?

This is a function of one or more variables (time, distance, temperature...).

A signal carries information!!

Signal processing is required to obtain a more desirable form.

Types of signals:

Natural (lighting) or synthetic (from a lab);

one dimension or more;

Continuous/discrete.

4




Conclusion

Introduction

Classi�cation of signals

Continuous-time vs Discrete-time signals:→ A continuous-time signal contains a value for all real numbers along the time axis;→ A discrete-time signal only has values at equally spaced intervals along the time axis.

Analog vs Digital:→ Analog signal corresponds to a continuous set of possible function values;→ Digital signal corresponds to a discrete set of possible function values.

Deterministic vs Random Signals:→ The past, present and future of a deterministic signal are known with certainty (its

values can be determined by a mathematical expression, rule or table). Otherwise, itis called Random and its properties are explained by statistical techniques.

Periodic vs Aperiodic: periodic signals repeat with some period T ;

Finite vs In�nite length.

5




Conclusion

Introduction

Classi�cation of signals

Sampling = discrete time (or space) Quantization = discrete amplitude

6




Conclusion

Introduction

There are many di�erent ways to describe or represent a signal mathematically.

Most of the time a linear combination of elementary time functions. These functionsare called basis functions.

Let the set of basis functions be de�ned by: {φp}, (di�erent posssibilities for therange of p)

x(t) =∑n

anφn(t)

7




Conclusion

Notations and reminder

a continuous signal x(.);

a discrete signal x[.];

a matrix A.

8




Conclusion

Notations and reminder

f is the frequency (Hz); w = 2πf is the pulsation (radian/s); T = 1fis the period

(s);

x ∈ Lp(R) means that∫ +∞−∞ |x(t)|p dt < +∞. For instance, the space L2(R) is

composed of the functions x with a �nite energy∫ +∞−∞ |x(t)|2 dt < +∞;

Euler's formulas: exp(jθ) = cos θ + j sin θ;

exp(jθ) = cos θ + j sin θexp(−jθ) = cos θ − j sin θ

exp(jθ) + exp(−jθ) = 2 cos θexp(jθ)− exp(−jθ) = 2j sin θ

From Wikepedia.

9




Conclusion

Reminder

sinc(x) = sinπxπx

;

Inner product:

〈x , y〉 =

∫ +∞

−∞x(t)× y∗(t)dt

〈x , y〉 =+∞∑

n=−∞x(n)× y∗(n)

Condition of orthogonality for real basis function:∫ t2

t1

φn(t)φm(t)dt =

{0 n 6= m

λm n = m

where λm is a constant (λm = 1, orthonormal).

Condition of orthogonality for complex basis function:∫ t2

t1

φn(t)φ∗m(t)dt =

{0 n 6= m

λm n = m

where λm is a constant (λm = 1, orthonormal).

10




Conclusion

Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation

Image transformation

1 Introduction

2 Image transformationPresentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation


4 Time sampling




8 Conclusion

11




Conclusion



What is a transformation?

im[x , y ]T−→ IM[u, v ]

im is the original image;

IM is the transformed image;

x , y (or u, v) represents the spatial coordinates of a pixel.

Goal of a transformation

The goal of a transformation is to get a new representation of the incoming picture.This new representation can be more convenient for a particular application or canease the extraction of particular properties of the picture.

12




Conclusion



There exist 3 types of transformation:

Point to point transformation:The output value at a speci�c coordinateis dependent only on one input value butnot necessarily at the same coordinate;

Local to point transformation:The output value at a speci�c coordinateis dependent on the input values in theneighborhood of that same coordinate;

Global to point transformation:The output value at a speci�c coordinateis dependent on all the values in the inputimage.

Note that the complexity increases with the size of the considered neighborhood...

13




Conclusion


Point to point transformation

Geometric transformation such as rotation, scaling...Example of rotation:

Scalar quantization

LUT (Look-Up-Table)

Remark: a point to point transformation doesn't require extra memory...

14




Conclusion


Example of local to point transformation

Note that local to point transformation is also called neighborhood operators.

Examples of local to point transformation:

Linear transformation:For instance, a tranverse �lter (FIR) IM[x , y ] =

∑i

∑j h(i , j)im[x − i , y − j],

where h is the 2D impulse response.

Non-linear transformation such as adaptive linear �ltering and rank �ltering(median)...

Morphological �ltering (erosion, dilatation...) based on the de�nition of astructuring element.

Remark: Non linear �lters are not reversible in general.

15




Conclusion


Global to point transformation

One of the most important transformation is the Fourier transform that gives afrequential representation of the signal.

Fourier series (development in real term):

Every signal x (x ∈ L2(t1, t1 + T )) can be decomposed into a linear combination ofsinusoidal and cosinusoidal component functions.

x(t) = b0 ++∞∑n=1

ancos(2πnt) ++∞∑n=1

bnsin(2πnt)

an and bn are used to weight the in�uence of each frequency component.

16




Conclusion



Fourier series (development in complex term):

x(t) =+∞∑

n=−∞Cnexp(j2πn

t

T)

with,

Cn =1

T

∫ t1+T

t1

x(t)exp(−j2πnt

T)dt

Remark:

Cn = an−jbn2

, n ≥ 1 ; Cn =a−n+jb−n

2, n ≤ 1 ; C−n = C∗n ;

Note that if the function to be represented is also T -periodic, then t1 is anarbitrary choice. Two popular choices are t1 = 0, and t1 = −T/2.

17




Conclusion



18




Conclusion

De�nitionTrivial propertiesParseval FormulaPropertiesSome examples

Fourier transformation

1 Introduction


3 Fourier transformationDe�nitionTrivial propertiesParseval FormulaPropertiesSome examples

4 Time sampling




8 Conclusion

19




Conclusion



Fourier transformation in L2(R)

x(t)F−→ X (f )

Inverse Fourier Transform:

x(t) =

∫ +∞

−∞X (f )exp(j2πft)df

Fourier Transform:

X (f ) =

∫ +∞

−∞x(t)exp(−j2πft)dt

provided that x ∈ L2(R) and X ∈ L2(R).

20




Conclusion



From the output of the Fourier transform, we de�ne:

The frequency spectrum: Real (X (f )) + jImg ((X (f )))The Fourier transform of a function produces a frequency spectrum whichcontains all of the information about the original signal, but in a di�erent form.

Magnitude spectrum: |Real (X (f )) + jImg (X (f ))|

Phase spectrum: Arctg(Img(X (f ))Real(X (f ))

)Power spectrum: Real (X (f ))2 + Img (X (f ))2

21




Conclusion



Linearity:

ax(t)F−→ aX (f )

ax1(t) + bx2(t)F−→ aX1(f ) + bX2(f )

Complex conjugate: x∗(t)F−→ X∗(−f )

x∗(t) =

(∫ +∞


)∗x∗(t) =

∫ +∞

−∞X∗(f )exp(−j2πft)df

In the same way, x∗(−t) F−→ X∗(f ) and x(−t) F−→ X (−f ).

22




Conclusion



Hermitian symmetry: if x(t) ∈ R, we deduce X (−f ) = X∗(f )

X (f ) =

∫ +∞


X∗(f ) =

(∫ +∞


)∗X∗(f ) =

∫ +∞

−∞x∗(t)exp(j2πft)dt

Given that x(t) ∈ R, we have x∗(t) = x(t) that implies X∗(f ) = X (−f ).

23




Conclusion



Note that, as we use the space L2(R), the inner product of x ∈ L2(R) and y ∈ L2(R)is given by:

〈x , y〉 =

∫ +∞

−∞x(t)y∗(t)dt

The norm is then given by:

‖x‖2 = 〈x , x〉 =∫ +∞−∞ |x(t)|2 dt∫ +∞

−∞|x(t)|2 dt =

∫ +∞

−∞x(t)x∗(t)dt

=

∫ +∞

−∞x(t)

[∫ +∞

−∞X∗(f )exp(−j2πft)df

]dt

=

∫ +∞

−∞X∗(f )

[∫ +∞


]df

=

∫ +∞

−∞X∗(f )X (f )df∫ +∞

−∞|x(t)|2 dt =

∫ +∞

−∞|X (f )|2 df

24




Conclusion



∫ +∞

−∞|x(t)|2 dt =

∫ +∞

−∞|X (f )|2 df

This formula (Parseval's theorem or energy conservation) proves that theenergy is conserved by the Fourier transform.

Loosely, the sum (or integral) of the square of a function is equal to thesum (or integral) of the square of its transform.

25




Conclusion



Translation in time/space domain: x(t − t0)F−→ exp(−j2πt0f )X (f )

x(t − t0)F−→

∫ +∞

−∞x(t − t0)exp(−j2πtf )dt

k = t − t0

x(t − t0)F−→

∫ +∞

−∞x(k)exp(−j2π(k + t0)f )dk

x(t − t0)F−→ exp(−j2πt0f )

[∫ +∞

−∞x(k)exp(−j2πfk)dk

]Displacement in time or space induces a phase shift proportional to frequency and

to the amount of displacement.

Frequency shift: x(t)exp(±j2πf0t)F−→ X (f ± f0)

Displacement in frequency multiplies the time/space function by a unit phasorwhich has angle proportional to time/space and to the amount of displacement.

Amplitude modulation.

26




Conclusion



Convolution in time/space domain: x1(t) ∗ x2(t)F−→ X1(f )× X2(f )

x1(t) ∗ x2(t)F−→

∫ +∞

−∞

[∫ +∞

−∞x1(τ)x2(t − τ)dτ

]exp(−j2πft)dt

x1(t) ∗ x2(t)F−→

∫ +∞

−∞x1(τ)

[∫ +∞

−∞x2(t − τ)exp(−j2πft)dt

]dτ

Translation

x1(t) ∗ x2(t)F−→

[∫ +∞

−∞x1(τ)exp(−j2πf τ)dτ

]× X2(f )

Convolution in frequency domain: x1(t)× x2(t)F−→ X1(f ) ∗ X2(f )

These properties mean that if two functions are multiplied in one domain, thentheir Fourier transforms are convolved in the other domain.

27




Conclusion



Time and frequency scaling: x(λt)F−→ 1|λ|X ( f

λ)

x(λt)F−→

∫ +∞

−∞x(λt)exp(−j2πtf )dt

k = λt

x(λt)F−→

∫ +∞

−∞x(k)exp(−j2πk

f

λ)dk

|λ|

x(λt)F−→

1

|λ|

[∫ +∞

−∞x(k)exp(−j2πk

f

λ)dk

]

Multiplication of the scale of the time/space reference frame changes by thefactor λ scales the frequency axis of the spectrum of the function.

All the previous properties are what makes Fourier transforms so usefuland practical.

28




Conclusion



Translated Dirac, x(t) = δ(t − T )

X (f ) =

∫ +∞

−∞x(t)exp(−j2πtf )dt

X (f ) = exp(−j2πTf )

If T = 0, X (f ) = 1. The Fouriertransform is constant whatever thefrequency.

x(t) = 1, X (f ) = δ(f )

x(t) = Arect(t|T )

X (f ) =

∫ +∞

−∞x(t)exp(−j2πtf )dt

= A

∫ +T2

−T2

exp(−j2πtf )dt

= −A

j2πf[exp(−jfTπ)− exp(jfTπ)]

X (f ) = ATsinc(fT )

29




Conclusion



Ideal low-pass �lter in frequency domain: X (f ) = rect(f |F2

) selects low

frequencies over[−F

4, F4

]. The impluse response of the �lter is calculated with

the inverse Fourier integral:

x(t) =

∫ +∞


=

∫ + F4

− F4

exp(j2πft)df

=1

j2πt

[exp(j2π

F

4t)− exp(−j2π

F

4t)

]sinθ =

exp(jθ)− exp(−jθ)

2j

x(t) =F

2sinc(

π

2tF )

30




Conclusion



31




Conclusion

What is the objective?Reminder on DiracSamplingSampling TheoremIn practice

Time sampling

1 Introduction



4 Time samplingWhat is the objective?Reminder on DiracSamplingSampling TheoremIn practice




8 Conclusion

32




Conclusion


Time sampling

To be more �exible and to gain in accuracy...

Digital algorithms are now ... everywhere from television, CD, embeddedelectronic (car...).

Whether sound recordings or images, most discrete signals are obtainedby sampling an analog signal. But, be careful, we have to follow some

constraints in order to be able to reconstruct the analog signal.

The simplest way to discretize an analog signal x is to record its samplevalues at intervals T : x(t) can be approximated by {x(nT )}n∈Z .

33




Conclusion


De�nition

Diracs are essential to make the transition from functions of real variable to discretesequences.

A Dirac can be de�ned as: ∫ +∞

−∞δ(t)x(t)dt = x(0)

δ(t) =

{0 if t 6= 0

+∞ for t = 0∫ +∞

−∞δ(t)dt = 1.

Remark: {δτ : δ(t − τ)}τ∈Z is an orthonormal basis:

〈δ(t − n), δ(t −m)〉 =

{0 if m 6= n

1 for m = n.

34




Conclusion


Properties of a Dirac

Scaling and symmetry: δ(λt) = δ(t)|λ|∫ +∞

−∞δ(λt)dt =

∫ +∞

−∞δ(k)

dk

|λ|

δ(λt) =δ(t)

|λ|

Note that if λ = −1, δ(−t) = δ(t).

Product: x(t)× δ(t − T ) = x(T )× δ(t − T )

Convolution: x(t) ∗ δ(t − T )

x(t) ∗ δ(t − T ) =

∫ +∞

−∞x(τ)δ(t − T − τ)dτ

=

∫ +∞

−∞x(τ)δ(τ − (t − T ))dτ

= x(t − T )

The e�ect of convolving with a translated Dirac is to time-delay x(t), by thesame amount (copy of the signal x(t) at the position T ).

35




Conclusion


Dirac comb: x(t) =∑

n∈Z δ(t − nT ). Notation: ΠT (t)

The product of a comb with a signal x(t) conducts to a sampling:

The convolution of a comb with a signal x(t) conducts to a periodization of x(t):

36




Conclusion


FT of a Dirac

Dirac: x(t) = δ(t), δ(t)F−→ 1, constant function whatever the frequency

X (f ) =

∫ +∞

−∞δ(t)exp(−j2πtf )dt

X (f ) = exp(−j2π0f ) = 1

Translated Dirac: x(t) = δ(t − T ), δ(t − T )F−→ exp(−j2πTf )

X (f ) =

∫ +∞

−∞δ(t − T )exp(−j2πtf )dt

X (f ) = exp(−j2πTf )

Constant signal: x(t) = 1, 1F−→ δ(f ). The FT of a constant signal is a weight at

the null frequency.

37

Dirac comb: x(t) =∑

n∈Z δ(t − nT )This is a periodic function, since x(t +T ) = x(t), ∀t. Its Fourier serie is given by:

x(t) =+∞∑

n=−∞Cnexp(j2πn

t

T)

Cn =1

T

∫ T2

−T2

δ(t)exp(−j2πnnt

T)dt =

1

T

x(t) =1

T

+∞∑n=−∞

exp(j2πnt

T)

By using the Fourier Trasnform,

F(x(t)) = F

1

T

+∞∑n=−∞

exp(j2πnt

T)

F(x(t)) =

1

T

+∞∑n=−∞

F(exp(j2πn

t

T)

)

F(x(t)) =1

T

+∞∑n=−∞

∫ +∞

−∞exp(−j2πt(f −

n

T))dt

∑n∈Z

δ(t − nT )F−→

1

T

∑n∈Z

δ(f −n

T)




Conclusion


Sampling

Sampling a signal is equivalent to multiplying it by a grid of impulses.

39




Conclusion


Sampling

Let x(t) be a continous function, its sampled version y(t) is given by:

y(t) = x(t)× ΠT (t)

= x(t)×∑n∈Z

δ(t − nT )

=∑n∈Z

x(t)δ(t − nT )

y(t) =∑n∈Z

x(nT )δ(t − nT )

where ΠT (t) is the sampling function (sampling frequency 1T).

40




Conclusion


Fourier Transform of a sampled signal

Let x(t) be a continous function, its sampled version y(t), y(t) = x(t)× ΠT (t)

y(t) = x(t)×∑n∈Z

δ(t − nT )

F(y(t)) = F

x(t)×∑n∈Z

δ(t − nT )

F(y(t)) = X (f ) ∗ F

∑n∈Z

δ(t − nT )

F(y(t)) = X (f ) ∗

1

T

∑n∈Z

δ(f −n

T)

F(y(t)) =

1

T

∑n∈Z

X (f ) ∗ δ(f −n

T)

F(y(t)) =1

T

∑n∈Z

X (f −n

T)

The Fourier representation of the sampled function is a superposition of the Fourier transformX (f ) and all its replicas at positions n

T . For n = 0, the spectrum is the primary spectrum andfor n 6= 0 we have copies of the spectrum.

41




Conclusion



Let x(t) be a continous function, its sampled version y(t), y(t) = x(t)× ΠT (t)Let X (f ) be the Fourier transform of x(t) and Y (f ) the Fourier transform of itssampled version y(t).

Y (f ) =1

T

∑n∈Z

X (f −n

T)

The sampling operation replicates the primary spectrum periodically in the Fourierdomain.

42




Conclusion



43




Conclusion



Aliasing e�ect...

We won't be able to recover the originalsignal with a good quality. The

transformation is no more inversible!

The sampling frequency is too small.

44




Conclusion


Shannon-Nyquist theorem

To ensure a perfect reconstruction of a signal x(t) (to avoid aliasinge�ect), the sampling frequency fe must verify:

fe > 2× fmax

where fmax is the maximal frequency of the spectrum X (f ).

The periodization doesn't provoke an aliasing e�et.

45




Conclusion


Whittaker-Shannon theorem

If x(t) is bandlimited to afrequency range [−fmax , fmax ],

that is, its Fourier transform X (f )is zero for |f | > fmax , then x(t)can be completely speci�ed bysamples taken at the Nyquist

sample rate of 2fmax .

46




Conclusion


Whittaker-Shannon theorem

To recover the signal x(t), we can use alow-pass �lter in frequency domain. Forinstance, the ideal low-pass �lter in the

frequency domain is given byG(f ) = Te rect(f /fe)

X (f ) = Y (f )× rect(f /fe)

x(t) =+∞∑

k=−∞x(kTe)sinc((t − kTe)fe)

47




Conclusion


The sampling of a physical signal always induces an aliasing e�ect, even with an idealsampler. Paley-Wiener theorem indeed showed that there is no bandlimited signalhaving a �nite energy...

In many cases, the spectrum of the signal is not perfectly known and can be corruptedby noise. It is thus required to �lter the analog signal before the sampling operation.

1 Pre-�ltering to obtain a bandlimited signal;

2 Sampling by checking the value of the sampling frequency;

3 Low-pass �ltering to retrieve the signal.

48




Conclusion

De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples

Discrete Fourier Transform

1 Introduction



4 Time sampling

5 Discrete Fourier TransformDe�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples



8 Conclusion

49




Conclusion


De�nition of 1D DFT

The discrete Fourier transform (DFT) is the transform of a �nite sequence. Since thetime-domain is discrete the spectrum is periodic

The DFT of a sequence of N complex numbers x0, . . . , xN−1 is given by:

X [k] = 1N

∑N−1n=0 x[n]exp(−j 2π

Nnk), k = 0, . . . ,N − 1

⇐⇒X [k] = 〈x , ek〉, where ek(n) = exp(j 2π

Nnk).

The family{ek(n) = exp(j 2π

Nnk)}0≤k<N

is an orthogonal basis of the space of

signals of period N.

The IDFT of a sequence of N complex numbers X0, . . . ,XN−1 is given by:

x[n] =∑N−1

n=0 X [k]exp(j 2πNnk), n = 0, . . . ,N − 1

The DFT computes the X [k] from the x[n], while the IDFT shows how to compute

the x[n] as a sum of sinusoidal components X [k]exp( j2πN

kn) with frequency k/Ncycles per sample.

50




Conclusion


De�nition of 1D DFT

X [k] =1

N

N−1∑n=0

x[n]exp(−j2π

Nnk)

The weighting coe�cient 1N

can be removed or replaced by 1√N. In the �rst case, the

weighting coe�cient of the IDFT is equal to 1N

whereas in the second case it is equal

to 1√N.

What is important is that DFT ◦ IDFT = Id .

51




Conclusion


Discrete bi-dimensional Fourier transformation

im[k, l ] is a discrete picture having a size N ×M.

Inverse Fourier Transform:

im[k, l ] =N−1∑u=0

M−1∑v=0

IM[u, v ]exp(j2π(k

Nu +

l

Mv))

Fourier Transform:

IM[u, v ] =1√NM

N−1∑k=0

M−1∑l=0

im[k, l ]exp(−j2π( kNu +

l

Mv))

u = 0, . . . ,N − 1, υk = uN, υk ∈ [0, 1];

v = 0, . . . ,M − 1, υl =vM, υl ∈ [0, 1].

The unit is cycles per unit length (pel−1).

52




Conclusion


Discrete bi-dimensional Fourier transformation

A frequential decomposition in terms of exponential basis images ...

im[k, l ] = IM[0, 0]× exp(j2π(0× k + 0× l)) +

IM[0, 1]× exp(j2π(0× k + l)) +

IM[1, 0]× exp(j2π(k + 0× l)) + . . .

im[k, l ] = IM[0, 0] +

IM[0, 1] (cos(2πl) + jsin(2πl)) +

IM[1, 0] (cos(2πk) + jsin(2πk)) +

IM[1, 1] (cos(2π(k + l)) + jsin(2π(k + l))) + . . .

IM[u, v ] are the weighting coe�cients of the spatial frequencies presentin the signal.

53




Conclusion


Some remarks:

1 A picture having a size of N ×M is a linear combination of N ×Mexponential basis images!!!!

2 The DFT of a picture yields a complex picture of the same size;

54




Conclusion


Some remarks:

3 IM[u, v ] and im[k, l ] are periodic of in�nite extent. Period = N for k, u. Period =M for l , v :

The image imt is bandlimited and is spatially sampled, for example, with arectangular array of N ×M pels:

im[k, l ] =

N−1∑i=0

M−1∑j=0

imt(k, l)δ(k − iTK )δ(l − jTL)

55




Conclusion


Some remarks:

56




Conclusion


Translation

Translation in the frequency domain

im[k, l ]exp

(j2π

N(u0k + v0l)

)DFT−→ IM[u − u0, v − v0]

Translation in the spatial domain

im[k − k0, l − l0]DFT−→ IM[u, v ]exp

(−j 2π

N(k0u + l0v)

)

The multiplication by an exponential function in a representationmodi�es the origin in the other representation.

57




Conclusion


Average value

im =1

NM

N−1∑k=0

M−1∑l=0

im[k, l ]

IM[u, v ] =1√NM

N−1∑k=0

M−1∑l=0

im[k, l ]exp(−j2π( kNu +

l

Mv))

IM[0, 0] =1√NM

N−1∑k=0

M−1∑l=0

im[k, l ]

im =1√NM

IM[0, 0]

IM[0, 0] is proportional to the average value of the picture.

58

Separable transformation = separable basis decomposition

The 2D DFT can be computed as the separable product of two one-dimensionaldiscrete Fourier transforms.

IM[u, v ] =1√NM

N−1∑k=0

M−1∑l=0

im[k, l ]exp(−j2π(k

Nu +

l

Mv))

Initialy the family{ek,l [u, v ] = exp(−j2π( k

Nu + l

Mv))}0≤k<N,0≤l<M

is used, but we

can also use the family {ek(u)el (v)}0≤k<N,0≤l<M .

IM[u, v ] =1√N

N−1∑k=0

exp(−j2πk

Nu)

[1√M

M−1∑l=0

im[k, l ]exp(−j2πl

Mv)

]

IM[u, v ] =1√N

N−1∑k=0

exp(−j2πk

Nu)[√

MDFT1D(im[k, l ])]

IM[u, v ] =√NDFT1D

[√MDFT1D(im[k, l ])

]We start by computing the Fourier coe�cients on the image rows int the basis

{el}0≤l<M . A second transform is applied on the columns of the transformed images

in the basis {ek}0≤k<N .




Conclusion



60




Conclusion



For a picture having a size N × N, there are three ways to compute its DFT:

DFT 1D: N2 multiplications and N(N − 1) additions, O(N2)

X [k] =1

N

N−1∑n=0

x[n]exp(−j2π

Nnk)

DFT 2D: N2 × N2 multiplications and N2(N − 1)2 additions, O(N4)

IM[u, v ] =1

N

N−1∑k=0

N−1∑l=0

im[k, l ]exp(−j2π(k

Nu +

l

Nv))

DFT 2D separable transform: 2N × N2 multiplications and 2N × N(N − 1)2

additions, O(N3)

61




Conclusion


Examples

Remark: the spatial frequency amplitude spectra of natural images are well describedby a power law X (f ) = k

f β. Estimated average values of β are ranged over 0.9 and

1.2. The power always falls with spatial frequency.

Power spectrum of a natural image (solid line) compared with 1f 2

(dashed line).

From E.P. Simoncelli and B.A.Olhausen, Natural Image Statistics and Neural Representation, Annun. Rev.

Neuroscience, 2001.

62




Conclusion


Examples

The spectra may be more or lessanisotropic.

From R.M. Balboa, N.M. Grzywacz, Power spectra and distribution of contrasts of natural images from di�erent

habitats, Vision Research, 43, pp. 2527-2537, 2003.

63




Conclusion


Examples

From left hand-side to right: original picture, spectrum, contour lines.

Remark: For display purpose, a logarithm is applied on the spectrum.

64




Conclusion

PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer

Scalar Quantization

1 Introduction



4 Time sampling


6 Scalar QuantizationPrincipleUniform quantizationOptimal quantization, Lloyd-MaxalgorithmExamples: uniform, semi-uniformand optimal quantizer


8 Conclusion

65




Conclusion


Principle

The quantization is a process to represent a large set of values with a smaller set.Scalar quantization:

Q : X 7−→ C = {yi , i = 1, 2, ...N}x Q(x) = yi

N is the number of quantization level;

X could be continue (ex: R) or discret;C is always discret (codebook,dictionnary);

card(X ) > card(C);

As x 6= Q(x), we will lost some information (lossy compression).

66




Conclusion


Uniform quantization

In the uniform quantization, the quantization step size ∆ is �xed, no matter what thesignal amplitude is.

input x

output y

t6 t7 t8 t9t4t3t2t1

y2

y1

∆

{ti }, decision levels

{yi }, representative levels

De�nition:

The quantization thresholds areuniformly distributed:∀i ∈ {1, 2, ...,N}, ti − ti+1 = ∆

The output values are the center ofthe quantization interval:

∀i ∈ {1, 2, ...,N}, yi =ti+ti+1

2

Example of the nearest neighborhoodquantizer (see on the left):

Q(x) = ∆×⌊x∆

+ 0.5⌋.

67




Conclusion



A uniform quantization is completely de�ned by the number of levels, the quantizationstep and if it is a mid-step or mid-riser quantizer.

A mid-step quantizerZero is one of the representative levels yk

input x

output y

t6 t7 t8 t9t4t3t2t1

y2

y1

∆

A mid-riser quantizerZero is one of the decision levels tk

input x

output y

t6 t7 t8 t9t4t3t2t1

y2

y1

Usually, a mid-riser quantizer is used if the number of representative levels is even and a mid-stepquantizer if the number of level is odd.

68




Conclusion


Uniform quantization with dead zone


input x

output y

t6 t7 t8 t9t4t3t2t1

y2

y1

∆

Uniform quantization with dead zone

input x

output y

t6 t7 t8 t9t4t3t2t1

y2

y1

Deadzone

Interest:To remove small coe�cients by favouring the zero value. Increase the codinge�ciency with a small visual impact.

69




Conclusion


Example of an uniform quantization

Example (Original picture quantized to 8, 7, 6, 4 bits/pixels)

(a) Original (8 bits per pixel)

(b) 7 bpp (∆ = 2) (c) 6 bpp (∆ = 4) (d) 4 bpp (∆ = 8)

70




Conclusion


Optimal quantization

Optimal quantizationAn optimal quantization of a random variable X having a probability distribution p(x)is obtained by a quantizer that minimises a given metric:

Linf -norm: D = max |X − Q(X )|;L1-norm: D = E [|X − Q(X )|];L2-norm: D = E

[(X − Q(X ))2

]called the Mean-Square Error (MSE). This is

the most used.

Considering the MSE, we have:

if the random variable X is continue: D =∫ xmaxxmin

p(x)(x − Q(x))2dx ;

if the random variable X is discret: D =∑n

k=1 p(xk)(xk − Q(xk))2.

71




Conclusion


Example

Example (Quantization error)

Hypothesis:

Uniform quantization with a quantization step ∆;

p(x) is a uniform probability distribution of a random variable X ;

N is the number of representative levels.

xmax − xmin

∆

xmin xmax∆2−∆

2

Quantization step: ∆ =xmax−xmin

N

Probability distribution: p(x) = 1xmax−xmin

= 1N∆

72




Conclusion


Example

Example (Quantization error)

D =

∫ xmax

xmin

p(x)(x − Q(x))2dx

= N ×∫ ∆/2

−∆/2p(x)(x − 0)2dx , 0 the mid-point

= N ×∫ ∆/2

−∆/2

1

N∆x2dx

=∆2

12

73




Conclusion


Optimal quantizer in a nutshell

Uniform quantizer is not optimal if the source is not uniformly distributed.

Optimal quantizer

To �nd the decision levels {ti} and the representative levels {yi} to minimize thedistortion D.

To reduce the MSE, the idea is to decrease the bin's size when the probability ofoccurrence is high and to increase the bin's size when the probability is low. For Nrepresentative levels and with a probability density p(x), the distortion is given by:

D =

∫ xmax

xmin

p(x)(x − Q(x))2dx

=

N−1∑k=1

∫ tk+1

tk

p(x)(x − yk)2dx

74




Conclusion


Optimal quantizer in a nutshell

The optimal {ti} and {yi} satisfy:

∂D∂ti

= 0 and ∂D∂yi

= 0

Lloyd-Max quantizer

∂D∂ti

= 0 ⇒ ti =yi+yi+1

2.

ti is the midpoint of yi and yi+1.

∂D∂yi

= 0 ⇒ yi =

∫ titi−1

p(x)xdx∫ titi−1

p(x)dx.

yi is the centroid of the interval [ti−1, ti ].

⇒ given the {ti}, we can �nd the corresponding optimal {yi}.⇒ given the {yi}, we can �nd the corresponding optimal {ti}.

How can we �nd the optimal {ti} and {yi} simultaneously?

75




Conclusion


Lloyd-Max algorithm in a nutshell

The Lloyd-Max algorithm is an algorithm for �nding the representative levels {yi} and thedecision levels {ti} to meet the previous conditions, with no prior knowledge.

Principle of the iterative process

1 The iterative process starts for k = 0 with a set of initial values for the representative

levels{y

(0)1 , ..., y

(0)N

}.

2 New values for decision levels are determined t(k+1)i =

y(k)i

+y(k)i+1

2

3 Compute the distortion D(k) and the relative errors δ(k);

4 Depending on the stopping criteria (δ(k) < ε), stop the process or update the

representative levels y(k+1)i =

∫ t(k+1)i

t(k+1)i−1

xp(x)dx

∫ t(k+1)i

t(k+1)i−1

p(x)dx

and go back to step 2.

76




Conclusion


Uniform, Semi-uniform and optimal quantizer

Suppose we have the following 1D discrete signal:

X = {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}

Example (Uniform quantizer: N = 4, Mid-riser and ∆ = 2)

input x

output y

02 4 6 8

1

3

5

7

ti ∈ T = {t0 = 0, t1 = 2, t2 = 4, t3 = 6, t4 = 8}

ri ∈ R = {r0 = 1, r1 = 3, r2 = 5, r3 = 7}

Check the de�nition of a uniform quantizer...

Quantized vector X = {1, 1, 3, 3, 1, 3, 5, 3, 5, 7, 7} (MSE = 0.42)

77




Conclusion


Uniform, semi-uniform and optimal quantizer


X = {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}

Example (Semi-uniform quantizer: N = 4, Mid-riser and ∆ = 2)

input x

output y

02 4 6 8

1

3

5

7

ti ∈ T = {t0 = 0, t1 = 2, t2 = 4, t3 = 6, t4 = 8}

ri ∈ R = {r0 = 2/3, r1 = 3.25, r2 = 4.75, r3 = 7.5}

Quantized vector X = {2/3, 2/3, 3.25, 3.25, 2/3, 3.25, 4.75, 3.25, 4.75, 7.5, 7.5}(MSE = 0.31)

78




Conclusion


Uniform, semi-uniform and optimal quantizer


X = {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}

Example (Lloyd-Max Algorithm)

Given that T = {0, 1.5, 3.87, 6.125, 8} and R = {0.005, 2.998, 4.75, 7.5}, thequantized vector isX = {0.005, 0.005, 2.998, 2.998, 2.998, 2.998, 4.75, 2.998, 4.75, 7.5, 7.5} (MSE=0.18).

input x

output y

02 4 6 8

1

3

5

7Uniform

Semi-uniform

Optimal

xi ∈ X

79




Conclusion


Uniform vs optimal quantizer

Example (N=5)

(a) Original (b) Uniform (c) Optimal

(d) Histo. (e) Decision levels (f) Decision levels

80




Conclusion

TaxonomySignal �delityPerceptual metricExamples

Distortion/quality assessment

1 Introduction



4 Time sampling



7 Distortion/quality assessmentTaxonomySignal �delityPerceptual metricExamples

8 Conclusion

81




Conclusion


Taxonomy

Distortion/quality metrics can be divided into 3 categories:

1 Full-Reference metrics (FR) for which both the original and the distorted imagesare required (benchmark, compression) ;

2 Reduced-Reference metrics (RR) for which a description of the original and thedistorted image is required (network monitoring);

3 No-Reference (NR) metrics for which the original image is not required (networkmonitoring).

Each category can be divided into two subcategories: metrics based on signal �delityand metrics based on properties on the human visual system.

82




Conclusion


Peak Signal to Noise Ratio (PSNR)

The PSNR is the most popular quality metric. This simple metric just calculates themathematical di�erence between each pixel of the degraded image and the originalimage.

PSNR:Let I and D the original and impaired images, respectively. These images having a size

of M pixels are coded with n bits.

PSNR = 10log10((2n−1)2)

MSEdB,

with the Mean Squared Error MSE =

∑(x,y)(I (x,y)−D(x,y))2

M.

A high value indicates that the amount of impairment is small. A small value indicates

that there is a strong degradation.

83




Conclusion



The PSNR is not always well correlated with the human judgment (MOS Mean OpinionScore). The reason is simple: this metric does not take into account the properties of thehuman visual system.

Example

(a) (b) (c) Original (d) Original+uniform noise

(a) and (b) from [?]: impact of Gabor patch on our perception.

84




Conclusion



Example (These three pictures have the same PSNR...)

(a) Original (b) Contrast stretched

(c) Blur (d) JPEG

85




Conclusion


Metric based on the error visibility

For this type of metric, the behavior of the visual cells are simulated:

Perceptual Color Space PSD CSF Masking

−

Perceptual Color Space PSD CSF Masking

Pooling Quality

I

D

PSD: Perceptual Subband Decomposition (Wavelet, Gabor, Fourier);

CSF: Contrast Sensitivity Function.

86




Conclusion


Metric based on the error visibility

Example

VDP (Visible Di�erences Predictor) [Daly,93]:

WQA (Wavelet-based Quality Assessment) [Ninassi et al.,08a]

VQM (Video Quality Model) [Pinson et al.,04]...

87




Conclusion


Metric based on the structural similarity

SSIM standing for Structural Similarity index. Image degradations are con-

sidered here as perceived structural information loss instead of perceived errors [Wang et al.,04a].

Let I and D the original and the degraded images, respectively.

S(x, y) = l(x, y) × c(x, y) × s(x, y) (1)

The luminance comparison measure l(x, y) =2µxµy

µ2x + µ2y

The contrast comparison measure c(x, y) =2σxσy

σ2x + σ2y

The structural comparison measure s(x, y) =σxy

σxσy

SSIM(x, y) =(2µxµy + C1)(2σxy + C2)

(µ2x + µ2y + C1)(σ2x + σ2y + C2)

SSIM → 1, the best quality and SSIM → 0 indicates a poor quality.

88




Conclusion


Metric based on the structural similarity

All pictures have the same MSE. (a) Original image; (b) Contrast-stretched image,MSSIM = 0.9168; (c) Mean-shifted image, MSSIM = 0.99; (d) JPEG,

MSSIM = 0.6949; (e) Blurred image, MSSIM = 0.7052; (f) Impluse noise,MSSIM = 0.7748. Extracted from [Wang et al.,04a].

89




Conclusion


Example of distortion maps

Example

(a) Original (b) Degraded

(c) MSE (d) WQA (e) SSIM

90




Conclusion

Distortion/quality assessment

1 Introduction



4 Time sampling




8 Conclusion

91




Conclusion

Conclusion

Image Transformation: global to point;

Fourier transformation;

Time sampling with Shannon's theorem;

Scalar quantization, a lossy process;

Quality and distortion.

92

Suggestion for further reading...

S. Daly. The visible di�erences predictor: An algorithm for the assessment ofimage �delity. Digital Images and Human Vision, pp. 179-206, 1993, MIT Press.

A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba. On the performance ofhuman visual system based image quality assessment metric using waveletdomain. Proc. SPIE Human Vision and Electronic Imaging XIII., Vol. 6806, pp.680610-12, 2008.

M. Pinson and S. Wolf. A new standardized method for objectively measuringvideo quality. IEEE Trans. Broadcasting, Vol. 50, N. 3, pp. 312-322, 2004.

Z. Wang, A. C. Bovik, H.R. Sheikh and E.P. Simoncelli. Image qualityassessment: from error visibility to structural similarity. IEEE Trans. on ImageProcessing, Vol. 13, pp. 600-612, 2004.

92

digital image processing image transform (fourier ... · properties some examples fourier...

Documents