Table of Content
Digital Image ProcessingImage Transform (Fourier) / Quantization / Quality
O. Le [email protected]
Univ. of Rennes 1http://www.irisa.fr/temics/staff/lemeur/
December 6, 2010
1
Table of Content
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
2
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Introduction
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
3
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Introduction
What is a signal?
This is a function of one or more variables (time, distance, temperature...).
A signal carries information!!
Signal processing is required to obtain a more desirable form.
Types of signals:
Natural (lighting) or synthetic (from a lab);
one dimension or more;
Continuous/discrete.
4
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Introduction
Classi�cation of signals
Continuous-time vs Discrete-time signals:→ A continuous-time signal contains a value for all real numbers along the time axis;→ A discrete-time signal only has values at equally spaced intervals along the time axis.
Analog vs Digital:→ Analog signal corresponds to a continuous set of possible function values;→ Digital signal corresponds to a discrete set of possible function values.
Deterministic vs Random Signals:→ The past, present and future of a deterministic signal are known with certainty (its
values can be determined by a mathematical expression, rule or table). Otherwise, itis called Random and its properties are explained by statistical techniques.
Periodic vs Aperiodic: periodic signals repeat with some period T ;
Finite vs In�nite length.
5
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Introduction
Classi�cation of signals
Sampling = discrete time (or space) Quantization = discrete amplitude
6
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Introduction
There are many di�erent ways to describe or represent a signal mathematically.
Most of the time a linear combination of elementary time functions. These functionsare called basis functions.
Let the set of basis functions be de�ned by: {φp}, (di�erent posssibilities for therange of p)
x(t) =∑n
anφn(t)
7
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Notations and reminder
a continuous signal x(.);
a discrete signal x[.];
a matrix A.
8
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Notations and reminder
f is the frequency (Hz); w = 2πf is the pulsation (radian/s); T = 1fis the period
(s);
x ∈ Lp(R) means that∫ +∞−∞ |x(t)|p dt < +∞. For instance, the space L2(R) is
composed of the functions x with a �nite energy∫ +∞−∞ |x(t)|2 dt < +∞;
Euler's formulas: exp(jθ) = cos θ + j sin θ;
exp(jθ) = cos θ + j sin θexp(−jθ) = cos θ − j sin θ
exp(jθ) + exp(−jθ) = 2 cos θexp(jθ)− exp(−jθ) = 2j sin θ
From Wikepedia.
9
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Reminder
sinc(x) = sinπxπx
;
Inner product:
〈x , y〉 =
∫ +∞
−∞x(t)× y∗(t)dt
〈x , y〉 =+∞∑
n=−∞x(n)× y∗(n)
Condition of orthogonality for real basis function:∫ t2
t1
φn(t)φm(t)dt =
{0 n 6= m
λm n = m
where λm is a constant (λm = 1, orthonormal).
Condition of orthogonality for complex basis function:∫ t2
t1
φn(t)φ∗m(t)dt =
{0 n 6= m
λm n = m
where λm is a constant (λm = 1, orthonormal).
10
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Image transformation
1 Introduction
2 Image transformationPresentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
11
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Image transformation
What is a transformation?
im[x , y ]T−→ IM[u, v ]
im is the original image;
IM is the transformed image;
x , y (or u, v) represents the spatial coordinates of a pixel.
Goal of a transformation
The goal of a transformation is to get a new representation of the incoming picture.This new representation can be more convenient for a particular application or canease the extraction of particular properties of the picture.
12
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Image transformation
There exist 3 types of transformation:
Point to point transformation:The output value at a speci�c coordinateis dependent only on one input value butnot necessarily at the same coordinate;
Local to point transformation:The output value at a speci�c coordinateis dependent on the input values in theneighborhood of that same coordinate;
Global to point transformation:The output value at a speci�c coordinateis dependent on all the values in the inputimage.
Note that the complexity increases with the size of the considered neighborhood...
13
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Point to point transformation
Geometric transformation such as rotation, scaling...Example of rotation:
Scalar quantization
LUT (Look-Up-Table)
Remark: a point to point transformation doesn't require extra memory...
14
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Example of local to point transformation
Note that local to point transformation is also called neighborhood operators.
Examples of local to point transformation:
Linear transformation:For instance, a tranverse �lter (FIR) IM[x , y ] =
∑i
∑j h(i , j)im[x − i , y − j],
where h is the 2D impulse response.
Non-linear transformation such as adaptive linear �ltering and rank �ltering(median)...
Morphological �ltering (erosion, dilatation...) based on the de�nition of astructuring element.
Remark: Non linear �lters are not reversible in general.
15
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Global to point transformation
One of the most important transformation is the Fourier transform that gives afrequential representation of the signal.
Fourier series (development in real term):
Every signal x (x ∈ L2(t1, t1 + T )) can be decomposed into a linear combination ofsinusoidal and cosinusoidal component functions.
x(t) = b0 ++∞∑n=1
ancos(2πnt) ++∞∑n=1
bnsin(2πnt)
an and bn are used to weight the in�uence of each frequency component.
16
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Global to point transformation
Fourier series (development in complex term):
x(t) =+∞∑
n=−∞Cnexp(j2πn
t
T)
with,
Cn =1
T
∫ t1+T
t1
x(t)exp(−j2πnt
T)dt
Remark:
Cn = an−jbn2
, n ≥ 1 ; Cn =a−n+jb−n
2, n ≤ 1 ; C−n = C∗n ;
Note that if the function to be represented is also T -periodic, then t1 is anarbitrary choice. Two popular choices are t1 = 0, and t1 = −T/2.
17
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation
Global to point transformation
18
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
1 Introduction
2 Image transformation
3 Fourier transformationDe�nitionTrivial propertiesParseval FormulaPropertiesSome examples
4 Time sampling
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
19
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Fourier transformation in L2(R)
x(t)F−→ X (f )
Inverse Fourier Transform:
x(t) =
∫ +∞
−∞X (f )exp(j2πft)df
Fourier Transform:
X (f ) =
∫ +∞
−∞x(t)exp(−j2πft)dt
provided that x ∈ L2(R) and X ∈ L2(R).
20
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
From the output of the Fourier transform, we de�ne:
The frequency spectrum: Real (X (f )) + jImg ((X (f )))The Fourier transform of a function produces a frequency spectrum whichcontains all of the information about the original signal, but in a di�erent form.
Magnitude spectrum: |Real (X (f )) + jImg (X (f ))|
Phase spectrum: Arctg(Img(X (f ))Real(X (f ))
)Power spectrum: Real (X (f ))2 + Img (X (f ))2
21
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Linearity:
ax(t)F−→ aX (f )
ax1(t) + bx2(t)F−→ aX1(f ) + bX2(f )
Complex conjugate: x∗(t)F−→ X∗(−f )
x∗(t) =
(∫ +∞
−∞X (f )exp(j2πft)df
)∗x∗(t) =
∫ +∞
−∞X∗(f )exp(−j2πft)df
In the same way, x∗(−t) F−→ X∗(f ) and x(−t) F−→ X (−f ).
22
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Hermitian symmetry: if x(t) ∈ R, we deduce X (−f ) = X∗(f )
X (f ) =
∫ +∞
−∞x(t)exp(−j2πft)dt
X∗(f ) =
(∫ +∞
−∞x(t)exp(−j2πft)dt
)∗X∗(f ) =
∫ +∞
−∞x∗(t)exp(j2πft)dt
Given that x(t) ∈ R, we have x∗(t) = x(t) that implies X∗(f ) = X (−f ).
23
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Note that, as we use the space L2(R), the inner product of x ∈ L2(R) and y ∈ L2(R)is given by:
〈x , y〉 =
∫ +∞
−∞x(t)y∗(t)dt
The norm is then given by:
‖x‖2 = 〈x , x〉 =∫ +∞−∞ |x(t)|2 dt∫ +∞
−∞|x(t)|2 dt =
∫ +∞
−∞x(t)x∗(t)dt
=
∫ +∞
−∞x(t)
[∫ +∞
−∞X∗(f )exp(−j2πft)df
]dt
=
∫ +∞
−∞X∗(f )
[∫ +∞
−∞x(t)exp(−j2πft)dt
]df
=
∫ +∞
−∞X∗(f )X (f )df∫ +∞
−∞|x(t)|2 dt =
∫ +∞
−∞|X (f )|2 df
24
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
∫ +∞
−∞|x(t)|2 dt =
∫ +∞
−∞|X (f )|2 df
This formula (Parseval's theorem or energy conservation) proves that theenergy is conserved by the Fourier transform.
Loosely, the sum (or integral) of the square of a function is equal to thesum (or integral) of the square of its transform.
25
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Translation in time/space domain: x(t − t0)F−→ exp(−j2πt0f )X (f )
x(t − t0)F−→
∫ +∞
−∞x(t − t0)exp(−j2πtf )dt
k = t − t0
x(t − t0)F−→
∫ +∞
−∞x(k)exp(−j2π(k + t0)f )dk
x(t − t0)F−→ exp(−j2πt0f )
[∫ +∞
−∞x(k)exp(−j2πfk)dk
]Displacement in time or space induces a phase shift proportional to frequency and
to the amount of displacement.
Frequency shift: x(t)exp(±j2πf0t)F−→ X (f ± f0)
Displacement in frequency multiplies the time/space function by a unit phasorwhich has angle proportional to time/space and to the amount of displacement.
Amplitude modulation.
26
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Convolution in time/space domain: x1(t) ∗ x2(t)F−→ X1(f )× X2(f )
x1(t) ∗ x2(t)F−→
∫ +∞
−∞
[∫ +∞
−∞x1(τ)x2(t − τ)dτ
]exp(−j2πft)dt
x1(t) ∗ x2(t)F−→
∫ +∞
−∞x1(τ)
[∫ +∞
−∞x2(t − τ)exp(−j2πft)dt
]dτ
Translation
x1(t) ∗ x2(t)F−→
[∫ +∞
−∞x1(τ)exp(−j2πf τ)dτ
]× X2(f )
Convolution in frequency domain: x1(t)× x2(t)F−→ X1(f ) ∗ X2(f )
These properties mean that if two functions are multiplied in one domain, thentheir Fourier transforms are convolved in the other domain.
27
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Time and frequency scaling: x(λt)F−→ 1|λ|X ( f
λ)
x(λt)F−→
∫ +∞
−∞x(λt)exp(−j2πtf )dt
k = λt
x(λt)F−→
∫ +∞
−∞x(k)exp(−j2πk
f
λ)dk
|λ|
x(λt)F−→
1
|λ|
[∫ +∞
−∞x(k)exp(−j2πk
f
λ)dk
]
Multiplication of the scale of the time/space reference frame changes by thefactor λ scales the frequency axis of the spectrum of the function.
All the previous properties are what makes Fourier transforms so usefuland practical.
28
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Translated Dirac, x(t) = δ(t − T )
X (f ) =
∫ +∞
−∞x(t)exp(−j2πtf )dt
X (f ) = exp(−j2πTf )
If T = 0, X (f ) = 1. The Fouriertransform is constant whatever thefrequency.
x(t) = 1, X (f ) = δ(f )
x(t) = Arect(t|T )
X (f ) =
∫ +∞
−∞x(t)exp(−j2πtf )dt
= A
∫ +T2
−T2
exp(−j2πtf )dt
= −A
j2πf[exp(−jfTπ)− exp(jfTπ)]
X (f ) = ATsinc(fT )
29
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
Ideal low-pass �lter in frequency domain: X (f ) = rect(f |F2
) selects low
frequencies over[−F
4, F4
]. The impluse response of the �lter is calculated with
the inverse Fourier integral:
x(t) =
∫ +∞
−∞X (f )exp(j2πft)df
=
∫ + F4
− F4
exp(j2πft)df
=1
j2πt
[exp(j2π
F
4t)− exp(−j2π
F
4t)
]sinθ =
exp(jθ)− exp(−jθ)
2j
x(t) =F
2sinc(
π
2tF )
30
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nitionTrivial propertiesParseval FormulaPropertiesSome examples
Fourier transformation
31
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Time sampling
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time samplingWhat is the objective?Reminder on DiracSamplingSampling TheoremIn practice
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
32
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Time sampling
To be more �exible and to gain in accuracy...
Digital algorithms are now ... everywhere from television, CD, embeddedelectronic (car...).
Whether sound recordings or images, most discrete signals are obtainedby sampling an analog signal. But, be careful, we have to follow some
constraints in order to be able to reconstruct the analog signal.
The simplest way to discretize an analog signal x is to record its samplevalues at intervals T : x(t) can be approximated by {x(nT )}n∈Z .
33
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
De�nition
Diracs are essential to make the transition from functions of real variable to discretesequences.
A Dirac can be de�ned as: ∫ +∞
−∞δ(t)x(t)dt = x(0)
δ(t) =
{0 if t 6= 0
+∞ for t = 0∫ +∞
−∞δ(t)dt = 1.
Remark: {δτ : δ(t − τ)}τ∈Z is an orthonormal basis:
〈δ(t − n), δ(t −m)〉 =
{0 if m 6= n
1 for m = n.
34
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Properties of a Dirac
Scaling and symmetry: δ(λt) = δ(t)|λ|∫ +∞
−∞δ(λt)dt =
∫ +∞
−∞δ(k)
dk
|λ|
δ(λt) =δ(t)
|λ|
Note that if λ = −1, δ(−t) = δ(t).
Product: x(t)× δ(t − T ) = x(T )× δ(t − T )
Convolution: x(t) ∗ δ(t − T )
x(t) ∗ δ(t − T ) =
∫ +∞
−∞x(τ)δ(t − T − τ)dτ
=
∫ +∞
−∞x(τ)δ(τ − (t − T ))dτ
= x(t − T )
The e�ect of convolving with a translated Dirac is to time-delay x(t), by thesame amount (copy of the signal x(t) at the position T ).
35
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Dirac comb: x(t) =∑
n∈Z δ(t − nT ). Notation: ΠT (t)
The product of a comb with a signal x(t) conducts to a sampling:
The convolution of a comb with a signal x(t) conducts to a periodization of x(t):
36
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
FT of a Dirac
Dirac: x(t) = δ(t), δ(t)F−→ 1, constant function whatever the frequency
X (f ) =
∫ +∞
−∞δ(t)exp(−j2πtf )dt
X (f ) = exp(−j2π0f ) = 1
Translated Dirac: x(t) = δ(t − T ), δ(t − T )F−→ exp(−j2πTf )
X (f ) =
∫ +∞
−∞δ(t − T )exp(−j2πtf )dt
X (f ) = exp(−j2πTf )
Constant signal: x(t) = 1, 1F−→ δ(f ). The FT of a constant signal is a weight at
the null frequency.
37
Dirac comb: x(t) =∑
n∈Z δ(t − nT )This is a periodic function, since x(t +T ) = x(t), ∀t. Its Fourier serie is given by:
x(t) =+∞∑
n=−∞Cnexp(j2πn
t
T)
Cn =1
T
∫ T2
−T2
δ(t)exp(−j2πnnt
T)dt =
1
T
x(t) =1
T
+∞∑n=−∞
exp(j2πnt
T)
By using the Fourier Trasnform,
F(x(t)) = F
1
T
+∞∑n=−∞
exp(j2πnt
T)
F(x(t)) =
1
T
+∞∑n=−∞
F(exp(j2πn
t
T)
)
F(x(t)) =1
T
+∞∑n=−∞
∫ +∞
−∞exp(−j2πt(f −
n
T))dt
∑n∈Z
δ(t − nT )F−→
1
T
∑n∈Z
δ(f −n
T)
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Sampling
Sampling a signal is equivalent to multiplying it by a grid of impulses.
39
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Sampling
Let x(t) be a continous function, its sampled version y(t) is given by:
y(t) = x(t)× ΠT (t)
= x(t)×∑n∈Z
δ(t − nT )
=∑n∈Z
x(t)δ(t − nT )
y(t) =∑n∈Z
x(nT )δ(t − nT )
where ΠT (t) is the sampling function (sampling frequency 1T).
40
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Fourier Transform of a sampled signal
Let x(t) be a continous function, its sampled version y(t), y(t) = x(t)× ΠT (t)
y(t) = x(t)×∑n∈Z
δ(t − nT )
F(y(t)) = F
x(t)×∑n∈Z
δ(t − nT )
F(y(t)) = X (f ) ∗ F
∑n∈Z
δ(t − nT )
F(y(t)) = X (f ) ∗
1
T
∑n∈Z
δ(f −n
T)
F(y(t)) =
1
T
∑n∈Z
X (f ) ∗ δ(f −n
T)
F(y(t)) =1
T
∑n∈Z
X (f −n
T)
The Fourier representation of the sampled function is a superposition of the Fourier transformX (f ) and all its replicas at positions n
T . For n = 0, the spectrum is the primary spectrum andfor n 6= 0 we have copies of the spectrum.
41
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Fourier Transform of a sampled signal
Let x(t) be a continous function, its sampled version y(t), y(t) = x(t)× ΠT (t)Let X (f ) be the Fourier transform of x(t) and Y (f ) the Fourier transform of itssampled version y(t).
Y (f ) =1
T
∑n∈Z
X (f −n
T)
The sampling operation replicates the primary spectrum periodically in the Fourierdomain.
42
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Fourier Transform of a sampled signal
43
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Fourier Transform of a sampled signal
Aliasing e�ect...
We won't be able to recover the originalsignal with a good quality. The
transformation is no more inversible!
The sampling frequency is too small.
44
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Shannon-Nyquist theorem
To ensure a perfect reconstruction of a signal x(t) (to avoid aliasinge�ect), the sampling frequency fe must verify:
fe > 2× fmax
where fmax is the maximal frequency of the spectrum X (f ).
The periodization doesn't provoke an aliasing e�et.
45
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Whittaker-Shannon theorem
If x(t) is bandlimited to afrequency range [−fmax , fmax ],
that is, its Fourier transform X (f )is zero for |f | > fmax , then x(t)can be completely speci�ed bysamples taken at the Nyquist
sample rate of 2fmax .
46
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
Whittaker-Shannon theorem
To recover the signal x(t), we can use alow-pass �lter in frequency domain. Forinstance, the ideal low-pass �lter in the
frequency domain is given byG(f ) = Te rect(f /fe)
X (f ) = Y (f )× rect(f /fe)
x(t) =+∞∑
k=−∞x(kTe)sinc((t − kTe)fe)
47
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
What is the objective?Reminder on DiracSamplingSampling TheoremIn practice
The sampling of a physical signal always induces an aliasing e�ect, even with an idealsampler. Paley-Wiener theorem indeed showed that there is no bandlimited signalhaving a �nite energy...
In many cases, the spectrum of the signal is not perfectly known and can be corruptedby noise. It is thus required to �lter the analog signal before the sampling operation.
1 Pre-�ltering to obtain a bandlimited signal;
2 Sampling by checking the value of the sampling frequency;
3 Low-pass �ltering to retrieve the signal.
48
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Discrete Fourier Transform
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier TransformDe�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
49
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
De�nition of 1D DFT
The discrete Fourier transform (DFT) is the transform of a �nite sequence. Since thetime-domain is discrete the spectrum is periodic
The DFT of a sequence of N complex numbers x0, . . . , xN−1 is given by:
X [k] = 1N
∑N−1n=0 x[n]exp(−j 2π
Nnk), k = 0, . . . ,N − 1
⇐⇒X [k] = 〈x , ek〉, where ek(n) = exp(j 2π
Nnk).
The family{ek(n) = exp(j 2π
Nnk)}0≤k<N
is an orthogonal basis of the space of
signals of period N.
The IDFT of a sequence of N complex numbers X0, . . . ,XN−1 is given by:
x[n] =∑N−1
n=0 X [k]exp(j 2πNnk), n = 0, . . . ,N − 1
The DFT computes the X [k] from the x[n], while the IDFT shows how to compute
the x[n] as a sum of sinusoidal components X [k]exp( j2πN
kn) with frequency k/Ncycles per sample.
50
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
De�nition of 1D DFT
X [k] =1
N
N−1∑n=0
x[n]exp(−j2π
Nnk)
The weighting coe�cient 1N
can be removed or replaced by 1√N. In the �rst case, the
weighting coe�cient of the IDFT is equal to 1N
whereas in the second case it is equal
to 1√N.
What is important is that DFT ◦ IDFT = Id .
51
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Discrete bi-dimensional Fourier transformation
im[k, l ] is a discrete picture having a size N ×M.
Inverse Fourier Transform:
im[k, l ] =N−1∑u=0
M−1∑v=0
IM[u, v ]exp(j2π(k
Nu +
l
Mv))
Fourier Transform:
IM[u, v ] =1√NM
N−1∑k=0
M−1∑l=0
im[k, l ]exp(−j2π( kNu +
l
Mv))
u = 0, . . . ,N − 1, υk = uN, υk ∈ [0, 1];
v = 0, . . . ,M − 1, υl =vM, υl ∈ [0, 1].
The unit is cycles per unit length (pel−1).
52
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Discrete bi-dimensional Fourier transformation
A frequential decomposition in terms of exponential basis images ...
im[k, l ] = IM[0, 0]× exp(j2π(0× k + 0× l)) +
IM[0, 1]× exp(j2π(0× k + l)) +
IM[1, 0]× exp(j2π(k + 0× l)) + . . .
im[k, l ] = IM[0, 0] +
IM[0, 1] (cos(2πl) + jsin(2πl)) +
IM[1, 0] (cos(2πk) + jsin(2πk)) +
IM[1, 1] (cos(2π(k + l)) + jsin(2π(k + l))) + . . .
IM[u, v ] are the weighting coe�cients of the spatial frequencies presentin the signal.
53
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Some remarks:
1 A picture having a size of N ×M is a linear combination of N ×Mexponential basis images!!!!
2 The DFT of a picture yields a complex picture of the same size;
54
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Some remarks:
3 IM[u, v ] and im[k, l ] are periodic of in�nite extent. Period = N for k, u. Period =M for l , v :
The image imt is bandlimited and is spatially sampled, for example, with arectangular array of N ×M pels:
im[k, l ] =
N−1∑i=0
M−1∑j=0
imt(k, l)δ(k − iTK )δ(l − jTL)
55
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Some remarks:
56
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Translation
Translation in the frequency domain
im[k, l ]exp
(j2π
N(u0k + v0l)
)DFT−→ IM[u − u0, v − v0]
Translation in the spatial domain
im[k − k0, l − l0]DFT−→ IM[u, v ]exp
(−j 2π
N(k0u + l0v)
)
The multiplication by an exponential function in a representationmodi�es the origin in the other representation.
57
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Average value
im =1
NM
N−1∑k=0
M−1∑l=0
im[k, l ]
IM[u, v ] =1√NM
N−1∑k=0
M−1∑l=0
im[k, l ]exp(−j2π( kNu +
l
Mv))
IM[0, 0] =1√NM
N−1∑k=0
M−1∑l=0
im[k, l ]
im =1√NM
IM[0, 0]
IM[0, 0] is proportional to the average value of the picture.
58
Separable transformation = separable basis decomposition
The 2D DFT can be computed as the separable product of two one-dimensionaldiscrete Fourier transforms.
IM[u, v ] =1√NM
N−1∑k=0
M−1∑l=0
im[k, l ]exp(−j2π(k
Nu +
l
Mv))
Initialy the family{ek,l [u, v ] = exp(−j2π( k
Nu + l
Mv))}0≤k<N,0≤l<M
is used, but we
can also use the family {ek(u)el (v)}0≤k<N,0≤l<M .
IM[u, v ] =1√N
N−1∑k=0
exp(−j2πk
Nu)
[1√M
M−1∑l=0
im[k, l ]exp(−j2πl
Mv)
]
IM[u, v ] =1√N
N−1∑k=0
exp(−j2πk
Nu)[√
MDFT1D(im[k, l ])]
IM[u, v ] =√NDFT1D
[√MDFT1D(im[k, l ])
]We start by computing the Fourier coe�cients on the image rows int the basis
{el}0≤l<M . A second transform is applied on the columns of the transformed images
in the basis {ek}0≤k<N .
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Separable transformation = separable basis decomposition
60
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Separable transformation = separable basis decomposition
For a picture having a size N × N, there are three ways to compute its DFT:
DFT 1D: N2 multiplications and N(N − 1) additions, O(N2)
X [k] =1
N
N−1∑n=0
x[n]exp(−j2π
Nnk)
DFT 2D: N2 × N2 multiplications and N2(N − 1)2 additions, O(N4)
IM[u, v ] =1
N
N−1∑k=0
N−1∑l=0
im[k, l ]exp(−j2π(k
Nu +
l
Nv))
DFT 2D separable transform: 2N × N2 multiplications and 2N × N(N − 1)2
additions, O(N3)
61
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Examples
Remark: the spatial frequency amplitude spectra of natural images are well describedby a power law X (f ) = k
f β. Estimated average values of β are ranged over 0.9 and
1.2. The power always falls with spatial frequency.
Power spectrum of a natural image (solid line) compared with 1f 2
(dashed line).
From E.P. Simoncelli and B.A.Olhausen, Natural Image Statistics and Neural Representation, Annun. Rev.
Neuroscience, 2001.
62
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Examples
The spectra may be more or lessanisotropic.
From R.M. Balboa, N.M. Grzywacz, Power spectra and distribution of contrasts of natural images from di�erent
habitats, Vision Research, 43, pp. 2527-2537, 2003.
63
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
De�nition of 1D DFTDe�nition of 2D DFTRemarksPropertiesSeparable transformationExamples
Examples
From left hand-side to right: original picture, spectrum, contour lines.
Remark: For display purpose, a logarithm is applied on the spectrum.
64
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Scalar Quantization
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier Transform
6 Scalar QuantizationPrincipleUniform quantizationOptimal quantization, Lloyd-MaxalgorithmExamples: uniform, semi-uniformand optimal quantizer
7 Distortion/quality assessment
8 Conclusion
65
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Principle
The quantization is a process to represent a large set of values with a smaller set.Scalar quantization:
Q : X 7−→ C = {yi , i = 1, 2, ...N}x Q(x) = yi
N is the number of quantization level;
X could be continue (ex: R) or discret;C is always discret (codebook,dictionnary);
card(X ) > card(C);
As x 6= Q(x), we will lost some information (lossy compression).
66
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform quantization
In the uniform quantization, the quantization step size ∆ is �xed, no matter what thesignal amplitude is.
input x
output y
t6 t7 t8 t9t4t3t2t1
y2
y1
∆
{ti }, decision levels
{yi }, representative levels
De�nition:
The quantization thresholds areuniformly distributed:∀i ∈ {1, 2, ...,N}, ti − ti+1 = ∆
The output values are the center ofthe quantization interval:
∀i ∈ {1, 2, ...,N}, yi =ti+ti+1
2
Example of the nearest neighborhoodquantizer (see on the left):
Q(x) = ∆×⌊x∆
+ 0.5⌋.
67
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform quantization
A uniform quantization is completely de�ned by the number of levels, the quantizationstep and if it is a mid-step or mid-riser quantizer.
A mid-step quantizerZero is one of the representative levels yk
input x
output y
t6 t7 t8 t9t4t3t2t1
y2
y1
∆
A mid-riser quantizerZero is one of the decision levels tk
input x
output y
t6 t7 t8 t9t4t3t2t1
y2
y1
Usually, a mid-riser quantizer is used if the number of representative levels is even and a mid-stepquantizer if the number of level is odd.
68
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform quantization with dead zone
Uniform quantization
input x
output y
t6 t7 t8 t9t4t3t2t1
y2
y1
∆
Uniform quantization with dead zone
input x
output y
t6 t7 t8 t9t4t3t2t1
y2
y1
Deadzone
Interest:To remove small coe�cients by favouring the zero value. Increase the codinge�ciency with a small visual impact.
69
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Example of an uniform quantization
Example (Original picture quantized to 8, 7, 6, 4 bits/pixels)
(a) Original (8 bits per pixel)
(b) 7 bpp (∆ = 2) (c) 6 bpp (∆ = 4) (d) 4 bpp (∆ = 8)
70
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Optimal quantization
Optimal quantizationAn optimal quantization of a random variable X having a probability distribution p(x)is obtained by a quantizer that minimises a given metric:
Linf -norm: D = max |X − Q(X )|;L1-norm: D = E [|X − Q(X )|];L2-norm: D = E
[(X − Q(X ))2
]called the Mean-Square Error (MSE). This is
the most used.
Considering the MSE, we have:
if the random variable X is continue: D =∫ xmaxxmin
p(x)(x − Q(x))2dx ;
if the random variable X is discret: D =∑n
k=1 p(xk)(xk − Q(xk))2.
71
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Example
Example (Quantization error)
Hypothesis:
Uniform quantization with a quantization step ∆;
p(x) is a uniform probability distribution of a random variable X ;
N is the number of representative levels.
xmax − xmin
∆
xmin xmax∆2−∆
2
Quantization step: ∆ =xmax−xmin
N
Probability distribution: p(x) = 1xmax−xmin
= 1N∆
72
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Example
Example (Quantization error)
D =
∫ xmax
xmin
p(x)(x − Q(x))2dx
= N ×∫ ∆/2
−∆/2p(x)(x − 0)2dx , 0 the mid-point
= N ×∫ ∆/2
−∆/2
1
N∆x2dx
=∆2
12
73
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Optimal quantizer in a nutshell
Uniform quantizer is not optimal if the source is not uniformly distributed.
Optimal quantizer
To �nd the decision levels {ti} and the representative levels {yi} to minimize thedistortion D.
To reduce the MSE, the idea is to decrease the bin's size when the probability ofoccurrence is high and to increase the bin's size when the probability is low. For Nrepresentative levels and with a probability density p(x), the distortion is given by:
D =
∫ xmax
xmin
p(x)(x − Q(x))2dx
=
N−1∑k=1
∫ tk+1
tk
p(x)(x − yk)2dx
74
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Optimal quantizer in a nutshell
The optimal {ti} and {yi} satisfy:
∂D∂ti
= 0 and ∂D∂yi
= 0
Lloyd-Max quantizer
∂D∂ti
= 0 ⇒ ti =yi+yi+1
2.
ti is the midpoint of yi and yi+1.
∂D∂yi
= 0 ⇒ yi =
∫ titi−1
p(x)xdx∫ titi−1
p(x)dx.
yi is the centroid of the interval [ti−1, ti ].
⇒ given the {ti}, we can �nd the corresponding optimal {yi}.⇒ given the {yi}, we can �nd the corresponding optimal {ti}.
How can we �nd the optimal {ti} and {yi} simultaneously?
75
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Lloyd-Max algorithm in a nutshell
The Lloyd-Max algorithm is an algorithm for �nding the representative levels {yi} and thedecision levels {ti} to meet the previous conditions, with no prior knowledge.
Principle of the iterative process
1 The iterative process starts for k = 0 with a set of initial values for the representative
levels{y
(0)1 , ..., y
(0)N
}.
2 New values for decision levels are determined t(k+1)i =
y(k)i
+y(k)i+1
2
3 Compute the distortion D(k) and the relative errors δ(k);
4 Depending on the stopping criteria (δ(k) < ε), stop the process or update the
representative levels y(k+1)i =
∫ t(k+1)i
t(k+1)i−1
xp(x)dx
∫ t(k+1)i
t(k+1)i−1
p(x)dx
and go back to step 2.
76
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform, Semi-uniform and optimal quantizer
Suppose we have the following 1D discrete signal:
X = {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}
Example (Uniform quantizer: N = 4, Mid-riser and ∆ = 2)
input x
output y
02 4 6 8
1
3
5
7
ti ∈ T = {t0 = 0, t1 = 2, t2 = 4, t3 = 6, t4 = 8}
ri ∈ R = {r0 = 1, r1 = 3, r2 = 5, r3 = 7}
Check the de�nition of a uniform quantizer...
Quantized vector X = {1, 1, 3, 3, 1, 3, 5, 3, 5, 7, 7} (MSE = 0.42)
77
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform, semi-uniform and optimal quantizer
Suppose we have the following 1D discrete signal:
X = {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}
Example (Semi-uniform quantizer: N = 4, Mid-riser and ∆ = 2)
input x
output y
02 4 6 8
1
3
5
7
ti ∈ T = {t0 = 0, t1 = 2, t2 = 4, t3 = 6, t4 = 8}
ri ∈ R = {r0 = 2/3, r1 = 3.25, r2 = 4.75, r3 = 7.5}
Quantized vector X = {2/3, 2/3, 3.25, 3.25, 2/3, 3.25, 4.75, 3.25, 4.75, 7.5, 7.5}(MSE = 0.31)
78
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform, semi-uniform and optimal quantizer
Suppose we have the following 1D discrete signal:
X = {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}
Example (Lloyd-Max Algorithm)
Given that T = {0, 1.5, 3.87, 6.125, 8} and R = {0.005, 2.998, 4.75, 7.5}, thequantized vector isX = {0.005, 0.005, 2.998, 2.998, 2.998, 2.998, 4.75, 2.998, 4.75, 7.5, 7.5} (MSE=0.18).
input x
output y
02 4 6 8
1
3
5
7Uniform
Semi-uniform
Optimal
xi ∈ X
79
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
PrincipleUniform quantizationOptimal quantization, Lloyd-Max algorithmExamples: uniform, semi-uniform and optimal quantizer
Uniform vs optimal quantizer
Example (N=5)
(a) Original (b) Uniform (c) Optimal
(d) Histo. (e) Decision levels (f) Decision levels
80
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Distortion/quality assessment
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessmentTaxonomySignal �delityPerceptual metricExamples
8 Conclusion
81
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Taxonomy
Distortion/quality metrics can be divided into 3 categories:
1 Full-Reference metrics (FR) for which both the original and the distorted imagesare required (benchmark, compression) ;
2 Reduced-Reference metrics (RR) for which a description of the original and thedistorted image is required (network monitoring);
3 No-Reference (NR) metrics for which the original image is not required (networkmonitoring).
Each category can be divided into two subcategories: metrics based on signal �delityand metrics based on properties on the human visual system.
82
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Peak Signal to Noise Ratio (PSNR)
The PSNR is the most popular quality metric. This simple metric just calculates themathematical di�erence between each pixel of the degraded image and the originalimage.
PSNR:Let I and D the original and impaired images, respectively. These images having a size
of M pixels are coded with n bits.
PSNR = 10log10((2n−1)2)
MSEdB,
with the Mean Squared Error MSE =
∑(x,y)(I (x,y)−D(x,y))2
M.
A high value indicates that the amount of impairment is small. A small value indicates
that there is a strong degradation.
83
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Peak Signal to Noise Ratio (PSNR)
The PSNR is not always well correlated with the human judgment (MOS Mean OpinionScore). The reason is simple: this metric does not take into account the properties of thehuman visual system.
Example
(a) (b) (c) Original (d) Original+uniform noise
(a) and (b) from [?]: impact of Gabor patch on our perception.
84
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Peak Signal to Noise Ratio (PSNR)
Example (These three pictures have the same PSNR...)
(a) Original (b) Contrast stretched
(c) Blur (d) JPEG
85
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Metric based on the error visibility
For this type of metric, the behavior of the visual cells are simulated:
Perceptual Color Space PSD CSF Masking
−
Perceptual Color Space PSD CSF Masking
Pooling Quality
I
D
PSD: Perceptual Subband Decomposition (Wavelet, Gabor, Fourier);
CSF: Contrast Sensitivity Function.
86
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Metric based on the error visibility
Example
VDP (Visible Di�erences Predictor) [Daly,93]:
WQA (Wavelet-based Quality Assessment) [Ninassi et al.,08a]
VQM (Video Quality Model) [Pinson et al.,04]...
87
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Metric based on the structural similarity
SSIM standing for Structural Similarity index. Image degradations are con-
sidered here as perceived structural information loss instead of perceived errors [Wang et al.,04a].
Let I and D the original and the degraded images, respectively.
S(x, y) = l(x, y) × c(x, y) × s(x, y) (1)
The luminance comparison measure l(x, y) =2µxµy
µ2x + µ2y
The contrast comparison measure c(x, y) =2σxσy
σ2x + σ2y
The structural comparison measure s(x, y) =σxy
σxσy
SSIM(x, y) =(2µxµy + C1)(2σxy + C2)
(µ2x + µ2y + C1)(σ2x + σ2y + C2)
SSIM → 1, the best quality and SSIM → 0 indicates a poor quality.
88
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Metric based on the structural similarity
All pictures have the same MSE. (a) Original image; (b) Contrast-stretched image,MSSIM = 0.9168; (c) Mean-shifted image, MSSIM = 0.99; (d) JPEG,
MSSIM = 0.6949; (e) Blurred image, MSSIM = 0.7052; (f) Impluse noise,MSSIM = 0.7748. Extracted from [Wang et al.,04a].
89
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
TaxonomySignal �delityPerceptual metricExamples
Example of distortion maps
Example
(a) Original (b) Degraded
(c) MSE (d) WQA (e) SSIM
90
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Distortion/quality assessment
1 Introduction
2 Image transformation
3 Fourier transformation
4 Time sampling
5 Discrete Fourier Transform
6 Scalar Quantization
7 Distortion/quality assessment
8 Conclusion
91
IntroductionImage transformationFourier transformation
Time samplingDiscrete Fourier Transform
Scalar QuantizationDistortion/quality assessment
Conclusion
Conclusion
Image Transformation: global to point;
Fourier transformation;
Time sampling with Shannon's theorem;
Scalar quantization, a lossy process;
Quality and distortion.
92
Suggestion for further reading...
S. Daly. The visible di�erences predictor: An algorithm for the assessment ofimage �delity. Digital Images and Human Vision, pp. 179-206, 1993, MIT Press.
A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba. On the performance ofhuman visual system based image quality assessment metric using waveletdomain. Proc. SPIE Human Vision and Electronic Imaging XIII., Vol. 6806, pp.680610-12, 2008.
M. Pinson and S. Wolf. A new standardized method for objectively measuringvideo quality. IEEE Trans. Broadcasting, Vol. 50, N. 3, pp. 312-322, 2004.
Z. Wang, A. C. Bovik, H.R. Sheikh and E.P. Simoncelli. Image qualityassessment: from error visibility to structural similarity. IEEE Trans. on ImageProcessing, Vol. 13, pp. 600-612, 2004.
92