lossy image compressionlossy image compression and...
TRANSCRIPT
![Page 1: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/1.jpg)
Lossy Image CompressionLossy Image Compression and JPEG standard
Yao WangPolytechnic Institute of NYU, Brooklyn, NY 11201
With contribution from Zhu Liu, andGonzalez/Woods, Digital Image Processing, 2ed and A. K. Jain,
Fundamentals of Digital Image Processingg g g
![Page 2: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/2.jpg)
Lecture Outline• Introduction• Quantization Revisited• Quantization Revisited
– General description of quantizerUniform quantization– Uniform quantization
• Transform codingReview of linear and unitary transform 1D and 2D– Review of linear and unitary transform, 1D and 2D
– DCT– Quantization of transform coefficientsQuantization of transform coefficients– Runlength coding of DCT coefficients
• JPEG standard overview
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 2
![Page 3: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/3.jpg)
A Typical Compression System
Input Transformed Quantized Binary
Transfor- Quanti- Binary
InputSamples
Transformedparameters
Quantizedparameters
Binarybitstreams
Transformation
Quantization
BinaryEncoding
PredictionTransformsM d l fitti
Scalar QVector Q
Fixed lengthVariable length
Model fitting…...
g(Huffman, arithmetic, LZW)
• Motivation for transformation ---
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 3
To yield a more efficient representation of the original samples.
![Page 4: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/4.jpg)
Quantization (Review)
• General descriptionUniform q anti er• Uniform quantizer
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 4
![Page 5: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/5.jpg)
General Description of Quantization
Quantizerf Q(f)Q(f)
r7
r5
Decision Levels {tk, k = 0, …, L}
Reconstruction Levels {rk, k = 0, …, L-1}
)[f
r6
r7
ft1 t2 t5
If ),[ 1 kk ttfThen Q(f) = rk t6L levels need bits LR log
t3 t7r3
t4
r4
r1
Quantizererror
L levels need bits LR 2logr2
r3
x returns the smallest integerthat is bigger than or equal to x
r0
f
r1 r2 r3 r4 r5 r6 r7r0
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 5
t1 t2 t3 t4 t5 t6 t7t0=-∞ t8=∞
![Page 6: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/6.jpg)
Uniform Quantization• Equal distances between adjacent
decision levels and between adjacentdecision levels and between adjacent reconstruction levels– tl – tl-1 = rl – rl-1 = ql l 1 l l 1 q
• Parameters of Uniform Quantization– L: levels (L = 2R)( )– B: dynamic range B = fmax – fmin– q: quantization interval (step size)q q ( p )– q = B/L = B2-R
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 6
![Page 7: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/7.jpg)
Uniform Quantization: Functional Representation
Q(f)
r7=fmax-q/2t i (f f )/L
qff r5
r6
stepsize q=(fmax-fmin)/L
minmin
2)( fqq
qfffQ
r3
r4
f /2
r1
r2 x returns the biggest integer that is smaller than or equal to x
ft0 t1 t2 t3 t4 t5 t6 t7 t8
fmin fmax
r0=fmin+q/2than or equal to x
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 7
min max
.for used is leveltion reconstruc which indicates which index, leveltion reconstruc thecalled is )( min fqfffI
![Page 8: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/8.jpg)
Uniform Quantization on Images=2
56
32
Orig
inal
, L=
q=8,
L=
O16
, L=1
6
4, L
=4
q=1
q=64
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 8
![Page 9: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/9.jpg)
Implementation• Setup the quantization function
Q(f) for all possible input level into a look-up table first. Actual x = imread('lena256_8.bmp');
[h i ht idth] i ( )
Matlab code:
a look up table first. Actual quantization can be implemented as a table look-up.
[height, width] = size(x);B = 256;x = double(x);% This is the quantization tableQ (256 1)Q = zeros(256, 1);% Quantized to 64 levelsL = 16;q = B / L;f i 0 255for i = 0:255,
Q(i+1, 1) = floor(i / q) * q + q /2; endy = zeros(size(x));f i 1 h i htfor i = 1:height,
for j = 1:width,y(i, j) = Q(x(i,j) + 1);
endE d
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 9
L = 16, MSE = 21.6
EndMSE = mean(mean((x-y).^2));
![Page 10: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/10.jpg)
Transform Coding• Represent an image (or an image block) as the linear combination of
some basis images and specify the linear coefficients.
+t1 t2 t3 t4
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 10
![Page 11: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/11.jpg)
Components in Transform Coding
Bl kf t )(tI BBlocktransform Quantizer Binary Encoder
f t )(tI Bt
Encoder
Binary Decoder DequantizerInverseblock
transform
f̂)(tIBtt̂
Decoder
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 11
For block transform coders, we usually use Unitary (othonormal) transforms
![Page 12: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/12.jpg)
Why Using Transform?
• When the transform basis is chosen properlyproperly– Many coefficients have small values and can
be quantized to 0 w/o causing noticeablebe quantized to 0 w/o causing noticeable artifacts
– The coefficients are uncorrelated and henceThe coefficients are uncorrelated, and hence can be coded independently w/o losing efficiency.
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 12
![Page 13: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/13.jpg)
What Transform Basis to Use?• The transform should
– Minimize the correlation among resulting coefficients, so that scalar quantization can be employed without losing too much in coding efficiency compared to vector quantization
– Compact the energy into as few coefficients as possible
• Optimal transform – Karhunan Loeve Transform (KLT): signal statistics dependent
Suboptimal transform• Suboptimal transform– Discrete Cosine transform (DCT): nearly as good as KLT for
common image signals
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 13
![Page 14: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/14.jpg)
One Dimensional Linear Transform• Let CN represent the N dimensional complex
space.space.• Let h0, h1, …, hN-1 represent N linearly
independent vectors in CN.p• Any vector f є CN can be represented as a linear
combination of h0, h1, …, hN-1 :0 1 N 1
,)(1
0
ktN
kk Bthf
AffBt 1
.)1()0(
],,...,,[ 110
tt
where N
thhhB
AffBt
f and t form a transform pair
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 14
)1(
Nt
![Page 15: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/15.jpg)
Inner Product
• Definition of inner product1
* )()(N
H ffffff
• Orthogonal
0
212121 )()(,n
H nfnfffff
• Norm of a vector0, 21 ff
Norm of a vector
Normalized vector: unit norm
1
0
22 |)(|,N
n
H nffffff
12 f• Normalized vector: unit norm• Orthonomal = orthogonal + normalized
1f
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 15
![Page 16: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/16.jpg)
Orthonormal Basis Vectors (OBV)• {hk, k=0,…N-1} are OBV if
lklk
lklk 01
, ,hh
• With OBVNN 11
fhhhhhfh Hlkl
kk
kll ltktkt
)(,)()(,,00
h
H
0
AffBfhh
t
HH
1
0
h
HN 1
.,1 IBBBBBB HHH or B is unitary
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 16
., IBBBBBB or y
![Page 17: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/17.jpg)
Definition of Unitary Transform• Basis vectors are orthonormal• Forward transform
fh
N
fhk1
* )()()(
h
fh
H
nkk nfnhkt
0
0
* ,)()(,)(
I t f
AffBf
h
ht
H
H
H1
• Inverse transform h N 1
N
k nhktnf
1
,)()()(
tABtthhhhf HN
N
k
kk
kt
nhktnf
110
10
)(
,)()()(
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 17
k0
![Page 18: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/18.jpg)
Example of 1D Unitary Transform
2/12/1
2/12/1
2/12/1
2/12/1
,2/12/1
,2/12/1
,2/1
2/1,
2/12/1
3210 hhhh
51
2/12/12/12/1
0t
02
32 1
0
tt
f
10
43
3
2
tt
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 18
![Page 19: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/19.jpg)
1D DFT as a Unitary Transform
;1...,,1,0,)(1)(1
0
2
Nkenf
NkF
N
n
Nknj
.1...,,1,0,)(1)(1 2
0
NnekFN
nf
NN
Nknj
n
,1)(2
oreN
nh Nknj
k
0N k
1
12
N
Nkj
.1,...,1,0,1
)1(2
Nke
NN
kNj
N
k
h
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 19
e N
![Page 20: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/20.jpg)
Example: 1D DFT, N=2
)02exp(
:vectorsbasisonly two are there:case 2kj
N
hhh
1
12
1,11
21:
)12
2exp(
)02
2exp(
21
10kj
jk
fh
f
obtain we,, Using
, determine ,21
if 10
t
tt
kk
111
212*11*1
21
21
,1
12
1,2
32*11*12
121
,11
21
10 tt
kk
fhh
21
11
21
11
23 :Verify 1100 tt
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 20
![Page 21: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/21.jpg)
Another Example: 1D DFT, N=4
111
11
111
11)1
42exp(
)04
2exp(
1 jjkj
kj
hhhhh
12
1;
111
21;
121;
111
21:yields
)34
2exp(
)24
2exp(4
21 using :case 4 3210
j
j
j
j
kj
kjN k
f
);3(213542
21;73542
21
42
4
10 jjjtt
f
8)3()3(14
.3213542
21;03542
21
222
35
32
jj
jjjtt
fhhhh
122016
41
)3()3(14)3()3(14)3()3(14
41 :Verify 33221100
jjjjjj
jjjjtttt
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 21
![Page 22: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/22.jpg)
1D Discrete Cosine Transform (DCT)
2)12(cos)()(
:Vectors Basis
N
knknhk
1
0
)()()( :Transform ForwardN
k nhnfkT
1,...,10
2
1)(
2
Nkk
N
Nkwhere
N
1
0
0
)()()( :Transforms InverseN
uk
n
nhkTnf
27060983cos
50682cos
65330381cos
501381cos
:case 4tion RepresentaVector
k
N
hhhhh
3210
2706.06533.06533.0
2706.0
821cos
815cos
89cos
21;
5.05.05.0
5.0
814cos
810cos
86cos
21;
6533.02706.0
2706.06533.0
87cos
85cos
83cos
21;
5.05.05.05.0
1111
21:yields
87cos
85cos
83cos
)(
k
k
kkk
f
10
3827065330*)45(27060*)32(235421
;9239.02706.0*)54(6533.0*)32(;7354221
542
8888
tt
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 22
fhhhh
33221100
32
:Verify
.3827.06533.0*)45(2706.0*)32(;2354223
5
tttt
tt
![Page 23: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/23.jpg)
Two Dimensional Decomposition
• Decompose a MxN 2D matrix F=[F(m,n)] into a linear combination of some basicinto a linear combination of some basic images, Hk,l=[Hk,l(m,n)], so that:
1
0
1
0, ,),(
M
k
N
llklkT HF
1 1
,
0 0
),(),(),(M N
lk
k l
nmHlkTnmF 0 0k l
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 23
![Page 24: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/24.jpg)
Graphical Interpretation
+t1 t2 t3 t4t1 t2 t3 t4
Inverse transform: Represent a vector (e.g. a block of image samples) as the superposition of some basis vectors (block patterns)Forward transform: Determine the coefficients associated with each basis vector
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 24
![Page 25: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/25.jpg)
Two Dimensional Inner Product
• Inner Product
1 1
*M N
• Norm of a Matrix
0 0
2*
121 ),(),(,m n
nmFnmFFF
• Norm of a Matrix
1
0
1
0
2),(,M N
nmFFFF
• A set of basis images {Hk,l, k=0,1,…,M-1, l=0 1 N-1} is orthonormal if
0 0m n
l=0,1,…,N-1} is orthonormal if
otherwise
jlikifjlikjilk 0
,,1, ,,,, HH
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 25
otherwise,0
![Page 26: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/26.jpg)
Two Dimensional Unitary Transform
• {Hk,l} is an orthonormal set of basis imagesFor ard transform• Forward transform
1 1
* )()()(M N
nmFnmHlkT FH
• Inverse transform
0 0
,, ),(),(,),(m n
lklk nmFnmHlkT FH
Inverse transform
1
0
1
0, ,),(),(),(
M
k
N
llk ornmHlkTnmF
1
0
1
0,),(
M
k
N
llklkT HF
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 26
![Page 27: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/27.jpg)
Example of 2D Unitary Transform
5)0,0(
,2/12/12/12/1
,2/12/12/12/1
,2/12/1
2/12/1,
2/12/12/12/1
11100100
T
HHHH
0)1,1(1)0,1(2)1,0(
),(
4321
TTT
F
0)1,1(T
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 27
![Page 28: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/28.jpg)
Separable Unitary Transform
• Let hk, k=0, 1, …, M-1 represent orthonormal basis vectors in CMorthonormal basis vectors in C ,
• Let gl, l=0, 1, …, N-1 represent orthonormal basis vectors in CNorthonormal basis vectors in CN,
• Hk,l=hkglT, or Hk,l(m,n)=hk(m)gl(n).
• Then Hk,l will form an orthonormal basis set in CMxN.
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 28
![Page 29: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/29.jpg)
Example of Separable Unitary Transform
• Example 1 2/12/1
2/12/12/12/1
.2/1
2/1,2/12/1
10
TT hhHhhH
hh
2/12/12/12/1
2/12/12/12/1
2/12/12/12/1
11110110
10010000
TT hhHhhH
hhHhhH
2/12/12/12/1
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 29
![Page 30: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/30.jpg)
Example: 4x4 DFT
1
1
21;
11
1
21;
1
1
21;
111
21: are basis DFT 1D theRecall 3210
jjhhhh
1111
111111111
11111
111111111
1
:yields using
11
jjjj
jjjj
jjT
lkk,l hhH
1111
11111
11111
11111
1
11114
1,
111111114
1,
11114
1,
111111114
13,02,01,00,0
jjjj
jjjjjjjj
jjjj
jjjjjj
jjjjjj
HHHH
HHHH
1111
111111111
11111
111111111
1
11114
,11114
,
11114
,11114 3,12,11,10,1
jjjj
jjjj
jjjj
jjjjjjjj
jjjj
HHH
HHHH
1111
1,
11111,
1111
1,
11111
11114
,
111111114
,
11114
,
111111114
33231303
3,22,21,20,2
jjjj
jjjjjjjj
jjjj
jjjj
jjjj
HHHH
HHH
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 30
11
114,
11114,
11114
,11114 3,32,31,30,3
jjjj
jjjjjjjj
jjjj
![Page 31: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/31.jpg)
Example: 4x4 DFT
comp te13100221
For T
F
02211111
e.g, yields, , Using
compute ,
12211210
For
,,
,
T
T
lklk
lk
FH
F
022111
4181221121013100221
41
122112101310
,
111111111111
41,
*
0,00,0
jj
T
FH
)1(4122123221
41
122112101310
,
1111
1141,3,23,2 jjjjjjjj
jjjjjj
T
FH
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 31
![Page 32: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/32.jpg)
Basis Images of 8x8 DCT
01)(
2)12(cos
2)12(cos)()(),(,
kNkwhere
Nln
NkmlknmH lk
Low-Low High-Low
1,...,12)(
Nk
Nkwhere
o o
Low-High High-High
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 32
![Page 33: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/33.jpg)
Property of Separable Transform
• When the transform is separable, we can perform the 2D transform separatelyperform the 2D transform separately.– First, do 1D transform for each row using
basis vectors gbasis vectors gl,– Second, do 1D transform for each column of
the intermediate image using basis vectors hkthe intermediate image using basis vectors hk.– Proof:
11 11 1 MM NM N
0
*
0 0
**
0 0
*, ),()(),()()(),(),(),(
mm nl
m nlk lmUmhnmFngmhnmFnmHlkT
kk
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 33
![Page 34: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/34.jpg)
DCT on a Real Image Block
>>imblock = lena256(128:135,128:135)-128
imblock=imblock
54 68 71 73 75 73 71 45
47 52 48 14 20 24 20 -8
20 -10 -5 -13 -14 -21 -20 -21 >>dctblock =dct2(imblock)
-13 -18 -18 -16 -23 -19 -27 -28
-24 -22 -22 -26 -24 -33 -30 -23
-29 -13 3 -24 -10 -42 -41 5
-16 26 26 -21 12 -31 -40 23
dctblock=
31.0000 51.7034 1.1673 -24.5837 -12.0000 -25.7508 11.9640 23.2873
113.5766 6.9743 -13.9045 43.2054 -6.0959 35.5931 -13.3692 -13.0005
195.5804 10.1395 -8.6657 -2.9380 -28.9833 -7.9396 0.8750 9.5585
17 30 50 -5 4 12 10 5 35.8733 -24.3038 -15.5776 -20.7924 11.6485 -19.1072 -8.5366 0.5125
40.7500 -20.5573 -13.6629 17.0615 -14.2500 22.3828 -4.8940 -11.3606
7.1918 -13.5722 -7.5971 -11.9452 18.2597 -16.2618 -1.4197 -3.5087
1 4562 13 3225 0 8750 1 3248 10 3817 16 0762 4 4157 1 1041-1.4562 -13.3225 -0.8750 1.3248 10.3817 16.0762 4.4157 1.1041
-6.7720 -2.8384 4.1187 1.1118 10.5527 -2.7348 -3.2327 1.5799
Note that low-low coefficients are much larger than high-high coefficients
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 34
In JPEG, “imblock-128” is done before DCT to shift the mean to zero
Note that low low coefficients are much larger than high high coefficients
![Page 35: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/35.jpg)
DCT Coefficient Distribution
ianc
eVa
r
Coefficient Index (Zig-Zag Order)
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 35
Coefficient Index (Zig Zag Order)
![Page 36: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/36.jpg)
Approximation by DCT Basis
Original With 16/64Coefficients
With 8/64Coefficients With 4/64
Coefficients
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 36
![Page 37: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/37.jpg)
Quantization of DCT Coefficients• Use uniform quantizer on each coefficient• Different coefficient is quantized with different• Different coefficient is quantized with different
step-size (Q):– Human eye is more sensitive to low frequencyy q y
components– Low frequency coefficients with a smaller Q– High frequency coefficients with a larger Q– Specified in a normalization matrix (aka quantization
matrix)matrix)– Normalization matrix can then be scaled by a scale
factor (aka quality factor or QP)
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 37
![Page 38: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/38.jpg)
Default Normalization Matrix in JPEG
NormalizationMatrix
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 38
Actual step size for C(i,j): Q(i,j) = QP*M(i,j)
![Page 39: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/39.jpg)
Example: Quantized Indices
>>dctblock =dct2(imblock)( )
dctblock=
31.0000 51.7034 1.1673 -24.5837 -12.0000 -25.7508 11.9640 23.2873
113.5766 6.9743 -13.9045 43.2054 -6.0959 35.5931 -13.3692 -13.0005>>QP=1;
>>QM=Qmatrix*QP;195.5804 10.1395 -8.6657 -2.9380 -28.9833 -7.9396 0.8750 9.5585
35.8733 -24.3038 -15.5776 -20.7924 11.6485 -19.1072 -8.5366 0.5125
40.7500 -20.5573 -13.6629 17.0615 -14.2500 22.3828 -4.8940 -11.3606
7.1918 -13.5722 -7.5971 -11.9452 18.2597 -16.2618 -1.4197 -3.5087
>>qdct=floor((dctblock+QM/2)./(QM))
qdct =
2 5 0 -2 0 -1 0 0
9 1 -1 2 0 1 0 0-1.4562 -13.3225 -0.8750 1.3248 10.3817 16.0762 4.4157 1.1041
-6.7720 -2.8384 4.1187 1.1118 10.5527 -2.7348 -3.2327 1.579914 1 -1 0 -1 0 0 0
3 -1 -1 -1 0 0 0 0
2 -1 0 0 0 0 0 0
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 39
Only 19 coefficients are retained out of 64
![Page 40: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/40.jpg)
Example: Quantized Coefficients%dequantized DCT block
>> iqdct=qdct.*QM
iqdct=
32 55 0 -32 0 -40 0 0
108 12 -14 38 0 58 0 0
196 13 -16 0 -40 0 0 0
42 17 22 29 0 0 0 042 -17 -22 -29 0 0 0 0
36 -22 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 Original DCT block0 0 0 0 0 0 0 0 dctblock=
31.0000 51.7034 1.1673 -24.5837 -12.0000 -25.7508 11.9640 23.2873
113.5766 6.9743 -13.9045 43.2054 -6.0959 35.5931 -13.3692 -13.0005
195 5804 10 1395 8 6657 2 9380 28 9833 7 9396 0 8750 9 5585195.5804 10.1395 -8.6657 -2.9380 -28.9833 -7.9396 0.8750 9.5585
35.8733 -24.3038 -15.5776 -20.7924 11.6485 -19.1072 -8.5366 0.5125
40.7500 -20.5573 -13.6629 17.0615 -14.2500 22.3828 -4.8940 -11.3606
7.1918 -13.5722 -7.5971 -11.9452 18.2597 -16.2618 -1.4197 -3.5087
Note that many coefficients become zero after quantization!
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 40
-1.4562 -13.3225 -0.8750 1.3248 10.3817 16.0762 4.4157 1.1041
-6.7720 -2.8384 4.1187 1.1118 10.5527 -2.7348 -3.2327 1.5799
q
![Page 41: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/41.jpg)
Example: Reconstructed Image
%reconstructed image block
>> qimblock=round(idct2(iqdct))Original image block
imblock=qimblock=
58 68 85 79 61 68 67 38
45 38 39 33 22 24 19 -2
21 2 11 12 13 19 24 27
imblock
54 68 71 73 75 73 71 45
47 52 48 14 20 24 20 -8
20 -10 -5 -13 -14 -21 -20 -2121 2 -11 -12 -13 -19 -24 -27
-8 -19 -31 -26 -20 -35 -37 -15
-31 -17 -21 -20 -16 -39 -41 0
-33 3 -1 -14 -11 -37 -44 1
-13 -18 -18 -16 -23 -19 -27 -28
-24 -22 -22 -26 -24 -33 -30 -23
-29 -13 3 -24 -10 -42 -41 5
-16 26 26 -21 12 -31 -40 23-16 32 18 -10 1 -16 -30 8
3 54 30 -6 16 11 -7 2317 30 50 -5 4 12 10 5
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 41
![Page 42: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/42.jpg)
Coding of DCT Coefficients• Many coefficients are zeros after quantization.• Specify which coefficients are non-zero by using• Specify which coefficients are non-zero by using
run-lengths of zeros.• Run-Length Coding - Used in JPEG standardRun Length Coding Used in JPEG standard
• Ordering coefficients in the zig-zag order• Each symbol=(length-of-zero, non-zero-value)• Code each symbol using Huffman coding
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 42
![Page 43: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/43.jpg)
Zig-Zag Ordering of DCT Coefficients
0 1 5 6 14 15 27 280 1 5 6 14 15 27 28
2 4 7 13 16 26 29 42
3 8 12 17 25 30 41 43
9 11 18 24 31 40 44 53
10 19 23 32 39 45 52 54
20 22 33 38 46 51 55 6020 22 33 38 46 51 55 60
21 34 37 47 50 56 59 61
35 36 48 49 57 58 62 63
Zig-Zag ordering: converting a 2D matrix into a 1D array, so that the frequency (horizontal+vertical) increases in this order and the coefficient variance decreases in this order
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 43
order, and the coefficient variance decreases in this order.
![Page 44: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/44.jpg)
Exampleqdct =
2 5 0 -2 0 -1 0 0
9 1 -1 2 0 1 0 0
14 1 -1 0 -1 0 0 0
3 -1 -1 -1 0 0 0 0
2 -1 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Run-length symbol representation:{2 (0 5) (0 9) (0 14) (0 1) (1 2) (0 1) (0 1) (0 3) (0 2) (0 1) (0 1) (0 2) (1{2,(0,5),(0,9),(0,14),(0,1),(1,-2),(0,-1),(0,1),(0,3),(0,2),(0,-1),(0,-1),(0,2),(1,-1),(2,-1), (0,-1), (4,-1),(0,-1),(0,1),EOB}
EOB: End of block, one of the symbol that is assigned a short Huffman
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 44
, y gcodeword
![Page 45: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/45.jpg)
What is JPEG
• The Joint Photographic Expert Group (JPEG), under both the International Standards Organization (ISO) and the International g ( )Telecommunications Union-Telecommunication Sector (ITU-T)
– www.jpeg.org
H bli h d l t d d• Has published several standards– JPEG: lossy coding of continuous tone still images
• Based on DCT
– JPEG-LS: lossless and near lossless coding of continuous tone still images
• Based on predictive coding and entropy codingp g py g
– JPEG2000: scalable coding of continuous tone still images (from lossy to lossless)
• Based on wavelet transform
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 45
Based on wavelet transform
![Page 46: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/46.jpg)
The 1992 JPEG Standard• Contains several modes:
– Baseline system (what is commonly known as JPEG!): y ( y )lossy
• Can handle gray scale or color images (8bit)– Extended system: lossyExtended system: lossy
• Can handle higher precision (12 bit) images, providing progressive streams, etc.
– Lossless versionLossless version• Baseline system
– Each color component is divided into 8x8 blocksp– For each 8x8 block, three steps are involved:
• Block DCT• Perceptual-based quantization
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 46
• Perceptual-based quantization• Variable length coding: Runlength and Huffman coding
![Page 47: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/47.jpg)
Coding of Quantized DCT CoefficientsCoefficients
• DC coefficient: Predictive coding– The DC value of the current block is predicted from that of the previous
block, and the error is coded using Huffman codingblock, and the error is coded using Huffman coding• AC Coefficients: Runlength coding
– Many high frequency AC coefficients are zero after first few low-frequency coefficients
– Runlength Representation:• Ordering coefficients in the zig-zag order• Specify how many zeros before a non-zero value• Each symbol=(length-of-zero non-zero-value)Each symbol (length of zero, non zero value)
– Code all possible symbols using Huffman coding• More frequently appearing symbols are given shorter codewords• For more details on the actual coding table, see Handout (Sec.8.5.3 in
[G l 02][Gonzalez02]• One can use default Huffman tables or specify its own tables.• Instead of Huffman coding, arithmetic coding can be used to achieve
higher coding efficiency at an added complexity
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 47
higher coding efficiency at an added complexity.
![Page 48: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/48.jpg)
Coding of DC Symbols• Example:
– Current quantized DC index: 2q– Previous block DC index: 4– Prediction error: -2
Th di ti i d d i t t– The prediction error is coded in two parts:• Which category it belongs to (Table of JPEG Coefficient
Coding Categories), and code using a Huffman code (JPEG Default DC Code)Default DC Code)
– DC= -2 is in category “2”, with a codeword “100”• Which position it is in that category, using a fixed length code,
length=category numberlength category number– “-2” is the number 1 (starting from 0) in category 2, with a fixed
length code of “01”.– The overall codeword is “10001”
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 48
![Page 49: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/49.jpg)
JPEG Tables for Coding DC
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 49
![Page 50: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/50.jpg)
Coding of AC Coefficients• Example:
– First symbol (0,5)Th l ‘5’ i t d i t t• The value ‘5’ is represented in two parts:
• Which category it belongs to (Table of JPEG Coefficient Coding Categories), and code the “(runlength, category)” using a Huffman code (JPEG Default AC Code)( )
– AC=5 is in category “3”, – Symbol (0,3) has codeword “100”
• Which position it is in that category, using a fixed length code, length=category numberlength=category number
– “5” is the number 5 (starting from 0) in category 3, with a fixed length code of “101”.
– The overall codeword for (0,5) is “100101”S d b l (0 9)– Second symbol (0,9)
• ‘9’ in category ‘4’, (0,4) has codeword ‘1011’,’9’ is number 9 in category 4 with codeword ‘1001’ -> overall codeword for (0,9) is ‘10111001’
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 50
– ETC
![Page 51: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/51.jpg)
JPEG Tables for Coding AC (Run,Category) Symbols(Run,Category) Symbols
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 51
![Page 52: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/52.jpg)
JPEG Performance for B/W images
65536 Bytes8 bpp
4839 Bytes0.59 bppCR=13.6CR 13.6
3037 Bytes0.37 bppCR=21.6
1818 Bytes0.22 bppCR=36.4
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 52
![Page 53: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/53.jpg)
JPEG for Color Images• Color images are typically stored in (R,G,B) format• JPEG standard can be applied to each component separately
– Does not make use of the correlation between color components– Does not make use of the lower sensitivity of the human eye to
chrominance samplesAlt t h• Alternate approach – Convent (R,G,B) representation to a YCbCr representation
• Y: luminance, Cb, Cr: chrominance D l th t h i t– Down-sample the two chrominance components
• Because the peak response of the eye to the luminance component occurs at a higher frequency (3-10 cpd) than to the chrominance components (0.1-0.5 cpd). (Note: cpd is cycles/degree)
• JPEG standard is designed to handle an image consists of many (up to 100) components
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 53
![Page 54: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/54.jpg)
RGB <-> YCbCr Conversion
0114.0587.0299.0 RY
128128
081.0419.0500.0500.0331.0169.0
r
b
BG
CC
128714034400001402.1001.0000.1
CY
GR
128128
001.0772.1000.1714.0344.0000.1
r
b
CC
BG
Note: Cb ~ Y-B, Cr ~ Y-R, are known as color difference signals.
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 54
![Page 55: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/55.jpg)
Chrominance Subsampling
4:2:04:2:24:4:4 4:1:1 4:2:0For every 2x2 Y Pixels
1 Cb & 1 Cr Pixel(Subsampling by 2:1 bothhorizontally and vertically)
4:2:2For every 2x2 Y Pixels
2 Cb & 2 Cr Pixel(Subsampling by 2:1
horizontally only)
4:4:4For every 2x2 Y Pixels
4 Cb & 4 Cr Pixel(No subsampling)
4:1:1For every 4x1 Y Pixels
1 Cb & 1 Cr Pixel(Subsampling by 4:1
horizontally only)
Y Pixel Cb and Cr Pixel
4 2 0 i th t f t
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 55
4:2:0 is the most common format
![Page 56: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/56.jpg)
Coding Unit in JPEG
4 8x8 Y blocks 1 8x8 Cb blocks 1 8x8 Cr blocks
Each basic coding unit (called a data unit) is a 8x8 block in any color component.In the interleaved mode, 4 Y blocks and 1 Cb and 1 Cr blocks are processed as a group (called a minimum coding unit or MCU) for a 4:2:0 image.
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 56
![Page 57: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/57.jpg)
Default Quantization Table
For luminance For chrominance
17 18 24 47 99 99 99 99
18 21 26 66 99 99 99 99
24 26 56 99 99 99 99 99
47 66 99 99 99 99 99 99
99 99 99 99 99 99 99 99
99 99 99 99 99 99 99 99
99 99 99 99 99 99 99 99
99 99 99 99 99 99 99 99
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 57
The encoder can specify the quantization tables different from the default ones as part of the header information
![Page 58: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/58.jpg)
Performance of JPEG• For color images at 24 bits/pixel (bpp)
– 0.25-0.5 bpp: moderate to good0.25 0.5 bpp: moderate to good– 0.5-0.75 bpp: good to very good– 0.75-1.5 bpp: excellent, sufficient for most
applications– 1.5-2 bpp: indistinguishable from original
F G K W ll Th JPEG Still i t– From: G. K. Wallace: The JPEG Still picture compression standard, Communications of ACM, April 1991.p
• For grayscale images at 8 bpp– 0.5 bpp: excellent quality
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 58
![Page 59: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/59.jpg)
JPEG Performance
487x414 pixels, Uncompressed, 600471 Bytes,24 bpp 487x414 pixels
41174 Bytes, 1.63 bpp, CR=14.7
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 59
85502 Bytes, 3.39 bpp, CR=7
![Page 60: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/60.jpg)
JPEG Pros and Cons
• ProsLow
• Cons– Single resolution– Low
complexityMemory
Single resolution– Single quality
No target bit rate– Memory efficient
– Reasonable
– No target bit rate– Blocking artifacts at low
bit rateReasonable coding efficiency
bit rate– No lossless capability
Poor error resiliencey – Poor error resilience– No tiling
No regions of interest
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 60
– No regions of interest
![Page 61: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/61.jpg)
Homework1. For the 2x2 image S given below, compute its 2D DCT, reconstruct it by retaining
different number of coefficients to evaluate the effect of different basis imagesdifferent number of coefficients to evaluate the effect of different basis images.a) Determine the four DCT basis images.b) Determine the 2D-DCT coefficients for S, Tk,l, k=0,1;l=0,1. c) Show that the reconstructed image from the original DFT coefficients equal to the original
imageimage.d) Modify the DCT coefficients using the given window masks (W1 to W5) and reconstruct
the image using the modified DCT coefficients. (for a given mask, “1” indicates to retain that coefficient, “0” means to set the corresponding coefficient to zero) What effect do you see with each mask and why?y
1001
,1000
,0100
,0010
,0001
,9119
54321 WWWWWS
2. For the same image S as given in Prob. 1, quantize its DCT coefficients using the quantization matrix Q. Determine the quantized coefficient indices and quantized values. Also, determine the reconstructed image from the quantized coefficients
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 61
coefficients.
5333
Q
![Page 62: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/62.jpg)
Homework (cnt’d)3. Describe briefly how JPEG compresses an image. You may want to break down your
discussion into three parts:a) How does JPEG compress a 8x8 image block (three steps are involved)b) H d JPEG l ib) How does JPEG compress a gray-scale imagec) How does JPEG compress a RGB color image
4. Suppose the DCT coefficient matrix for an 4x4 image block is as shown below (dctblock).
a) Quantize its DCT coefficients using the quantization matrix Q given below assuminga) Quantize its DCT coefficients using the quantization matrix Q given below, assuming QP=1. Determine the quantized coefficient indices and quantized values.
b) Represent the quantized indices using the run-length representation. That is, generate a series of symbols, the first being the quantized DC index, the following symbols each consisting of a length of zeros and the following non-zero index, the last symbol is EOB ( d f bl k)(end of block).
c) Encode the DC index and the runlength symbols from (b) using the JPEG coding method, with the coding tables given in the lecture note. For this problem, assuming the quantized index for the DC coefficient of the previous block is 60.
,
0.0085 0.0044 0.0086 0.0046- 0.0019 0.0036 0.0207- 0.0086- 0.0071 0.0877 0.0033- 0.0134 0.0912 0.0466- 0.0500- 1.3676
* 0030.1
edctblock ,
120 103 78 49 103 68 37 18 69 40 16 14 51 24 10 16
Q
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 62
![Page 63: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/63.jpg)
Computer Assignment1. Computer assignment: Write a program to examine the effect of
quantizing the DCT coefficients. For each 8 x 8 block in the image, your program should first calculate the DCT of the block, quantize th ffi i t d th t k th i DCT F ti tithe coefficients, and then take the inverse DCT. For quantization, using a scaled version of the JPEG quantization matrix as the stepsizes. Examine the resulting image quality with the followngvalues for the scaling factor: 0.5, 1, 2, 4, 8. What is the largest
l f th li f t t hi h th t t d ivalue of the scaling factor at which the reconstructed image quality is very close to the original image? Hint: you should make use of the ”blkproc” function in MATLAB to speed up your program.
2. Modify the program above, so that it keeps only first N coefficients in the zig-zag order. N should be a user-selectable parameter. Examine the resulting image quality when N=1, 4, 16, 32. For the image you have chosen, how many DCT coefficients should be g y yretained to maintain a sufficient quality?
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 63
![Page 64: Lossy Image CompressionLossy Image Compression and …eeweb.poly.edu/~yao/EL5123/lecture10_TransformCoding_JPEG.pdfLossy Image CompressionLossy Image Compression and JPEG standard](https://reader034.vdocument.in/reader034/viewer/2022042116/5e938459b564403247768eda/html5/thumbnails/64.jpg)
Reading
• R. Gonzalez, “Digital Image Processing,” Section 8 5 - 8 6Section 8.5 - 8.6
• A.K. Jain, “Fundamentals of Digital Image Processing ” Section 11 4 11 5Processing,” Section 11.4 – 11.5
Yao Wang, NYU-Poly EL5123: Transform Coding and JPEG 64