JPEGJPEG -595-

Motivations:Motivations:

1.1. Uncompressed video and Uncompressed video and audio data are huge. In audio data are huge. In HDTV, the bit rate easily HDTV, the bit rate easily exceeds 1 Gbps. --> big exceeds 1 Gbps. --> big problems for storage and problems for storage and network communications.network communications.

-596-

2. The 2. The compression ratiocompression ratio of of lossless methods (e.g., lossless methods (e.g., Huffman, Arithmetic, LZW) is Huffman, Arithmetic, LZW) is not high enough for image and not high enough for image and video compression, especially video compression, especially when distribution of pixel when distribution of pixel values is relatively flat. values is relatively flat.

-597-

• Spatial Redundancy Spatial Redundancy Removal -- Intraframe Removal -- Intraframe coding (JPEG)coding (JPEG)

• Spatial and temporal Spatial and temporal Redundancy Removal -- Redundancy Removal -- Intraframe and Interframe Intraframe and Interframe coding (H.261, MPEG)coding (H.261, MPEG)

-598-

What is What is JPEG ? ?

•"Joint Photographic Expert "Joint Photographic Expert Group". Voted as international Group". Voted as international standard in 1992. standard in 1992.

•Works with color and grayscale Works with color and grayscale images, e.g., satellite, medical, ...images, e.g., satellite, medical, ...

-599-

The JPEG standard for coding The JPEG standard for coding of digital images.of digital images.

• designed by the Joint designed by the Joint Photographic Experts GroupPhotographic Experts Group

• a collaborative effort a collaborative effort between ITU-T and ISObetween ITU-T and ISO

-600-

It aims to support It aims to support

• compression of continuous-compression of continuous-tone images tone images

• compression of still imagescompression of still images

-601-

• Adobe’s Postscript language Adobe’s Postscript language for printing systemsfor printing systems

• The Raster Content portion The Raster Content portion of the ISO Office document of the ISO Office document architecture and interchange architecture and interchange formatformat

-602-

• The ITU-T color facsimile The ITU-T color facsimile standardstandard

• the European ETSI videotext the European ETSI videotext standardstandard

• JPEG can also be used in JPEG can also be used in coding of viode, Motion coding of viode, Motion JPEG (UBR or internet)JPEG (UBR or internet)

-603-

JPEG overviewJPEG overview

• EncodingEncoding

• Decoding -- Reverse the order Decoding -- Reverse the order

-604-

Major StepsMajor Steps

1.1. DCT (Discrete Cosine DCT (Discrete Cosine Transformation)Transformation)

2.2. QuantizationQuantization

3.3. Zigzag ScanZigzag Scan

4.4. DPCM on DC componentDPCM on DC component

-605-

Major StepsMajor Steps

5. RLE on AC Components 5. RLE on AC Components

6. Entropy Coding 6. Entropy Coding -606-

-607-

-608-

1. Discrete Cosine 1. Discrete Cosine Transform (DCT)Transform (DCT)

• From spatial domain to From spatial domain to frequency domain frequency domain

-609-

The 64 (8 x 8) DCT basis functions

-610-

DEFINITIONS

Forward Discrete Cosine Transform (DCT):

-611-

Inverse Discrete Cosine Transform Inverse Discrete Cosine Transform (IDCT)(IDCT)

-612-

Computing the DCT Computing the DCT

Factoring reduces problem to a Factoring reduces problem to a series of 1D DCTsseries of 1D DCTs

-613-

-614-

-615-

• What is F[0,0]? -- define DC What is F[0,0]? -- define DC

• and AC componentsand AC components

-616-

2. Quantization2. Quantization

F'[u, v] = round F'[u, v] = round

( F[u, v] / q[u, v] )( F[u, v] / q[u, v] )

Why ? -- To reduce Why ? -- To reduce number of bits per number of bits per samplesample

-617-

QuantizationQuantization

Example: 101101 = 45 Example: 101101 = 45 (6 bits). (6 bits). q[u, v] = 4 --> Truncate q[u, v] = 4 --> Truncate to 4 bits: 1011 = 11. to 4 bits: 1011 = 11.

Quantization error is the main Quantization error is the main source of the Lossy source of the Lossy Compression. Compression.

-618-

Uniform quantizationUniform quantization

• Divide by constant Divide by constant NN and and round result (round result (NN = 4 or 8 in = 4 or 8 in examples above). examples above).

• Non powers-of-two gives fine Non powers-of-two gives fine control (e.g., control (e.g., NN = 6 loses 2.5 bits) = 6 loses 2.5 bits)

-619-

Non-uniform Non-uniform QuantizationQuantization

•Eye is most sensitive to low Eye is most sensitive to low frequencies (upper left corner), frequencies (upper left corner), less sensitive to high frequencies less sensitive to high frequencies (lower right corner)(lower right corner)

-620-

• In JPEG, each F[u,v] is divided In JPEG, each F[u,v] is divided by a constant q(u,v). by a constant q(u,v).

• Table of q(u,v) is called Table of q(u,v) is called quantization tablequantization table. .

-621-

Standard defines 2 default Standard defines 2 default quantization tables, one for quantization tables, one for luminance, one for chrominance.luminance, one for chrominance.

-622-

The Luminance Quantization Table q(u,v)The Luminance Quantization Table q(u,v)                   

16 11 10 16 24 40 51 61 16 11 10 16 24 40 51 61

12 12 14 19 26 58 60 5512 12 14 19 26 58 60 55

14 13 16 24 40 57 69 5614 13 16 24 40 57 69 56

14 17 22 29 51 87 80 6214 17 22 29 51 87 80 62

18 22 37 56 68 109 103 77 18 22 37 56 68 109 103 77

24 35 55 64 81 104 113 9224 35 55 64 81 104 113 92

49 64 78 87 103 121 120 10149 64 78 87 103 121 120 101

72 92 95 98 112 100 103 9972 92 95 98 112 100 103 99

-623-

The The Chrominance Quantization TableChrominance Quantization Table q(u, v) q(u, v)

17 18 24 47 99 99 99 9917 18 24 47 99 99 99 99

18 21 26 66 99 99 99 9918 21 26 66 99 99 99 99

24 26 56 99 99 99 99 9924 26 56 99 99 99 99 99

47 66 99 99 99 99 99 9947 66 99 99 99 99 99 99

99 99 99 99 99 99 99 9999 99 99 99 99 99 99 99

99 99 99 99 99 99 99 9999 99 99 99 99 99 99 99

99 99 99 99 99 99 99 9999 99 99 99 99 99 99 99

99 99 99 99 99 99 99 9999 99 99 99 99 99 99 99

-624-

The numbers in the The numbers in the above quantization above quantization tables can be scaled up tables can be scaled up (or down) to adjust the (or down) to adjust the so called so called quality factor. .

Custom quantization tables can Custom quantization tables can also be put in image/scan header.also be put in image/scan header.

-625-

3. Zig-zag Scan3. Zig-zag Scan

• Why? -- to group low Why? -- to group low frequency coefficients in top of frequency coefficients in top of vector. vector.

• Maps 8 x 8 to a 1 x 64 vector Maps 8 x 8 to a 1 x 64 vector

-626-

-627-

4. Differential Pulse 4. Differential Pulse Code Modulation Code Modulation (DPCM) on DC (DPCM) on DC componentcomponent

• DC component is large and DC component is large and varied, but often close to varied, but often close to previous value (like lossless previous value (like lossless JPEG).JPEG).

-628-

Differential Pulse Code Differential Pulse Code Modulation (DPCM) on Modulation (DPCM) on DC componentDC component

• Encode the difference from Encode the difference from previous 8x8 blocks -- DPCM previous 8x8 blocks -- DPCM

-629-

5. Run Length Encode 5. Run Length Encode (RLE) on AC components(RLE) on AC components

• 1x64 vector has lots of zeros in 1x64 vector has lots of zeros in itit

-630-

RLE on AC componentsRLE on AC components

• Encode as (Encode as (skip, valueskip, value) pairs, ) pairs, where where skipskip is the number of is the number of zeros and zeros and valuevalue is the next non- is the next non-zero component. zero component.

• Send (0,0) as end-of-block Send (0,0) as end-of-block sentinel value. sentinel value.

-631-

6. Entropy Coding6. Entropy Coding

• Categorize DC values into SSS (number of Categorize DC values into SSS (number of bits needed to represent) and actual bits. bits needed to represent) and actual bits.

                                ----------------------------------------                                Value SSS Value SSS                                 0 0 0 0                                 -1,1 -1,1 1 1                                 -3,-2,2,3 2 -3,-2,2,3 2                                 -7..-4,4..7 3 -7..-4,4..7 3                                 -------------------- --------------------

-632-

Entropy CodingEntropy Coding

ExampleExample: if DC value is 4, 3 bits are needed. : if DC value is 4, 3 bits are needed.

• Send off SSS as Huffman Send off SSS as Huffman symbol, followed by actual 3 symbol, followed by actual 3 bits.bits.

-633-

• For AC components (For AC components (skip, valueskip, value), encode ), encode the composite symbol (the composite symbol (skip,SSSskip,SSS) using the ) using the Huffman coding. Huffman coding.

• Huffman Tables can be custom (sent in Huffman Tables can be custom (sent in header) or default. header) or default.

-634-

Overview of the JPEG bitstreamOverview of the JPEG bitstream

-635-

-636-

• A "Frame" is a picture, A "Frame" is a picture,

• a "scan" is a pass through the pixels a "scan" is a pass through the pixels (e.g., the red component),(e.g., the red component),

• a "segment" is a group of blocks, a "segment" is a group of blocks,

• a "block" is an 8 x 8 group of pixels.a "block" is an 8 x 8 group of pixels.

-637-

• Frame header: Frame header:

- sample precision (width, height) of image - sample precision (width, height) of image - number of components - number of components - unique ID (for each component) - unique ID (for each component) - horizontal/vertical sampling factors (for - horizontal/vertical sampling factors (for each component) each component) - quantization table to use (for each - quantization table to use (for each component)component)

-638-

• Scan header Scan header - Number of components in scan - Number of components in scan component ID (for each component) component ID (for each component) - Huffman table for each component (for - Huffman table for each component (for each component) each component)

• Misc. (can occur between headers) Misc. (can occur between headers) - Quantization tables - Quantization tables - Huffman Tables - Huffman Tables - Arithmetic Coding Tables Comments - Arithmetic Coding Tables Comments - Application Data - Application Data

-639-

JPEG ModesJPEG Modes

• Baseline/Sequential -- the one that we Baseline/Sequential -- the one that we described in detail described in detail

• Lossless Lossless

• Progressive Progressive

• Hierarchical Hierarchical

• "Motion JPEG" -- Baseline JPEG "Motion JPEG" -- Baseline JPEG applied to each image in a video. applied to each image in a video.

-640-

1. Sequential Mode1. Sequential Mode

• Each image component is encoded Each image component is encoded in a single left-to-right, top-to-in a single left-to-right, top-to-bottom scan.bottom scan.

-641-

Sequential ModeSequential Mode

Baseline Sequential ModeBaseline Sequential Mode is a is a simple case of the Sequential simple case of the Sequential mode: mode:

o It supports only 8-bit images (not 12-bit It supports only 8-bit images (not 12-bit images) images)

o It uses only Huffman coding (not It uses only Huffman coding (not Arithmetic coding)Arithmetic coding)

-642-

-643-

2. Lossless Mode2. Lossless Mode

• A special case of the JPEG A special case of the JPEG where indeed there is no loss. where indeed there is no loss.

• Its block diagram is as below:Its block diagram is as below:

-644-

-645-

It does not use DCT-based method! Instead, It does not use DCT-based method! Instead, it uses a it uses a predictivepredictive (differential coding) (differential coding) method:method:

-646-

A predictor combines the values of up to A predictor combines the values of up to three neighboring pixels (not blocks as in three neighboring pixels (not blocks as in the Sequential mode) as the predicted value the Sequential mode) as the predicted value for the current pixel, indicated by "X" in for the current pixel, indicated by "X" in the figure below. the figure below.

The encoder then compares this prediction The encoder then compares this prediction with the actual pixel value at the position with the actual pixel value at the position "X", and encodes the difference (prediction "X", and encodes the difference (prediction residual) losslessly.residual) losslessly.

-647-

-648-

-649-

Predictor Prediction

1 A

2 B

3 C

4 A + B - C

5 A + (B - C) / 2

6 B + (A - C) / 2

7 (A + B) / 2

-650-

Since it uses only previously encoded Since it uses only previously encoded neighbors, the very first pixel I(0, 0) neighbors, the very first pixel I(0, 0) will have to use itself. will have to use itself.

Other pixels at the first row always use Other pixels at the first row always use P1, at the first column always use P2. P1, at the first column always use P2.

-651-

3. Progressive Mode3. Progressive Mode

• Goal: display low quality image Goal: display low quality image and successively improve.and successively improve.

-652-

• Two ways to successively Two ways to successively improve image: improve image:

- Spectral selection- Spectral selection: Send DC : Send DC component and first few AC component and first few AC coefficients first, then gradually coefficients first, then gradually some more ACs. some more ACs.

-653-

- Successive approximation- Successive approximation: send : send DCT coefficients MSB (most DCT coefficients MSB (most significant bit) to LSB (least significant bit) to LSB (least significant bit). significant bit). (Effectively, it is sending quantized (Effectively, it is sending quantized DCT coefficients frist, and then the DCT coefficients frist, and then the difference between the quantized difference between the quantized and the non-quantized coefficients and the non-quantized coefficients with finer quantization stepsize.) with finer quantization stepsize.)

-654-

-655-

4. Hierarchical Mode4. Hierarchical Mode

A Three-level Hierarchical A Three-level Hierarchical JPEG Encoder JPEG Encoder

-656-

• Can be repeated Can be repeated multiple times. multiple times.

• Good for viewing high Good for viewing high resolution image on low resolution image on low resolution display. resolution display.

-657-

(a) Down-sample by (a) Down-sample by factors of 2 in each factors of 2 in each dimension, e.g., reduce dimension, e.g., reduce 640 x 480 to 320 x 240 640 x 480 to 320 x 240

(b) Code smaller image using (b) Code smaller image using another JPEG mode another JPEG mode (Progressive, Sequential, or (Progressive, Sequential, or Lossless). Lossless).

-658-

(c) Decode and up-(c) Decode and up-sample encoded image sample encoded image

(d) Encode difference between (d) Encode difference between the up-sampled and the original the up-sampled and the original using Progressive, Sequential, or using Progressive, Sequential, or Lossless. Lossless.

-659-

JPEG 2000

•JPEG 2000 is the upcoming JPEG 2000 is the upcoming standard for Still Pictures (due standard for Still Pictures (due Year 2000). Year 2000).

•Major change from the current Major change from the current JPEG is that wavelets will JPEG is that wavelets will replace DCT as the means of replace DCT as the means of transform coding.transform coding.

-660-

Among many things it will Among many things it will address: address:

• Low bit-rate compression Low bit-rate compression performance,performance,

• Lossless and lossy compression Lossless and lossy compression in a single codestream,in a single codestream,

-661-

• Transmission in noisy Transmission in noisy environment where bit-error is environment where bit-error is high,high,

• Application to both gray/color Application to both gray/color images and bi-level (text) images and bi-level (text) imagery, natural imagery and imagery, natural imagery and computer generated imagery, computer generated imagery,

-662-

• Interface with MPEG-4, Interface with MPEG-4,

Content-based description. Content-based description.

-663-