an introduction to h.264/avc and 3d video coding

An Introduction to H.264/AVC and 3D Video Coding

Outline Video Coding Concepts

basic concept review image coding structure video coding structure

H.264/AVC Introduction history performance comparison

H.264/AVC Coding Tools inter prediction intra prediction transform & quantization de-blocking filter entropy coding

3D Video Coding 3D video format multiview video coding

Summary with Q&A

Video Coding Concept

-basic concept review

-image coding structure

-video coding structure

The Scope of Image and Video Coding Standardization

Only the Syntax and Decoder are standardized:

Images and Video

Needs for Video Compression

Without compression Visual telephony (e.g. CIF @ 15 frames/s):

325 (pels) x 288 (lines) x 15 (farmes/s) x 1.5 bytes = 18.25 Mbit/s

Digital TV (ITU-T 601 4:2:0 @30 frames/s): 720 (pels) x 480 (lines) x 30 (farmes/s) x 1.5 bytes = 124.

4 Mbit/s HDTV (e.g. 1280x720 pels 4:2:0 @ 60 frames/s):

Compression results in lower bit rates Lower transmission and storage cost

RGB vs. YCbCr [1/3]

RGB vs. YCbCr [2/3]

RGB vs. YCbCr [3/3]

Common YCbCr Formats

Subjective View

Block Based Coding [1/2]

Block Based Coding [2/2]

Group of Picture (GOP)

Video Coding Concept

-basic concept review

-image coding structure

-video coding structure

Image Coding Structure

Transform

Quantization

S: 0 1 2 3 4 5 6 7 (3 bits)

Quantization:

Quantization step-size Q=2: S/2

Quantization Levels (Q): 0 0 1 1 2 2 3 3 (2 bits)

Inverse quantization (x2): 0 0 2 2 4 4 6 6

Quantization error: 0 1 0 1 0 1 0 1

Quantization step-size Q=4: S/4

Quantization Levels (Q): 0 0 0 0 1 1 1 1 (2 bits)

Inverse quantization (x4): 0 0 0 0 4 4 4 4

Quantization error: 0 1 2 3 0 1 2 3

Effect of DCT + Quantization

Entropy coding

Video Coding Concept

-basic concept review

-image coding structure

-video coding structure

Temporal Redundancy [1/2]

The amount of data to be coded can be reduced significantly

Standard Video Encoder

Block Based Motion Compensation [1/2]

Algorithms for Motion Estimation

Full Search Guarantee find the global minimum SAD high computational complexity

Fast Search Local minimum SAD Low computational complexity Reduce candidate blocks Reduce matching pixels in candidate blocks

Diamond Search

Video coding structure

H.264/AVC Introduction

-History

-Performance comparison

History

Joint Video Team

MPEG-2 Has Hit A Wall

MPEG-4 in Comparison

H.26L Provides Focus

MPEG-4 “Adopts” H.264

State of the Art Standards

MPEG-2 DVD, DVT, since 1994

MPEG-4 DVR, Digital Still Camera, since 1999 ~1.5x coding gain over MPEG-2 (ASP)

MPEG-4 part 10, AVC (H.264) Mobile video, DVB-H, Blu-ray Disc and etc. 2~3x coding gain over MPEG-2

AVC Profiles

coding tools and profiles

H.264/AVC Introduction

-History

-Performance comparison

Compare to Other Standard

Fair comparisons of H.26L(TML-8.0) versus H.263v3,MPEG-2,and MPEG-4 TML-8.0 at half of the bit rate as MPEG-4 for the same visual fidelity Source from VCEG-N18.doc (Soptember,2001)

Objective evaluation Average improvement of TML-8.0over MPEG-2 (VM-5) of 5.8 dB PSNR

(peak gain 7.2 dB) for equal bandwidths TML-8.0 average gain of 3.1 dB relative to H.263++ (High-Latency

Profile) for equal bandwidths (up to 5.2 dB) Gain of 2.2 dB over MPEG-4 (Advanced Simple Profile) for equal

bandwidths (max. 3.6 dB)

Test Sets

“Streaming” Test: Four QCIF sequences coded at 10 Hz and 15 Hz (Foreman, Container, News,

Tempete) Four CIF sequences coded at 15 Hz and 30 Hz (Bus, Flower, Garden, Mobile a

nd Calendar, and Tempete) With B frame

“Real-Time Conversation” Test: Four QCIF sequences encoded at 10Hz and 15Hz (Akiyo, Foreman, Mother an

d Daughter, and Silent Voice) Four CIF sequences encoded at 15Hz and 30Hz (Carphone, Foreman, Paris, a

nd Sean) Without B frames

Objective evaluation [1/2]

Objective evaluation [2/2]

Subjective evaluation

Example: Sequence Mobile, frame 40

Perceptual Test of H.264/AVC High Profile

Objective Performance of H.264/AVC High Profile

Intra mode performance [1/2]

Average gain of H.264 to JPEG: 5.2 dB (luma) Average gain of H.264 to JPEG2000: 1.12 dB (luma) Average gain of Motion JPEG2000 to H.264: 1.42 dB (chroma) The smaller the bit rate, the higher the gain of H.264

Intra mode performance [2/2]

Intra mode performance [chroma]

Intra mode performance [FRExt]

a set of 8 photographic monochrome test images with resolutions from 512x512 up to 2048x3072 samples

Average gain of H.264/AVC HP to JPEG2000: 0.5 dB over the entire test image set and all bit-rates

JPEG2000 vs. H.264 Intra

H.264/AVC Coding Tools

-Inter prediction

-Intra prediction

-Transform and Quantization

-De-blocking Filter

-Entropy Coding

Basic Coding Structure

Standard Tools Comparison

Motion Compensation

Macro Block Partitions

Example – Frame 1

Example – Frame 2

Example – Residual [no MC]

Example – Residual [16x16]

Example – Residual [8x8]

Example – Residual [4x4]

Example of Variable Block Sizes

Large block means Less bits for MVs More bits on residuals

Small block means More bits for MVs Less bits on residuals

Summary

Key Features Enhances motion compensation Small blocks for transform coding De-blocking filter

50% bit rate saving against any other standards, by Better prediction More computation More memory

Video coding layer is still based on hybrid video coding algorithm, buy with important differences