![Page 1: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/1.jpg)
An Introduction to H.264/AVC and 3D Video Coding
![Page 2: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/2.jpg)
Outline Video Coding Concepts
basic concept review image coding structure video coding structure
H.264/AVC Introduction history performance comparison
H.264/AVC Coding Tools inter prediction intra prediction transform & quantization de-blocking filter entropy coding
3D Video Coding 3D video format multiview video coding
Summary with Q&A
![Page 3: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/3.jpg)
Video Coding Concept
-basic concept review
-image coding structure
-video coding structure
![Page 4: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/4.jpg)
The Scope of Image and Video Coding Standardization
Only the Syntax and Decoder are standardized:
![Page 5: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/5.jpg)
Images and Video
![Page 6: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/6.jpg)
Needs for Video Compression
Without compression Visual telephony (e.g. CIF @ 15 frames/s):
325 (pels) x 288 (lines) x 15 (farmes/s) x 1.5 bytes = 18.25 Mbit/s
Digital TV (ITU-T 601 4:2:0 @30 frames/s): 720 (pels) x 480 (lines) x 30 (farmes/s) x 1.5 bytes = 124.
4 Mbit/s HDTV (e.g. 1280x720 pels 4:2:0 @ 60 frames/s):
Compression results in lower bit rates Lower transmission and storage cost
![Page 7: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/7.jpg)
RGB vs. YCbCr [1/3]
![Page 8: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/8.jpg)
RGB vs. YCbCr [2/3]
![Page 9: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/9.jpg)
RGB vs. YCbCr [3/3]
![Page 10: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/10.jpg)
Common YCbCr Formats
![Page 11: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/11.jpg)
Subjective View
![Page 12: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/12.jpg)
Block Based Coding [1/2]
![Page 13: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/13.jpg)
Block Based Coding [2/2]
![Page 14: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/14.jpg)
Group of Picture (GOP)
![Page 15: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/15.jpg)
Video Coding Concept
-basic concept review
-image coding structure
-video coding structure
![Page 16: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/16.jpg)
Image Coding Structure
![Page 17: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/17.jpg)
Transform
![Page 18: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/18.jpg)
Quantization
S: 0 1 2 3 4 5 6 7 (3 bits)
Quantization:
Quantization step-size Q=2: S/2
Quantization Levels (Q): 0 0 1 1 2 2 3 3 (2 bits)
Inverse quantization (x2): 0 0 2 2 4 4 6 6
Quantization error: 0 1 0 1 0 1 0 1
Quantization step-size Q=4: S/4
Quantization Levels (Q): 0 0 0 0 1 1 1 1 (2 bits)
Inverse quantization (x4): 0 0 0 0 4 4 4 4
Quantization error: 0 1 2 3 0 1 2 3
![Page 19: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/19.jpg)
Effect of DCT + Quantization
![Page 20: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/20.jpg)
Entropy coding
![Page 21: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/21.jpg)
Video Coding Concept
-basic concept review
-image coding structure
-video coding structure
![Page 22: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/22.jpg)
Temporal Redundancy [1/2]
The amount of data to be coded can be reduced significantly
![Page 23: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/23.jpg)
Standard Video Encoder
![Page 24: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/24.jpg)
Block Based Motion Compensation [1/2]
![Page 25: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/25.jpg)
Algorithms for Motion Estimation
Full Search Guarantee find the global minimum SAD high computational complexity
Fast Search Local minimum SAD Low computational complexity Reduce candidate blocks Reduce matching pixels in candidate blocks
![Page 26: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/26.jpg)
Diamond Search
![Page 27: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/27.jpg)
Video coding structure
![Page 28: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/28.jpg)
H.264/AVC Introduction
-History
-Performance comparison
![Page 29: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/29.jpg)
History
![Page 30: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/30.jpg)
Joint Video Team
![Page 31: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/31.jpg)
MPEG-2 Has Hit A Wall
![Page 32: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/32.jpg)
MPEG-4 in Comparison
![Page 33: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/33.jpg)
H.26L Provides Focus
![Page 34: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/34.jpg)
MPEG-4 “Adopts” H.264
![Page 35: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/35.jpg)
State of the Art Standards
MPEG-2 DVD, DVT, since 1994
MPEG-4 DVR, Digital Still Camera, since 1999 ~1.5x coding gain over MPEG-2 (ASP)
MPEG-4 part 10, AVC (H.264) Mobile video, DVB-H, Blu-ray Disc and etc. 2~3x coding gain over MPEG-2
![Page 36: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/36.jpg)
AVC Profiles
![Page 37: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/37.jpg)
coding tools and profiles
![Page 38: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/38.jpg)
H.264/AVC Introduction
-History
-Performance comparison
![Page 39: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/39.jpg)
Compare to Other Standard
Fair comparisons of H.26L(TML-8.0) versus H.263v3,MPEG-2,and MPEG-4 TML-8.0 at half of the bit rate as MPEG-4 for the same visual fidelity Source from VCEG-N18.doc (Soptember,2001)
Objective evaluation Average improvement of TML-8.0over MPEG-2 (VM-5) of 5.8 dB PSNR
(peak gain 7.2 dB) for equal bandwidths TML-8.0 average gain of 3.1 dB relative to H.263++ (High-Latency
Profile) for equal bandwidths (up to 5.2 dB) Gain of 2.2 dB over MPEG-4 (Advanced Simple Profile) for equal
bandwidths (max. 3.6 dB)
![Page 40: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/40.jpg)
Test Sets
“Streaming” Test: Four QCIF sequences coded at 10 Hz and 15 Hz (Foreman, Container, News,
Tempete) Four CIF sequences coded at 15 Hz and 30 Hz (Bus, Flower, Garden, Mobile a
nd Calendar, and Tempete) With B frame
“Real-Time Conversation” Test: Four QCIF sequences encoded at 10Hz and 15Hz (Akiyo, Foreman, Mother an
d Daughter, and Silent Voice) Four CIF sequences encoded at 15Hz and 30Hz (Carphone, Foreman, Paris, a
nd Sean) Without B frames
![Page 41: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/41.jpg)
Objective evaluation [1/2]
![Page 42: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/42.jpg)
Objective evaluation [2/2]
![Page 43: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/43.jpg)
Subjective evaluation
Example: Sequence Mobile, frame 40
![Page 44: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/44.jpg)
Perceptual Test of H.264/AVC High Profile
![Page 45: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/45.jpg)
Objective Performance of H.264/AVC High Profile
![Page 46: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/46.jpg)
Intra mode performance [1/2]
Average gain of H.264 to JPEG: 5.2 dB (luma) Average gain of H.264 to JPEG2000: 1.12 dB (luma) Average gain of Motion JPEG2000 to H.264: 1.42 dB (chroma) The smaller the bit rate, the higher the gain of H.264
![Page 47: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/47.jpg)
Intra mode performance [2/2]
![Page 48: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/48.jpg)
Intra mode performance [chroma]
![Page 49: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/49.jpg)
Intra mode performance [FRExt]
a set of 8 photographic monochrome test images with resolutions from 512x512 up to 2048x3072 samples
Average gain of H.264/AVC HP to JPEG2000: 0.5 dB over the entire test image set and all bit-rates
![Page 50: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/50.jpg)
JPEG2000 vs. H.264 Intra
![Page 51: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/51.jpg)
H.264/AVC Coding Tools
-Inter prediction
-Intra prediction
-Transform and Quantization
-De-blocking Filter
-Entropy Coding
![Page 52: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/52.jpg)
Basic Coding Structure
![Page 53: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/53.jpg)
Standard Tools Comparison
![Page 54: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/54.jpg)
Motion Compensation
![Page 55: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/55.jpg)
Macro Block Partitions
![Page 56: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/56.jpg)
Example – Frame 1
![Page 57: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/57.jpg)
Example – Frame 2
![Page 58: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/58.jpg)
Example – Residual [no MC]
![Page 59: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/59.jpg)
Example – Residual [16x16]
![Page 60: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/60.jpg)
Example – Residual [8x8]
![Page 61: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/61.jpg)
Example – Residual [4x4]
![Page 62: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/62.jpg)
Example of Variable Block Sizes
Large block means Less bits for MVs More bits on residuals
Small block means More bits for MVs Less bits on residuals
![Page 63: An Introduction to H.264/AVC and 3D Video Coding](https://reader036.vdocument.in/reader036/viewer/2022062309/56649d955503460f94a7cf0c/html5/thumbnails/63.jpg)
Summary
Key Features Enhances motion compensation Small blocks for transform coding De-blocking filter
50% bit rate saving against any other standards, by Better prediction More computation More memory
Video coding layer is still based on hybrid video coding algorithm, buy with important differences