1. 1. problem statement 2. overview of h.264/avc scalable extension i. temporal scalability ii....
TRANSCRIPT
1
A Parallelism Encoding Framework for The Temporal Scalability of H.264/AVC Scalable Extension
Shu-Sian Yang, Sung-Wen Wang, Hong-Ming Chen, and Ja-Ling Wu
Department of Computer Science and Information Engineering
Graduate Institute of Networking and MultimediaNational Taiwan University, Taipei, Taiwan
E-mail:{pigyoung, song, blacksmith, wjl}@cmlab.csie.ntu.edu.tw
2
Outline
1. Problem Statement2. Overview of H.264/AVC Scalable Extension
I. Temporal ScalabilityII. Spatial ScalabilityIII. Complexity Reduction
3. Previous Parallel Encoding Scheme for Video Coding
1. MB-Level (Wave-front) Parallelism2. Frame-Level Parallelism
4. Parallel Encoding Based on Hierarchical B-Picture Structure
I. Frame-Level Parallel Scheme5. Conclusions and Future Work
3
SVC Encoder Structure Overview Combined scalability. H.264 based, layered video coding.
4
Higher Complexity
Base Layer (BL) is identical to the standard H.264 Enhancement Layers (EL) have “inter-layer”
predictions in additional:• H.264:
– Inter 16x16– Inter 8x16– Inter 16x8– Inter 8x8
• Inter 8x8• Inter 4x8• Inter 8x4• Inter 4x4
– Intra 16x16 (4 modes)– Intra 4x4 (9 modes)
• SVC additional:– BL Inter 16x16– BL Inter 8x16– BL Inter 16x8– BL Inter 8x8
• BL Inter 8x8• BL Inter 4x8• BL Inter 8x4• BL Inter 4x4
– BL Intra 16x16– BL Intra 4x4
– BL Inter 16x16 w. residue pred.
– BL Inter 8x16 w. residue pred.
– BL Inter 16x8 w. residue pred.
– BL Inter 8x8 w. residue pred..
• BL Inter 8x8 w. residue pred.
• BL Inter 4x8 w. residue pred.
• BL Inter 8x4 w. residue pred.
• BL Inter 4x4 w. residue pred.
– BL Intra 16x16 w. residue pred.
– BL Intra 4x4 w. residue pred.
5
Scalabilities
Three kinds of scalabilities: Quality (SNR) scalability▪ Fine-Grain-Scalability (FGS)▪ Bit-plane coding
Spatial scalability▪ Decimation▪ Wavelet transform
Temporal scalability▪ Hierarchical B-picture
30 fps15 fps7.5 fps
4CIF
CIFQCIF
6
Temporal Scalability
Hierarchical B-picture H.264 allows B pictures may or may not be used
as references. Hierarchical prediction. Temporal scalability can be achieved by
hierarchical truncating B pictures.
8
4
2
1 3
6
5 7
10
9 11
14
13 15
12
16Key Picture
Group of Pictures (GOP size = 16)
16Key Picture
Level 1
Level 2
Level 3
Level 4
7
• Higher temporal level, larger distance between current and reference frames.
• Frames at higher temporal level are the references frames of subsequent lower temporal level frames.
Temporal Scalability- Motion Characteristics of Different Temporal Levels
Level 1
Level 2
Level 3
Level 4
8
4
2
1 3
6
5 7
10
911
14
13
15
12
4 pictures away
8 pictures away
16
8 pictures away
8
• Statistical distribution of optimal MVs• Obtained from full search.• Total 7 test sequences.
• MVs are scattered sparsely at higher temporal levels.
Temporal Scalability- Motion Characteristics of Different Temporal Levels
(%)
Level 1 Level 2 Level 3 Level 4
Origin 18.74 19.67 21.18 26.05
Within 9x9 44.12 46.20 66.08 82.96
Within 15x15 56.35 66.04 84.38 91.34
Level 1Level 2Level 3Level 4
8
1
9
Level 3Series1 Series2 Series3 Series4Series5 Series6 Series7 Series8Series9 Series10 Series11 Series12Series13 Series14 Series15 Series16Series17 Series18 Series19 Series20Series21 Series22 Series23 Series24Series25 Series26 Series27 Series28Series29 Series30 Series31 Series32Series33
MV distribution
Perc
enta
ge
Level 4Series1 Series2 Series3 Series4Series5 Series6 Series7 Series8Series9 Series10 Series11 Series12Series13 Series14 Series15 Series16Series17 Series18 Series19 Series20Series21 Series22 Series23 Series24Series25 Series26 Series27 Series28Series29 Series30 Series31 Series32Series33
MV distribution
Perc
enta
ge
Level 2Series1 Series2 Series3 Series4Series5 Series6 Series7 Series8Series9 Series10 Series11 Series12Series13 Series14 Series15 Series16Series17 Series18 Series19 Series20Series21 Series22 Series23 Series24Series25 Series26 Series27 Series28Series29 Series30 Series31 Series32Series33
MV distribution
Perc
enta
ge
Temporal Scalability- Motion Characteristics of Different Temporal Levels
Level 1Series1 Series2 Series3 Series4Series5 Series6 Series7 Series8Series9 Series10 Series11 Series12Series13 Series14 Series15 Series16Series17 Series18 Series19 Series20Series21 Series22 Series23 Series24Series25 Series26 Series27 Series28Series29 Series30 Series31 Series32Series33
MV distribution
Perc
enta
ge
1616
-16
0-16 016
16
-16
0-16 0
1616
-16
0-16 0 16
16
-16
0-16 0
10
Parallel Encoding Based on Hierarchical B-Picture Structure
1. Data-Level Parallelism• GOP, Slice, Picture, Macroblock
• GOP: Extensive memory usage limits its scalability.• Picture: Difficult to identify independent pictures.• Slice: Coding efficiency degrades due to slice
boundaries.• MB: Extensive requirement of synchronizations.
• Applicable to all encoders
2. Function-Level Parallelism• Asymmetric workload• Depends on encoder implementations
11
Parallel Encoding Based on Hierarchical B-Picture Structure
• MB-Level (Wave-front) Parallelism:• Only MB-Level parallelism can be achieved in traditional
codecs.• Extensive controls and synchronizations required.
• Frame-Level Parallelism:• Using IBBPBBP pattern, set B pictures as non-reference
pictures.
12
• Proposed Picture Decomposition Based on Hierarchical B-Picture:• Utilizing the hierarchical B-Picture structure, picture-level
parallelism is allowed in SVC
Parallel Encoding Based on Hierarchical B-Picture Structure
Level 4
Level 3
Level 2
Level 1
13
• Experimental results: execution time of motion estimation
Bus Stefan Conoa Garden Dancer Toys and Cal. Vintage Car Average
Sequen-tial
100 100 100 100 100 100 100 100
MB-Level 64.915215072876
66.8130943672275
64.0276532137518
64.9564379336929
65.785350781534
62.0400888512799
62.5428296438884
64.4400956948931
Proposed 58.1437125748503
59.1463414634145
59.0262582056893
57.7388149939541
60.0809170600134
59.1362697866482
56.4955270322832
58.5382630166932
5152535455565758595
ME Module Performance
Exec
ution
Tim
e (%
)
Parallel Encoding Based on Hierarchical B-Picture Structure
14
Parallel Encoding Based on Hierarchical B-Picture Structure
Experimental results: coding efficiency comparison
15
Future Work
• For parallel video encoding, modules like motion compensation and up-sampling are good candidates for data level parallel processing. Along with data level parallelism, the function level one can also be integrated into a hybrid scheme.
• Platform dependent issues such as power consumption and load balancing on asymmetric architectures are also important research issues
16
Thank for your attendance!
Any Comment is Welcome