scalable video coding extension of h.264/avcmapl.nctu.edu.tw/course/vc_2009/files/07.special...
TRANSCRIPT
Scalable Video Coding Extension of H.264/AVC
Wen-Hsiao Peng, Ph.D
Multimedia Architecture Processing Laboratory (MAPL)Department of Computer Science, National Chiao Tung University
May 2008
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 1 / 68
Scalable Extension of H.264/AVC
Outline
Introduction
Scalable Video Coding Extension of H.264/AVC
Standardization ActivitiesCoding TechnologiesTransport Interface
R-D Performance
Complexity Analysis
Concluding Remarks
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 2 / 68
Introduction
Introduction
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 3 / 68
Introduction De�nition
Scalable Video Coding (SVC)
Embedded Video Representation
Spatial, Temporal, SNR, and Combined Scalability
40kbits/s
Enhancement-LayerBase-Layer
3Mbits/s
Time
SNRScalability
SpatialScalability
TemporalScalability
EmbeddedRepresentation
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 4 / 68
Introduction Concepts
Scalability
QCIF
QCIF@75Hz,L QCIF@15Hz,M QCIF@30Hz,H
CIF
CIF@15Hz,L CIF@30Hz,H
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 5 / 68
Introduction Rationales
Why Do We Need SVC?
Diversi�ed Clients
Di�erent Computation Power and Display Capabilities
Heterogeneous Networks
Di�erent Types of NetworksTime-varying Channel Bandwidth
Servers
Limited Storage Spaces - compression is requiredLimited Bandwidth - unicast may not be applicable
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 6 / 68
Introduction Rationales
Why Do We Need SVC?
Ethernet
Ethernet
Server
Wireless
Point-to-PointTransmission
Broadcasting
Router
Wireless
512 kbps
32 kbps
128 kbps
256 kbps
64 kbps
3 Mbps
1.5 Mbps
384 kbps
64 kbps
Bandwidth
Time
DiversifiedClients
Time-VaryingBandwidth
LimitedResources
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 7 / 68
Introduction Rationales
Graceful Degradation
Foreman
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 8 / 68
Introduction Applications
How is SVC Useful?
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 9 / 68
SVC Standardization Activities
SVC Standardization Activities
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 10 / 68
SVC Standardization Activities SVC Roadmap
Roadmap
MPEG-4 Part 10 Amd. 3
Started in MPEG-21 Part 13Moved to Joint Video Team (JVT) since 2004/10
Roadmap
2003/03, Call for Evidence (CfE)2004/04, Call for Proposal (CfP) - 12 Wavelet-based + 2 DCT-based2004/10, Activities Moved to JVT (Palma Meeting)2005/01, Wording Draft (WD) - HHI Proposal2006/01, Proposed Draft of Amendment (PDAM)2006/07, Final Proposed Draft of Amendment (FPDAM)2007/01, Final Draft of Amendment (FDAM)2007/07, Amendment (AMD)
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 11 / 68
SVC Standardization Activities SVC History
SVC Call-for-Proposal
AVC/H.264-based technology
HHI, NCTU
Wavelet-based schemes
University of BresciaDanae+ ThomsonMicrosoft Research Asia (MSRA)University of South Wales (UNSW)
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 12 / 68
SVC Standardization Activities SVC History
AVC/H.264-based Technology
Scalable Extension of AVC/H.264
Decomation
MCTF
MCTF
4x4Transform
4x4Transform
MotionRefinement
ResiduePrediction
Spatio-Temporal Transform
Scalable orCABACCoding
Entropy Coding
Scalable orCABACCoding
Bit-Stream
Input
Inter-LayerMotion
Prediction
SpatialDecimation
ScalableEntropyCodingTemporal
PredictionSpatialDCT
Inter-LayerResidue
Prediction
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 13 / 68
SVC Standardization Activities SVC History
Wavelet-based Technology
t+2D
TemporalMCTF
Spatial2D Wavelet
Spatio-Temporal Transform
ScalableCoding
Entropy Coding
Bit-StreamInput
TemporalWavelet
SpatialWavelet
ScalableEntropyCoding
2D+t
Spatial2D W avele t
In-bandT empora l
M CT F
In-bandT empora l
M CT F
S patio-Temporal Transform
S ca lableCoding
Entropy Cod ing
S calableCoding
Bit-Stream
InputHF
LL
Spatial 2DD ecom po sition
S patia l 2DDecompositio n
SpatialWavelet
In-BandTemporalWavelet
In-BandDCT
ScalableEntropyCoding
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 14 / 68
SVC Standardization Activities SVC History
Subjective Quality Evaluation
Single Stimulus Multimedia Test (SSMT)
Absence of unimpaired reference11 grade scale
T e s t s e q u e n c e 1 T e s t s eq u e n c e 2
V o te1
V o te2
1 0 s 5 s 1 0 s 5 s
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 15 / 68
SVC Standardization Activities SVC History
Subjective Quality Comparison
Harbour - QCIF 15Hz
0 1 2 3 4 5 6 7 8 9 10
192 kbps
128 kbps
96 kbps
MOS
BresciaDanae + ThomsonHHIMSRAUNSW
AVC/H.264-based
Wavelet-based
Quality
Bit Rate
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 16 / 68
SVC Standardization Activities SVC History
Summary
AVC/H.264-based Technologies
Better Viewing Quality but Worse PSNR
Wavelet-based Schemes
Worse Viewing Quality but Better PSNR
2004, AVC/H.264-based approach formally became starting point
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 17 / 68
SVC Technology
SVC Technology Overview
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 18 / 68
SVC Technology Technology Overview
SVC Technology Overview
Temporal Scalability
Motion Compensated Temporal Filtering (MCTF)Hierarchical B Pictures
Spatial Scalability
Pyramid and Stack StructureSeparate MCTF/Motion Descriptions
SNR Scalability
Coarse Granularity Scalability (CGS)Medium Granularity Scalability (MGS)
Adaptive Inter-layer Prediction
No need to decode multiple prediction loopsComparable decoding complexity than single layer
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 19 / 68
SVC Technology Technology Overview
SVC Encoder Block Diagram
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 20 / 68
SVC Technology Technology Overview
SVC Syntax Structure
NAL unitheader
Sliceheader Slice data
Slicetrail
Macroblock layer End ofslice
cabac_alignment
mb_skip_run
mb_skip_flag
mb_field_decoding_flag
Slice data
Sub_mb_pred
Mb_pred
residual_pred_flag
Macroblock layer
base_mode_flag mb_type
pcm sample
coded_block_pattern
mb_qp_delta Residual
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 21 / 68
SVC Technology Temporal Scalability
Temporal Scalability
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 22 / 68
SVC Technology Temporal Scalability
Dyadic Temporal Scalability
U
L0 L0 L0 L0 L0 L0 L0 L0 L0
-
H1
-
H1
-
H1
-0.5
0.5
H1
+ +
0.25+ +
L1 L1 L1 L1
P
GOP BoundaryInput Frames
0.25
L0 L0 L0 L0 L0 L0 L0 L0 L0
1
2 2
53 4 43
5 6 6 7 7 8 8
GOP Boundary
MCTF Hierarchical B Prediction
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 23 / 68
SVC Technology Temporal Scalability
Motion Compensated Temporal Filtering (MCTF)
Wavelet Transform along Motion Trajectory
H
G
2
2
InputHigh Pass Output
Low Pass Output
5/3 Lifting Scheme
Prediction Step, Pk = [�1/2, 1,+1/2]
hk|{z}HighPassOutputs
= S2k+1| {z }OddFrames
� Pk S2k| {z }EvenFrameFiltering
Update Step, Uk = [3/4, 1, 3/4]
lk|{z}LowPassOutputs
= S2k|{z}EvenFrames
+ Uk hk| {z }HighPassFrameFiltering
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 24 / 68
SVC Technology Temporal Scalability
MCTF Example
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 25 / 68
SVC Technology Temporal Scalability
Hierarchical B Predictions
Hierarchical B Predictions
Remove Update Step + Close-loop PredictionEnabled by H.264/AVC Syntax
Hierarchical B vs. P (Low Delay)
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 26 / 68
SVC Technology Temporal Scalability
E�ciency and Mismatch Trade-o�
Closed Loop (Hierarchical B)
Use reconstructed frames for predictionNo Mismatch but Worse Prediction E�ciency
Open Loop (MCTF)
Employ original pictures for predictionMismatch with Better Prediction E�ciency
−
In-LoopFrame Buffer
+
Closed-Loop Encoder
Prediction
Input Video
Reconstruction
Quantization−
Out-LoopFrame Buffer
Open-Loop Encoder
Prediction
Input Video
Quantization
InverseQuantization
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 27 / 68
SVC Technology Temporal Scalability
Encoder and Decoder Mismatch
Drifting Errors
−
Out-LoopFrame Buffer
Open-Loop Encoder
Prediction
Input Video
Quantization +
In-LoopFrame Buffer
InverseQuantization
Reconstruction
Closed-Loop Decoder
Output VideoPredictor Mismatch
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 28 / 68
SVC Technology Temporal Scalability
Loop Control Performance
Closed-loop is better in most cases
Foreman CIF 30 Hz
kbits/s
150 200 250 300 350 400 450
PSNR
-Y
34
35
36
37
38
39
MCTF with Open-LoopMCTF with Closed-LoopTraditional GOP with IBBP
Foreman QCIF 15 Hz
kbits/s
40 50 60 70 80 90 100 110
PSNR
-Y
33
34
35
36
37
38
39
MCTF with Open-LoopMCTF with Closed-LoopTraditional GOP with IBBP
Close > Open > IBBP Close > Open > IBBP
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 29 / 68
SVC Technology Spatial Scalability
Spatial Scalability
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 30 / 68
SVC Technology Spatial Scalability
Spatial Scalability
Stack Structure with Adaptive Inter-layer Predictions
Residual prediction performed in Spatial domain
Hiearchical Prediction
−
In-LoopFrame Buffer
+
Q
−
In-LoopFrame Buffer
+
Q
Interpolation
Inter-Layer Residual Prediction
Out-LoopFrame Buffer
EntropyCoder
Spatial Enh.-Layer
EntropyCoder
Decimation
MotionCompensation
MotionCompensation
{M.V}
MCTF
Spatial Base-Layer IQ
Scaling &Refinement
IQ
−
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 31 / 68
SVC Technology SNR Scalability
SNR Scalability
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 32 / 68
SVC Technology SNR Scalability
SNR Scalability
Coarse Granularity Scalability (CGS)
Distinctive Quality LevelsStack Structure of Spatial Scalability
Medium Granularity Scalability (MGS)
Packet-based Quality Scalable CodingKey Pictures
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 33 / 68
SVC Technology SNR Scalability
Coarse Granularity Scalability
Stack Structure (Similar to Spatial Scalability)
Residual prediction performed in Transform domain
Distortion
Bit RateR1 R2 R3
1λ
2λ
3λ
Hiearchical Prediction
−
In-LoopFrame Buffer
+
Q
−
In-LoopFrame Buffer
+
Q
Inter-Layer Prediction
Out-LoopFrame Buffer
EntropyCoder
EntropyCoder
MotionCompensation
MotionCompensation
{M.V}
MCTF
Quality Base-Layer IQ
Refinement
IQ
−
Quality Enh.-Layer 1
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 34 / 68
SVC Technology SNR Scalability
Medium Granularity Scalability
Packet-based Quality Scalable Coding
Distribute Transform Coe�cients among Several Slices
1514109
131183
12742
6510
1514109
131183
12742
6510
Key Picture Concepts
Trade-o� between Drift and Coding E�ciencyResynchronization Points between Encoder and Decoder
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 35 / 68
SVC Technology SNR Scalability
Drift Control
BL-only, EL-only, Two-Loop, and Key Pictures
(A) BL-only (B) EL-only
(C) Two-Loop (D) Key Pictures
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 36 / 68
SVC Technology SNR Scalability
Viewing Quality Comparison
Two Loops
TwoLoops
Key Pictures
KeyPictures
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 37 / 68
SVC Technology SNR Scalability
Performance Comparison of Prediction Structures
BL-only, EL-only, Two-Loop, and Key Pictures
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 38 / 68
SVC Technology SNR Scalability
Performance Comparison of SNR Scalability
CGS performance decreases with increasing number of layers
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 39 / 68
SVC Technology Adaptive Inter-layer Prediction
Adaptive Inter-layer Prediction
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 40 / 68
SVC Technology Adaptive Inter-layer Prediction
Adaptive Inter-Layer Prediction
Motion Prediction
Macroblock Type, Reference Index, Motion Vector
Residual Prediction
Subtraction of Base Layer from Enhancement Layer
Textural Prediction
Inter-layer Intra Prediction
Remarks
No Base Layer ReconstructionApproximately 0-0.5dB Coding Loss
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 41 / 68
SVC Technology Adaptive Inter-layer Prediction
Multi-level Inter-layer Prediction
GOP Boundary
Spatial Enh. Layer 1with MCTF
(Frame Rate = FR)
Base Layer with Hierarchical B
Pictures(Frame Rate = FR/2)
B2B1 B2A
H1H2 H1
H3 H1H2 H1
L3
H1H2 H1
H3 H1H2 H1
L3Spatial Enh. Layer 2with MCTF
(Frame Rate = FR)
A
L3
L3
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 42 / 68
SVC Technology Adaptive Inter-layer Prediction
Inter-layer Motion Prediction
Macroblock Type, Reference Index, Motion Vector
base_mode_flag
co-located 8x8 block
inter-layerintra-prediction
deriveMB partition,mv, ref index
mb_type
motion_prediction_flag
derive mvp,ref index
mvd
mv, refindex
residual signal
1 0
01
Inter-codedIntra-coded
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 43 / 68
SVC Technology Adaptive Inter-layer Prediction
Macroblock Partition Derivation
Partition Derivation across Spatial Resolutions
8x8 8x4
4x44x8
D irec t,16x16, 16x8
8x16Intra
16x16 16x8
8x88x16
16x16 16x16
16x16 16x16
Intra Intra
Intra Intra
Subordinate Layer
Spatial Enhancement Layer
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 44 / 68
SVC Technology Adaptive Inter-layer Prediction
Inter-layer Residual and Textural Prediction
Textural Prediction - Intra-coded Macroblocks
De-blocking Filtering before Spatial InterpolationSpatial Interpolation across Submacroblocks (8x8)
Residual Prediction - Inter-coded Macroblocks
Independent from Motion PredictionNo Spatial Interpolation across Transform Blocks
Textural Prediction Residual Prediction
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 45 / 68
SVC Technology Adaptive Inter-layer Prediction
Gain of Textural Prediction
Simulcast vs. SVC+Textural
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 46 / 68
SVC Technology Adaptive Inter-layer Prediction
Gain of Motion Prediction
SVC+Textural vs. SVC+Textural/Motion
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 47 / 68
SVC Technology Adaptive Inter-layer Prediction
Gain of Residual Prediction
SVC+Textural/Motion vs. SVC+Textural/Motion/Residual
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 48 / 68
Combined Scalability
Combined Scalability
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 49 / 68
Combined Scalability
Combined Scalability
CIF, 30 Hz, 115 - 256 kbit/sCIF, 15 Hz, 88 - 222 kbit/s
CIF, 7.5 Hz, 65 - 189 kbit/s
CIF, 3.75 Hz, 55 - 165 kbit/sCIF, 1.875 Hz, 48 - 139kbit/s
QCIF, 15Hz 41 - 80 kbit/s
QCIF, 7.5Hz 32 - 66 kbit/s
I
L3 H0 H1
B3 B2 B1
Spatialupsampling
SNR base 41 kbit/s
B2 PB3 B3 B3
I B3 B2 B1 B2 PB3 B3 B3
Layer 0: QCIF 15Hz
FGS 41 - 80 kbit/s
Layer 1: CIF 30 Hz
FGS 115 - 256kbit/s
SNR base 115kbit/s
FGS refinement
FGS refinement
Prediction
H0 H2 H0 H1 H0 H3 H0 H1 H0 H2 H0 H1 H0 L3
L3 H0 H1 H0 H2 H0 H1 H0 H3 H0 H1 H0 H2 H0 H1 H0 L3
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 50 / 68
Encoder Optimization
Encoder Optimization
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 51 / 68
Encoder Optimization
Bottm-up Encoder Control
Current JSVM
Bottom-up Encoding Process
Sequential Mode Selection
Base Layerp�B = arg minfPBg
[DB (PB ) + λBRB (PB )]
Enhancement Layer
p�E = arg minfPE jP�Bg
[DE (PE jP�B ) + λERE (PE jP�B )]
Remarks
Base layer is comparable to H.264/AVCEnhancement layer is much worse than H.264/AVC
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 52 / 68
Encoder Optimization
Multi-loop Encoder Control
Joint Base and Enhancement Layer Optimization
Base Layer Encoding
p�B = arg minfPE jPB ,PBg
�ω� [DB(PB) + λBRB(PB)]+
(1�ω)� [DE (PE ) + λERE (PE )]
�
Enhancement Layer Encoding (not modi�ed)
p�E = arg minfPE jP�Bg
[DE (PE jP�B ) + λERE (PE jP�B )]
Remarks
Trade-o� between Base and Enhancement Coding E�ciency
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 53 / 68
Encoder Optimization
Multi-loop Encoder Control
10% bit rate increase relative to H.264/AVC
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 54 / 68
SVC Transport Interface
SVC Transport Interface
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 55 / 68
SVC Transport Interface SVC NAL Units
SVC NAL HEADER
TraditionalNAL
Header
(D, T, Q)Information
DiscardableFlag
Priority IDof a NAL
No Inter-LayerPrediction Flag
Use BaseReference
IDR Flag
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 56 / 68
SVC Transport Interface SVC NAL Units
SVC NAL Grouping
Layer Representation, Dependency Layer, Scalable Layer, ScalableLayer Representation, Access Unit
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 57 / 68
SVC Transport Interface SVC NAL Units
SVC Inter-layer Dependency Hierarchy
(Layer Type, Dependency id, Quality id)
MGS_3_2
MGS_3_3
MGS_4_1
MGS_4_2
MGS_3_1
BASE_4_0BASE_0_0
CGS_1_0
CGS_2_0
BASE_3_0
CGS_4_0
FGS_4_1
BASE_5_0
FGS_5_1
FGS_4_2
Spatial layer 0 Spatial layer 1 Spatial layer 2
FGS_5_2
BASE_0_0
CGS_1_0
CGS_2_0
BASE_3_0
MGS_3_1
MGS_3_2
BASE_4_0
MGS_4_1
MGS_3_3
Spatial layer 0 Spatial layer 1 Spatial layer 2
MGS_4_2MGS_3_2
MGS_3_3
MGS_4_1
MGS_4_2
MGS_3_1
BASE_4_0BASE_0_0
CGS_1_0
CGS_2_0
BASE_3_0
CGS_4_0
FGS_4_1
BASE_5_0
FGS_5_1
FGS_4_2
Spatial layer 0 Spatial layer 1 Spatial layer 2
FGS_5_2
BASE_0_0
CGS_1_0
CGS_2_0
BASE_3_0
MGS_3_1
MGS_3_2
BASE_4_0
MGS_4_1
MGS_3_3
Spatial layer 0 Spatial layer 1 Spatial layer 2
MGS_4_2
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 58 / 68
SVC Transport Interface Scalability Information Signaling
Scalability Information SEI
Number of Scalable Layers
Information of Each Scalable Layer
Layer Identi�er + Decoding Dependency Informationlayer id = (Dependency id � 16+Quality id)� 8+ Temporal idBit Rate, Frame Size, Frame RateInitial Parameter SetsRegion-of-interest (ROI) Information
Priority Information
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 59 / 68
SVC Transport Interface Layer Switching
Layer Switching
Switching between Quality Re�nement Layers
Possible in each access unit
Switching between Dependency Layers
Possible at IDR access units
Down-switching
Possible in virtually any access unitRequire multiple-loop decoding
Up-switching
Wait for next IDR access unit
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 60 / 68
SVC Transport Interface Layer Switching
R-D Optimized Layer Switching
What to produce and what to use?
QCIF@30Hz or CIF@10Hz?Spatial quality vs. Temporal quality
RDO Without RDO
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 61 / 68
H.264/AVC vs. SVC
H.264/AVC vs. SVC
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 62 / 68
H.264/AVC vs. SVC
R-D Performance
CREW_Combine Scalability
34
34.5
35
35.5
36
36.5
37
37.5
0 400 800 1200 1600 2000 2400 2800 3200Bitrate (kbps)
PS
NR
Y (d
B)
SVC_4CIF60SVC_CIF30
SVC_QCIF15
AVC_4CIF60AVC_CIF30
AVC_QCIF15
AVC/H.264@4CIF SVC
@4CIF
AVC/H.264@CIF
SVC@CIF
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 63 / 68
SVC Complexity Analysis
SVC Complexity Analysis
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 64 / 68
SVC Complexity Analysis
How Complex is SVC?
Complexity of SVC depends on # of prediction loops
Encoder is slower than real-time by
60x for temporal scalability in CIF resolution146x for spatial scalability27x for CGS, 75x for FGS in CIF resolution1000x for combined scalability
Decoder is slower than real-time by
16x for spatial scalability1.7x for CGS, 5x for FGS in QCIF resolution92x for combined scalability
Decoder is faster than real-time for temporal scalability in QCIF
Ratio of memory access over computation is >200
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 65 / 68
Concluding Remarks
Concluding Remarks
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 66 / 68
Concluding Remarks
Concluding Remarks
Scalable Video Coding (SVC)
Spatial, Temporal, SNR, and Combined ScalabilityPower and Format Adaptation with Graceful Degradation
What is Next?
Encoder OptimizationDecoder ImplementationTransport MechanismBit-depth and Color ScalabilityInterlaced Videos
Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 67 / 68
References
References
1 H. C. Huang, W. H. Peng, T. Chiang, and H. M. Hang, \Advances inthe Scalable Amendment of H.264/SVC," IEEE CommunicationsMagazine, vol.45, pp. 68 - 76, 2007.
2 H. Schwarz, D. Marpe, and T. Wiegand, \Overview of the ScalableVideo Coding Extension of the H.264/AVC Standard," IEEETransactions on Circuits and Systems for Video Technology, vol. 17,no. 9, pp. 1103 - 1120, September 2007.
3 T. Wiegand, G. Sllivan, J. Reichel, H. Schwarz, and M.Wien, \JointDraft ITUT Rec. H.264 { ISO/IEC 14496-10/Amd.3 Scalable VideoCoding," ISO/IEC JTCI/SC29/WG11 and ITU-T SG16 Q.6,JVT-X201, July 2007.
4 T. Wiegand, \Scalable Video Coding," ISO/IEC JTCI/SC29/WG11and ITU-T SG16 Q.6, JVT-W132, April 2007.
5 Y.-K. Wang, M. Hannuksela, S. Pateux, A. Eleftheriadis, and S.Wenger, \System and Transport Interface of SVC," IEEETransactions on Circuits and Systems for Video Technology, vol. 17,no. 9, pp. 1149 - 1163, September 2007.Wen-Hsiao Peng, Ph.D (NCTU CS) MAPL May 2008 68 / 68