mm-4094, amd video compression engine: the route towards low-latency cloud gaming solutions, by...
DESCRIPTION
Presentation MM-4094, AMD Video Compression Engine: The Route towards Low-Latency Cloud Gaming Solutions, by Khaled Mammou and Ihab Amer at the AMD Developer Summit (APU13) November 11-13, 2013.TRANSCRIPT
AMD VIDEO CODING ENGINE THE ROUTE TOWARDS LOW-LATENCY CLOUD
GAMING SOLUTIONS
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 2
AGENDA
Introduction
Low Latency Encoding
AMD Video Coding Engine
Conclusions
Introduction
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 4
A LITTLE BIT OF HISTORY!
2008/9
2011/12
Perf/W
Year
CPU
CPU-based
Video Coding
CPU GPU
CPU HW
IP
GPU-
accelerated
Video Coding
HW-
accelerated
Video Coding
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 5
CPU VS. GPU VS. FIXED-FUNCTION-HW VIDEO COMPRESSION
Pros
‒ No extra dollars
‒ Higher achievable qualities at target bitrates (less architectural limitations)
‒ High Flexibility & Short Lead Time
‒ Can be optimized by advanced IS (e.g. MMX)
Cons
‒ Limited Operations per WATT
‒ Monopolized CPU
Examples
‒ Handbrake/x264
CPU Pros
‒ No extra dollars
‒ Part of CPU available for other tasks
‒ SW-based – Still Flexible & relatively Short Lead Time
Cons
‒ Limited Operations per WATT
‒ Major design and code changes to leverage parallelism
‒ Massive parallelism impacts quality/bitrate
Examples
‒ GPU-accelerated MainConcept Enc.
‒ GPU-accelerated x264
CPU+GPU Pros
‒ Fast!
‒ Power Efficient!
‒ Most of CPU available for other tasks
Cons
‒ Additional Area Cost
‒ Least Flexible (hard coded)
‒ Long lead time
Examples
‒ Applications in the market that support:
‒ AMD VCE
‒ Intel Quick Sync
‒ NVIDIA NVEnc
Fixed-Function HW
AMD Video Coding Engine
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 7
AMD VIDEO CODING ENGINE (VCE)
VCE is AMD’s dedicated fixed-function video coding engine for improved video encoding performance
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 8
VCE TARGET PLATFORMS
AMD APUs
AMD Discrete GPUs
Server Yes Yes
Desktop Yes Yes
All-in-one Yes Yes
Premium Notebook Yes Yes
Value Notebook Yes Possible
Tablet Yes N/A
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 9
VCE MAIN USE CASES
(**) Courtesy of CiiNow, Inc.
(*) Courtesy of Cyberlink, Inc. VCE
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 10
Up to 3x 1080p@~30fps per instance
Low-power budget
Multi-streaming support
Configurable speed/quality tradeoff
Flexible/programmable to meet various use-cases
VCE CAPABILITIES
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 11
DESIGN ASPECTS OF VCE
Bitra
te
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 12
VCE CREW – OUR EVERYDAY STORY!
“I don’t believe in perfection. I don’t think there is
such a thing. But the energy of wanting things to be
great is a perfectionist energy!”
Reese Witherspoon Hollywood Actress & Academy Award Winner
“Have no fear of perfection – you’ll never reach it!”
Salvador Dalí Spanish Painter/Artist
Low Latency Encoding
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 14
Latency is the elapsed time between the user’s input and his/her perception of the corresponding game reaction
LATENCY IS KEY! WHAT IS LATENCY?
Game client
Game server
Game Engine
Code Picture Buffer
Decoder
Decoded Picture Buffer
Display
User
Encoder
Buffer
Graphics Commands Rendered Frame Compressed Stream
Network Packets NAL Units Decoded Frame
Encoder
Ren
der
ing
Network
< 100–150 ms −
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 15
Game server
Encoding latency is the elapsed time between a frame is rendered on the server and it is decoded on the client
LATENCY IS KEY! WHAT IS ENCODING LATENCY?
Game client
Game Engine
Code Picture Buffer
Decoder
Decoded Picture Buffer
Display
User
Encoder
Buffer Encoder
Network
Rendered Frame Compressed Stream
Network Packets NAL Units Decoded Frame
Ren
der
ing
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 16
VCE Quality Presets
ENCODE SPEED VS. QUALITY HOW FAST CAN VCE ENCODE?
1080p 720p 480p
Speed 95 fps
215 fps
535
Balanced
80 fps
180 fps
470
Quality 40 fps 90 fps 250
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 17
0
10
20
30
40
50
60
70
0 10 20 30 40 50 60
Inst
anta
ne
ou
s b
itra
te (
mb
its/
s)
Frame number
CQP
Avg. Bitrate (6Mbit/s)
Constant Quantization Parameter (CQP)
BUFFERING DELAY VS. QUALITY WHY BUFFERING?
Tranmission over a Constant Bitrate (CBR) channel of 6 Mbit/s ?
Buffering
Transmission delay
(latency!)
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 18
Constant Bit Rate (CBR) Rate Control
‒ Control transmission delays (i.e., # bits per picture) by adjusting QPs
‒ Leaky bucket model
‒ Defined by the triplet
‒ Avoid encoder buffer underflow and overflow (i.e., transmission bitrate = encoding bitrate)
BUFFERING DELAYS VS. QUALITY WHAT IS CBR?
Encoder buffer fullness
time
B
…
b0
S0
b1
S1 S2
b2
S0+De
Encoder can predict the decoder
buffer fullness
Buffering latency is smaller than B/R
Output Rate (R)
Initial
Fullness (F)
Buffer
Size (B)
Input Rate
Decoder buffer fullness
time
B
F
b1
T1
b2
T2 … T0
b0
T0-Dd
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 19
Variable Bit Rate (VBR) Rate Control
‒ Channel can stop transmission without loosing synchronization (e.g., packet based networks)
‒ Leaky bucket model
‒ Defined by the triplet
‒ Avoid only encoder buffer overflow (i.e., transmission bitrate may be higher than the encoding bitrate)
LATENCY VS. QUALITY WHAT IS VBR?
Allows shorter buffering delay
AMD Media SDK exposes two VBR modes: Peak-Constrained VBR, and Latency-Constrained VBR
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 20
0
10
20
30
40
50
60
70
0 10 20 30 40 50 60
Inst
anta
ne
ou
s b
itra
te (
mb
its/
s)
Frame number
VBR High Buffering Latency
CQP
Avg. Bitrate (6Mbit/s)
High buffering delay
‒ Peak-Constrained VBR
‒ Target Bitrate = 6 Mbps
‒ Peak Bitrate = 6 Mbps
‒ VBVBufferSize = 6 Mbits (i.e., buffering latency of 1000 ms)
‒ IDRPeriod = 60
LATENCY VS. QUALITY HOW BUFFERING LATENCY IMPACTS QUALITY?
Reduce overshoots with
limited quality impact
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 21
0
10
20
30
40
50
60
70
0 10 20 30 40 50 60
Inst
anta
ne
ou
s b
itra
te (
mb
its/
s)
Frame number
VBR Low Buffering Latency
VBR High Buffering Latency
CQP
Avg. Bitrate (6Mbit/s)
Low buffering delay
‒ Latency-Constrained VBR
‒ Target Bitrate = 6 Mbps
‒ Peak Bitrate = 6 Mbps
‒ VBVBufferSize = 0.1Mbits (i.e., buffering latency of 16 ms)
‒ IDRPeriod = 60
LATENCY VS. QUALITY HOW BUFFERING LATENCY IMPACTS QUALITY?
Minimum buffering delays
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 22
Low buffering delay
LATENCY VS. QUALITY HOW BUFFERING LATENCY IMPACTS QUALITY?
Poor IDR frames quality
Replace IDR frames with partially intra-encoded frames
Frame 300 (IDR)
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 23
Intra-refresh principle
ERROR RESILIENCY VS. QUALITY WHAT IS INTRA-REFRESH?
IDR P P P
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 24
Intra-refresh principle
‒ Spread out Intra Units throughout successive pictures
‒ Constraint inter/intra-prediction to preserve error resiliency (i.e., Dirty/Clean Maps)
ERROR RESILIENCY VS. QUALITY WHAT IS INTRA-REFRESH?
Restriction on Search Region
Restriction on Search Region
I D D D C I D D C C I D C C C I
Restriction on Search Region
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 25
Intra-refresh with low buffering delay
‒ VBR Rate Control
‒ Target Bitrate = 6 Mbps
‒ Peak Bitrate = 6 Mbps
‒ VBVBufferSize = 0.1Mbits (i.e., buffering latency of 16 ms)
‒ IDRPeriod = 60
HOW INTRA-REFRESH IMPACTS QUALITY?
ERROR RESILIENCY VS. QUALITY
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 26
Intra-refresh with low buffering delay
HOW INTRA-REFRESH IMPACTS QUALITY?
ERROR RESILIENCY VS. QUALITY
Frame 300 (IDR)
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 27
Intra-refresh with low buffering delay
HOW INTRA-REFRESH IMPACTS QUALITY?
ERROR RESILIENCY VS. QUALITY
Improved IDR frames quality
Frame 300 (Intra-Refreshed)
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 28
VCE vs. Software Encoder
‒ VBR Rate Control
‒ Target Bitrate = 6 Mbps
‒ Peak Bitrate = 6 Mbps
‒ VBVBufferSize = 0.1Mbits (i.e., buffering latency of 16 ms)
‒ IDRPeriod = 60
‒ Intra-refresh
HOW GOOD IS VCE?
COMPARATIVE EVALUATION
Software Encoder
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 29
VCE vs. Software Encoder
HOW GOOD IS VCE?
COMPARATIVE EVALUATION
VCE Software Encoder
Conclusions
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 31
CONCLUSIONS
Fixed-Function HW acceleration is the prevalent technology for video compression
AMD MediaSDK and RapidFire allow application developers to configure VCE parameters
‒ Latency/Error Resiliency/Quality
‒ Number of encoded streams/Power/NW-efficiency
VCE offers an out-of-the-box solution for ultra-low latency cloud gaming
‒ Special rate control settings
‒ Intra-refresh support
Deployed in the solutions of various cloud-gaming partners
Thank You!
| AMD VCE FOR LOW-LATENCY CLOUD GAMING | NOVEMBER 19, 2013 33
DISCLAIMER & ATTRIBUTION
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.
The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.
AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. Other names are for informational purposes only and may be trademarks of their respective owners.