ai technology is changing the future of video compression · video live system video single...
TRANSCRIPT
IABM Copyright 2019@THEIABM www.theiabm.org
AI Technology is Changing the Future of
Video Compression
Stéphane Cloirec – Sr. Director, Video Appliances & Software
Harmonic
IABM Copyright 2019@THEIABM www.theiabm.org
Introduction
Harmonic, the worldwide leader in video delivery technology and services, enables media companies and service providers to deliver ultra-high-quality broadcast and OTT video services to consumers globally. The company has also revolutionized cable access networking via the industry’s first virtualized cable access solution, enabling cable operators to more flexibly deploy gigabit internet service to consumers’ homes and mobile devices.
As the Senior Director of Video Appliances & Software at Harmonic, Stephane Cloirec leads the product management and strategic direction of the company’s video appliances, including encoding, transcoding, stream processing, as well as playout, automation and storage solutions. Cloirec joined Harmonic with 15 years of expertise in product management from companies such as JDSU, Thales, Grass Valley and Thomson Video Networks, where he led product strategy and software transitions initiatives. He holds a master’s degree in signal processing from the University of Rennes and an engineering degree in telecommunications and computer science from Centrale-Supelec, Paris.
IABM Copyright 2019@THEIABM www.theiabm.org
Agenda
Video Compression History
Typical problems solved by AI
Video Compression Decisive Criteria
Implementations
AI Advantages
Machine Learning Types and Relation to Video Compression
Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
Video Compression History
Multipass look ahead encoding
Motion Compensated Temporal Filter
First multichannel 1RU encoder
Pure software encoder
Content-Aware Encoding (CAE)
A long process!
Continuous Improvement
IABM Copyright 2019@THEIABM www.theiabm.org
Artificial Intelligence:
A New Way Of Making Improvements!
IABM Copyright 2019@THEIABM www.theiabm.org
Typical Problems Solved By AI
Processing large data sets
Decision helping
Predictions!
Detection
IABM Copyright 2019@THEIABM www.theiabm.org
Benefits brought by AI and ML
• Speed
• Security
• Accuracy
Additional benefitsCost Reduction
• Machine replaces human
• For detection/prediction based activities
• Allows to do much more of it since it becomes cheap
IABM Copyright 2019@THEIABM www.theiabm.org
Video Compression Decisive Criteria
Density
• Decisions
• Frame or field coding
• Macroblock size
• Picture type
• Bit rate allocation
• VQ measure
Latency
VQ/Bit rate trade off
IABM Copyright 2019@THEIABM www.theiabm.org
Implementation!
IABM Copyright 2019@THEIABM www.theiabm.org
Machine Learning applied to Video Compression
• Improved compression
Supervised learning
• Clustering video sources
Unsupervised learning
• Continuous improvements
Reinforcement learning
IABM Copyright 2019@THEIABM www.theiabm.org
Implementing AI/ML for Video Compression
Learning parameters
Live System
Encoder Packager Mux
Offline Learning SystemTest Files
IABM Copyright 2019@THEIABM www.theiabm.org
AI Advantages and Opportunities
• New Services deployed Quickly
• Customization
New opportunities, appropriate for SaaS
Development savings
• Replace years dev. by CPU time
Density improvements
• AI implementation using less CPU than human developed algorithms
IABM Copyright 2019@THEIABM www.theiabm.org
Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
AI
Dynamic Resolution Encoding
Dynamic Encoding Style Dynamic Frame rate
Encoding
Encoding Pre processing and Encoding
AI Encoding Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
• Goal is to reduce video bitrate • Goal is to improve user QoE
Encoding Style
AI chooses encoding parameters depending on video input
Dynamic Resolution Encoding
AI chooses best resolution depending on video input
Can be combined all together
• Goal is to improve encoding density
Dynamic Frame Rate Encoding
AI chooses a lower frame rate depending on video input
• Any application (ABR, CBR, VBR)
480p 1080p 60fps 30fps5 Mbps 4 Mbps
• Application to Mobile (zero rating) • Application to IPTV (enable HD on SD lines)• Application to OTT streaming on Main screen
AI Encoding Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
AI
Dynamic Resolution Encoding
Dynamic Encoding Style Dynamic Frame rate
Encoding
Encoding Pre processing and Encoding
AI Encoding Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
Dynamic Encoding Style
Offline Learning System
Test Files
Encoding Styles
Bitrate and VQ stats
Video Analysis
AIEncoding Style
Prediction Model
Encoder
CompressedVideo
Live system
Video
Encoding Core
Encoding StylePrediction Model
Video AnalysisCompressed Video
IABM Copyright 2019@THEIABM www.theiabm.org
Dynamic Encoding Style
Offline Learning System
Test Files
Encoding Styles
Bitrate and VQ stats
Video Analysis
AIEncoding Style
Prediction Model
Encoder
CompressedVideo
Live system
Video
Encoding Core
Encoding StylePrediction Model
Video AnalysisCompressed Video
Demonstrates bitrate saving vs traditional VBRup to 20% for the most complex content
IABM Copyright 2019@THEIABM www.theiabm.org
AI
Dynamic Resolution Encoding
Dynamic Encoding Style Dynamic Frame rate
Encoding
Encoding Pre processing and Encoding
AI Encoding Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
Encoder
CompressedVideo
Live system
Video
Single Encoding Core
ResolutionPrediction Model
Video AnalysisCompressed Video
Offline Learning System
Test FilesVideo Analysis
AIResolution
Prediction Model
x Resolution Encoding VQ stats
Dynamic Resolution Encoding
IABM Copyright 2019@THEIABM www.theiabm.org
Encoder
CompressedVideo
Live system
Video
Single Encoding Core
ResolutionPrediction Model
Video AnalysisCompressed Video
Offline Learning System
Test FilesVideo Analysis
AIResolution
Prediction Model
x Resolution Encoding VQ stats
Dynamic Resolution Encoding
Perceived video quality improvement
for all type of contents
IABM Copyright 2019@THEIABM www.theiabm.org
AI
Dynamic Resolution Encoding
Dynamic Encoding Style Dynamic Frame rate
Encoding
Encoding Pre processing and Encoding
AI Encoding Application Examples
IABM Copyright 2019@THEIABM www.theiabm.org
Encoder
CompressedVideo
Live system
Video
Single Encoding Core
Frame ratePrediction Model
Video Analysis Compressed Video
Offline Learning System
Test Files Video Analysis
AIFrame rate
Prediction Model
x FR EncodingVQ stats
Dynamic Frame Rate Encoding
IABM Copyright 2019@THEIABM www.theiabm.org
Encoder
CompressedVideo
Live system
Video
Single Encoding Core
Frame ratePrediction Model
Video Analysis Compressed Video
Offline Learning System
Test Files Video Analysis
AIFrame rate
Prediction Model
x FR EncodingVQ stats
Dynamic Frame Rate Encoding
Measured 35% CPU usage optimization
on average for typical video content mix
IABM Copyright 2019@THEIABM www.theiabm.org
BW saving QoE Scalability
Dynamic Resolution Encoding
Dynamic Encoding Style
Dynamic Frame rate Encoding
V V X
X X V
V VX
Summary
IABM Copyright 2019@THEIABM www.theiabm.org
AI research started several
years ago at Harmonic
EyeQ is the first product to use AI techniques for
OTT (ABR)
AI techniques expended to other
applications(CBR, VBR)
Exciting Future! More optimization, more customization, nextgen video standards, …
Conclusion
IABM Copyright 2019@THEIABM www.theiabm.org
• https://www.pyimagesearch.com/2016/06/20/detecting-cats-in-images-with-opencv/
• https://medium.com/@skychain.global/skychain-ai-for-lung-cancer-detection-released-e246ba694228
• http://didarc.com/en/news/artificial-intelligence-and-autonomous-vehicles
Picture Credits
IABM Copyright 2019@THEIABM www.theiabm.org
Thank You.
Broadcast Asia 2019
©2019 Harmonic Inc. All rights reserved worldwide.