Download - Overview of Selected Current MPEG Activities
Overview of SelectedCurrent MPEG Activities
Christian Timmerer
Klagenfurt University (UNIKLU) Faculty of Technical Sciences (TEWI)Department of Information Technology (ITEC) Multimedia Communication (MMC)
http://research.timmerer.com http://blog.timmerer.com mailto:[email protected]
January 2010
A report from the 91st MPEG meeting, Kyoto, Japan
Christian Timmerer, Klagenfurt University, Austria 2
Outline
• Background / Introduction– MPEG, who the f*** is MPEG?– Benefits for using MPEG technologies and contributing to MPEG
• Selected Current MPEG Activities– Req: MVS, MMT, AIT– Sys: MPEG-M/-U/-V– Vid: 3DVC, HVC– Aud: SAOC
• ConclusionsJanuary 2010
Christian Timmerer, Klagenfurt University, Austria
Moving Picture Experts Group (MPEG)
January 2010 4-3
MPEG Family of Standards: Applications and Technologies
MPEG-1,2,4: have fueled tremendous growth in digital media content
MPEG-7: makes digital content searchable (value in the metadata!)
MPEG-21: transactions of digital items
MPEG Standards Evolution: – Coding Content Description Transactions
“Media ContentDescription”
(MPEG-7)
EnterpriseContentMgmt
“Transactions of Digital Items”
(MPEG-21)
E-Commerce Of Digital Content
ContentAdaptation
“Digital ContentCoding”
(MPEG-1,2,4)
Streaming
Broadband
VideoStorage
• Compression• Coding• Communications
• Automatic indexing• Multimedia search engines• Content-based retrieval• Personalization & summarization
Object-basedManipulation
MediaLogging
FlexibleBusinessModels
Applications
Technologies
Repurposing
© John R. Smith, Tutorial at WWW’04
Rights management Media mining and decision support
Christian Timmerer, Klagenfurt University, Austria 4
Multimedia Middleware
(MPEG-E/-M)
Moving Picture Experts Group (MPEG)
January 2010
“Umbrella” standards – Multimedia
Application Formats(MPEG-A) Tools – MPEG
System, Video, Audio
(MPEG-B/-C/-D)
Immersive User Experience
(MPEG-U/-V)
MPEG Standards Evolution: – Framework Integration Tool Sets Middleware
Immersive User Experience Platform
Platform(Advanced
IPTV Terminal)
MPEG-A: value combinations of existing standards MPEG-B/-C/-D: have fueled tremendous growth in digital
media content MPEG-E/-M: make standards accessible enabling tiny
applications MPEG-U/-V: increase the Quality of Experience
• Combination of Standards / Profiles / Levels for applications
• Payload / file / delivery formats• Generic coding tools
• Architecture• Application Programming Interfaces• Resource / quality management• Fault / integrity management• Protocol payload formats
• Widgets & Advanced User Interaction• Sensory Information• Virtual World Object Characteristics
• Elementary / Aggregated Services• APIs, Protocols, Payload Formats
Technologies
MPEG Family of Standards: The Success Story continues …
Applications
Music Player
PhotoPlayer
Streaming
Archival
Broadcasting
Surveillance Surround Sound
Streaming
Video Tool Library
Object-based Manipulation
Metadata Coding
Tiny, Fast Application
Deployments
Multimedia Value Chain
Streaming in Heterogeneous Environments
Content Sharing, the smart way
Virtual Worlds
Multi-DeviceRendering + Interaction
Streaming
Internet-TV
Media Value
Networks
Mobile-TV
TV-Everywhere
PaymentCashing
Interactive Personalized TV
Christian Timmerer, Klagenfurt University, Austria 5
Why using MPEG and contributing to MPEG?
Why using MPEG standards?
• Leading edge SDO for multimedia coding and systems
• Open IPR, world, access (public), meeting, interface, use
• On-going support• Reference software and
conformance bitstreams available
Why contributing to MPEG standards?
• Direct, tough feedback from the experts improve➪
• Visibility of your research results impact➪
• Good networking, contact to industry research ➪project(s) (proposals)
• Patents / Licensing $$$➪
January 2010
Christian Timmerer, Klagenfurt University, Austria 6
Mobile Visual Search (MVS)
January 2010
Requirements
Applications:• SnapTell service by Amazon/A9• Point & Find service by Nokia• kooaba application by kooaba ltd• Google Goggles application by Google
Scope: Bitstream enabling efficient transmission of local image features in mobile visual search and mobile augmented reality applications.
Goals:• Interoperability between visual
search applications and visual database servers
• Improve performance of mobile visual search applications
• Facilitate efficient implementation of visual search functionality on mobile phones http://www.chiariglione.org/mpeg/working_documents.htm#Explorations
Christian Timmerer, Klagenfurt University, Austria 7
MPEG Media Transport (MMT)
January 2010
Requirements
Multimedia Applications
MPEG Media Transport (MMT)MPEG Media Transport (MMT)
Objectives• Efficient delivery of media• Enable content and
network adaptive• Enable cross-layer
optimization• Support integrated services• Support DRM tools• Support content forwarding
and retransmission
Multimedia Applications
MPEG Media Transport (MMT)
Data Link Interface
Timeline Call for Proposal in its next meeting in April 2010Evaluation of submissions in its July 2010 meetingPhase I: Application layer protocols to be finalized by the first quarter of 2011
http://www.chiariglione.org/mpeg/working_documents.htm#Explorations
Christian Timmerer, Klagenfurt University, Austria 8
Advanced IPTV Terminal (AIT)
January 2010
Requirements
Service: a set of content and functions to be offered to a User within the AIT value chain.
Elementary Service: a service for which protocols and APIs are defined within AIT.
Aggregated Service: two or more elementary services.
Aim: definition of elementary services with corresponding set of protocols and APIs to enable any user in an AIT value chain to access those services in an interoperable fashion. The service definition and its composition is also within the scope of AIT.
An AIT value chain is a collection of Users, including Creators, End Users and Service Providers .
http://www.chiariglione.org/mpeg/working_documents.htm#Explorations
Christian Timmerer, Klagenfurt University, Austria 9
AIT: Elementary Services
January 2010
Requirements
Christian Timmerer, Klagenfurt University, Austria 10
MPEG-M: MPEG-Extensible Middleware (MXM)• A set of APIs to access possibly all MPEG
technologies• Simple methods to call complex functionalities
inside MXM engines• “Thin” applications because the complexity is in
the MXM engines• Replacement of MXM engines with better
performing ones at no cost• Creation of a global market of MXM Engines,
MXM Applications and MXM Devices
January 2010
Systems
• Part 1 – MXM Architecture and Technologies• Part 2 – MXM Application Programming Interfaces (APIs)• Part 3 – MXM Reference Software• Part 4 – MXM Protocols: specifies a set of protocols
enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection information
http://mxm.wg11.sc29.org/
Christian Timmerer, Klagenfurt University, Austria 11
MPEG-U: Rich Media User Interfaces (RMUI)
January 2010
Systems
Objectives• Enable one to consistently deliver
a rich experience across consumer electronics and mobile devices
• Broad range of app domains (home network, broadcast, Web, mobile)
• Update UI continuously• Integration w/ media types (audio,
video, text, 2D and 3D graphics) Use Cases
Remote UI and customized/personalized UIUI aggregation from different applicationsWidget Communication, downloadable widget,secured widgetMultimodal widget http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-U
http://www.chiariglione.org/mpeg/technologies/mpeg-u/mpu-widgets/index.htm
Christian Timmerer, Klagenfurt University, Austria 12
MPEG-V: Media Context and Control (MCC)
January 2010
System Architecture
Pt. 1: Architecture
Pt. 3: Sensory Information
Pt. 4: Virtual World Object Characteristics
Pt. 2: Control Information
Systems
Pt. 6: Common Types and Tools
Pt. 7: Conformance and Reference Software
Pt. 5: Data Formats for Interaction Devices
http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-V
Christian Timmerer, Klagenfurt University, Austria 13
3D Video Coding (3DVC)
Timeline• Done: Test material defined• Final depth maps + preliminary anchor coding for new sequences (until April 2010)• Final Anchor coding for new sequences (until July 2010)• Draft CfP (April 2010), internal• Preliminary and Final CfP (July/Oct. 2010)
January 2010
Video
State of the art:• ISO/IEC 23002-3 (MPEG-C Pt. 3): 2D+Depth
o Backward compatible w/ legacy deviceso Agnostic of coding formatso Limited depth range (doesn’t handle
occlusions)• ISO/IEC 14496-10 Amd: Multiview Video
Coding (MVC)o Direct coding of multiple viewso Exploits inter-camera redundancyo Rate of MVC encoded video is proportional
to the number of views
http://www.chiariglione.org/mpeg/working_documents.htm#Explorations
Christian Timmerer, Klagenfurt University, Austria 14
High-performance Video Coding (HVC)• Final CfP issued jointly: MPEG + VCEG• Timeline
– 01/22-02/15: formal registration period (33 pre-registered)– 02/22: coded test material shall be available at the test site – 03/02: Start of subjective tests– 04/12-13: registration/submission of documents– 04/17: subjective test results available– 04/17-23: evaluation of proposals at standardization meeting
• Test classes (* … only PSNR evaluation)– Class A*: Cropped areas of size 2560x1600 (30fps):– Class B: 1920x1080p 24 fps & 1920x1080p 50-60 fps– Class C: 832x480p 30-60 fps (WVGA)– Class D: 416x240p 30-60 fps (WQVGA)– Class E: 1280x720p 60fps
• Testing fee: 8900 CHF plus 7.6% VAT• Test coordinator: Vittorio Baroncini (FUB)
January 2010
Video
Applications ➪ Everything but nothing new!? ➪ Just more performance!?
Requirements ➪ Compression Performance ➪ Picture formats (8Kx4K) ➪ Color spaces + sampling ➪ Frame rates ➪ Scanning modes ➪ Complexity ➪ Low delay ➪ Random access, trick modes ➪ Error resilience ➪ Buffer models ➪ Interface to system layers ➪ Scalable Video Coding
http://www.chiariglione.org/mpeg/working_documents.htm#Explorations
Christian Timmerer, Klagenfurt University, Austria 15
Spatial Audio Object Coding (SAOC)Transmission of high-quality multi-channel audio signals• @bitrates previously used for mono/stereo signals• Backward compatibility with existing receivers
January 2010
Audio
http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-D
Spatial Audio Object Coding (SAOC)
MPEG-4 Object-Based Audio Representation• Bitrate & computational decoding complexity
grows approximately proportionally with the number of transmitted objects
• Structural complexity of the decoder can be an obstacle to implementation in many real-world applications
• High compression efficiency due to using a downmix signal plus accompanying parametric side information
• Backward compatibility with respect to existing trans- mission structures
Christian Timmerer, Klagenfurt University, Austria 16
Conclusions
• What is MPEG and what are they doing (right now)?– SDO in the area of multimedia coding & systems– MVS, MMT, AIT, MXM, RMUI, MCC, 3DVC, HVC, SAOC,
etc.
• Long time commitment but sustainability
• Next meeting: 19-23 April 2010, Dresden (not that far away)
January 2010
Christian Timmerer, Klagenfurt University, Austria 17
Thank you for your attention
... questions, comments, etc. are welcome …
Ass.-Prof. Dipl.-Ing. Dr. Christian TimmererKlagenfurt University, Department of Information Technology (ITEC)
Universitätsstrasse 65-67, A-9020 Klagenfurt, [email protected]
http://research.timmerer.com/Tel: +43/463/2700 3621 Fax: +43/463/2700 99 3621
© Copyright: Christian Timmerer
January 2010