mpeg-4 standard mpeg-4 multimedia standard olivier dechazal

MPEG-4 standard

MPEG-4 Multimedia Standard

Olivier Dechazal

MPEG-4 standard2

Agenda

• Overview

• Audio coding

• Video coding

• System

MPEG-4 standard3

Definition of MPEG-4 applications

“A coded, streamable representation of audio-visual objects and their associated time-variant data along with a description of how they are combined”

MPEG-4 standard4

History

• MPEG-4 is an ISO/IEC standard developed by the Moving Picture Experts Group.

• MPEG-4 was finalized in October 1998 and became an International Standard in the first months of 1999

• Fully backward compatible extensions under the title of MPEG-4 Version 2 were frozen at the end of 1999, to acquire the formal International Standard Status early in 2000.

MPEG-4 standard5

Applications for MPEG-4

• Create content once, play on any network

• 3 levels of quality– Multimedia database, video games– Video phone, Multimedia authoring– Mobile videophone, wireless LAN

MPEG-4 standard6

Examples of possible scenes

(0,1,0)

Z

Y

X

2D inside a 3D plane

MPEG-4 standard7

Examples of possible scenes

Animated Text+ Video + Still Images

MPEG-4 standard8

Main characteristics

• Allows more interaction by the user

• Scalable

• Object based, allows scenes to be composed of natural and synthetic objects

• Very different from MPEG-1&2

MPEG-4 standard9

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA

Natural coding Synthetic coding

GA CELP Parametric TTS

MPEG-4 standard10

GA

• Stands for General Audio

• The input signal is first decomposed into a time/frequency spectral representation by means of an analysis filter bank

• Then subsequently quantized and coded with AAC and TwinVQ coders

MPEG-4 standard11

GA coder scheme

MPEG-4 standard12

AAC

• High quality

• Higher bit rate (above 32 kbp/s)

• Twice more compressed than MP3 for same quality

MPEG-4 standard13

TwinVQ

Transform-domain Weighted InterleaveVector Quantization (lower bit rate)

MPEG-4 standard14

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA


GA CELP Parametric TTS

MPEG-4 standard15

CELP

• Code Excited Linear Predictive

• Voice coding technique

• Used with bitrates between 6-24 kbit/s

• 2 sampling rates: 8 and 16 kHz

MPEG-4 standard16

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA


CELP Parametric TTSGA

MPEG-4 standard17

Parametric coders

• Very low bit rate (2 to 16 kbit/s )

• Decompose the input signal into components which are described by appropriate source models and represented by model parameters

• Certain aspects of the coded representation can be manipulated independently

• HVXC (Harmonic Vector Excitation Coding) for speech

• HILN (Harmonic and Individuals plus Noise) for music

MPEG-4 standard18

Parametric-HVXC

• Harmonic coding of LPC residual signals for voiced segments

• Vector eXcitation Coding for unvoiced segments

• 2.0 and 4.0 kbit/s of fixed bit rate mode

• Less than 2.0 kbit/s of variable rate mode

MPEG-4 standard19

HVXC decoder scheme

MPEG-4 standard20

Parametric-HILN

• Models parameters: – Harmonics lines: fundamental freq+

amplitudes of the harmonics components– Individual Lines: frequency and amplitude of

each individual line– Noise: spectral shape of the noise (gotten by

LPC method)

• Bit rate : 6-16 kbit/s

MPEG-4 standard21

HILN coder/decoder scheme

MPEG-4 standard22

Natural Audio Coders

Quality

Cellular

Telephone

AM

FM

CD

2 4 8 16 32 64 kbit/s

Parametric speech(HVXC)

High quality speech(CELP)

General audio(AAC, TwinVQ)

Parametric audio(HILN)

MPEG-4 standard23

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA



MPEG-4 standard24

SA

• Structured Audio (use the structural redundancy in the creation of the sounds)

• SAOL (Structured Audio Orchestra Language)software-synthesis language for any kind of synthesis (FM,

sampling, physical-modeling,…)

• SASL (Structured Audio Score Language) coding of the note desired for SAOL, the time of occurrence, and

the parameters controlling the differentiating algorithm (how loud the sound is, how long it is, how it varies)

• SASBF (Structured Audio Sample Bank Format) format for efficiently transmitting banks of sound samples

MPEG-4 standard25

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA



MPEG-4 standard26

TTS

• Text To Speech• TTS coders bit rate range from 200 bit/s to 1.2 Kbit/s• From a text or a text with prosodic parameters (pitch contour,

phoneme duration, and so on) it generates intelligible synthetic speech.

• Lip synchronization control with phoneme information. • Trick mode functionality: pause, resume, jump forward/backward. • International language and dialect support for text. (i.e., it can be

signaled in the bitstream which language and dialect should be used)

• International symbol support for phonemes, and support for specifying age, gender, speech rate of the speaker

MPEG-4 standard27

Scalable audio

• SNR / NMR (Noise to Mask Ratio) Scalability

• Audio Bandwidth Scalability

• Restriction of GeneralityEx: CELP + AAC

• Implementation ComplexityCore layer easier to decode

MPEG-4 standard28

MPEG-4 Video

MPEG 4

audio systemvideo

MPEG-4 standard29

Video coding-Main features

• Coding and animation of synthetic and natural hybrid video object

• Same approach as MPEG1/2 algorithms– Macro-block based DCT motion compensation– Uses I,P,B frames and variable length codes

• Wide range of bit rate 5 kbit/s to 5 Mbit/s available• Wide range of resolutions available (from a few pel per

line to TV resolution)• Supports the coding of arbitrary object shape (non

rectangular)• Allowed face and body animation • Coding of 2D and 3D Meshes with Implicit Structure• Supports coding of SPRITE objects

MPEG-4 standard30

MPEG-4 system

MPEG 4

audiosystemvideo

BIFS Obj Descrip DMIF

MPEG-4 standard31

BIFS

Binary Format for Scene

MPEG-4 standard32

BIFS

Tree scene description

MPEG-4 standard33

BIFS Features

• VRML concepts : set of nodes to represent the primitive scene objects to be composed, the behavior and interactivity

• Integration of streams• Integration of 2D and 3D video and audio

objects• Advanced Audio Features• Update protocol to modify the scene in time• Compression efficiency

MPEG-4 standard34

MPEG-4 system

MPEG 4

audiosystemvideo

Obj DescripBIFS DMIF

MPEG-4 standard35

Object descriptors

• Contain pointers to :– Scalably coded content streams– Alternate quality content streams– Object Content Information (locations, transparency,

…)– IPR Information

• Sub descriptors for :– Decoder Configuration– Sync. Layer Header Configuration– Quality of Service Information– Extension Information

MPEG-4 standard36

ObjectDescriptorID (OD_ID)

ObjectDescriptor{ OD_ID_1 List of { Elementary- Stream- Descriptors }}

ES_Descriptor{ ES_ID_1 .......}



Object descriptors

MPEG-4 standard37

MPEG-4 system

MPEG 4

audiosystemvideo

DMIFObj DescripBIFS

MPEG-4 standard38

DMIF

• Delivery Multimedia Integration Framework• It is the interface between the MPEG4

application and the transport network• Irrespective of whether the peer is a remote

interactive peer, broadcast or local storage media

• Open the different channels for the elementary streams with different bandwidth and QoS

• Use of different networks (IP, ATM, narrowband, mobile,…)

MPEG-4 standard39

System scheme

MPEG-4 standard40

Conclusion

• MPEG-4 provides a lot of tools to code audio and video objects for a whole range of applications

• In addition to this set of tools, MPEG-4 is a structure to manipulate interactively these objects

• MPEG-4 has been evolving (more audio and video coders)

MPEG-4 standard41

Main references

• N1683 MPEG4 Overview • N1695 MPEG4 Systems FAQ • http://garuda.imag.fr/MPEG4/syssite/syspub/main.html• http://www.chiariglione.org/mpeg/faq/mp4-aud/mp4-

aud.htm• http://www.tnt.uni-hannover.de/project/mpeg/audio/faq/

mpeg4.html• http://sound.media.mit.edu/mpeg4/sa-tech.html• http://faac.sourceforge.net/wiki/index.php?page=HVXC• http://wwwam.hhi.de/mpeg-video/standards/mpeg-

4.htm#E11E16

MPEG-4 standard42

Questions?????

Let’s go yellow jackets!!!!!!!!!

mpeg-4 standard mpeg-4 multimedia standard olivier dechazal

Documents