iee 5037 multimedia communications lecture 1:...

IEE 5037 Multimedia CommunicationsLecture 1: Introduction

Dr. Tian-Sheuan [email protected]. Electronics EngineeringNational Chiao-Tung University

Dept. Electronics Engineering,N

ational Chiao T

ung University Special Thanks to Prof. Hang to provide most of the slides

in this lecture, except highlighted by other sources.

2

Chapter 1 IntroductionWhy need compression

How compression possible

Principles of Compression Techniques

Multimedia Standards

Subjects to be discussed

3

Why Compression?

-- Massive dataSpeech: 8 bits (per sample) x 8K (samples/sec) = 64Kbits/sCD audio:

16 bits x 44.1K (samples/sec) x 2 (channels) = 1.411Mbits/sec(44.1K = 60 (fields) x 245 (lines) x 3 (samples) (J. Watkinson,The Art of Digital Audio, p.28, Focal Press, 1989) )

Digital TV: (4:2:2, NTSC in CCIR 601)Still picture: 720 (pels) x 483 (lines) x 2.0 bytes = 5.564 MbitsMotion picture: 5.564 Mbits x 29.97 (frames/sec) = 167Mbits/sec

Digital HDTV: (ATSC)1920 (pels) x 1080 (lines) x 1.5 bytes x 30 (frames/sec)=746Mbits

4

Picture: “Hotel“, 720x576, 414,720 Byte

Source: T. Wiegand “Digital image communication”

5

JPEG-2000 Compressed to 12,960 Byte


6

JPEG-2000 Compressed to 3,240 Byte


7

The Power of Compression


8

Digital Image Formats


9

Digital Image Formats


10

Digital Image Formats and Applications


11

Examples of Storage Media


12

Examples of Transmission Media


13

An Example of Comm. Systems< A Digital TV System >

Our focus


14

Examples of Data Sources-- Classified based upon compression techniques

General digital data: various types of computer filesTextSpeech: human conversationAudio: musicBi-level images: fax, dithered (two-tone) imagesStill gray-level pictures: graphics, natural imagesMotion pictures: video conferencing, television.

-- Data characteristics can be used to reduce bit rate.

15

How Compression Possible?Characteristics of data:

Stationary statistical model -- Shannon information theoryNon-stationary properties such as local correlation

Characteristics of human perception:Finite resolution of hearing and visionAuditory masking effectColor representationVisual masking effect

16

Signals (Waveforms)

Time/ Space

Amp.

Analog Signals x(t)

Conti. Conti.

Discrete-time (discrete-space) (sampled-data) signal x(m)

Discrete Conti.

Digital signals x(m)

Discrete Discrete

17

Audio SamplesPiano (fur Elise) samples and spectrogram (McClellan et al., DSP First, Prentice-Hall, 1998)

18

Data Compression TechniquesInformation lossless: Redundancy reduction ─ The

original data can be completely recovered.Direct: Huffman codes, arithmetic coding, Ziv-Lempel coding, … (narrow-sense data compression)Predictive: Run length coding, …

Information lossy: Information (entropy) reduction --The reproduced data are approximations of the original data. This may not be meaningful for a computer file.Block coding: vector quantization, transform coding, …Sequential: DPCM, tree coding, …Multi-resolution (non-block): sub-band, wavelet, ...

19

Compression TechniquesWaveform coding: Reproduce waveform,e.g., DPCM, transform, sub-band, …-- Universal but lower efficiency.Content-based coding: Reproduce contents, e.g., (speech) vocoder, (image) contour-texture coding, (video) model-based coding.International standards: JPEG, MPEG, H.261/3, …

20

Compression Algorithm Development

Phase 1: Modeling. Describe redundancy in the form of a model. E.g., prob. distribution, Markov model, …--Techniques are then invented to remove redundancy; e.g.,: predictor, transform, …Phase 2: Coding. Representations of “model”and “residual” (unpredictable) information. --Techniques: e.g., quantization and entropy coding of the prediction errors, …

21

Elements in Waveform CodingDecorrelation: Reduce spatial and temporal redundancy. Techniques: prediction, transform, …Selecting representatives: Reduce the number of possible signals. Techniques: quantization, …Entropy coding: Equalize the probability distribution of the output symbols. Techniques: Huffman codes, Ziv-Lempel coding, ...

22

Multimedia Coding Standards

Complete, practical coding algorithms-- A balance between (compression) performance and (implementation) complexity (and a compromise among various interest parties)Critical for telecommunication products and consumer audio/video media products

23

Standards Organizations

CCITT – Comité Consultaitif International Télégraphique et Téléphonique (International Telegraph and Telephone Consultative Committee) ITU – International Telecommunication UnionISO – International Standardization

OrganizationIEC – International Electrotechnical

Commission

24

Image/Video StandardsISO/IEC JTC1 SC29 – ISO and IEC Joint Technical Committee (on Information Technology) Subcommittee 29 (Coding of audio, picture, multimedia and hypermedia)– Working Group (WG) 1:

JBIG (Joint Bi-level Image Group) – 1-bit to 4/5-bit still pictures

JPEG (Joint Photographic Experts Group) – 8-bit or more still picturesISO/IEC JTC1 SC29– WG 11: MPEG (Moving Picture Experts Group) – Motion pictures– WG 12: MHEG (Multimedia-Hypermedia Experts Group) –Multi/Hyper-media exchange format

25

Video Coding StandardsStandards Typical rates ApplicationsITU-T (CCITT) H.261 128 384k bits/s Videophone over ISDN ISO MPEG-1 (11172-2) 1.2 Mbits/s Video CDISO MPEG-2 (13818-2) 4–10 Mbits Digital TV/HDTV (ITU-T H.262) 20 Mbits/s Over air/networksITU-T H.263 < 64k bits/s Videophone over PSTNISO MPEG-4 (14496-2) Low/high-rates Object-orientedISO MPEG-7 (15938) Database Content descriptionITU-T H.263 v2 < 64k bits/s PSTN/wireless VideophoneITU-T H.264 (JVT,AVC) < 40k bits/s Net/wireless Videophone

ISDN: Integrated Services Digital NetworkPSTN: Public Switched Telephone Network

26

MPEG Audio StandardsStandards (Year) Coding rates Sampling rates/

(channel)

MPEG-1 Layer 1 (92) 32k – 448 kbits/s 32, 44.1, 48k (1-2 chs)



MPEG-2 Layers 1,2,3 (94) 32, 44.1, 48, 16, 22.05, 24k (1-5.1)

MPEG-2 AAC (97) 8-64 kbits/s/ch 8, …, 96k (1-96 chs)

MPEG-4 AAC (99) 8-64 kbits/s/ch 8, …, 96k (1-96 chs)

27

Speech Coding StandardsStandards Typical rates (Year) Quality: MOS (1-5)PCM 64 kbits/s (1972) 4.4 (PSTN)G.721 ADPCM 32 kbits/s (1984) 4.1 (PSTN)GSM 13 kbits/s (1991) 3.6 (Cellular) G.728 (low delay) 16 kbits/s (1992) 4.0IS-96A (CDMA) 0.8-8.55 kbits/s (1993) ~3.4 (Cellular) G.729 8 kbits/s (1995) ~4.2G.723.1 5.3, 6.3 kbits/s (1995) ~4.0Half-rate GSM 5.6 kbits/s (1995) ~3.4 (Cellular) AMR 5.15-12.2 kbits/s (1999) ~3.9 (3GPP)

MOS: Mean Opinion Score -- 5=excellent, 4=good, 3=fair, 2=poor, 1=bad

28

Subjects To Be DiscussedBasic theory: Shannon theory (outline)Lossless coding algorithms: Huffman, arithmetic, run-length.Lossy coding algorithms: (vector) quantization, differential coding, Transform coding, subband coding, wavelet-coding, motion estimation/compensation. Standards: JPEG, JPEG2000, H.261/3 basic, MPEG-1/2/4 video.

29

Organization - from ISO/IEC to WG

Where does MPEG fit into the scheme of things

ISOInternation Standards Organization

IECInternational Electrotechnical Commission

ISO/IECJTC 1

Joint Technical Committee 1

Source: MPEG 101, N6706

30

Organization - from ISO to WG

Where does MPEG fit in the scheme of things

ITTFInternational Technical Task Force

Geneva, CH

SC xxSubcommittee

WG 1JPEG

WG11MPEG

Moving Picture Experts GroupMilan, IT

WG12MHEG

(last meeting 2001/03

SC 29Subcommittee 29

Secretariat - ITSJC/IPSJTokyo, Japan

JTC 1Joint Technical Committee - Information Technology

Secretariat - ANSINew YorK City

ISO/IECGeneva, CH


31

Organization - from WG to adhoc

WG11 MPEG Committee

Leonardo Chiariglione

WG11 MPEG Committee


SystemsOlivier Avaro


VideoJen-Rainer Ohm

VideoJen-Rainer Ohm

AudioSchuyler Quackenbush


MDSIan Burnett

MDSIan Burnett

TestTobias Oelbaum

TestTobias Oelbaum Requirments

Rob KoenenRequirmentsRob Koenen

LiaisonJan Borman

LiaisonJan Borman ISG

Marco MattevelliISG

Marco Mattevelli

IntegrationJean-Claude Durford


SNHCMickael Borgas-Sevenier



32

Organization - from WG to adhoc

WG11 MPEG Committee


WG11 MPEG Committee




VideoJen-Rainer Ohm

VideoJen-Rainer Ohm



MDSIan Burnett

MDSIan Burnett

TestTobias Oelbaum

TestTobias Oelbaum Requirments

Rob KoenenRequirmentsRob Koenen

LiaisonJan Borman

LiaisonJan Borman ISG

Marco MattevelliISG

Marco Mattevelli






33

Membership

MPEG experts

National Bodies

WG11

SC29 NB members of SC29

JTC1 NB members of JTC1

ISO/IECNational Standard

Organisations


34

The MPEG Modus Operandi

•ExplorationThe search for new technologySeek Industry expertsOpen seminars

•RequirementsEstablish the scope of workCall for Proposals

•Competitive phaseDo HomeworkResponse to CfPInitial technology selection

•Collaborative phaseCore ExperimentsWorking Drafts


35

The MPEG Modus Operandi (cont’d)

•StandardizationBallotsNational Body Comments

•AmendmentAdding new technology

•CorrigendaCorrective actions

•New subdivisionsAdd new non-compatible technology


36

Where to from here?

• Ongoing search for technology enhancements

• Call for evidence of new/improved technology

• Significant benefits need to be demonstrated

• Actions include:•amendments or •new subdivisions to existing standards or •entirely new standards


37

Where to from here

Amendments • compatible enhancements to an existing specification

Corrigenda• Corrective actions to any element of the specification

Collaboration• Establish liaison with other duly constituted organizations• Exchange requirements and specification.

New subdivision of specification• non-compatible enhancements and additions to technology


38

Specification development time-lines

No fix duration specified or required -

•Exploration6-12 months depending on extent of search

•Requirements development6-12 months partly in parallel with exploration

•Competitive phase3-6 months partly in parallel with requirements

•Collaborative phase1 year following completion of competitive phase

total approximately 2 yearsSource: MPEG 101, N6706

39

Completion phase of development

Fixed minimum timeline - balloting

3 months 2 months publish

Specification CD FCD FDIS IS

Amendment PDAM FPDAM FDAM AMD

Corrigenda X DCOR X COR

TechnicalReport PDTR X DTR TR

Documenttype

4 months

All documents start out in life as working drafts (WD)


40

Alphabet soup

WD - Working Draft

CD - Committee DraftFCD - Final Committee DraftFDIS© - Final Draft International StandardIS © - International Standard© indicates ISO copyright

PDAM - Proposed Draft AmendmentFPDAM - Final Proposed Draft AmendmentFDAM © - Final Draft AmendmentAMD © - Amendment


41

More Alphabet soup

DCOR - Draft CorrigendaCOR © - Corrigenda

PDTR - Proposed Draft Technical ReportDTR © - Draft Technical ReportTR © - Technical Report

DoC - Disposition of Comments


42

Step by step

WDs become CDs (PDAM, PDTR, DCOR)(changes from here on out are by ballot / comments only)

CDs are balloted by National Bodies

Ballots are submitted by NB’s to SC 29 with or without comments(at FDIS ballots are JTC 1 level)

Comments are processed and approved at MPEG meetings (DoC)

Specification moves on to next stage


iee 5037 multimedia communications lecture 1:...

Documents