wk 12 -- multimedia systems

7/24/2019 Wk 12 -- Multimedia Systems

1/46

Multimedia Systems 1CS502

Multimedia Systems

CS-502 Operating Systems


2/46


Outline

Requirements and challenges fr audi and!ide in cmputer systems

Systems fr multimedia

Cmpressin and "and#idth

$rcessr scheduling

%ile& dis'& and net#r' management

(anen"aum& Chapter )Sil"ershat*& Chapter 20


3/46

Multimedia Systems +CS502

,hat d #e mean "y multimedia.

/udi and !ide #ithin a cmputer system Cs 3 4s

Cmputer hard dri!e

i!e "radcast 3 #e" casts

,e"cams& S'ype&

4ide n demand $ause& fast fr#ard& re!erse& etc7

8nteracti!e meetings

$resentatins #ith 2-#ay audi (elecnferencing

8nteracti!e gaming


4/46


Requirements

Smth. audi and !ideeteriratin in quality :: ;er'y play"ac'

/udi?!ide n $Cs ding smething else

Multiple cncurrent streams

4ide 3 multimedia ser!ers(i4& etc7

,ide range f net#r' "and#idths


5/46


System and OS Challenges

@and#idths and Cmpressin

Aitter

$rcessr Scheduling

is' Scheduling


6/46


Sme System /rchitectures

Simple=ata paths fr audi?!ide that are separate

frm cmputatinal data paths

Mdern%ast system "us& C$B& de!ices

4ide ser!eris' farm and multiple streams


7/46

Multimedia Systems )CS502

audi stream

C-ROM

dri!e

System Organi*atin simpleD

Separate data path fr audi stream

Main system "us and C$B #ere t "usy?sl# t

handle real-time audi

C$B

Memry

memry "us

e!ice Sund

card


8/46

Multimedia Systems ECS502

!ide stream !ia8S/ 3 "ridge t

graphics card

audi stream !ia

8S/ "ridge tsund card

System Organi*atin typical $entiumD

8S/

"ridgeIDE

disk

MainMemry

C$B

e!el

2cache

@ridge Moni-

tor

Fraphics

card

BS@

Gey-

"ardMuse

Hther-net SCS8

MdemSund

card$rinter

$C8 "us

8S/ "us

/F$ $rt


9/46

Multimedia Systems ICS502

4ide Ser!er

Multiple C$Bs

is' farm

1000s f dis's

Multiple high-"and#idth net#r' lin's

Ca"le (4

4ide n demand

8nternet


10/46


,hy CmpressinJ C-quality audi

22&050 K* 99&100 samples?sec 16 "its per sample

(# channels 1)6&000 "ytes?sec

179 m"its?sec O'ay fr a mdern $C

Cmpressin rati !aries #ith type f music


11/46


,hy CmpressinJ 4ide

Standard. (4 frame L 690 9E0 piels N 25-+0frames?sec fpsDI&216&000 piels?sec L 2)&69E&000 "ytes?sec

K(4 L 12E0 )20 piels N +0 fpsE2&I99&000 "ytes?sec

(ypical m!ie 1++ minutesappr7 220 giga"ytes>

4 hlds 97) giga"ytesa!erage f 620 'il"ytes?sec>

Standard. m!ie f 1++ minutes requires seriuscmpressin ;ust t fit nt 4


12/46


4ide Cmpressin Requirements

Cmpressin rati : 50=1 i7e7& 220 giga"ytes=97) giga"ytes

4isually indistinguisha"le frm riginal H!en #hen paused

%ast& cheap decder Sl# encder is 'ay

4CR cntrls $ause& fast fr#ard& re!erse


13/46

Multimedia Systems 1+CS502

4ide Cmpressin Standards

M$HF MtinPictureEperts GrupD @ased n A$HF JintPhtgraphicEperts GrupD Multi-layer

ayer 1 L system and timing infrmatin

ayer 2 L !ide stream ayer + L audi and tet streams

(hree standards M$HF-1 +52290 framesP Q 175 m"?sec Q 4KS qualityD

ayer + L M$+ /udi standard

M$HF-2 standard (4 3 K(4P 175-15 m"?sec 4 encding

M$HF-9 cm"ined audi& !ide& graphics

2 3 + animatins


14/46


A$HF cmpressin single frameD

17 Cn!ert RF@ int 8 YL luminancei7e7& "rightnessD "lac'-#hite (4

I& QL chrominancesimilar tsaturationand hueD

Reasn= Kuman eye is mre sensiti!e tluminance than t clr rds vs.cnesD

27 #n-sampleI, Q channels i7e7& a!erage !er 22 piels t reduce reslutin

lssy cmpressin& "ut "arely nticea"le t eye

+7 $artitin each channel int EE "lc's 9E00 Y"lc's& 1200 eachI3 Q"lc's


15/46


A$HF cntinuedD


16/46


A$HF cntinuedD


17/46

Multimedia Systems 1)CS502

A$HF cntinuedD

97 CalculateDiscrete Cosine Transform

C(D f each EE "lc' ,hat is a iscrete Csine (ransfrmJ

57 i!ide EE "lc' f C( !alues "y

!uanti"ation ta#le Hffecti!ely thr#ing a#ay higher frequencies

67 ineari*e EE "lc'& run-length encde&and apply a Kuffman cde t reduce t a

small fractin f riginal si*e in "ytesD


18/46

Multimedia Systems 1ECS502

A$HF cncludedD

)7 Stre r transmit EE quanti*atin ta"lefll#ed "y list f cmpressed "lc's

/chie!es 20=1 cmpressin #ith gd !isualcharacteristics

Kigher cmpressin ratis pssi"le #ith !isi"le degradatin

A$HF algrithm eecuted "ac'#ards t rec!erimage

4isually indistinguisha"le frm riginal N 20=1

A$HF algrithm is symmetric Same speed fr#ards and "ac'#ards


19/46

Multimedia Systems 1ICS502

M$HF

A$HF-li'e encding f each frame

(a'es ad!antage f tem$oral localit%

87e7& each frame usually shares similarities#ith pre!ius frameencde and transmit nly differences

Smetimes an ";ect m!es relati!e t

"ac'grundfind ";ect in pre!ius frame& calculate

difference& apply motion vector


20/46


(empral cality eampleD

Cnsecuti!e 4ide %rames


21/46


M$HF rgani*atin

(hree types f framesI&frame'Intracded rIndependent7

%ull A$HF-encded frame

Occurs at inter!als f a secnd r s

/ls at start f e!eryscene

P&frame'Predicti!e frame ifference frm pre!ius frame

(&frame'(idirectinal frame

i'e$&frame"ut difference frm "th$reviousand ne)tframe

I ( (( P ( (( P ( (( P ( (( I ( (( P ( (( P


22/46


M$HF Characteristics

Cmpressin ratis f 50=1 E0=1 are

readily "taina"le

/symmetric algrithm

%ast decde li'e A$HFD

Hncde requires image search and analysis t

get high quality differences

ecding chips n graphics cards a!aila"le


23/46

Multimedia Systems 2+CS502

M$HF $r"lem %ast %r#ard?Re!erse

Cannt simply s'ip frames


24/46


M!ie. %ile Organi*atin

One M$HF-2 !ide stream

Multiple audi streams Multiple languages

Multiple tet streams Su"titles in multiple languages

/ll interlea!ed


25/46


Challenge

K# t get the cntents f a m!ie file frm dis'

r 4 dri!e t !ide screen and spea'ers7

%ied frame rate 25 r +0 fpsD

Steady audi rate @undedjitter

Classical pr"lem in real&time sche*ulin+

O"scure niche "ecme mainstream>

See Sil"ershat*& Chapter 1I


26/46


$rcessr Scheduling fr Real-(ime

ate Monotonic Sche*ulin+RMSD

/ssume mperidic prcesses

$rcess i requires Cimsec f prcessing time e!eryPi

msec7

Hqual prcessing e!ery inter!al U li'e clc'#r'>

/ssume

et pririty f prcess i"e

et pririty f nn-real-time prcesses "e -

11

=

m

i i

i

P

C

iP

1


27/46

Multimedia Systems 2)CS502

Rate Mntnic Scheduling cntinuedD

(hen using these pririties in scheduler guarantees

the needed Qualit% of Service SD& pr!ided that

D12C1

1

=

m

m

i i

im

P

C

/symttically appraches ln as m

87e7& must maintain smeslac/in scheduling

/ssumes fied amunt f prcessing per peridictas/


28/46

Multimedia Systems 2ECS502

$rcessr Scheduling fr Real-(ime

Earliest Dea*line 0irst H%DSche*ulin+

,hen each prcess i"ecme ready& it

annunces deadlineDifr its net tas/7

Scheduler al#ays assigns prcessr t

prcess #ith earliest deadline7 May pre-empt ther real-time prcesses


29/46

Multimedia Systems 2ICS502

Harliest eadline %irst Scheduling cntinuedD


30/46

Multimedia Systems +0CS502

Harliest eadline %irst Scheduling cntinuedD

H% is mre cmple scheduling algrithm $ririties are dynamically calculated

$rcesses must 'n# deadlines fr tas's

H% can ma'e higher use f prcessr thanRMS

Bp t 100V

K#e!er& it is usually a gd idea t "uild

in sme slac'


31/46

Multimedia Systems +1

CS502

Multimedia %ile 3 is' Management

Single m!ie r multimedia file n $C dis'

8nterlea!e audi& !ide& etc7

S temprally equi!alent "lc's are near each ther

/ttempt cntiguus allcatin /!id see's #ithin a frame

Text

FrameAudio

Frame


32/46


CS502

%ile rgani*atin %rame vs.@lc'

%rame rgani*atin Small dis' "lc's 9-16 G"ytesD

%rame inde entries pint t starting "lc' fr each frame

%rames !ary in si*e M$HFD

/d!antage= !ery little strage fragmentatin isad!antage= large frame ta"le in R/M

@lc' rgani*atin arge dis' "lc' 256 G"ytesD

@lc' inde entries pint t firstI&framef a sequence

Multiple frames per "lc'

/d!antage= much smaller "lc' ta"le in R/M

isad!antage= large strage fragmentatin n dis'


33/46

Multimedia Systems ++

CS502

%rame vs.@lc' rgani*atin

smaller larger


34/46


CS502

%ile $lacement n Ser!er

Randm

Striped

Organ pipe. allcatin

Mst ppular !ide in center f dis'


35/46


CS502

is' Scheduling ser!erD

Scheduling dis' acti!ity is ;ust as imprtant

as scheduling prcessr acti!ity

/d!antage= $redicta"ility Bnli'e dis' acti!ity f rdinary cmputing

8n ser!er& there #ill "e multiple dis'

requests in each frame inter!al

One request per frame fr each cncurrent !idestream


36/46


CS502

is' Scheduling cntinuedD

SC/< Hle!atrD algrithm fr each frame

inter!al Srt "y cylinder W

Cmplete in time fr start f net frame inter!al

4ariatin SC/


37/46

Multimedia Systems +)

CS502


38/46

Multimedia Systems +E

CS502

$ush vs.$ull ser!er


39/46

Multimedia Systems +I

CS502

@and#idth


40/46

Multimedia Systems 90

CS502

Cnclusin

Multimedia cmputing is challenging

$ssi"le #ith mdern cmputers Cmpressin is essential& especially fr !ide

Real-time cmputing techniques m!e intmainstream

$rcessr and dis' scheduling

(here is much mre t this su";ect than fitsint ne class


41/46


CS502

@rea'


42/46


CS502

igressin n (ransfrms

%uriers therem=H!ery cntinuus peridic functin can "e

reduced t the sum f a series f sine #a!es

(he0ouriertransformis a representatin fthat functin in terms f the frequencies fthse sine #a!es

Original functin can "e rec!ered frm its

%urier transfrm %urier transfrms ccur frequently in

nature>


43/46

Multimedia Systems 9+

CS502


44/46


CS502

iscrete Csine (ransfrm

/ frm f the %urier (ransfrm

,hen applied t an EE "lc' f samples

i7e7 piel !aluesD yields an EE "lc' f

s$atial fre!uencies

Original EE "lc' f samples can "e

rec!ered frm its C(7

/ssuming infinite arithmetic precisin


45/46


46/46


Return t A$HF

wk 12 -- multimedia systems

Documents