A DataAwareAdmissionControlTechniquefor SocialLive

Streams(SOLISs)

SumitaBarahmandandShahramGhandeharizadeh

DatabaseLaboratoryTechnicalReport2012-03

ComputerScienceDepartment,USC

Los Angeles,California90089-0781

July20,2012

Abstract

A SOcialLIve Stream,SOLIS, is a live streamproducedby a device whoseowner is sharingthe

streamwith herfriends,grantingeachfriendto performtimeshiftedviewing for apre-specifiedduration.

Thesystembuffersthischasedatato facilitateits browsinganddisplay. In thepresenceof many SOLISs,

memorymayoverflow andpreventdisplayof somechasedata.This paperpresentsa novel data-aware

admissioncontrol, DA-AdmCtrl, techniquethat summarizeschasedatapro-actively to maximizethe

numberof admissibleSOLISswith no memoryoverflow. It is designedfor usewith multi-coreCPUs

andmaximizesutility of datawhenevertheuser’slevel of satisfaction(utility) with differentdataformats

is available. We useanalyticalmodelsandsimulationstudiesto quantify the tradeoff associatedwith

DA-AdmCtrl.

A Introduction

Socialnetworking sitessuchasRAYS [5], Qik [23] or Bambuser[4] enabletheir membersto invite their

friends to view a live streamfrom their devices. A streamingdevice might be a smartphonesuchasan

iPhoneor an inexpensive wirelessconsumerelectronicdevice by a vendorsuchasLinksys or Panasonic.

With RAYS, theownerof aDevice�mayspecifyachaseduration( ����� � ) for anInvitee � . ThisallowsInvitee� to time shift view [12, 16, 17] datafor thespecifiedduration ����� � . ThetermSOcialLive Stream,SOLIS,

refersto device producedstreamswith thischaracteristic[6], seeFigure1.

The motivation for usingstreamingdeviceswith a socialnetworking web site is two folds. First, to

sharea SOLISwith a largenumberof invitees.Second,to supportfunctionalitysuchaschasedisplayand

recordingof streamsfor future recall andsharing.With the first, a device typically supportsa handfulof

simultaneousviewersdueto eitherlimited network bandwidth,processingcapability, or both.For example,

1

Figure1: A SOLISwith two chasedisplays.

thePanasonicBL-C131cameraconfiguredto useits wirednetwork connectionsupportsamaximumof 9 to

12 simultaneousdisplays.Whenusedwith a socialnetworking website,a streammight besharedamong

thousandsof viewers.

With the second,the constantlyevolving chasedatacorrespondingto the past ����� � time units must

be bufferedon behalfof Invitee � . The infrastructureof a socialnetworking site provides this memory1.

Moreover, it may provide sophisticatedindexing techniques(usingtapestries[7]) to enablethe invitee to

retrieve the relevant portionsof chasedata. With a singleSOLIS�

andmany inviteeswith different ����� �values,RAYS maintainsdatapertainingto the maximum ����� � value. All viewerssharethis copy of data

andthesoftwareenforcesthevalueof ����� � specified(by theownerof thedevice) for Invitee � . This means

inviteesof a SOLISmayjoin an in progressSOLIS latewithout requiringadditionalmemory. However, a

new SOLISrequest� requiresmemoryequalto themaximum ��� � specifiedby theownerof Device � for

invitee .A key deployment questionis the configurationof the server and its memorysize. One may over

provision for the maximumnumberof SOLISsandtheir � values. This is a reasonableapproachgiven

today’s inexpensive priceof memory. However, if eithertheanticipatednumberof SOLISsis unknown or

thesizeof thedevice formatteddatachangesdynamically, thena deploymentmayrun out of memoryand

evict chasedata.Every timeaninviteereferencesthisevicteddata,thesystemmayeitherignoretherequest

and/orreportthedatais missing,reducingthequalityof service.

Oneapproachto this limitation is to requirethesystemadministratorto detectmemoryoverflows and

extend the systemwith additionalmemory. This detective approachhasits own limitations. First, from

the first time an overflow is detecteduntil additionalmemoryis obtained,SOLIS requestsmay overflow

memoryrepeatedly. Second,configuringaserver with additionalmemorymayrequiresomedown time.1An alternative is to buffer this dataon the client device. A limitation of this approachis that the client device may have

insufficient physicalmemoryto buffer � time unitsof data.

2

An alternative approachis to employ aDataAwareAdmissionControl,DA-AdmCtrl, technique.When

a new SOLISrequestarrivesto thesystemandthereis insufficient memoryto supportits chasedata,DA-

AdmCtrl startsto summarizepastand incoming datapro-actively in order to admit the new request. A

SummarizationTechnique, ST, mayeitherreducetheresolution(quality)of datain orderto reduceits mem-

ory requirementor convert device produceddatainto a formatthatcanbeusedby anotherST to reduceits

size,seediscussionsof SectionE. DA-AdmCtrl admitsa new SOLISrequestaslong asthereis sufficient

memoryto accomodateits chasedata.Otherwise,it rejectstheSOLISrequest.Thesystemmayallow this

requestto proceedasa livestreamwith nochasedisplay.

With multi-coreCPUs,DA-AdmCtrl employs severalST processesto summarizedifferentdatachunks

simultaneouslywhereeachdatachunkcorrespondsto a portion of chasedisplayof oneor moreSOLISs.

Theremay exist dependenciesbetweentheSTssuchthat thesummaryformat producedby oneST is the

input to anotherST. For example,Figure3.ashows oneinter-dependency namedDeep-Desc.It consistsof

four differentdataformats: ��� producedby adevicewith averagesizeof 1000KB perunit of time,andthree

summarizedformats ��� , ��� , and ��� with their sizesshown in eachnode(numberon an edgedenotesthe

averageservicetime of ST to processits input dataformatto produceits outputdataformat).DA-AdmCtrl

is flexible enoughto supportarbitrarily complex inter-dependenciesbetweenSTsaslong asthey aretrees.

Theprimarycontribution of this paperis a dataawareadmissioncontrol techniquethatadmitsSOLIS

requestswith theobjective to maximizeutility of datawith nomemoryoverflow. Utility quantifiestheuser’s

level of satisfactionwith adataformat(producedby eitherthedeviceor anST).Whentheutility for all data

formatsis identical(chaseviewer doesnot discriminatebetweenthedifferentdataformats),theadmission

controlselectsthosedataformat(s)thatpreventmemoryoverflow. In additionto consideringtheavailable

memoryat oneinstancein time, the admissioncontrol ensuresthe summarizationratefor the chasedata

is eitherequalto or higherthantherateof dataarrival by thecurrentlyactive SOLISs,preventingmemory

over-flow. Thisanalysisconsidersthenumberof cores,theservicetimeof STs,andtheirinter-dependencies.

Thefollowing exampleillustratestheseconcepts.

Example 1: ConsiderthreeSOLISs,eachwith ����� time units. EachSOLIS produces1000KB

datain format ��� in eachtime unit, ����� =1000KB (seeTable1). The systemis configuredwith oneST

thatsummarizesthe incominghigh resolutiondatato a low resolutionformat � � that is ten timessmaller,���"! =100 KB. Assumethe summarizationprocessrequires2 time units. At first glance,the amountof

requiredmemoryfor these3 SOLISsmight appearto be 1500KB, #%$&�'$(���"! . However, asshown in

Figure2, assuminga systemwith morethansix cores,the total amountof requiredmemoryduringsteady

stateis 7200KB. At any instanceof time,say7th timeunit, theoccupiedmemorymustaccommodatethree

differentSOLISsproducing�)� data(3 $ 1000KB), �)� dataproducedat time 6 that is beingsummarized

actively (3 $ 1000KB), ��� datathat is beingproducedactively by thesummarizationtechnique(3 $ 100

KB), and � � datapertainingto summarydatacorrespondingto Time 5, 4, and3 (3 $ 3 $ 100 KB). The

3

0 1 2 3 4 5 6 7 80

2000

4000

6000

8000Memory(KB)

Time

7200690066006300

Figure2: Memoryusagefor threeSOLISsof Example1.

latermustbemaintainedin memorybecause� =5.

The numbersin this simple examplescale. The amountof requiredmemorywith 3000 SOLISsis

approximately7 GB assumingasystemwith 6000cores.If thesystemis configuredwith lessmemorythan

what is required,thentheadmissioncontrolmight beforcedto rejectsomeof thearriving SOLISrequests

eventhoughthesystemis configuredwith many coresto summarizedata.At least6000coresarerequired

becausethe time to summarize��� formatteddatais twice the inter-arrival time for data. Fewer than6000

coresrequiresmorememorythan7 GB, seediscussionsof Equation1. *Therestof this paperis organizedasfollows. SectionB presentsrelatedwork. Detailsof DA-AdmCtrl

areprovidedin SectionC. We quantifythetradeoffs associatedwith DA-AdmCtrl in SectionD. SectionE

describesflexibility of DA-AdmCtrl andits usein two differentcontexts. SectionF offersbrief conclusions

andfutureresearchdirections.

B Related Work

Admissioncontrol techniquesprevent a newly arrived requestfrom disruptingcurrently active requests

and their observed quality of service[11, 25]. They do so by monitoringeither the amountof available

memory, disk [26, 18, 28, 1] and network [2, 8] bandwidth,or a combinationof theseresources.For

example,admissioncontrol techniquesdevelopedfor video-on-demandserversconsiderdisk bandwidthin

combinationwith theavailablememoryto controlhow many simultaneousrequestsmaystreamdatafrom

thediskwithoutoverflowing memory[13, 10, 22, 9, 15, 20]. Similarly, multi-layerencodingtechniquesare

employedto controlthebandwidthrequirementsof astreamin thepresenceof burstybackgroundloadthat

exhauststheavailablenetwork bandwidth[21, 24, 27]. Thesestudiesstrive to deliver continuousmediain

a timely mannerto prevent a client’s displayfrom starvingfor data. Our proposedtechniqueis different

becauseit focuseson live streamingdataand how to summarizethis dataproactively in order to admit

requestswithout overflowing memory. Note thatoncedatais lost (dueto memoryoverflows), theoriginal

cannotbere-constructed.Similarly, oncea lossysummarizationtechniquereducestheresolutionof datato

reducethememoryrequirementof aSOLIS,theoriginaldatais lostpermanently.

4

Severalstudiesfocusedonrecordingof continuousmediaproducedby camerasusedby news organiza-

tions[19, 3, 14]. Theirproposedtechniqueswrite datato magneticdiskdriveswithoutoverflowing memory

of a server asa temporarystagingarea. Thesestudieshighlight the limited bandwidthof magneticdisk

drives,restrictingthenumberof admittedrequestsseverely. While thebandwidthof diskscontinuesto be

a limiting factor today, the costof memoryhasdroppedsignificantly to deliver inexpensive serverswith

tensof Gigabytesof memory2. Thesetrendsaretheprimarymotivationfor ourproposedadmissioncontrol

technique.By usingmemoryonly, wecansupportamuchlargernumberof concurrentrequeststhan[19, 3].

Ouradmissioncontroltechniqueis novel for two reasons.First, it quantifiesmemoryrequirementsof alter-

native summarizationtechniqueswith multi-coreCPUs.Second,it is flexible enoughto considerarbitrarily

complex inter-dependenciesbetweendifferentsummarizationtechniquesaslong asthey aretrees.

We introducedtheconceptof SOLIS, its userinterface,anda buffer replacementtechniquethat sum-

marizes(insteadof evicting) datawhenmemorybecomesfully occupiedin [6]. Thereplacementtechnique

cannotcontrolthenumberof SOLISsand,oncethenumberof SOLISsexceedsathreshold,is forcedto evict

a significantamountof chasedatafrom memory. The proposedadmissioncontrol techniqueaddressthis

limitation in two ways.First, it summarizesdatapro-actively. Second,it doesnotadmita SOLISrequestif

it resultsin memoryoverflow.

C Admission Control

Admissioncontrol is invoked every time a new SOLISrequestarrivesto thesystem.It canbeconfigured

to eitherpreventmemoryoverflow or maximizeutility of datawhile preventingmemoryoverflow. Memory

overflow occurswhendatapertainingto chasedisplayportionof activeSOLISsexceedstheavailablemem-

ory, forcing thesystemto dropchasedatathatmight be referencedby a viewer. It is undesirablebecause

whenever theviewer referencesthis data,theviewer’s displaybecomesblank asthe referenceddatais no

longeravailable.Datautility refersto theuser’s satisfactionwith thequality of displayeddata.An ST may

reducethesizeof incomingdataby producinga lower resolution(quality) format. This dataformatmight

beperceivedasundesirableby auserwhomaystopviewing it. Theuser’ssatisfactionis applicationspecific

andweassumeasystemadministratorquantifiesit by assigningautility to eachdataformat.A higherutility

valuereflectsabetterqualityof service(amoresatisfieduser).Weassumethedeviceproduceddatahasthe

highestutility.

Inputsto DA-AdmCtrl includethe amountof availablememory, the alternative STssupportedby the

serverandtheir inter-dependencies, andthecharacteristicsof eachSTsuchasits compressionfactor, antici-

patedservicetime,andutility of its outputdata.Given � dataformats,eachformatis representedasanode2At the time of this writing, onemay purchasea ZT DesktopPCwith 16 GB of memoryanda 2 TB disk drive for lessthan

$900.

5

andanedgerepresentsanSTthatconsumesonedataformatto produceanother. Thus,aninter-dependency

is a directedacyclic graphconsistingof nodesandedgesannotatedwith meta-data.Figure3 shows four

inter-dependenciesdescribingtheaveragesizeof adataformat(numberin thenode)andservicetime of an

ST (valueassignedto anedge).While the inter-dependenciesof Figure3 aresimple,our proposedadmis-

sioncontrolsupportsarbitrarily complex inter-dependenciesaslong asthey aretrees,i.e., directedacyclic

graphswith oneroot, �)� .DA-AdmCtrl musteitheradmitor rejectSOLISrequestssuchthatit meetsits configuredobjectiveusing

its availableresources.Theserver startsbuffering thechasedatafor a SOLISassoonasit is admittedby

theDA-AdmCtrl. Thechaseportionof a rejectedSOLIS is not bufferedandtherequestmayproceedasa

livestream.Thedecisionto admit requestsis trivial whentheavailablememoryexceedsthechasedisplay

requirementsof theactive SOLISsandtheir device produceddata.Theproblembecomesinterestingonce

the numberof active SOLISsis suchthat a newly arrived SOLIS requestexhauststhe availablememory.

Now, DA-AdmCtrl mustdecidewhich STsto employ pro-actively to accommodatethe incomingSOLIS.

In essence,DA-AdmCtrl is solvinga two dimensionalbin packingproblem.Ononedimension,it strivesto

meetthememoryrequirementsof theactiveSOLISsandSTsusedto compactthem.Onaseconddimension,

it solveshow to scheduleSTsusingtheavailableCPUcores.

Wesolvetheproblemusingatwostepheuristic.Thefirst stepidentifieshow many SOLISsaresupported

givena combinationof ( + ) dataformats. Eachcombinationresultsin mixesof datafor the identifieddata

formats.A mix definesthenumberof SOLISsin eachformatwithin the + possibleformats,+-,.� . Each

mix supportsa fixed numberof SOLISsand hasa utility. Different mixes are storedin a sortedarray,

identifying themaximumnumberof SOLISs(/ ) supportedby systemresources(memoryandcores).

In thesecondstep,DA-AdmCtrl usesthis arrayto decidewhetherto admitor rejecta new request.At

any instancein time,DA-AdmCtrl is awareof thenumberof activeSOLISs.If themaximumnumber(/ ) is

reachedandanew SOLISrequestarrives,DA-AdmCtrl rejectsthisnew requestto preventmemoryoverflow

for theexisting SOLISs.Otherwise,it computesthenew numberof SOLISsin thesystemandlooksup the

tableto identify themix thatsupportsthisnew number. It proceedsto scheduleCPUresourcesto realizethe

identifiedmix. Similarly, whenaSOLISleavesthesystem,DA-AdmCtrl decrementsthenumberof SOLISs

in thesystemandlooksupthetableto identify themostsuitablemix. It maystopscheduledsummarizations

thatareeithernotnecessaryor reducetheutility of dataunnecessarily.

Below, weelaborateon thesetwo stepsin turn.

C.1 Step 1: Mix of STs

Thefirst stepof theheuristiccomputesdifferentmixesof + dataformats.Eachmix supportsafixednumber

of SOLISsand may offer a different utility. To simplify discussionand without loss of generality, we

assumeeachSOLIShasa fixedchasedisplayduration � . SectionC.1.1extendsthis discussionto SOLISs

6

Figure3: Exampleinter-dependencies:(a) Deep-Desc,(b)Deep-Asc,(c)Shallow-Desc,(d) Shallow-Asc.Numbersin eachnodedenotethesizeof adataformatperunit of time. Numbersnext to anedgedenotetheservicetime of a summarizationtechnique,chosento highlight the tradeoffs associatedwith DA-AdmCtrl(seediscussionsof Figure6).

with varying � values.

Thevalueof + dictatesthenumberof dataformatsthatDA-AdmCtrl mayemploy ataninstancein time.

It is an integergreaterthanor equalto zeroandlessthanor equalto � . If +-�10 thenDA-AdmCtrl must

admit requestsbasedon device produceddataformatandmaynot useanST. If +.�32 thenDA-AdmCtrl

mayadmitrequestsby usingoneof the � dataformats.If +4�65 thenDA-AdmCtrl mayadmitrequestsby

usinga combinationof eitheroneor two (out of the � data)formats.And when +4�7� , DA-AdmCtrl may

utilize all availabledataformatssimultaneously.

Thenumberof STsusedtogenerateacombinationof + dataformatsisdependentontheinter-dependency

Term Definition8Chasedisplayduration.9Numberof cores,i.e.,simultaneoussummarizations.9;:=<Numberof coresto summarizedataformat >@? .ATotalavailablememory.BNumberof dataformats.BDCDeviceproduceddataformat.B ? An availabledataformat E .

ST A summarizationtechnique.F G <Sizeof dataformat

B ? in onetime unit.H G <Timerequiredto generate

B ? , E�I(J .K Numberof dataformatsusedsimultaneously, LNM K M B .OMaximumnumberof admissibleSOLISs.

Table1: List of termsandtheir definitions.

7

betweenthe dataformats. To illustrate, considerthe Deep-Descinter-dependency of Figure 3.a. When+P�Q2 , DA-AdmCtrl may utilize any oneof thesedataformatsto admit requests.To producethe most

compactrepresentation��� with averagesize of 1 KB, DA-AdmCtrl must utilize 3 STs andapply them

in succession.With the Shallow-Descinter-dependency (seeFigure3.c), DA-AdmCtrl may use1 ST to

producethemostcompactrepresentation��� .A combinationis a setconsistingof + dataformats: RS� , RT� , ..., RVU , RW�YX[Z\�)� , � � , ..., � �^] , RW�`_�6R@� . With+a�b0 , the setonly consistsof ��� andDA-AdmCtrl mustuse ��� . With +dcd0 , the numberof possible

combinationsis e U �gf��)h � �\i . Largervaluesof + increasethenumberof combinations.

When +6�j0 , thenumberof admissibleSOLISsis klnm �po�q where r is the sizeof availablememory,����� is the size of device produceddatain one time unit, and � is the durationof chasedisplay in the

sametime unit granularity. When +s�a2 , thereare � possiblecombinations,eachsetwith oneuniquedata

format RW� . Eachcombinationhasonemix consistingof one t value,thenumberof SOLISssupportedby

its correspondingdataformat. Theexact valueof t dependson theavailablememory, thesummarization

servicetime uwvyx , andthenumberof cores,seeEquation1.

When +zc{2 , eachcombinationsupportsa set of admissiblerequests: ZVt"� , tV� , ..., t@UD] where t\� is

thenumberof SOLISssupportedby its correspondingdataformat. Eachsetis nameda mix andsupportse U �gf�� tp� admissibleSOLISs. For example,with one of the inter-dependencies of Figure 3 and +b�{5 ,a combinationmight be � � and ��� . Assuminga systemwith 10,000KB of memoryconfiguredwith the

granularityof timesetto 1 secondand � =1second,amix of thiscombinationmightbe Z 91,90] , supporting

a total of 181SOLISs.This mix requiresDA-AdmCtrl to commitapproximately90%of thememoryto be

occupiedby datain format � � while theremaining10%is occupiedby datain format ��� . Thetwo extreme

mixes for this combinationare (1) Z 100,0] : 100 SOLISsin format � � and0 SOLISsin � � , and(2) Z 0,

1000] : 0 SOLISsin format ��� and1000SOLISsin ��� . Thus,themixesproducedfor one + value(say � )areasupersetof thosewith a lower + value(lessthan � ). In thisexample,themixesproducedfor + =2 area

supersetof thoseproducedwith + =1.

A mix maynotbefeasiblegiventheavailablememory, theservicetimeof differentSTs,andthenumber

of CPU cores. An ST allocatesmemoryfor its produceddataprior to startingits summarization.Thus,

invocationof anSTincreasestherequiredmemoryduringits servicetime,seeExample1 in SectionA. The

averageincreaseby anSTis theaverageoutputsizeof thatST. To quantifytheamountof requiredmemory,

webreaktime into fix sizedintervals,say100milliseconds.Moreover, werepresent� andservicetime uwv|x

8

of eachST in theseunits.Theamountof requiredmemoryis:

r&�~}|�� ����$��������w�U��gf��

q^�n�W� x��� ��� v|xuwvyx $���� ��x �w�� q^� ��S� �b� U�

�gf��q^�n� � x�"� � vyxuDv|x ��$���� � (1)

This equationconsistsof 3 termsandwe explain themin turn. During eachtime unit, a device producesa

fixedamountof data,����� . With � SOLISs,theamountof requiredmemoryis ��$������ . During onetime

unit, thenumberof in-progresssummarizationfor format RW� is estimatedto be �b�6��� � x�W��x�� where � vyx is the

numberof employedcores(independentsummarizationprocesses)to producesummaryformat RW� . For the

remaining�P��uDv|x time units, thesummarizeddatamustbemaintainedin memory. If � c�� then � is

resetto � . Thispreventsthelasttermfrom producingnegative values.

A mix is valid aslong as(1) its requiredmemoryis lessthanor equalto theavailablememory, r(�~}yN,r , and(2) its requirednumberof coresis lessthanor equalto theavailablecores,e U �gf�� � vyx^, � . Step1

discardsinvalid mixes.

Givena combination,we enumerateits mixesstartingwith the leastcompactrepresentationamongall

theformatswithin a combination.This in combinationwith thesizeof memorylimits thechoiceof values

for thecompactformats,reducingthenumberof enumeratedmixes. Theenumerationprocessterminates

eitheroncea mix requiresmemoryin excessof r or all possiblemixes have beenenumerated.As an

example,recall theearlierexamplewith a combinationconsistingof ��� and � � with a 10,000KB memory

size,we enumeratethe possiblecombinationsstartingwith the highestnumberof SOLISssupportedby� � . This would be Z 100,0] . Next, we decreasethenumberof SOLISsfor ��� andcomputehow many are

supportedby � � . This producesZ 99,10] anddiscardsvalues1 to 9 for � � . Thisprocessrepeatsuntil either

themix Z 0,1000] is encounteredor anenumeratedmix hasamemoryrequirementin excessof 10,000KB.

Oncethe memoryrequirementof a mix, say Z 99,10] , exceeds10,000KB thenthe availablememoryin

combinationwith CPUresourcescannotsupportmorethan10 summarizations.Hence,thereis no point in

considering20 summarizations.Theseearlypruningof mixesexpeditesStep1, seediscussionsof Figure4

in thefollowing paragraphs.

Valid mixesareinsertedaskey-valuepairsin a hashtable. Thehashtablemaintainsthemix with the

highestutility for � SOLISs.Thehashtablekey is theuniquevalue � that is a candidatevaluefor � , the

maximumnumberof SOLISs.Its valueis adatastructurecontainingarepresentationof amix andits utility

rating(if any). Beforeeachkey-valueinsertfor a key � , Step1 performsa hashlookupfor � . If no entry

existsthenin proceedsto insertthenew key-valuepair in thehashtable.Otherwise,thenew mix overwrites

9

8 16 32 64 128 18510

2

104

106

108

1010

W=1

W=4

States

Cores

W=2W=3

0 2 4 8 16 32 64 128 1850

500

1000

1500

2000

W=2, 3, 4

W=1

Nmax

Cores

4.a)Numberof visitedstates. 4.b)MaximumadmissibleSOLISs.

Figure4: Impactof + on numberof visitedstatesand � with Shallow-Asc and � =2 minutes.

theexistingoneonly if its utility is higher.

Thespacecomplexity of this algorithmis O(� ) dueto its useof a simplehashtableto storekey-value

pairsthatmaintaindifferentmixesfor eachuniquevalueof � . DA-AdmCtrl usesthis tableto eitheradmit

or rejectSOLISrequests,seediscussionsof Step2 in SectionC.2.

An upperboundon the complexity of this stepis O(� U $4� U���� ). This is an upperboundbecause

it ignoresthe inter-dependency betweenthe differentdataformats,the numberof cores,andour pruning

heuristic.Figure4.ashows thenumberof visitedstateswith different + valueswhenpruningheuristicsare

appliedasa function of thenumberof CPU cores(1 GB of memoryandthe Deep-Ascinter-dependency

graph). The y-axis denotesthe numberof visited statesand is log-scale. The numberof visited states

increasesasa function of the numberof coresbecauseadditionalcoressupporta larger mix of requests.

185coresarerequiredto summarizedatato thelowestformatpossibleto realizethemaximumnumberof

admittedSOLISs,1800. With anexhaustive search,this stepwould have visited14 thousand,76 million,

182billion, 163 trillion stateswith + =1, 2, 3, and4, respectively. Our techniquevisits significantlyfewer

states.

Figure4.b shows the maximumnumberof admissibleSOLISswith differentnumberof coresand +values. While increasingthe valueof + from 1 to 2 enhancesthe numberof admittedSOLISs,valuesof+ beyond2 providedno improvements.This meansincurringthecomplexity overheadof + setto 3 and4

is not beneficialwhentheobjective is to maximize � . Notethatwhenthenumberof SOLISsis below � ,

with + valueshigherthan2, themixesin thehashtablemayconsistof morethan2 formats,seediscussions

of Figure5 in SectionD.

C.1.1 Varying � values

A new SOLISrequestmay specifythe valueof its � uponits arrival andthis valuemay not be available

in advance. Moreover, differentSOLIS requestsmay usea different � value. Typically, the systemhas

sufficient memoryto accommodatechasedataof a few SOLISsin their device produceddataformat, ��� .This enablesDA-AdmCtrl to admit the first few arriving SOLISswithout computingthe mix of format

10

combinations.As freememoryfallsbelow acertainthreshold(say10%of r ) thenDA-AdmCtrl computes

theaverage� value( � ) usedby theactiveSOLISsto computethetableof mixes.Thisestablishesboththe

tableandits assumed� , i.e., � � . As SOLISsarrive (leave) thesystem,� maybecomedifferentthan � � .� valuessmallerthan � � areundesirablebecausethey force DA-AdmCtrl to useSTsmoreaggressively

thannecessary, reducingutility of data. � valuesgreaterthan � � arelessdesirablebecausethey mislead

DA-AdmCtrl to utilize STsin amannerthatmayexhaustavailablememory.

To preventdataloss,everytime � exceeds� � , were-computethetable.To avoid frequentcomputation

of thetable,we usean inflation valueto establishthenew valueof � � , � � = ��$ � , usedto computethe

table. The valueof � may inflate � � by 10%, � =1.1. This means� must increaseby 10% in order to

triggercomputationof anew table.

To preventaggressive useof STsandtheir impactondatautility, were-computethetableevery timethe

valueof � differs from � � significantly. For example,we re-computethe tablewhen q �q is morethana

factorof two.

With theinter-arrival timeof SOLISsanticipatedto bein theorderof minutes(andhours),� shouldnot

changethatoften.Thus,there-computationof tableof mixesis notexpectedto beeitherfrequentor taxing

of CPUresources.

C.2 Step 2: Combination Lookup

The hashtable of Step1 is usedby DA-AdmCtrl to either admit or reject SOLIS requestsas follows.

DA-AdmCtrl maintainsthenumberof active SOLISrequests,� . Whena new SOLISrequestarrives,DA-

AdmCtrl increments� by 1 and usesthis value to probethe hashtable for a mix (which provides the

highestutility). If the hashtable producesno output then the new requestis rejectedbecause� is the

maximumnumberof supportedSOLISs, � = � , and � is restoredto its original value. Otherwise,DA-

AdmCtrl proceedsto admit the requestand realizesthe new mix by schedulingthe latestarriving data

chunksfor summarizationfirst. Thesechunksresidein the memoryfor the longestamountof time and

Equation1 assumesthey aresummarizedfirst.

D Evaluation

We useda simulationstudyto characterizethebehavior of DA-AdmCtrl with differentamountof memory,

CPU cores,summarizationtechniquesand their inter-dependencies. This sectionreportson oneexperi-

mentalsettingandour obtainedresults. Table2 shows the assumeddataandsystemsettings. We focus

on inter-dependencies of Figure3 due to their simplicity. The numbersshown in eachnodeof an inter-

dependency areat the granularityof blockswherethe sizeof a block is 250 KB. The first threereported

experimentsassumethe utility of datais the samefor all dataformats,characterizingDA-AdmCtrl and

11

DataSOLISBit Rate 2 MbpsChasedisplayduration,� 2 minutes

SystemCacheMemorySize, r 1 GBNumberof Dataformats,� 4Granularityof timeunit 1 second

Table2: Simulationparametersfor theexperiments.

� (#of Cores) Simple DA-AdmCtrl32 33 573256 256 18641024 1024 22592048 2048 27873574 3574 3574

Table 3: Number of admittedSOLISs with Simple and DA-AdmCtrl ( + =2) using Shallow-Asc inter-dependency of Figure3.d, � =2 minutes.

its behavior with differentsettings.The lastexperimentreportson utility of a SOLISastheDA-AdmCtrl

admitsa largernumberof SOLISrequests.

Thefirst experimentcomparesour admissioncontrolwith analternative thatutilizesthemostcompact

datarepresentationalways,namedSimple. It assumesthe utility of all dataformatsis identical. Simple

providesthesame3 performanceasDA-AdmCtrl with theShallow-Descinter-dependency, seeFigure3.c.

With otherinter-dependencies, Simpleadmitsfewer SOLISsbecauseit is too time consumingto produce

themostcompactrepresentationandthesystemmayhave insufficient numberof coresto freememoryat a

fastenoughrateto meettheproductionrateof databy theactive SOLISs.This is shown in Table3 with the

Shallow-Asc inter-dependency asa functionof thenumberof cores,i.e., numberof simultaneoussumma-

rizations. With 32 cores,Simpleadmitslessthan10%of SOLISssupportedby DA-AdmCtrl becausethe

time to summarize� � datato Format � � is 1000msec,seeFigure3.d. DA-AdmCtrl utilizesa combination

of formats� � and � � with servicetime of 1 and100msec,respectively. This tenfold differencein summa-

rization time enablesAdmCtrl to admitmorethantentimesasmany SOLISs.As we increasethenumber

of cores,thegapbetweenSimpleandDA-AdmCtrl decreasesbecausemany moresummarizationprocesses

may proceedsimultaneously. With 3574cores,both techniquesadmit the samenumberof SOLISs. No

additionalSOLISsmaybeadmittedbecausethememory(1 GB, seeTable2) is fully occupiedwith datain

format ��� .3Simpleis inferior whentheobjective is to maximizeutility of dataandthemostcompactrepresentationprovidesthe lowest

utility.

12

200 400 600 800 1000 1200 1400 1600 18000

2000

4000

6000

8000

10000

12000

W=1

Avg Memory Utility Per SOLIS

W=2, 3, 4

Nmax

Figure5: Averageutility of dataoccupying memoryperadmittedSOLIS,Deep-Asc,� =2 minutes.

When memoryis not fully occupied,the numberof coresplays an importantrole in the numberof

admittedSOLISs.Typically, thenumberof admittedSOLISsincreasesmonotonicallyasa functionof the

numberof cores.This is thecasewith Simplewhich exhibits a linear relationshipbecauseit summarizes�)� to ��� always,incurringa 1000msecservicetime for eachsummarization.However, this functionis not

necessarilylinearwith DA-AdmCtrl. Its characteristicdependsontheservicetimeof thedifferentSTsused

by a mix pickedby AD-AdmCtrl. For example,in Table3, we increasethenumberof coresby a factorof

eight from 32 to 256coreswhile thenumberof SOLISswith AdmCtrl increasesby only a factorof three.

This is becauseDA-AdmCtrl employs a combinationof � � and ��� with 32 coresanda combination4 of � �and ��� with 256cores.Theaverageservicetimeof STsdonotexhibit alinearrelationshipwith thesemixes.

Whenboth the memoryand the numberof coresare limited, increasingthe value of + from 1 to 2

enhancesthe numberof SOLISsadmittedby DA-AdmCtrl, seeFigure4.b. When +1��2 , the admission

controltechniqueis forcedto useonly oneformat. With a handfulof cores,it selects� � which reducesthe

sizeof deviceproduceddataby a factorof 10. With tensof cores,it selects� � andappliestwo STslinearly

to producethisdataformat,admittinga largernumberof SOLISs.

With +s�65 anda handfulof cores,DA-AdmCtrl employs a combinationof ��� and � � . As thenumber

of coresincreasesfrom 2 to 8, DA-AdmCtrl usestheadditionalprocessingcapabilityto summarizemore

data into format � � , admitting more SOLISs. With 32 cores, +z��5 supportsapproximatelytwice as

many SOLISswhencomparedwith +��{2 . With additionalcores,the gapbetween+P�{2 and +��z5closesbecausethecoresareno longerthelimiting factor. With both + values,DA-AdmCtrl employs � � to

maximizethenumberof admittedSOLISs.

Figure5 shows theaverageutility of dataoccupying memoryper admittedSOLIS aswe increasethe

numberof SOLISs,i.e., systemload. When + is setto 1, with Deep-Asc,DA-AdmCtrl is forcedto usethe

mostcompactrepresentation,droppingtheutility of datadramatically. With + setto 2, DA-AdmCtrl usesa

mix of two dataformatsto realizeadropin datautility asa functionof thenumberof admittedSOLISs.

In thethird experiment,weanalyzedtheinter-dependency of summarizationtechniquesandtheir impact4With 32 cores,DA-AdmCtrl allocates45%of memoryto � ! and55%to � � . With 256cores,DA-AdmCtrl allocates77%to� � andtheremaining23%to ��¡ .

13

0

1000

2000

3000

4000N (Max Num SOLISs)

Shallow−Desc

1 2 4 8 32 128 256 3575Number of Cores

Deep−Asc

Deep−DescShallow−Asc

Figure6: Numberof admissibleSOLISs( � ) asa functionof thenumberof coresfor all four techniques

0 500 1000 1500 2000 2500 3000 35000

20,000

40,000

60,000

80,000

100,000

120,000

Number of Admitted SOLISs

Avg Memory Utility Per SOLIS

Figure7: Utility of dataasa functionof systemload,admittedSOLISs.Shallow-Asc, � =2 minutes,+ =2.

on DA-AdmCtrl. Figure 6 shows the numberof admissibleSOLISsby the four inter-dependencies of

Figure3 with differentnumberof cores.Shallow-Descsupportsthehighestnumberof SOLISsandrealizes

this with thefewestnumberof cores,� =4. This is dueto thefastsummarizationtime (1 msec)to produce

themostcompactrepresentation,��� . With all otherinter-dependencies, thetimeto producethisdataformat

is either1000 msecor higher. Deepinter-dependenciessupportthe fewestnumberof admittedSOLISs

becausethey preventDA-AdmCtrl from freeingmemoryfastenoughto meettherateof dataproductionby

SOLISs.

Our final experimentanalyzedtheutility of dataasa functionof thenumberof SOLISsadmittedin the

system.In thisexperiment,STsarevery fastandtheirprocessingtime is nota limiting factor. Wereporton

theresultswith theShallow-Asc inter-dependency graphwith utility of datasetto 1000,100,10,and1 for

Formats��� , ��� , � � , and ��� , respectively. Thus,theuserperceivesdevice produceddatato bea thousand

timesbetterthanthe mostcompactrepresentation��� . We report the averagememoryutility per SOLIS,

computedby dividing thetotal utility of datain memoryby thenumberof active SOLISs.Figure7 shows

this metricasa functionof thenumberof SOLISrequestsissuedto thesystem.With a few ( , 33)SOLISs,

DA-AdmCtrl realizesthe highestutility per SOLISby maintainingchasedataproducedby all SOLISsin

their device produceddataformat. With more than33 SOLISs,DA-AdmCtrl employs a combinationof�)� with one of � � or ��� (either Z\���p¢£� �T] or Z\�)�\¢£���W] ) becauseutility of �)� is ten times higher. With

308 SOLISs,DA-AdmCtrl hasinsufficient memoryto useformat �)� andswitchesto usea combination

consistingof � � with oneof theothertwo dataformats, � � and ��� . It abandonsuseof � � with morethan

1827SOLISs,usinga combinationof � � and ��� . Finally, with 3574SOLISs,it switchesto use ��� only.

14

Beyond3574SOLISs,thereis insufficient memorywith ��� andDA-AdmCtrl rejectsadditionalrequests.

The smoothdeclinein datautility shown in Figure 7 is becauseDA-AdmCtrl picks mixes with the

objective to maximizeutility of datawhile preventingmemoryoverflow.

E Discussion

Inter-dependenciesof Figure3 highlight theflexibility of DA-AdmCtrl to supportarbitrarily complex sum-

marizationDAGs(directedacyclic graphs).We now presenttwo concreteexampleusesof thedeepinter-

dependenciesshown in Figures3.aand3.b. While the first is implicit anddueto arrival of a new SOLIS

request,thesecondis dueto formatmismatchbetweendifferentsoftwarepackages.We describethesein

turn:

1. Considerascenarioin whichtheavailablememorysupports5 SOLISsin theirdeviceproducedformat

(F1), 6 in formatF2 and7 in formatF3. With 5 active SOLISs,DA-AdmCtrl maintainsall in their

original device producedformat(F1). Now whenthe6th SOLISrequestarrives,DA-AdmCtrl must

summarizeall chasedataof theSOLISsto F2. Whenthe7th SOLISrequestarrives,onceagainDA-

AdmCtrl is invoked andit startsto summarizedatainto F3. Soapartfrom theincomingchunks,the

previously summarizedchunksin F2 aresummarizedto F3. In essence,this is traversingthetreeof

Figure3.a/bon-demand(whena new SOLIS requestarrives) to summarizechasedatainto a more

compactrepresentation.

2. XugglerAPI providestoolsfor manipulatingmediastreams.XugglersupportsAdobe’sRTMPstreams

as an input whereassomenetwork camerascreatean RTSPstream. In order to manipulatevideo

streamsusingXuggler, the RTSPstreamsmustbe convertedto RTMP streams,i.e., usingFFMpeg

andthe RED5 mediaserver. This conversioncanbe consideredasan intermediatesummarization

stepconvertingF1 (theRTSPstreamgeneratedby thedevice) to F2 (RTMP stream)which canthen

be usedby Xuggler to createother lower resolutionformats. With FFMpeg, onemay specify the

numberof framesandtheir resolutionin orderto ensurethe RTMP formattedoutput is not greater

thantheoriginal RTSPformat.

F Conclusions and Future Research Directions

DA-AdmCtrl managesthe format of chasedataoccupying main memorywith the objective to maximize

thenumberof admissibleSOLISswhile preventingoverflow of memory. Whentheuser’s satisfactionwith

differentdataformats(utility of data)is available,DA-AdmCtrl selectsdataformatswith the objective to

maximizedatautility. It considersarbitraryinter-dependencies betweendifferentdataformatsaslong as

15

they aretrees.In addition,it considersthenumberof coresandschedulessummarizationsin a mannerthat

preventsmemoryoverflow. Ourevaluationsectionshowsdifferentinter-dependencies haveanimpactonthe

numberof admissibleSOLISs.In addition,it highlightstheimportanceof theservicetimeto produceadata

format.Useof themostcompactrepresentationmaynotbefeasibleif theservicetimeof thesummarization

technique(s)to produceit is toohigh.

Currently, we areundertakingan implementationof DA-AdmCtrl in RAYS. This implementationen-

visionsa modularandconfigurablecomponentthat canbe pluggedinto differentsystemsthat requirean

admissioncontrol techniqueto managetheir memory. It must considerdifferent policies that might be

definedby anapplication.For example,consideranapplicationthatassignsa negative costto both refer-

encesfor missingchasedataandSOLISrequestrejection.DA-AdmCtrl mustintelligently reject(admitby

droppingexisting) SOLISsto maximizetheoverall utility observedby theapplication.

References

[1] D. Ahmed, N. Chowdhury, and M. Akbar. Admissioncontrol algorithm for multimediaserver: a

hybrid approach. InternationalJournal of Computers and Applications, 29(4):414–419,September

2007.

[2] N. Alon, Y. Azar, andS.Gutner. Admissioncontrol to minimizerejectionsandonlinesetcover with

repetitions.ACM Trans.Algorithms, 6(1):11:1–11:13,December2009.

[3] W. G. Aref, I. Kamel, andS. Ghandeharizadeh.Disk Schedulingin Video Editing Systems. IEEE

Transactionson Knowledge andDataEngineering, 2000.

[4] Bambuser. LiveFromYourMobile, http://bambuser.com.

[5] S. BarahmandandS. Ghandeharizadeh.RAYS: RecallAll You See,GraceHopperCelebrationof

Womenin Computing,Oregon2011.

[6] S.BarahmandandS.Ghandeharizadehd.ChaseDisplayof SocialLiveStreams.In Proceedingsof the

3rd ACM SIGMMInternationalWorkshoponSocialMedia, November2011.

[7] ConnellyBarnes,DanB Goldman,Eli Shechtman,andAdamFinkelstein.Videotapestrieswith con-

tinuoustemporalzoom.ACM TransactionsonGraphics(Proc.SIGGRAPH), 29(3),August2010.

[8] A. Blum, A. Kalai, andJ.Kleinberg. Admissioncontrol to minimizerejections.In In Proceedingsof

the7th InternationalWorkshopon AlgorithmsandData Structures(WADS2001),LNCS2125, pages

155–164.Springer, 2001.

16

[9] E. ChangandH. Garcia-Molina.BubbleUp:Low Latency Fast-Scanfor MediaServers. In Proceed-

ingsof thefifth ACM InternationalConferenceonMultimedia, pages87–98,November1997.

[10] M. Chen,D. D. Kandlur, andP. S.Yu. Optimizationof theGroupedSweepingScheduling(GSS)with

HeterogeneousMultimedia Streams. In Proceedingsof the First ACM InternationalConferenceon

Multimedia, August1993.

[11] S. Cheng,C. Chen,andI. Chen. PerformanceEvaluationof an AdmissionControl Algorithm: Dy-

namicThresholdwith Negotiation.PerformanceEvaluation, 52(1),March2003.

[12] M. Christel,M. Smith, C. Taylor, andD. Winkler. Evolving Video Skims into Useful Multimedia

Abstractions.In CHI, April 1998.

[13] J. GemmellandS. Christodoulakis.Principlesof Delay-Sensitive MultimediaDataStorageandRe-

trieval. In ACM Transactionson InformationSystems, 10(1):51–90,January1992.

[14] S. Ghandeharizadeh,L. Huang,andI. Kamel. A CostDrivenDisk SchedulingAlgorithm for Multi-

mediaObjectRetrieval. IEEE Transactionson Multimedia, 5(2):186–196,2003.

[15] S.GhandeharizadehandR.Muntz.DesignandImplementationof ScalableContinuousMediaServers.

In Parallel Computing, 24:91–122,1998.

[16] L. He, E. Sanocki,A. Gupta,andJ. Grudin. Auto-Summarizationof Audio-VideoPresentations.In

Proceedingsof ACM Multimedia, November1999.

[17] K. Inkpen,R. Hegde,S. Junuzovic, C. Brooks,andJ. Tang. AIR Conferencing:AcceleratedInstant

Replayfor In-MeetingMultimodalReview. In Proceedingsof ACM Multimedia, October2010.

[18] X. JiangandP. Mohapatra.Efficientadmissioncontrolalgorithmsfor multimediaservers.Multimedia

Syst., 7(4),July 1999.

[19] I. Kamel,T. Niranjan,andS.Ghandeharizedah.A Novel DeadlineDrivenDisk SchedulingAlgorithm

for Multi-Priority MultimediaObjects.In Proceedingsof IEEEData EngineeringConference, 2000.

[20] J.Lui andX. Wang.An AdmissionControlAlgorithm for Providing Quality-of-ServiceGuaranteefor

Individual Connectionin a Video-on-DemandSystem.In Proceedingsof theFifth IEEE Symposium

on Computers andCommunications(ISCC2000), 2000.

[21] S. McCanne,V. Jacobson,andM. Vetterli. Receiver-driven layeredmulticast. In SIGCOMM, pages

117–130.ACM, 1996.

17

[22] R. T. Ng andJ. Yang. Maximizing Buffer andDisk Utilizations for News On-Demand. In VLDB,

pages451–462,1994.

[23] qik. RecordandShareVideoLiveFromYourMobile Phone,http://qik.com.

[24] R. Rejaie,M. Handley, andD. Estrin. RAP: An End-to-EndRate-BasedCongestionControlMecha-

nismfor RealtimeStreamsin theInternet.In INFOCOM, pages1337–1345,1999.

[25] P. Shenoy, P. Goyal, andH. Vin. Issuesin MultimediaServerDesign.ACM Comput.Surv., 27(4):636–

639,December1995.

[26] F. Tobagi,J. Pang,R. Baird, andM. Gang. StreamingRAID: a disk arraymanagementsystemfor

videofiles. In Proceedingsof thefirst ACM internationalconferenceon Multimedia, pages393–400,

1993.

[27] D. TurnerandK. Ross.Adaptive Streamingof Layer-EncodedMultimediaPresentations.Journal of

VLSISignalProcessing, 34(1):83–99,May 2003.

[28] H. Vin, P. Goyal, andA. Goyal. A StatisticalAdmissionControlAlgorithm for MultimediaServers.In

Proceedingsof thesecondACM internationalconferenceon Multimedia, MULTIMEDIA ’94, pages

33–40,1994.

18