Optimal Quality Adaptation for Optimal Quality Adaptation for MPEG-4 Fine-Grained Scalable MPEG-4 Fine-Grained Scalable

VideoVideoTaehyun Kim and Mostafa H. AmmarTaehyun Kim and Mostafa H. Ammar

College of Computing, Georgia Institute of TechnoloCollege of Computing, Georgia Institute of Technology gy

ProblemsProblems

Rate smoothing is not useful for a Rate smoothing is not useful for a best effort network, since the best effort network, since the Internet does not provide any Internet does not provide any information about the bandwidth information about the bandwidth evolution in advance.evolution in advance.

A smooth data rate does not always A smooth data rate does not always guarantee a smooth quality for VBR guarantee a smooth quality for VBR video.video.

ProblemsProblems

Frequent adding and dropping of Frequent adding and dropping of layers can incur significant quality layers can incur significant quality variability.variability. Quality Adaptation for minimizing the Quality Adaptation for minimizing the

perceptual video quality by using perceptual video quality by using bidirectional optimum layer selection.bidirectional optimum layer selection.

ProblemsProblems

Small time scale variabilitySmall time scale variability Receiver bufferReceiver buffer

Large time scale variabilityLarge time scale variability Scalable (Layered) video encodingScalable (Layered) video encoding

GoalGoal

Trying to accommodate the mismatch Trying to accommodate the mismatch caused by both caused by both the available bandwidth the available bandwidth variabilityvariability and and the encoded video the encoded video variabilityvariability..

To develop an optimal algorithm that To develop an optimal algorithm that minimizes the quality variabilityminimizes the quality variability while at while at the same time the same time maximizing the maximizing the utilization of the variable network utilization of the variable network bandwidthbandwidth..

Rate variability in MPEG-4 Rate variability in MPEG-4 FGSFGS

Base Layer FGS Layer (SNR).Fixed quantization step size

A river runs through it , GOP=12

.Max variation = 7.4 kBytes.Max variation = 33.6 kBytes

Rate variability in MPEG-4 FGSRate variability in MPEG-4 FGS

FGST Layer (SNR).Max variation = 29.9 kBytes

Two hybrid temporal-SNR Two hybrid temporal-SNR scalability structures scalability structures FGS-

FGST

Two hybrid temporal-SNR scalabiTwo hybrid temporal-SNR scalability structures lity structures FGST-FGS

Video Quality – Base LayerVideo Quality – Base Layer

100 VOPs, 45 dB

Video Quality – Base+FGS LayeVideo Quality – Base+FGS Layerr

100 VOPs, improved more than 20 dB

Video Quality – Base+FGST LayerVideo Quality – Base+FGST Layer

300 VOPs, inconsistent quality

Video Quality – All LayerVideo Quality – All Layer

300 VOPs, 67.3 dB

Quality Adaptation Quality Adaptation AlgorithmAlgorithm

Quality adaptationQuality adaptation is defined by a is defined by a mechanism that adds and drops mechanism that adds and drops layers based on the layers based on the available available network bandwidthnetwork bandwidth while while maximizing maximizing the perceptual video qualitythe perceptual video quality..

Consistent “long runs” of the same Consistent “long runs” of the same quality video.quality video.

Composed AlgorithmComposed Algorithm

The The quality smoothing algorithmquality smoothing algorithm proposed in [13] accomplishes the proposed in [13] accomplishes the maximum reduction of quality variabilitymaximum reduction of quality variability for layered for layered CBRCBR video using bidirectional video using bidirectional layer selection.layer selection.

Rate smoothing algorithmRate smoothing algorithm presented in presented in [18] enables a sender to transmit a [18] enables a sender to transmit a piecewise CBR sequencepiecewise CBR sequence by using the by using the work-ahead smoothing technique.work-ahead smoothing technique.

Composed AlgorithmComposed AlgorithmL: Number of

layers

N: Number of VOPs

xi[k]: size of VOP kSik : a feasible sequence of layer i

The cumulative selected data defined by

Optimal Quality AdaptationOptimal Quality Adaptation

The cumulative

capacity

The receiver buffer size for storing unplayed i -th layer video

The VOP size of i -th layer, at

time k

The available bandwidth : the residual bandwidth after accommodating layers 1, 2, …, i-1.i -th layer, at time k .

Framework of quality Framework of quality adaptationadaptation

Display bufferedvideo + prefetch

No display video

Framework of quality adaptatiFramework of quality adaptationon

The constraint of The constraint of rate adaptationrate adaptation is is determined by the determined by the receiver buffer receiver buffer sizesize and the and the source video ratesource video rate, , whereas the main constraint of whereas the main constraint of quality adaptationquality adaptation is is transmission transmission resourcesresources..

State transition diagram State transition diagram specifying the quality specifying the quality

adaptation mechanismadaptation mechanism

Select Discard

No capacity

Available cumulative capacity ≥ threshold

Available cumulative capacity <thresholdEnough capacity

Optimal Optimal AdaptationAdaptation

Available network bandwidth is known

Residual bandwidth for higher layer

Stay as long as possible

Theorem 1Theorem 1

In the framework of the optimal In the framework of the optimal quality adaptation, a quality adaptation, a threshold valuethreshold value equal to the equal to the receiver buffer sizereceiver buffer size satisfiessatisfies 1) minimum video quality variability1) minimum video quality variability 2) the necessary condition of maximum 2) the necessary condition of maximum

network utilizationnetwork utilization

Online HeuristicOnline Heuristic The optimal quality adaptation algorithm assumes

the available bandwidth information is known in advance.

An algorithm that minimizes quality variability without using future bandwidth information.

The differences between the online heuristic and the optimal adaptation 1) the online heuristic makes a decision on which layer a

nd which VOP to be transmitted in real time (lines 4-6) 2) a sender makes a receiver prefetch the next selected

VOPs when there is a transition from the select state to the discard state (line 15).

Online Online Heuristic Heuristic AlgorithmAlgorithm

Receiver prefetch the next selected VOPs

Make decision on which layer and which VOP to be transmitted in real time

How to determine the next prefetch point at the transition time ?

An MA (Moving Average) type estimator to determine the prefetch point. simple and widely known for the usage of T

CP retransmission timeout estimation in [7].

Experimental modelExperimental model

TFRC throughput

TFRC/UDP (TCP-Friendly Rate Control)

Quality transition of the i th layer is defined by

Ratesmoothing

Qualitysmoothing

Receiver buffer

Performance over TFRC (1)QT=121 QT=13

QT=87 QT=9

Slow response time of TFRC

Composed

Optimal adaptation

Performance over TFRC (2)

Onlineheuristic

Thresholdbased

QT=126

Target on minimize loss probability

QT=16

TCP throughput

Small time scale variability is significant as much as 3 Mbps

Performance over TCP (1)Composed

Optimal adaptation

Performance over TCP (2)

Thresholdbased

Onlineheuristic

Two reasons contribute to Two reasons contribute to superiority of TCPsuperiority of TCP

TCP achieves more throughput than TCP achieves more throughput than TFRC in dynamic condition.TFRC in dynamic condition.

Although TCP exhibits significant Although TCP exhibits significant small time scale variability, it can be small time scale variability, it can be successfully accommodated by the successfully accommodated by the receiver buffer.receiver buffer.

Experiment results for 4 video Experiment results for 4 video streamsstreams

Average Quality Transition

Average Run Length [13]

ConclusionConclusion

Considering a problem of providing perceptuaConsidering a problem of providing perceptually good quality for layered VBR streaming videlly good quality for layered VBR streaming video.o.

An optimal adaptation algorithm that minimizAn optimal adaptation algorithm that minimizes quality variability while increasing the usages quality variability while increasing the usage of the available bandwidth.e of the available bandwidth.

Companion web siteCompanion web site http://www.cc.gatech.edu/computing/Telecomm/http://www.cc.gatech.edu/computing/Telecomm/

people/Phd/tkim/qa.htmlpeople/Phd/tkim/qa.html