adaptable applications towards balancing network and terminal resources to improve video quality

1

TU/e

Adaptable applicationsTowards Balancing Network and Terminal Resources to Improve Video Quality

D. Jarnikov

2

TU/e

Contents

• Introduction (www)

• New solutions

• Subjective evaluation

• Conclusions and future plans

3

TU/e

Introduction of in-home network

4

TU/e

Problem description

Video data transmission

Video data decoding

Network condition

good bad

Pe

rce

ive

d

vid

eo

qua

lity

bad

good

Pe

rce

ive

d

vid

eo

qua

lity

bad

good

Resource consumptionlow high

5

TU/e

Objective: networked terminal

• Resource-constrained terminal : CPU• Resource-constrained network : bandwidth

• Wireless network has fluctuations

Source Terminal

Wireless network

6

TU/e

Solution

• Scalable video technique– Choose size of base layer such, that we can almost guarantee the

transmission

– The enhancement layer is transmitted if there is available bandwidth

– The number of layers to be decoded can be chosen for every frame

• Controller– Choose how much video data (e.g. layers) should be processed.

– Optimizes perceived quality when looking at available input data AND available CPU power

7

TU/e

Past: Conclusions & Future plans

• The usage of scalable video enables trade-offs between user perceived quality and network and terminal resources

• A controller can be used to optimized perceived quality with respect to the available CPU power and amount of input data

• We developed the controller that doesn’t depend on the scalability technique

• The correctness of controller behavior depends on rightness of parameters

• Take into account other terminal resources

• Organize a feedback from the terminal to the source

• Create MPEG2 to scalable video transcoder

8

TU/e

Contents

• Introduction (www)

• New solutions

– System view

– Transcoder

– Controller

– Network sender-receiver

– Summary

• Subjective evaluation

• Conclusions and future plans

9

TU/e

System: Present view

Source Terminal

Wireless network

TranscoderNetwork

Sender-ReceiverTerminal --

?

10

TU/e

Contents

• Introduction (www)

• New solutions

– System view

– Transcoder

– Controller

– Network sender-receiver

– Summary

• Subjective evaluation

• Conclusions and future plans

11

TU/e

Transcoder: General Info

Separates video stream on most important information and least important information (refinement)

• Input: MPEG2 video• Output: Scalable MPEG2 video• Parameters: number of output layers, sizes of the layers

VLDInverse Quantization

Q-1

stream QuantizationQ

VLC

-

ELstream

BLstream

QuantizationQ’

VLC

12

TU/e

Transcoder: New enhancement layers

From I to B frames:• empty macroblocks can be skipped• variable length coding tables of B frames are better suited to

encode residual values

Advantages: • lower importance of base layer size• less syntax overhead

Both approaches are compliant with new MPEG System proposal

13

TU/e

Contents

• Introduction (www)

• New solutions

– System view

– Transcoder

– Controller

– Network sender-receiver

– Summary

• Subjective evaluation

• Conclusions and future plans

14

TU/e

Controller: General view

• Controller chooses how much video data (e.g. layers) should be processed. Takes into account:– amount of available resources (CPU)– amount of video data available (e.g. how many layers have we

received)

• Objective: maximize perceived quality– MAX number of layers to be processed– MIN deadline misses– MIN quality changes

Terminal

Scalable VideoInput

Decoder

Controller

Post-processing

Controller

15

TU/e

Controller: Markov decision process

State: progress w.r.t. deadline

number of layers decoded

maximal number of layers for the next frame

Revenue:Reward: number of layers

Penalty: deadline misses

Penalty: quality change

Penalty: quality change, caused be the network

16

TU/e

Controller: Challenges

Processing times- depends on content (stochastic)- depends on layer size

Probabilities change for every possible layer size

=> Unique strategy for every layer size

Maximal number of layers for the next frame- network-dependent parameter (stochastic)

We need appropriate network behavior model Unique strategy for every network condition

Number of layers in total Unique strategy for every number of layers

17

TU/e

Contents

• Introduction (www)

• New solutions

– System view

– Transcoder

– Controller

– Network sender-receiver

– Summary

• Subjective evaluation

• Conclusions and future plans

18

TU/e

Network sender-receiver (NSR)

• Streams layers over the network

• Takes care about layer prioritization

• Sends feedback about receiving conditions

Streaming conditions + Receiving conditions = Network conditions

μ, BER / PER

19

TU/e

NSR: Channel Model

• Using μ, BER / PER we can build a simple channel model [example: Gilbert-Elliott channel (GEC)]

• We run a simulations for different network conditions and layer configurations

• For every pair μ, BER / PER we can estimate what is an optimal layer configuration (lookup table)

Layer configuration is a run-time input for the transcoder

20

TU/e

NSR: Controller input

Processing times- depends on content (stochastic)- depends on layer size

Probabilities change for every possible layer size layer size

=> Unique strategy for every layer size

Maximal number of layers for the next frame- network-dependent parameter (stochastic) number of layers

We need appropriate network behavior model layer size Unique strategy for every network conditions μ, BER / PER

Number of layers in total Unique strategy for every number of layers number of layers

BUT! Layer sizes and number of layers have one-to-one relation with μ, BER / PER

21

TU/e

Contents

• Introduction (www)

• New solutions

– System view

– Transcoder

– Controller

– Network sender-receiver

– Summary

• Subjective evaluation

• Conclusions and future plans

22

TU/e

System: Global view

TranscoderNetwork

Sender-ReceiverTerminal --

μ, BER / PER

number of layerslayers sizes

strategy

Calculatedoffline

Alt

erna

tive

23

TU/e

Contents

• Introduction (www)

• New solutions

• Subjective evaluation

• Conclusions and future plans

24

TU/e

User tests

How does proportion between sizes of BL and EL influence perceived quality?

What is better: a large EL with high losses or small EL with low losses?

RESULTS: • BL size is very important• With decrease of overall bit-rate the importance of BL size

increases• With increase of EL size married to a lower frequency is

perceived slightly worse

25

TU/e

Contents

• Introduction (www)

• New solutions

• Subjective evaluation

• Conclusions and future plans

26

TU/e

Conclusions

• Made a transcoder that produces less bit per EL for the same quality

• Made a network simulation that allows a better choice of layer configuration

• Enhanced a terminal controller with realistic network behavior model

• User tests were performed for perceived quality evaluation of a scalable video

27

TU/e

Future plans

• Take into account other terminal resources

• Enhance the feedback mechanism from the terminal to the source with terminal capabilities information

• Allow source-terminal resource negotiations

• Perform subjective test for dynamic behavior of scalable video scheme

• Implement transcoder on CE platform

28

TU/e

?

29

TU/e

Transcoder: Comparison of the approaches

0

2

4

6

8

10

12

14

BL BL&EL

PS

NR

dif

fere

nc

e w

ith

re

fere

nc

e (d

B)

1 + 4 2 + 3 2.5 + 2.5 3 + 2 4 + 1

0

2

4

6

8

10

12

14

BL BL&EL

PS

NR

dif

fere

nce

wit

h r

efe

ren

ce

(dB

)

1 + 4 2 + 3 2.5 + 2.5 3 + 2 4 + 1

Difference in PSNR between one-layer reference and two-layer scalable coding

(the overall bitrate is 5 MBps)

I-frame approach

B-frame approach

30

TU/e

NSR: Channel Model

• Using μ, BER / PER we can build a simple channel model [example: Gilbert-Elliott channel (GEC)]

• We model the channel as a two-state discrete time Markov chain (DTMC) with states G (good) and B (bad) and four probabilities P, Q, EG, and EB

G B

P

Q

1-P 1-Q

)QP( 1

1

50 P.BER

31

TU/e

NSR: Channel Model Simulation

32

TU/e

NSR: Channel Model Outcome

For every pair μ, BER / PER we can estimate what is an optimal layer configuration (lookup table)

Layer configuration is a run-time input for the transcoder

Example: maximal BL bitrate as a function of network conditions