rate distortion optimized streaming
DESCRIPTION
Rate Distortion Optimized Streaming. Maryam Hamidirad CMPT 820 Simon Fraser Univerity. Outline. Introduction Rate Distortion Optimized Framework Basic Framework Receiver Driven Streaming Rich Acknowledgements Multiple Deadlines Dependant Packet Delay - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/1.jpg)
1
Rate Distortion Optimized Streaming
Maryam HamidiradCMPT 820
Simon Fraser Univerity
![Page 2: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/2.jpg)
2
OutlineIntroductionRate Distortion Optimized Framework
Basic Framework Receiver Driven Streaming
Rich AcknowledgementsMultiple DeadlinesDependant Packet DelayCongestion Distortion Optimized
StreamingConclusion
![Page 3: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/3.jpg)
3
IntroductionInternet Packet Delivery
Loss Throughput Delay
Challenge : Maximize quality of audio and video considering transmission rate and latency constraints
Network Adaptive
![Page 4: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/4.jpg)
4
Introduction(cntd.)Media Streaming System :
Client ApplicationError Detection and Concealment
Transport MechanismCongestion Control by Retransmission and Packet
Drops
Media Server Intelligent Transport by sending the right packet at
the right time
EncoderRate Scalable Coding
![Page 5: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/5.jpg)
5
Rate Distortion Optimized Framework
Framework has been propose by Chao and Miao
Goal : Compute which packets to send and when to minimize the reconstruction distortion
![Page 6: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/6.jpg)
6
Basic FrameworkMedia Server has media streams
packetized into data units
Framework chooses optimal set of data units to transmit at successive transmission opportunities
Scheduler decides based on an entire optimized plan
![Page 7: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/7.jpg)
7
Basic Framework (cntd.)Parameters :
Data unit : l Size : B(l) Deadline :
The maximum arrive time to be useful for decoding
: distortion reduction which is decrease in distortion rate if l is decoded
N : transmission opportunities π : transmission policy
It has N binary vector π(l) for each data unit l
ε( l): error probability data unit l received late or not at all
P(l) : number of times packet has been sent
![Page 8: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/8.jpg)
8
Basic Framework(cntd.)• Policy π wants to find the best tradeoff
between expected transmission rate and distortion construction
• Formally , minimize Lagrangian function:
(1)
(2)
(3)
![Page 9: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/9.jpg)
9
Basic Framework(cntd.)Packet loss and delay are considered
independently Packet Loss Bernoulli Function Delay Shifted τ-distribution
Exhaustive Search is not useful. The search space grows exponentially
Chao and Miao proposed radio framework
![Page 10: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/10.jpg)
10
Basic Framework(cntd.)The radio framework uses
conjugate direction searchThe Iterative Sensitivity
Algorithm minimizes the Lagrangian function◦The policy for π(l) is optimized while
others are fixed.◦It runs for every l in round robin
fashion in order for π to achieve a local minimum
![Page 11: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/11.jpg)
11
Basic Framework (cntd.)
where is the rate distortion tradeoff multiplier
is the data unit size
is the sensitivity of overall distortion to error probability of data unit l
(4)
![Page 12: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/12.jpg)
12
Basic Framework (cntd.)
Figure 1 . improved video streaming performance achieved by RadioTaken from Multimedia over IP and Wireless Networks
![Page 13: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/13.jpg)
13
Receiver Driven StreamingTransmitting many video and
audio make the server become computationally overwhelmed.
Shift Computation to the Client as much as possible
![Page 14: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/14.jpg)
14
Receiver Driven Streaming (cntd.)Strategy :
Client will be provided with information about size , distortion reduction values , interdependencies of data. Rate distortion preamble is small
It computes optimized scheduler and compute sequence of requests that specify data units.
![Page 15: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/15.jpg)
15
Receiver Driven Streaming (cntd.)
Figure 2. Sender driven and receiver driven streaming comparison
Taken from Multimedia over IP and Wireless Networks
![Page 16: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/16.jpg)
16
Receiver Driven Streaming (cntd.)Combining sender driven and
receiver driven approach can be used to R-D optimized algorithm to diverse network topologies.
Example : Using radio
framework in a proxy between network backbone and last hop link.
![Page 17: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/17.jpg)
17
Receiver Driven Streaming (cntd.)
Figure 3. Proxy Driven Radio StreamingTaken from Multimedia over IP and Wireless Networks
![Page 18: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/18.jpg)
18
Receiver Driven Streaming (cntd.)Proxy uses hybrid of sender and
receiver streaming.
It improves the end-to-end performance. The traffic caused by retransmission of lost packets is not traversed to server and stays in last hop
![Page 19: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/19.jpg)
19
Rich AcknowledgementsInstead of sending ACK for each
received packet , send the state of received packets periodically that ACKs received packets and NACKs lost packets.
Needs changes in the basic framework
![Page 20: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/20.jpg)
20
Rich Acknowledgements(cntd.)Transmission policy of a data unit
can be understood in terms of Markov Decision Process.
At time t(i) , server makes observation o(i) and takes action a(i) which is send or don’t send .
The sequence of (o(i) , a(i)) is a Markov decision tree.
![Page 21: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/21.jpg)
21
Rich Acknowledgements(cntd.)Possible actions :
◦ Send ◦ don’t send
Possible observations : ◦∅ no relevant feedback has arrived◦ACK feedback packet acknowledged
received data unit◦NACK feedback packet indicate lost
data unit
![Page 22: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/22.jpg)
22
Rich Acknowledgements(cntd.)
Figure 4. State space for Markov Decision process when rich acknowledgement usedTaken from Multimedia over IP and Wireless Networks
![Page 23: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/23.jpg)
23
Rich Acknowledgements(cntd.)
Optimization algorithm calculates the probabilities through each path given the policy and find the best tradeoff between expected number of transmission p(l) and loss probability e(l)
![Page 24: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/24.jpg)
24
Rich Acknowledgements(cntd.)
Figure 5. Rich vs. conventional acknowledgement rate distortion optimized streaming using QCIF foreman
Taken from Multimedia over IP and Wireless Networks
![Page 25: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/25.jpg)
25
Rich Acknowledgements(cntd.)Improved Performance :
Effect of lost packet is mitigated because subsequent feedback packets contain same information
Less ambiguity for server by having NACKs
![Page 26: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/26.jpg)
26
Multiple DeadlinesInstead o discarding the frame
arrive later than the associated deadline , we consider that frame will be useful for decoding other frames or at least itself.
We associate multiple
deadlines to one frame
![Page 27: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/27.jpg)
27
Multiple Deadlines (cntd.)Example :
We have a set of frames IBBBP . If frame p arrives later than deadline , it still can help in decoding the next B frames.
Decoders that allow Accelerated Retroactive Decoding (ARD) It allows many streaming clients to
decode video faster than real time. When the late frame arrives, it goes back to dependant frames and decode the dependant parts up to play-out time.
![Page 28: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/28.jpg)
28
Multiple Deadlines (cntd.)Need changes in the
formulation : We have to add the error probability
for each deadline
(5)
In the above equation, is the sensitivity factor that depends on each deadline and is the sensitivity of overall distortion if data unit arrives in .
![Page 29: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/29.jpg)
29
Multiple Deadlines (cntd.)
Figure 6. Rate distortion performance scheduler for a foreman stream sequenced over a simulated channel with 20% packet lossAnd shifted -distribution delay. PSNR improvement up to 3.15 db can be seen in ARD with multiple deadlines .Taken from Multimedia over IP and Wireless Networks
![Page 30: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/30.jpg)
30
Multiple Deadlines (cntd.)Multiple deadline approach take
the benefit of using the information of late arrived packet , therefore they improve PSNR compared to single-deadline scheme.
![Page 31: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/31.jpg)
31
Dependant packet delaysIn the framework :
Packet Delay : Shifted -distribution Loss : Bernoulli Model
Packet delays are assumed to be independent of each other which simplifies the calculation of error probability
This is not realistic Suboptimal performance
![Page 32: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/32.jpg)
32
Dependant packet delays(cntd.)
Figure 7. Rate distortion performance scheduler for a foreman sequence streamed over measured internet delay traceTaken from Multimedia over IP and Wireless Networks
![Page 33: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/33.jpg)
33
Dependant packet delays(cntd.)In higher rate , heuristic ARD will
outperform iid- model: The algorithm mistakenly
believes that if a data unit arrives late , other data units will arrive on time or earlier. Therefore, it sends packets multiple times even though packet loss in low (0-14%)
![Page 34: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/34.jpg)
34
Dependant packet delays(cntd.)Improvement : Model the delay
at successive transmission time slots as a first order Markov random process.
Feedback packets will inform server about the delay over channel in the recent past.
![Page 35: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/35.jpg)
35
Congestion Distortion Optimized Streaming
In radio streaming approach, packet delay is not affected by transmitted packets.
Delay is a random variable with parameterized distribution that adapts slowly according to feedback information. Media steam transmitted at
negligible rate to bandwidth , model is acceptable. But , in higher rates it is not.
![Page 36: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/36.jpg)
36
Congestion Distortion Optimized Streaming (cntd.)
Improvement: Congestion distortion optimized streaming (Codio) Effects of transmitted packets is
considered. It gets an optimal tradeoff between congestion and reconstruction distortion.
It assumes a succession of high bandwidth link followed by a bottleneck last hop used by media streams.
![Page 37: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/37.jpg)
37
Congestion Distortion Optimized Streaming (cntd.)
Figure 8. Performance Comparison of Codio and Radio Streaming for video streaming over a bottleneck link.Taken from Multimedia over IP and Wireless Networks
![Page 38: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/38.jpg)
38
Congestion Distortion Optimized Streaming (cntd.)
Codio outperforms Radio: It transmits packets as late as safely
possible. This reduces the congestion in backlog and therefore end-to-end delay.
![Page 39: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/39.jpg)
39
ConclusionTo get better quality of audio and video streams ,
media streaming should be network adaptive
Media server can decide which packets and when to send to optimized the distortion of decoded video
Radio Framework proposed to avoid exhaustive search
There are extensions to the basic framework which improve performance
Rich Acknowledgement Multiple Deadline Dependant Packet Delay Congestion Distortion Optimized Streaming
![Page 40: Rate Distortion Optimized Streaming](https://reader035.vdocument.in/reader035/viewer/2022062500/56814ea5550346895dbc5055/html5/thumbnails/40.jpg)
40
Thank You