441-mmedia.ppt
Post on 06-May-2015
573 Views
Preview:
TRANSCRIPT
Multimedia Networking
Ashwin BharambeCarnegie Mellon University
15-441 Networking, Spring 2003Tuesday, April 22, 2003
Ashwin Bharambe Carnegie Mellon University
What media are we talking about?
Audio
and/or
Video
This lecture concerned about:
Ashwin Bharambe Carnegie Mellon University
Lecture Overview
Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies
Streaming Stored Media Real Time Interactive Media (Voice over IP)
Protocols Real-Time Transport Protocol (RTP) Real-Time Transport Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Application Classes Streaming Stored Media
Pipeline reception of stored files over the network No need to wait for entire file before viewing Example: On-demand video
Unidirectional Real-Time Media Similar, but data is generated on the fly Non-interactive: just listen/view, no fast-forward! Example: news broadcast
Interactive Real-Time Media Example: video conference
Ashwin Bharambe Carnegie Mellon University
Streaming Media Applications
Characteristics Interactive: Users can play, pause Even rewind/fast-forward
Requirements Delay: from client request until display start can
be 1 to 10 seconds Time is used to buffer data to reduce chance of
stalling Delay constraints on data delivery to maintain
continuous playout.
On demand Video
Ashwin Bharambe Carnegie Mellon University
Unidirectional Real-Time Apps
Characteristics By storing media: can pause, rewind Often used combined with multicast
Requirements As with streaming media, delay constraints on
data delivery to maintain continuous playout
News Broadcast
Ashwin Bharambe Carnegie Mellon University
Real-Time Interactive Applications
Requirements Very stringent delay requirements because of
real-time interactive nature Limit to how much people are willing to adapt to
the delay Video: < 150 msec acceptable (one-way) Audio: < 150 msec not perceived, < 400 msec
acceptable (one-way)
Internet Telephony
Ashwin Bharambe Carnegie Mellon University
Roadmap
Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies
Streaming Stored Media Real Time Interactive Media (Voice over IP)
Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Review: Network Basics
Source
Dest
Throughput = Amount of data arriving per unit
time.
Jitter = Variance of interpacket arrival
times
Interpacket Arrival Time
Delay
Loss
Best-effort delivery!!
Ashwin Bharambe Carnegie Mellon University
End-to-End Delay
Interactive applications Video conferencing
Human communication starts to break down at ~500 msec delays
Feedback loops Delay affected by encoding/decoding
Today’s Internet Best effort service and end-to-end mechanisms Provide little opportunity for controlling delay
Longer term (only?) solution: Quality of Service
Ashwin Bharambe Carnegie Mellon University
Packet Jitter
Real-Time systems must deal with this Jitter is hard to bound even with QoS
Partial solution: Buffer management Buffers are used to smooth jitter, but how big
should the buffer be? Unexpected jitter can lead to starvation or buffer
overflow if buffer is too small But over-provisioning buffers increases delay
Ashwin Bharambe Carnegie Mellon University
Throughput
Estimating throughput is problematic Media coding may be tunable, but changes in
quality are generally perceptually annoying Throughput will often vary faster than feedback
delay which makes estimation hard Multicast makes this worse Possible Solutions:
QoS mechanisms Scalable encoding combined with congestion
control (i.e., dynamically adapt compression to bandwidth)
Ashwin Bharambe Carnegie Mellon University
Packet Loss
Effect of loss is media and encoding specific Loss of one packet in a video stream may result
in the loss of multiple frames Other packets received can be rendered useless Retransmission often not feasible (no time)
Possible Solutions: Encoding that alleviates packet loss effects Forward Error Correction (FEC)
Ashwin Bharambe Carnegie Mellon University
Roadmap
Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies
Streaming Stored Media Real Time Interactive Media (Voice over IP)
Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Media Encoding
Digitization essential, of course! Sample signal at some frequency
8KHz, for example Quantize
represent each sample as some fixed #bits Compress: make number of bits small
Audio: GSM, G.729, G.723.3, MP3, … Video: MPEG 1/2/4, H.261, …
Then: Send -> Receive, decompress/convert, play
Ashwin Bharambe Carnegie Mellon University
Audio Encoding
Traditional telephone quality encoding 8KHz samples of 8 bits each 64kbps
CD quality encoding: 44.1 KHz of 16 bits 1.41 Mbs uncompressed
MP3 compression Frequency ranges that are divided in blocks, which are
converted using DCT, quantized, and encoded
Range Ratio
Layer 1 384 kbps 4
Layer 2 192 kbps 8
Layer 3 128 kbps 12
Ashwin Bharambe Carnegie Mellon University
Image Encoding: JPEG
Divide digitized image in 8 x 8 pixel blocks The DCT phase converts the pixel block into
a block of frequency coefficients Discrete Cosine Transform – similar to FFT
The quantization phase limits the precision of the frequency coefficient
This is based on a quantization table, which controls the degree of compression
The encoding phase packs this information in a dense fashion
Ashwin Bharambe Carnegie Mellon University
Video Encoding
Once frames are captured ("raw" video) resulting file is very large:
320 x 240 x 24-bit color = 230,400 bytes/frame 15 frames/second = 3,456,000 bytes/second 10 seconds takes around 30 Mbytes! (no audio)
Commonly-used encodings: AVI, MPEG Exploit spatial locality inside a single image Temporal locality across images Layered encoding
Ashwin Bharambe Carnegie Mellon University
Roadmap
Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies
Streaming Stored Media Real Time Interactive Media (Voice over IP)
Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Streaming Media Applications
Important and growing Reduction of storage costs High-speed access networks (e.g., DSL) Enhancements to network caching of video using
content distribution networks Operation
Segment audio/video file; send over UDP/TCP Real-Time Transport Protocol (RTP)
User-Interactive Control Real-Time Transport Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Streaming Helper Application
Displays content Typically requested via a Web browser Example: RealPlayer
Typical functions: GUI for user control Decompression Jitter removal Error correction
Ashwin Bharambe Carnegie Mellon University
Streaming From Web Servers (1)
Video sent as HTTP object(s) Options:
Interleaved audio and video in one file Two separate files with client synchronizing the
display
Ashwin Bharambe Carnegie Mellon University
Streaming From Web Servers (2)
Alternative Set up connection between server and player,
then download
Ashwin Bharambe Carnegie Mellon University
Using a Streaming Server Application layer protocol can be better tailored to
streaming Gets us around HTTP, allows a choice of UDP vs. TCP
Ashwin Bharambe Carnegie Mellon University
Streaming Media ApplicationClient Side Issues
Ashwin Bharambe Carnegie Mellon University
Roadmap
Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies
Streaming Stored Media Real Time Interactive Media (Voice over IP)
Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Voice Over IP (VoIP) Internet phone applications generate
packets during talk spurts No traffic during silence
Operation Sample, quantize, encapsulate and send
Protocol choices UDP: some packets may be lost (up to ~20% loss
is tolerable) TCP: eliminates loss but creates variance in delay TCP slow start can also create periods of very
small throughput
Ashwin Bharambe Carnegie Mellon University
Voice Over IP (VoIP): Issues
Delay End-to-end delays > 400 msec cannot be tolerated Packets that are that delayed are ignored (i.e., lost)
Jitter Use timestamps, sequence numbers Delay playout at receivers (fixed or adaptive)
Packet loss Forward Error Correction (FEC) Interleaving Receiver-based repair
Ashwin Bharambe Carnegie Mellon University
Compensating for Jitter:VoIP with Fixed Playout Delay
Playout delay
Ashwin Bharambe Carnegie Mellon University
Compensating for Jitter:Adaptive Playout Delay
Disadvantages of fixed playout delay More packet loss (delayed packets), if too small High latency, if too large
Solution: track network delay performance Estimate average delay and delay variance Similar to RTT estimation in TCP
Adjust playout delay during silent periods Silent periods get shortened/expanded Hardly noticeable!
Ashwin Bharambe Carnegie Mellon University
Recovery from Packet Loss: Forward Error Correction (FEC) Send redundant encoded chunk every n chunks (XOR original n chunks)
If 1 packet in this group lost, can reconstruct If > 1 packet lost, cannot recover
Disadvantages: The smaller the group size, larger the overhead Playout delay increased
Ashwin Bharambe Carnegie Mellon University
Recovering from Packet Loss:Piggybacking Lo-fidelity Stream
Ashwin Bharambe Carnegie Mellon University
Recovering from Packet Loss: Interleaving
Divide 20 msec of audio data into smaller units of 5 msec each and interleave
Upon loss, have a set of partially filled chunks
Ashwin Bharambe Carnegie Mellon University
Recovering from Packet Loss:Receiver-based Repair Exploit temporal similarity
Replicate packets Smart solutions
Interpolation
Ashwin Bharambe Carnegie Mellon University
Roadmap
Multimedia Applications Application Requirements / Challenges Media Encodings Case Studies
Streaming Stored Media Real Time Interactive Media (Voice over IP)
Protocols Real Time Protocol (RTP) Real Time Control Protocol (RTCP)
Ashwin Bharambe Carnegie Mellon University
Real Time Transport Protocol (RTP)
Recall: Multimedia apps. append headers to audio/video
chunks Format, sequence numbers, timestamps, …
RTP: public domain standard for such headers Chunks encapsulated in RTP packets before being
sent to UDP Real Time Control Protocol (RTCP) for control
RTP logically extends UDP. Sits between UDP and application Thin Protocol
Ashwin Bharambe Carnegie Mellon University
RTP: What does it do?
Framing Multiplexing “Real-time delivery” (sort’a) Synchronization Feedback
Ashwin Bharambe Carnegie Mellon University
RTP Packet Format
Source/Payload type Different formats assigned different codes E.g., GSM -> 3, MPEG Audio -> 14
Sequence numbers Time stamps Synchronization source ID Miscellaneous fields
Ashwin Bharambe Carnegie Mellon University
Timestamp vs. Sequence Number
Timestamp relates packet to real time Timestamp values sampled from a media specific
clock Sequence number relates packet to other
packets Allows many packets to have the same
timestamp but different sequence numbers
Ashwin Bharambe Carnegie Mellon University
Audio silence example
Consider audio data: what do you want to do during silence? Not send anything
Why might this cause problems? Other side needs to distinguish between loss and
silence How does the timestamp/seq# mechanism
help? Big jump in timestamps, but correct next
sequence number Silence!
Ashwin Bharambe Carnegie Mellon University
SSRC: Synchronization Source
Identifies the stream Not just the participant All packets with the same SSRC go together
Picked at random Why do we need this?
No assumption of stream association from underlying network mechanism
Possible problems? Collision
Ashwin Bharambe Carnegie Mellon University
Real Time Control Protocol (RTCP)
RTCP packets transmitted by each participant in RTP session to all others using multicast
Distinct port number from RTP Reports on:
Loss rate Inter-arrival jitter Identity of receivers Delay, (indirectly).
Ashwin Bharambe Carnegie Mellon University
Control Bandwidth Scaling
Needed for scalability Consider one speaker and 2 receivers as
compared to one speaker with 20 or 200 receivers
Recommendation: 5% of data bandwidth Sending delay should be randomized Should account for overhead of underlying
network service (i.e., UDP and IP)
Ashwin Bharambe Carnegie Mellon University
Summary
Supporting multimedia applications in today’s Internet is difficult Several tricks to alleviate problems posed by best-
effort delivery In future, perhaps bandwidth will be extremely
plentiful everywhere Or – we will need Quality of Service some day or
the other! Bunch of Real-Time protocols
standardizing some aspects of real-time communication
top related