internet video streaming
TRANSCRIPT
Video Communications over the Internet
Jozsef Vass
1/16/2000
Multimedia Communications and Visualization LaboratoryDept. of Computer Engineering and Computer Science
University of Missouri-ColumbiaColumbia, MO 65211, USA
http://meru.cecs.missouri.edu
2
Presentation Overview
References
Multimedia Requirements
Internet Protocol
Real-Time Transport Protocol
Resource Reservation Protocol
Multicast
Multiresolution Coding
Packetization
Error Control
Flow Control
Conclusions
3
References
S. McCanne, M. Vetterli, and V. Jacobson, “Low-complexity video coding for
receiver driven layered multicast,” IEEE Journal on Selected Areas in
Communications, vol. 15, no. 6, pp. 983-1001, Aug. 1997.
W.-T. Tan and A. Zakhor, “Real-time Internet video using error resilient
scalable compression and TCP-friendly transport protocol,” IEEE Transaction
on Multimedia, vol. 1, no. 2, pp. 172-186, June 1999.
T. Turletti, S.F. Parisis, and J.-C. Bolot, “Experiments with a layered
transmission scheme over the Internet,” INRIA Technical Report, Nov. 1997.
F. Kuo, W. Effelsberg, and J.J. Gracia-Luna-Aceves, Multimedia
Communications, Prentice Hall, 1998.
A.S. Tanenbaum, Computer Networks, Prentice Hall, 1998.
4
Multimedia Requirements
High bandwidth
Delay (critical for interactive applications)
Variation of delay (jitter)
Reliability
Commonly referred as quality-of-service (QoS) parameters
Low complexity codecs: Software only encoder/decoder
Multicasting: Bandwidth reduction
Support for heterogeneous system
5
Connection Type
Connectionless (Datagram): Each packet is routed individually
Currently used in the Internet
Efficient for short connections
Robustness
Easy internetworking
Connection-oriented (Virtual Circuit): Route is established at
connection setup
Currently used in X.25, Frame Relay, ATM
Efficient routing at runtime
Call acceptance to avoid congestion
6
Internet Protocol version 4
Provide reliable transmission in the case of link failure
Datagram service, each packet is routed independently
Best effort service, does not support QoS (fair)
Unicast (multicast addresses are reserved)
No provisions for supporting continuous media
No flow control
No error control
7
Internet Protocol version 4
Source Address
Destination Address
Header Checksum
Type of Service
TransportProtocol
Identification
Time to Live
HeaderLength
Version(4) Total Length
MF
DF
Fragment Offset
Options (optional)
20 byte header
8
Internet Protocol version 4
Packets can be segmented as traversing through the network
Identification: Fragmentation information
DF: Do not fragment. Each router shall handle packets of 576 bytes
MF: More fragments
Fragment Offset: Tells the place of the fragment
9
Internet Protocol version 6
New version of Internet Protocol
New features
Introduction of streams: Flow => Packet stream between sender and receiver
Larger address space: 128 bit
Improved multicast addressing
Priority
Security (authentication, integrity, and encryption)
Backward compatible to IPv4
Connectionless
No flow control
No error control
10
Internet Protocol version 6
Version(6) Priority
Payload Length
Source Address
Next Header Hop Limit
Source Address
Source Address
Source Address
Destination Address
Destination Address
Destination Address
Destination Address
Flow Label
40 byte header
11
Internet Protocol version 6
Flow label: Multimedia flow can be identified
Special handling by the router
Soft state: Keep the state at the router until a time-out is reached
12
Migration to IPv6
Requires infrastructure changes
Internet providers work with small profit margins
IPv4 is good enough
May take several years
13
User Datagram Protocol
Source Port
UDP ChecksumUDP Length
Application interface to IP Send datagrams without establishing connection
Eight byte header
Destination Port
14
Real-Time Transport Protocol (RTP)
Transport protocol for multimedia streams over the Internet
Main functionality: Timing and synchronization
Used over UDP
Lightweight protocol No error correction No flow control No packet reordering No retransmission of lost packets
Does not support either resource reservation or QoS
Multicast capable
Associated control protocol: Real-Time Control Protocol (RTCP) Measure connection parameters and send transmission records (participants can
monitor the quality of the media flow) Parameter negotiation
15
Real-Time Transport Protocol (RTP)
Timestamp
Sequence NumberPayload TypeMCCV P
Contributing Source (CSRC) Identifier(Optional, up to 15 fields)
Synchronization Source (SSRC) Identifier
V Version NumberP Padding On/OffCC CSRC CounterM Mark
12 byte header
16
Real-Time Transport Protocol (RTP)
Timestamp: Intrastream and interstream synchronization
Synchronization Source (SSRC) Identifier: Random number generated
by the source, unique identification of the stream
Contributing Source (CSRC) Identifier: If a router (mixer) combines
media streams, the SSRC of contributing sources are recorded
17
Real-Time Transport Protocol (RTP)
Implemented as part of the application
Relatively new protocol
Used in vic and vat
Expected to be used in next generation WWW browsers
18
Resource Reservation Protocol
Earlier protocols ST-II (Internet Stream Protocol): Parallel to IP
Tenet Protocol Suit (UC Berkeley)
Resource Reservation Protocol (RSVP)
To be used with IPv6 (may also be used with IPv4)
Reservations are made for flows (specified in IP header)
Based on the flow label, the router schedules transmission in accordance with the reservation setup
RSVP keeps soft states Large amount of information
Must be refreshed within a given period
19
Resource Reservation Protocol
Receiver oriented: Each receiver may join or leave session
Each receiver decides based on its own characteristics and
requirements => Heterogeneous reservation (multicast)
The path may change: Cannot provide hard QoS guarantees
Tunneling: Traffic may flow through routers that do not support RSVP
=> no guarantees can be given
Broken links are not handled
20
Multicast Distribution
Unicast: Send a copy of each packet to each receiver individually
Multicast: Send each packet simultaneously to all the interested
receivers
Multicast Backbone (MBone) of Internet
Multicast IP address space was reserved (group addresses)
Routers must be extended
Understand group addresses
New routing mechanism
Duplicate incoming packets if necessary
Forward packets on the correct links
21
Multicast Distribution
IGMP: Internet Group Management Protocol
Multicast routing
Joining and leaving multicast groups
Multicasting still considered experimental
Multicasting will be fully integrated to IPv6
Experimental tools: vat, vic, wb, etc.
22
Multiresolution Coding
Network heterogeneity Local network Dial up network (PSTN, ISDN, etc.) Wide area network
Earlier video coding techniques (H.261, MPEG 1, H.263, etc.) do not
support scalable coding
Scalability in recent video coding standards
MPEG-2: Spatial and quality scalability
H.263+: Spatial, quality, and temporal scalability
MPEG-4: Spatial, quality, temporal, and object scalability
23
Multiresolution Coding
Simulcast: Set of independent encoders each producing a different
output rate => Low compression ratio since cross stream correlation is
not exploited
Multiresolution coding: Correlation across streams is exploited
resulting in high compression efficiency
The more layer the decoder receives the better the visual quality
Each layer may be carried by a different multicast group =>
Multiresolution-multicast framework
24
Multiresolution Coding
Hierarchical encoding:
Used in MPEG-2, H.263+, etc.
Base layer => Coarsest resolution signal, received by all receivers
Enhancement layers continuously refine the quality
Only layers received that the physical link can support
Layers must be received in importance order
25
Multiresolution Coding
Hierarchical encoding: Some layers are mandatory to reconstruct the
signal
The network must provide prioritized transmission => In the case of
packet loss, always the lowest priority layer must be dropped
ATM supports two levels of priority
IPv4 does not support priorities
IPv6 support priorities
26
Multiresolution Coding
Solutions for IPv4
Produce substreams that are of equal importance. No matter which
substream is received, the quality can be continuously improved (Multiple
Description Coding)
Ensure that higher priority streams are more reliably transmitted => Use
unequal error protection (UEP)
27
Multiresolution Coding
Audio coding: Subsampling
Each flow is separately encoded
Each flow has the same importance
The larger number of flows received, the better the quality
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 8 kHz
2.7 kHz
2.7 kHz
2.7 kHz
28
Multiresolution Video Coding Block-Based Techniques
High coding performance
Most standards (H.261, H.263, MPEG-1, MPEG-2, MPEG-4, etc.)
based on this technique
Temporal correlation is exploited by block-based motion estimation
and compensation
Remaining spatial redundancy is exploited by transform coding
High complexity of encoder
Error sensitivity
Error propagation
Scalability is hard to support
29
Multiresolution Video Coding Block-Based Techniques
Conditional replenishment
Only blocks are encoded that change from previous frame
Blocks are encoded in intra mode, no prediction is applied
Lower performance than prediction
Higher error resilience
Only good for videoconferencing (large portion of the scene is
unchanged)
30
Multiresolution Video Coding Three-Dimensional Techniques
Block-based techniques does not naturally support scalability
Wavelet decomposition naturally provides multiresolution
representation
Wavelets can be extended to 3-D for video coding
Error resilience
No recursive loop: Reduces temporal error propagation
No spatial prediction: Reduces spatial error propagation
31
Multiresolution Video Coding Three-Dimensional Techniques
Nine substreams: Each substream is of approximately equal importance
32
Packetization
Before transmission, the stream must be packetized
Due to significant overhead of IP, large packet size is preferred
IPv4+UDP+RTP=20+8+12=40 bytes
IPv6+UDP+RTP=40+8+12=60 bytes
Packetization delay: Time to wait for data to fill the packet
Critical for low bit rate interactive applications (especially for speech)
Multilayer transmission increases the packetization delay
Error protection increases the packetization delay
33
Packetization
Speech example:
Sampling rate 8 kHz, eight bits/sample: 64 kbps
It is not beneficial to use high compression efficiency algorithms
Buffer sizeProtocol
20 ms 40 ms 80 ms
IPv4 16 kbps 8 kbps 4 kbpsIPv6 24 kbps 12 kbps 6 kbps
34
Error Control
Type of errors Bit errors (rare due to high quality fiber)
Packet loss due to congestion
Retransmission Most widely used in the current infrastructure (TCP, etc.) Receiver send ACK or NACK Computationally inexpensive Introduces delay: May not be tolerated in real-time application Requires backchannel Does not work for multicast
Eventually all the packets need to be transmitted NACK impulsion problem
35
Error Control
Forward Error Correction (FEC) Requires processing at the host Increases the bandwidth => May worsen the problem during congestion Interlaced protection for packet loss (introduces packetization delay)
Reed-Solomon codes Maximum error correction capability Low computational complexity
Information Packet
Parity Packet
Information Packet
Information Packet
Parity Packet
36
Flow Control
Rate of insertion packets into the network
Each coded layer is transported by a different flow
How many flow to subscribe
On congestion => Drop a layer (easy to detect)
On spare capacity => Add a layer (hard to detect)
Join experiments => May overload the network for large multicast groups
Layer 1
Layer 2
Layer 3
Layer 4
37
Flow Control
The most dominant traffic of Internet is TCP (WWW)
TCP is adaptive flow: Reduces rate when network is congested
Fair sharing with TCP is required: Match the rate of TCP
k: [0.7,1.3]
MSS: Maximum segment size
RTT: Round trip time
p: Packet loss rate
These parameters must be measured
pRTT
MSSkT
38
Conclusions
Internet video is very challenging
Large overhead of Internet protocols
Best effort service
Low delay for interactive applications
Error control
Multicasting
Lack of prioritization
The situation is improving
IPv6 enables prioritization
Resource Reservation Protocol