multimedia systems networking
Post on 13-Jan-2016
61 Views
Preview:
DESCRIPTION
TRANSCRIPT
MULTIMEDIA SYSTEMS IREK DEFEE
MULTIMEDIA SYSTEMSNETWORKING
MULTIMEDIA SYSTEMS IREK DEFEE
SERVER CLIENTNETWORK
SOFTWARE
MULTIMEDIA SYSTEM
MULTIMEDIA SYSTEMS IREK DEFEE
• WE CONSIDER MULTIMEDIA SYSTEMS BUILT-UP ON LARGE SCALE: REACHING PRACTICALLY
EVERYBODY, LIKE TV, RADIO AND
TELEPHONE TODAY, THAT IS THOUSANDS AND MILIONS OF USERS
• MULTIMEDIA MEANS BROADBAND
STREAMING AND INTERACTIVITY
MULTIMEDIA SYSTEMS IREK DEFEE
MAIN PROBLEMS IN NETWORKING • NETWORKS FOR MULTIMEDIA REQUIRE:- SUFFICIENT/CONTROLLED BANDWIDTH - RESERVATION OF NETWORK
RESOURCES (WE CAN NOT LOSE DATA)- STREAMING - FAST INTERACTIVITY DELIVERY CAN BE WIRED OR WIRELESS:
WIRED - USES PHYSICAL LINKS
WIRELESS – USES RADIO
MULTIMEDIA SYSTEMS IREK DEFEE
• A MODEL OF INTERACTIVE SYSTEM IS INTERNET
• A MODEL OF BROADBAND STREAMING ARE TV, RADIO
• MULTIMEDIA SYSTEMS WOULD
BE SOMETHING LIKE INTEGRATION
OF THEM BOTH
• SOMETHING LIKE BECAUSE WE CAN NOT PREDICT IN DETAIL WHAT IT MIGHT BE
MULTIMEDIA SYSTEMS IREK DEFEE
• COMMUNICATION SERVICE TYPES- BROADCAST: SINGLE SOURCE, MANY USERS – LIKE TV, 1-WAY DISTRIBUTION- MULTICAST: LIKE BROADCAST BUT USERS SIGN TO A SESSION, ONE
WAY CONNECTION- RETRIEVAL, UNICAST –
ASYMMETRIC CONNECTION, USERS RECEIVE MUCH MORE THAN SEND, E.G. WWW
MULTIMEDIA SYSTEMS IREK DEFEE
• COMMUNICATIVE SERVICE –
SYMMETRIC 2-WAY CONNECTION
LIKE TELEPHONE
IN MULTIMEDIA SYSTEMS WE ARE MOSTLY INTERESTED IN THE RETRIEVAL AND COMMUNICATIVE SERVICES WITH HIGH BIT RATE
MULTIMEDIA SYSTEMS IREK DEFEE
• STREAMING
STREAMING MEANS DATA ARE SEND
SYNCHRONOUSLY IN TIME AND WILL
REACH USER PRESERVING THEIR TIME
ORGANIZATION. NO DATA LOSS OR
DELAYS ARE ALLOWED.
EXAMPLE: TELEPHONE NETWORK
IN MULTIMEDIA STREAMING IS
ABSOLUTELY NECESSARY
MULTIMEDIA SYSTEMS IREK DEFEE
• THIS IS VERY MUCH RELATED TO THE PROBLEM OF NETWORK RESOURCE MANAGEMENT AND
CONTROL• THERE ARE TWO TYPES OF NETWORKS- CONNECTION-ORIENTED- CONNECTIONLESS- IN CONNECTION-ORIENTED
NETWORK RESOURCES ARE ALLOCATED BEFORE
MULTIMEDIA SYSTEMS IREK DEFEE
THE DATA TRANSFER IS STARTED.
IN CONNECTIONLESS NETWORK
RESERVATION IS NOT DONE: DATA
MAY BE TRANSFERRED WITHOUT ANY
GUARANTEE OF RESOURCE
ALLOCATION, OR WITH SOME
GUARANTEE ONLY.
MULTIMEDIA SYSTEMS IREK DEFEE
• HOW STREAMING CAN BE ENSURED IN NETWORKING?
THERE ARE THREE BASIC TYPES OF NETWORKING
- CIRCUIT SWITCHING - CONNECTION IS ’SWITCHED’ BEFORE SENDING DATA (”PHONE”)
- PACKET SWITCHING – DATA ARE
ORGANIZED IN PACKETS. EACH PACKET CARRIES ADDRESSES
MULTIMEDIA SYSTEMS IREK DEFEE
- CELL SWITCHING – BETWEEN CIRCUIT AND PACKET SWITCHING
”PACKETS ARE SWITCHED”
(TECHNOLOGY CALLED ATM –
ASYNCHRONOUS TRANSFER MODE)
ALL THESE SYSTEMS HAVE VARIOUS
POSIIVE AND NEGATIVE ASPECTS FOR MULTIMEDIA
MULTIMEDIA SYSTEMS IREK DEFEE
IN CIRCUIT SWITCHING NETWORK RESOURCES ARE FULLY RESERVED FOR STREAM TRANSMISSION
IN PACKET SWITCHING PACKETS ARE
FLOWING IN LITTLE CONTROLLED WAY
MAKING STREAMING PROBLEMATIC
IN CIRCUIT SWITCHING INTERACTIVITY IS SLOW (TIME FOR CONNECTION NEEDED)
IN PACKET SWITCHING INTERACTIVITY IS
FAST
MULTIMEDIA SYSTEMS IREK DEFEE
• EXAMPLES:- TELEPHONE SYSTEM – SWITCHING
(connection establishment) IS DONE IN EXCHANGES, SWITCHBOARDS OR SWITCHING CENTERS
- INTERNET - PACKETS ARE DIRECTED IN ROUTERS WHICH READ THEIR
ADDRESSES AND DIRECT THEMThe difference is that only circuit switchingfully guarantees streaming but it is expensive
MULTIMEDIA SYSTEMS IREK DEFEE
• ADVANTAGES OF CIRCUIT SWITCHING:
- GUARANTEED DATA DELIVERY
- STREAMING AND TIMING OF DATA
NO PROBLEM
- DISADAVANTAGES:
- CIRCUIT ESTABLISHMENT (”call”)
TAKES TIME
- EXPENSIVE INFRASTRUCTURE
MULTIMEDIA SYSTEMS IREK DEFEE
• PACKET SWITCHING
EXAMPLE: INTERNET
DATA ARE ORGANIZED IN PACKETS
PACKETS CARRY HEADERS WITH
SOURCE AND DESTINATION ADDRESS
PACKETS ARE ROUTED IN THE
NETWORK BASING ON THE ADDRESSESS
MULTIMEDIA SYSTEMS IREK DEFEE
• ADVANTAGES OF PACKET SWITCHING
- SIMPLICITY, EVEN A PC CAN BE ROUTER
- PACKETS CAN BE ROUTED IMMEDIATELY
- PACKETS CAN BE SEND USING DIFFERENT ROUTES, OR STORED
EASY ADAPTATION TO THE AMOUNT OF TRAFFIC
- IDEAL FOR TIME NON-CRITICAL DATA
MULTIMEDIA SYSTEMS IREK DEFEE
• DISADVANTAGES:
- NO GUARANTEE FOR
PACKET DELIVERY
- NO GUARANTEE FOR RESOURCES
NO GUARANTEE FOR PACKET DELIVER ON-TIME, NO STREAMING
MULTIMEDIA SYSTEMS IREK DEFEE
• EXAMPLE – WWW. WE GET INSTANTRESPONSE FROM THE PS INTERNETIF THE INTERNET WOULD BE CS, WEWOULD NEED TO WAIT FORCONNECTION ESTABLISHMENT.BUT THERE IS NO RESOURCERESERVATION FOR STREAMS IN PSA SOLUTION FOR THIS PROBLEMCOULD BE INTELLIGENT PACKETSWITCHING AND NETWORKING
MULTIMEDIA SYSTEMS IREK DEFEE
• IN INTELLIGENT PS PACKETS HAVE
PRIORITIES ASSIGNED, MULTIMEDIA
PACKETS GET HIGH PRIORITY SO
THEY WILL NOT BE DISTURBED BY
OTHER PACKETS
IF THE NETWORK WOULD HAVE ENOUGH CAPACITY, MULTIMEDIA PACKETS WILL FLOW IN STREAMS
(if there is not enough capacity, no help)
MULTIMEDIA SYSTEMS IREK DEFEE
• THUS PACKET SWITCHING CAN BE IN PRINCIPLE BETTER BECAUSE OF STREAMING AND FAST INTERACTIVITY
• CURRENT NETWORKING IS THUS EVOLVING IN THIS DIRECTION
(INTERNET IS BECOMING VERY
POWERFUL)
MULTIMEDIA SYSTEMS IREK DEFEE
CELL SWITCHING OVERVIEW • THIS TECHNOLOGY IS REALIZED IN
ONE PARTICULAR FORM CALLED • ATM – ASYNCHRONOUS TRANSFER• MODEIT IS A COMBINATION OF CIRCUITSWITCHING AND PACKET SWITCHINGPACKETS HAVE CONSTANT, VERYSHORT LENGTH – 48DATA+5HEADER=53BYTES/PACKET
MULTIMEDIA SYSTEMS IREK DEFEE
• ATM ENABLES COMBINATION OF
CIRCUIT AND PACKET SWITCHING
• BASIC CONCEPTS ARE VIRTUAL
CHANNEL AND VIRTUAL PATH
H PAYL. H PAYL. H PAYL. H
FLOW OF ATM CELLS H – HEADER 5 BYTESPAYLOAD 48 BYTES
MULTIMEDIA SYSTEMS IREK DEFEE
• HEADERS ESTABLISH WHERE THE
PACKETS ARE TRANSMITTED
GFC VPI VCI PT CL HEC
STRUCTURE OF ATM CELL HEADER:
GFC - GENERIC FLOW CONTROLVPI - VIRTUAL PATH IDENTIFIERVCI - VIRTUAL CIRCUIT IDENTIFIERPT - PAYLOAD TYPECL - CALL LOSS PRIORITYHEC – HEADER ERROR CHECK
4 8 16 3 1 8 bits
MULTIMEDIA SYSTEMS IREK DEFEE
• FOR MULTIMEDIA DATA
ONE VP COULD BE ALLOCATED
AND SEVERAL VC FOR E.G.
VC=1 VIDEO
VP=5 VC=2 AUDIO
VC=3 TEXT
MULTIMEDIA SYSTEMS IREK DEFEE
• IN THE ATM NETWORK THERE ARE
SWITCHES WHICH DIRECT PACKETS
ACCORDING TO THEIR VP AND VC
SWITCH 1 SWITCH 2VP=1
VC=5VP=6VC=13
VP=4VC=2
VP AND VC BETWEEN THE SWITCHES (THERE CAN BE MANY OF THEM HAS TO BE ASSIGNED TO ENABLE END-TO-END TRANSMISSION
MULTIMEDIA SYSTEMS IREK DEFEE
• HOW THE ASSIGNMENT IS DONE?
IT CAN BE DONE BY HAND
OR IT CAN BE DONE AUTOMATICALLY USING SPECIAL
SYSTEM FOR MAPPING END POINT
NUMBERS TO VP AND VC OF
INTERMEDIATE SWITCHES
MULTIMEDIA SYSTEMS IREK DEFEE
• IN ATM THIS IS SOLVED IN A WAY
WHICH IS SIMILAR TO TELEPHONE
NETWORKS
IT IS POSSIBLE TO ESTABLISH
CONNECTION BETWEEN ANY
END-POINTS BECAUSE THEY HAVE
UNIQUE NUMBERS
MULTIMEDIA SYSTEMS IREK DEFEE
• THE ATM CALLS CAN SPECIFY
BANDWIDTH AND OTHER PARAMTERS
THERE ARE SEVERAL CLASSES OF
SERVICES
AAL – ATM ADAPTATION LAYER
AAL1- CONSTANT BITRATE WITH
TIMING
MULTIMEDIA SYSTEMS IREK DEFEE
• AAL2 – VARIABLE BIT RATE WITH TIMING
• AAL3/4- VARIABLE BIT RATE WITH
NO TIMING
• AAL5 VARIABLE BIT RATE WITH NO TIMING, CONNECTIONLESS
• FOR THESE AAL’S IT IS SPECIFIED HOW DIFFERENT PROTOCOLS ARE
MAPPED ON THEM
MULTIMEDIA SYSTEMS IREK DEFEE
• FOR EXAMPLE MPEG-2 TRANSPORT
STREAM IS MAPPED INTO
2 X188 MPEG-2 PACKETS -> 8 ATM
CELLS
AAL2 IS IDEAL FOR MULTIMEDIA
BUT IT IS DIFFICULT TO IMPLEMENT
(TIMING)
LARGE ATM NETWORKS ONLY RARELY ARE IMPLEMENTED
MULTIMEDIA SYSTEMS IREK DEFEE
• BUT IN NEW THIRD GENERATION WIRELESS NETWORKS
- WIRELESS LAN
- WIRELESS CELLULAR
ATM AND AAL2 (WIRELESS) IS SPECIFIED AND IMPLEMENTED
THUS, THEY MAY OPERATE IN THE
ATM MODE
MULTIMEDIA SYSTEMS IREK DEFEE
PACKET SWITCHING – IP INTERNET
IN STANDARD PACKET SWITCHING
THERE IS NO NETWORK RESOURCE
RESERVATION BEFORE THE
PACKETS ARE SEND.
THE BASIC METHOD OF
TRANSFERRING PACKETS ON THE
INTERNET IS THE
UDP – USER DATAGRAM PROTOCOL
MULTIMEDIA SYSTEMS IREK DEFEE
IPv4 Header Structure
Ip version Header_length Tos Total length
Identification Flags and fragmentation
TTL Protocol Header check sum
32 bit Source Ip address
32 bit Destination address
Options(if any)
Data
MULTIMEDIA SYSTEMS IREK DEFEE
• UDP/IP PACKETS ARE ROUTED ACCORDING TO THEIR ADRESSES,
ONCE SENT, THEY ARE UNCONTROLLED
THIS CAN BE VERY UNRELIABLE
BUT ON RELIABLE AND HIGH
BANDWIDTH LINKS IT CAN BE
SUFFICIENT FOR MULTIMEDIA
MULTIMEDIA SYSTEMS IREK DEFEE
• THE TCP/IP PROTOCOL IS USED TOPROVIDE INFORMATION THAT PACKETS REACHED THEIR
DESTINATION, THERE IS CONFIRMATION SEND ABOUT THEIR
RECEPTION• IF THERE IS NO CONFIRMATION,
PACKETS ARE RESEND • TCP/IP IS GOOD FOR NON-TIME
CRITICAL DATA (EMAIL)
MULTIMEDIA SYSTEMS IREK DEFEE
IP NETWORKING FOR
MULTIMEDIA • FOR MULTIMEDIA DATA PACKET
SWITCHING INTERNET CAN BE VERY BAD
• ON THE OTHER HAND INTERNET IS CHEAP, AVAILABLE AND UBIQUITOUS
MULTIMEDIA SYSTEMS IREK DEFEE
FOR SOLVING PROBLEMS WITH MULTIMEDIA DATA OVER IP PACKET
NETWORK, THERE ARE THREE APPROACHES:
- INTRODUCING RESOURCE RESERVATION, PROTOCOL CALLED
RSVP – RESOURCE RESERVATION PROTOCOL AIMS TO DO IT
- INTRODUCE PACKET PRIORITY LABELLING (DONE IN ROUTERS)
MULTIMEDIA SYSTEMS IREK DEFEE
• BOTH WOULD REQUIRE IMPLEMENTATION IN ROUTERS
AND ARE COMPLICATED
• THIRD APPROACH - SIMPLER
- TRYING TO FOLLOW HOW PACKETS
ARE FLOWING THROUGH THE NETWORK AND SEND INFORMATION
TO THE SENDING SIDE
MULTIMEDIA SYSTEMS IREK DEFEE
• REAL-TIME PROTOCOL RTP
IN THIS PROTOCOL EACH PACKET
GETS TIME STAMP AND NUMBER
WHEN IT IS SEND. ON THE RECEIVING
SIDE IT IS POSSIBLE TO DETECT IF
PACKETS ARE LOST AND WHAT ARE
THE PACKETS DELAYS. THECOMPLETE
PACKET LOOKS LIKE THIS
MULTIMEDIA SYSTEMS IREK DEFEE
• THERE IS A PROTOCOL CALLED
RTCP – REAL TIME CONTROL PROTOCOL WHICH COLLECTS STATISTICAL INFORMATION ABOUT
PACKET DELAYS AND LOSS AND SENDS IT TO THE SENDING SIDE
WHICH MAY REACT IN SOME WAY
MULTIMEDIA SYSTEMS IREK DEFEE
• THE RTP/RTCP COMBINATION IS
SIMPLE AND REALISTIC FOR THE
CURRENT INTERNET:
- DOES NOT CHANGE ANYTHING
IN THE SYSTEM
- PROVIDES CONCISE REPORTING
- SENDING SIDE MAY ESTIMATE
QUALITY OF CONNECTION AND REACT E.G. BY REDUCING BITRATE
MULTIMEDIA SYSTEMS IREK DEFEE
RTP INTRODUCTIONprovides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio or video. does not address resource reservation and does not guarantee quality-of-service for real-time services.
independent of the underlying transport and network layers.
RTP packet consists fixed RTP header, a possibly empty list of contributing sources and the payload data
MULTIMEDIA SYSTEMS IREK DEFEE
RTP header
T
TIMESTAMP
SYNCHRONIZATION SOURCE IDENTIFIER (SSRS)
V P MCC PTX SEQUENCE NUMBER
CONTRIBUTING SOURCE IDENTIFIERS (CSRC)...
MULTIMEDIA SYSTEMS IREK DEFEE
RTP header
• Version (2 bits) identifies the version of RTP
• Padding (1 bit) packet contains one or more additional
padding octets • Extension (1 bit) the fixed header must be followed by one
header extension
• CSRC Count (4 bits) contains the number of CSRC identifiers that follow the fixed header.
V P CCX
MULTIMEDIA SYSTEMS IREK DEFEE
RTP header
• Marker (1 bit) the packet contains marker bits
• Payload Type (7 bits) identifies the format of the RTP payload
• Sequence number (16 bits) increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence.
M PT SEQUENCE NUMBER
MULTIMEDIA SYSTEMS IREK DEFEE
RTP header
• Timestamp (32 bits) reflects the sampling instant of the first octet in the RTP data packet
• SSRC (32 bits) field identifies the synchronization source.
• CSRC list (0 to 15 items, 32 bits each) identifies contributing sources for the payload. The number of identifiers is given by the CC field.
TIMESTAMP
SYNCHRONIZATION SOURCE IDENTIFIER (SSRS)
CONTRIBUTING SOURCE IDENTIFIERS (CSRC)...
MULTIMEDIA SYSTEMS IREK DEFEE
RTP Applications
• Designed for multiparticipant multimedia conferences– Storage of continuos data– Interactive distributed simulation– Control and measurement applications
MULTIMEDIA SYSTEMS IREK DEFEE
Implementation of RTP
• Typically integrated into application processing• No direct coupling at the RTP level between
different media– For example, audio and video are in different sessions
– Enables the choice of receiving only audio signal
• With UDP, RTP uses an even port number and the corresponding RTCP stream uses the next higher odd port number
MULTIMEDIA SYSTEMS IREK DEFEE
RTP/UDP/IP
MULTIMEDIA SYSTEMS IREK DEFEE
Application dependent protocol
• Complete implementation of RTP for a particular application requires a profile and a payload format (e.g., media encodings) specification
MULTIMEDIA SYSTEMS IREK DEFEE
Profile
• Defines – Mapping of a set of payload formats to payload type
values– Extensions into the RTP and RTCP headers
• RTP Profile for – Audio and Video Conferences with Minimal Control
(IETF 1890)– Source Authentication and Non-Repudiation of Audio
and Video Conferences (Internet Draft)– Interactive media (proposed in journal paper)
MULTIMEDIA SYSTEMS IREK DEFEE
Payload Types• RFCs
– H.261 and H.263 video streams; Redundant Audio Data; MPEG1/MPEG2 Video; Bundled MPEG; JPEG-compressed Video; Generic Forward Error Correction
– MPEG-4 IN PREPARATION
• Internet Drafts:– MPEG-4 Streams; Interleaved Media; DV Format Video;
MPEG-2 AAC Streams; DTMF Digits, Telephony Signals; Text Conversation; Real-Time Pointers; DV Audio; MP3 Audio, Shared Multicast Virtual Worlds (SMVW)
• Also proprietary streams can be used
MULTIMEDIA SYSTEMS IREK DEFEE
RTCP
• RTP control protocol (RTCP)
• Periodic transmission of control packets to all participants in session
MULTIMEDIA SYSTEMS IREK DEFEE
Features of RTCP
• Feedback on the quality of data distribution– Evaluate whether problems are local or global
– Transmission rate can be changed according to network situation
• Persistent transport-level identifier for an RTP source called the canonical name or CNAME– Keep track of participants
– Associate multiple data streams from a given participant in a set of related RTP sessions
MULTIMEDIA SYSTEMS IREK DEFEE
Features of RTCP (2)
• Observe the number of participants• Scaling of control information transmission rate in
order for RTP to suit also to a large number of participants– Maximum of 5 % of session bandwidth allocated to
RTCP
• Transport minimal session control information– For example display participant identification in the
user interface
MULTIMEDIA SYSTEMS IREK DEFEE
RTCP Packet Format
• SR: Sender report, for transmission and reception statistics from participants that are active senders
• RR: Receiver report, for reception statistics from participants that are not active senders
• SDES: Source description items– describes participant using ASCII text
• BYE: Indicates end of participation • APP: Application specific functions
MULTIMEDIA SYSTEMS IREK DEFEE
Compound RTCP Packet
• Typically multiple RTCP packets are sent together as a compound RTCP packet
• In the following format:– Encryption prefix if packet is encrypted– SR or RR– SDES– Other RTCP packet types
MULTIMEDIA SYSTEMS IREK DEFEE
Sender report (SR)
• First section of SR– Version, padding, reception report count,
packet type, and SSRC– Similar to the beginning of RTP header
MULTIMEDIA SYSTEMS IREK DEFEE
Second section of SR
• NTP timestamp: 64 bits – Real time when report was sent – Used in combination with timestamps returned in
reception reports to measure round-trip propagation to those receivers
• RTP timestamp: 32 bits – Same time as the NTP timestamp but with the same
random offset as the RTP timestamps in data packets– Used by media- independent receivers to estimate the
nominal RTP clock frequency
MULTIMEDIA SYSTEMS IREK DEFEE
Second section of SR (2)
• Sender's packet count: 32 bits – Total number of RTP data packets transmitted
by the sender since starting transmission
• Sender's octet count: 32 bits – Total number of payload octets transmitted in
RTP data packets by the sender since starting transmission
– Used to estimate the average payload data rate
MULTIMEDIA SYSTEMS IREK DEFEE
Third section of SR
• SSRC_n (source identifier): 32 bits – SSRC identifier of the source to which the information
in this reception report block pertains
• Fraction lost: 8 bits – Fraction of RTP data packets from source SSRC_n lost
since the previous SR or RR packet was sent
• Cumulative number of packets lost: 24 bits – Total number of RTP data packets from source
SSRC_n that have been lost since the beginning of reception
MULTIMEDIA SYSTEMS IREK DEFEE
Third section of SR (2)
• Extended highest sequence number received: 32 bits – Low 16 bits contain the highest sequence number
received in an RTP data packet from source SSRC_n– Most significant 16 bits extend sequence number with
the corresponding count of sequence number cycles
• Interarrival jitter: 32 bits – Estimate of the statistical variance of the RTP data
packet interarrival time– J=J+(|D(i-1,i)|-J)/16
MULTIMEDIA SYSTEMS IREK DEFEE
Third section of SR (3)
• Last SR timestamp (LSR): 32 bits – NTP timestamp received as part of the most
recent RTCP sender report (SR) packet from source SSRC_n
• Delay since last SR (DLSR): 32 bits – Delay between receiving the last SR packet
from source SSRC_n and sending this reception report block
MULTIMEDIA SYSTEMS IREK DEFEE
Sender Report
• Profile may define extensions to SR
• Format of the receiver report (RR) packet is the same as that of the SR packet except sender information is omitted
MULTIMEDIA SYSTEMS IREK DEFEE
Why RRs and SRs?• Sender may modify its transmission rate based on the
feedback• Receivers can determine whether problems are local,
regional or global• Network managers may use profile-independent
monitors that receive only RTCP packets to evaluate the performance of their networks for multicast distribution
• Cumulative counts are used to make measurements over both short and long time periods, and to provide resilience against the loss of a report
MULTIMEDIA SYSTEMS IREK DEFEE
Why RRs and SRs? (2)
• Since timestamp is independent of the clock rate for the data encoding, it is possible to implement encoding- and profile-independent quality monitors
• A third-party monitor can calculate the average payload data rate and the average packet rate over an interval without receiving the data
• Jitter measure may indicate congestion before it leads to packet loss– necessary to analyze a number of reports from one
receiver over time or from multiple receivers
MULTIMEDIA SYSTEMS IREK DEFEE
SDES: Source description RTCP packet
• CNAME: Canonical end-point identifier SDES item– CNAME should be fixed for a given participant in order
to provide a binding across multiple media tools
• NAME: User name, EMAIL,PHONE,LOC: Geographic user location, TOOL: Application or tool name
• NOTE: Notice/status SDES item– Intended for transient messages describing the current
state, e.g., "on the phone, can't talk"
• PRIV: Private extensions SDES item
MULTIMEDIA SYSTEMS IREK DEFEE
RTP Translators and Mixers
• Firewalls and low bandwidth connections• Connects transport-level "clouds"
– Each cloud is defined by a common network and transport protocol, multicast address or pair of unicast addresses, and transport level destination port
• May add or removes encryption• May change encoding of data or underlying
protocols
MULTIMEDIA SYSTEMS IREK DEFEE
Translators• Translator passes through the data streams from
different sources separately• Forwards RTP packets with their SSRC identifier
intact• Receivers cannot detect the presence of a translator• Replicator from multicast to unicast• Application-level filter in firewalls• May, for example, filter non-CNAME SDES
information if bandwidth is limited
MULTIMEDIA SYSTEMS IREK DEFEE
Mixer• Intermediate system that receives streams of RTP
packets from sources, combines them and then forwards the combined stream
• Makes timing adjustments among the streams and generates its own timing for the combined stream
• Packets marked with the mixer's own SSRC identifier
• Generates its own reception reports for sources in each cloud and sends them out only to the same cloud
MULTIMEDIA SYSTEMS IREK DEFEE
Pros and cons of mixers
• Output bandwidth is limited to that of one source even when multiple sources are active on the input side
• Receivers on the output side don't have any control over which sources are passed through or muted
• Receivers can't do inter-media synchronization of the original streams
MULTIMEDIA SYSTEMS IREK DEFEE
Loop detection
• Translators and mixers may create transmission loops
• A loop of data packets to a multicast destination can cause severe network flooding. – All mixers and translators are required to implement a
loop detection algorithm
• SSRCs are kept globally unique for each RTP session, they can be used to detect loops that may be introduced by mixers or translator
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP (The Real Time Streaming Protocol)
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP introduction
• application level protocol for control over the delivery of data with real-time properties.
• provides an extensible framework to enable controlled, on demand delivery of real-time data, such as audio and video.
• protocol provides means for choosing delivery channels
such as UPD, multicast UDP and TCP
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP introduction
• provides means for choosing delivery mechanisms based on RTP
• does not depend on the mechanism used to carry continuous media.
• similar in syntax and operation to HTTP
• control may occur on TCP connection while the data flow via UDP.
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message
• ANNOUNCE • DESCRIBE• GET_PARAMETER• OPTIONS• PAUSE• PLAY• RECORD• REDIRECT• SETUP• SET_PARAMETER• TEARDOWN
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message (describe)
• DESCRIBE method retrieves the description or a presentation or media object identified request URL from the server.
– DESCRIBE rtsp://server.example.com/foo RTSP/1.0– Cseg: 312– Accept: application/sdp, application/rtsl,
application/mheg
• response– RTSP/1.0 200 OK– Cseg: 312– Date: 23 Jan 1999 15:35:06 GMT– Content-Type: application/sdp– Content-Length: 376
– SDP part
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message (announce)
• two purposes:
• 1. When sent from client to server, ANNOUNCE posts the description of a presentation or media object by the request URL to a server.
• 2. When sent from server to client, ANNOUNCE updates the session description in real-time.
• The whole presentation description should be sent, rather than just the additional components.
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message (options)
• query the server about options it may or may not support
• OPTIONS * RTSP/1.0• Cseq: 1• Require: implicit-play• Proxy-Require: gzipped-message
• Response could be• RTSP/1.0 200 OK• cseg: 1• Public: DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE
• or• RTSP/1.0 551 Option not supported• cseq:1• unsupported: implicit-play
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message (setup)
• specifies the transport mechanism to be used for the streamed media
• change transport parameter
• SETUP rtsp://example.com/foo RTSP/1.0• Cseg: 302• Session: 47112344• Transport: RTP/AVP;unicast;client_port=…
• If the change is not allowed response is• RTSP/1.0 455 Method Not Valid In This State
•Cseq: 302
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message (play)
• tells the server to start sending data • PLAY request may be queued• PLAY rtsp://example.com/foo RTSP/1.0• Cseg: 832• Session: 47112344• Range: npt=10-15
• PLAY rtsp://example.com/foo RTSP/1.0• Cseg: 833• Session: 47112344• Range: npt=20-25
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message
• The PAUSE request causes the stream delivery to be
interrupted temporarily.
• The TEARDOWN request stops the stream delivery freeing the resource associated with it.
• The REDIRECT request informs the client that it must connect to another server location.
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP request message
• The GET_PARAMETER request retrieves the value of a parameter of presentation or stream.
• The SET_PARAMETER requests to set the value of a
parameter for a presentation or stream.
• The RECORD request initiates recording a range of media
data according to the presentation description.
MULTIMEDIA SYSTEMS IREK DEFEE
RTSP example
DESCRIBE
200 OK
SETUP (video)
200 OK
SETUP (audio)
200 OK
PLAY
200 OK
PAUSE
460 (error message)
PAUSE
200 OK
MULTIMEDIA SYSTEMS IREK DEFEE
HIGHER LEVEL PROTOCOLS
DEVELOPED BY MMUSIC GROUP OF IETF INCLUDE SDP AND SIP PROTOCOLS IN ADDITION TO RTP/RTCP/RTSP
MULTIMEDIA SYSTEMS IREK DEFEE
Introduction
.
• MMUSIC (Multiparty Multimedia Session Control)
• Internet standard track protocols to support Internet teleconferencing sessions.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP (Session Initiation Protocol)
• application-layer control protocol for creating, modifying and terminating sessions with one or more participants
• Internet multimedia conference, Internet telephone calls and multimedia distribution.
MULTIMEDIA SYSTEMS IREK DEFEE
SDP (Session Description Protocol)
• describing multimedia sessions
• session announcement, session invitation, and other forms of multimedia session initiation.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP introduction
• supports name mapping and redirection services
• allows implementation of ISDN and Intelligent Network telephony subscriber service.
• Internet telephony gateways that connect Public Switched Telephone Network (PSTN) parties can also use SIP to set up calls.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP introduction cont.
• an application-layer control protocol for creating, modifying and terminating sessions with one or more participants.
• invite both persons and ”robots”, such as a media storage service.
• can be used to initiate sessions as well as invite members to sessions that have been advertised and established by other means.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP introduction
• does not offer conference control services such as floor control or voting
• doesn’t prescribe how a conference is to be managed• can be used to introduce conference control protocol
• does not reserve resources• can convey to the invited system the information
necessary to do this.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP Addressing
• The SIP URL similar to a mailto or telnet URL i.e. user@host
• User part is a user name or a telephone number • the host part is either a domain name or a numeric
network address
• Sip:jorma.ollila@nokia.com
MULTIMEDIA SYSTEMS IREK DEFEE
SIP entities
• SIP client is an application program that is able to send SIP methods (invite).
• SIP server is an application that accepts SIP methods and replies with status code.
• SIP proxy is an intermediary program that acts as both a server and client for the purpose of making requests on behalf of the other client
MULTIMEDIA SYSTEMS IREK DEFEE
SIP entities• SIP registrar is a server that accepts REGISTER
requests.
• SIP redirect server is a server that accepts a SIP request, maps a address into zero or more new addresses and returns these addresses to the client.
• Does not initiate or accepts call by itself.
• Location server is used by SIP request or proxy server to obtain information about a callee’s possible locations
MULTIMEDIA SYSTEMS IREK DEFEE
SIP methods
• INVITE (call user)• ACK (confirm
connection)• CANCEL (cancel
operation)• BYE (disconnect)• REGISTER (sign up
with server)• OPTIONS (capability
query)
MULTIMEDIA SYSTEMS IREK DEFEE
SIP invite
• The INVITE method indicates that the user or service is being invited to participate in a session.
• message body contains a description of the session
• type of media it is able to receive • media it is willing to send • parameters such as network destination.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP invite• INVITE sip:watson@boston.bell-tel.com SIP/2.0• Via: SIP/2.0/UDP kton.bell-tel.com• From: A. Bell <sip:a.g.bell@bell-tel.com>• To: T. Watson <sip:watson@boston.bell-tel.com>• Call-ID: 3298420296@kton.bell-tel.com• Cseq: 1 INVITE• Subject: Mr. Watson, come here.• Content-type: application/sdp• Content-length: …
• (SDP)• v=0• o=bell 53655765 2353687637 IN IP4 128.3.4.5• s=Mr. Watson, come here.• c=IN IP4 kton.bell-tel.com• m=audio 3456 RTP/AVP 0 3 4 5
MULTIMEDIA SYSTEMS IREK DEFEE
SIP ack• confirms that the client has received a final response
to an INVITE request
• ACK is used only with INVITE request.
• ACK request may contain final session descriptions to be used by callee.
• If the ACK message body is empty, the callee uses the session description in the INVITE request.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP ack
• ACK sip:watson@boston.bell-tel.com SIP/2.0• Via: SIP/2.0/UDP kton.bell-tel.com• From: A. Bell <sip:a.g.bell@bell-tel.com>• To: T. Watson <sip:watson@boston.bell-
tel.com>• Call-ID: 3298420296@kton.bell-tel.com• Cseq: 1 ACK
MULTIMEDIA SYSTEMS IREK DEFEE
SIP cancel, bye and options
• cancels a pending request with the same header field values, but doesn’t affect a completed request.
• release the call• may be issued by either caller or callee.
• find out the capabilities of the potential callee• without ”ringing” the designated address
MULTIMEDIA SYSTEMS IREK DEFEE
SIP register
• register the address listed in the To header field with the SIP server.
• REGISTER sip:bell-tel.com SIP/2.0• Via: SIP/2.0/UDP saturn.bell-tel.com• From: sip:watson@bell-tel.com• To: sip:watson@.bell-tel.com>• Call-ID: 70710@saturn.bell-tel.com• Cseq: 1 Register• Contact: <sip:watson@saturn.bell-
tel.com:3890;transport=udp>• Expires: 7200
MULTIMEDIA SYSTEMS IREK DEFEE
SIP status messages (responses)• 1xx: Informational - request received,
continuing to process the request• 2xx: Success - the action was successfully
received, understood and accepted
• 3xx: Redirection - further action needs to be taken in order to complete the request.
• 4xx: Client error - request contains bad syntax or cannot be fulfilled at this server.
• 5xx: Server error - the server failed to fulfill an apparently valid request.
• 6xx: Global failure - the request cannot be fulfilled at any server.
MULTIMEDIA SYSTEMS IREK DEFEE
SIP status messages (responses)
• 100 Trying
• 180 Ringing
• 181 Call is being forwarded
• 182 Queued
• 200 OK
• 300 Multiple choices
• 301 Moved permanently
• 302 Moved Temporarily
• 303 See other
• 304 Use proxy
• 380 Alternative Service
• 400 Bad request
• 401 Unauthorized
• 402 Payment required
• 403 Forbidden
• 404 Not Found ...
• 480 Temporarily not available
• ….
• 500 Internal server error
• 501 Not implemented
• 502 Bad gateway
• 503 Service unavailable
• 504 Gateway time-out
• 505 SIP version not supported
• 600 Busy everywhere
• 603 Decline
• 604 Does not exist anywhere
• 605 Not acceptable
MULTIMEDIA SYSTEMS IREK DEFEE
SIP status messages (responses)
• SIP/2.0 180 Ringing• Via: SIP/2.0/UDP kton.bell-tel.com• From: A. Bell <sip:a.g.bell@bell-
tel.com>• To: <sip:watson@boston.bell-
tel.com>• Call-ID: 3298420296@kton.bell-
tel.com• Cseq: 1 INVITE• Content-length:0
• SIP/2.0 200 OK• Via: SIP/2.0/UDP kton.bell-
tel.com• From: A. Bell
<sip:a.g.bell@bell-tel.com>• To: <sip:watson@boston.bell-
tel.com>• Call-ID: 3298420296@kton.bell-
tel.com• Cseq: 1 INVITE• Contact:
sip:watson@boston.bell-tel.com• Content-type: application/sdp• Content-length: …
• v=0• o=bell 53655765 2353687637 IN
IP4 128.3.4.5• s=Mr. Watson, come here.• c=IN IP4 kton.bell-tel.com• m=audio 3456 RTP/AVP 0 3 4 5
MULTIMEDIA SYSTEMS IREK DEFEE
SIP operation in proxy modepulver.com
Proxy server
nortel.com
user@nortel.com
1. INVITE sip:jeff.pulver@pulver.com SIP/2.0 From: sip:user@nortel.com
Location Server
jeff
.pu
lve
r
pu
lve
r@v
on
1
2. INVITE sip:pulver@von1 SIP/2.0 From: sip:user@nortel.com
3. SIP/2.0 200 ok From: sip:pulver@von1
pulver@von1
4. SIP/2.0 100 OK From: sip:jeff.pulver@pulver.com
5. ACK sip:jeff.pulver@pulver.com SIP/2.0 From: sip:user@nortel.com
6. ACK sip:pulver@von1 SIP/2.0 From: sip:user@nortel.com
MULTIMEDIA SYSTEMS IREK DEFEE
SIP operation in redirect mode
1. INVITE sip:jeff.pulver@pulver.com From: sip:user@nortel.com
2. SIP/2.0 320 Moved temporarily Contact: sip:pulver@von1.pulver.com
nortel.com
user@nortel.com
pulver.com
Redirect Server
Location Server
Je
ff.p
ulv
er
pu
lve
r@v
on
1
4. INVITE sip:pulver@von1.pulver.com From: user@nortel.com
3. ACK sip:jeff.pulver@pulverr.com From: sip:user@nortel.com5. SIP/2.0 200 OK
To: user@nortel.com
6. ACK sip:pulver@von1.pulver.com From: sip:user@nortel.com
Pulver@von1
MULTIMEDIA SYSTEMS IREK DEFEE
SIP and H.323
• SIP• 60+ pages• Firewall - friendly
• SIP address = email address
• Personal mobility, IN services
• H.323• 200+ pages• complex, multiple
protocols• yet another address
• Not (yet)
MULTIMEDIA SYSTEMS IREK DEFEE
SDP introduction• describing multimedia sessions for the purposes of
session announcement, session invitation, and other forms of multimedia session initiation
• advertise multimedia conference addresses and conference tool-specific information necessary to participation.
• does not incorporate a transport protocol, and is intended to use with other protocols like SIP and RTSP.
MULTIMEDIA SYSTEMS IREK DEFEE
SDP includes:
• Session name and purpose• Time(s) the session is active (start, stop, repeat
time and so on• The media comprising the session (type, transport
protocol, format and so on)• Information to receive those media (addresses,
ports, formats and so on)• Bandwidth information• Contact information
MULTIMEDIA SYSTEMS IREK DEFEE
SDP example– v=0– o=mhadley 2890844526 2890842807 IN IP4 126.16.64.4– s=SDP seminar– i=A seminar on the session description protocol– u=http://www.cs.ucl.ac.uk/staff/M.Hadley/dsp.03.ps– e=mjh@isi.edu (Mark Hadley)– p=+44 171 380 7777– c=IN IP4 224.2.17.12/127– b=CT:128– t=2873397496 2873404696– a=recvonly– m=audio 49170 RTP/AVP 0– m=video 51372 RTP/AVP 31– m=application 32461 udp wb– a=orient:portrait
• V,O,S,T & M tags are required
MULTIMEDIA SYSTEMS IREK DEFEE
APPLICATION TO MULTIMEDIA – MPEG-4
EXAMPLE MPEG-4 SYSTEM FITS AND REQUIRES SOMETHING LIKE THE RTP/RTCP/RTP AND SDP/SIP IN MPEG-4 GENERIC NETWORKING SCHEME IS PROVIDED, ABOVE PROTOCOLS CAN BE ADAPTED TO IT THIS IS ONGOING WORK
MULTIMEDIA SYSTEMS IREK DEFEE
Streaming Media Formats
MULTIMEDIA SYSTEMS IREK DEFEE
...
Scene Description Stream
Object Descriptor Stream
Visual Stream
Visual Stream
Visual Stream
Audio Stream
Interactive Scene Description
MPEG-4 Systems Principle
STREAMSWHICH CAN BE TRANSPORTEDOVER NETWORK
MULTIMEDIA SYSTEMS IREK DEFEE
Example of MPEG-4 scene
MULTIMEDIA SYSTEMS IREK DEFEE
Object-based compression and delivery
MULTIMEDIA SYSTEMS IREK DEFEE
...
Decoding
MPEG-4 InteractiveScene
Composition andRendering
PrimitiveAV Objects
Scene DescriptionInformation
...
ElementaryStreams
FlexMuxNetwork
TransMux
...
Object Descriptor
Display andLocal UserInteraction
MPEG-4: An integrated Multimedia System
MULTIMEDIA SYSTEMS IREK DEFEE
Interactive AudiovisualScene
Elementary Streams
Composition and Rendering
Display andUser
Interaction
Transmission/Storage Medium
(RTP)UDP
IPMP4
DABMux
DeliveryLayer
DeliveryLayer
FlexMux
DMIF Application Interface
SL SLSL SL ... SyncLayerSyncLayer
Elementary Stream Interface
SceneDescriptionInformation
ObjectDescriptor
... CompressionLayer
CompressionLayer
SL
SL-Packetized Streams
(PES)MPEG-2
TS
AAL2ATM
UpchannelInformation
SLSL
...
Multiplexed Streams
FlexMuxFlexMux
AV ObjectsData
MPEG-4NETWORKING
MULTIMEDIA SYSTEMS IREK DEFEE
Example of MPEG-4 over IP
MULTIMEDIA SYSTEMS IREK DEFEE
Carriage of MPEG-4 contents over IP Networks
• MPEG-4 is a standard designated for the representation and delivery of multimedia information over a variety of transport protocols;
• MPEG-4 includes:
– Interactive scene management;
– Visual & Audio representations;
– Functional systems (multiplexing, synchronization);
– Object descriptor framework;
• Transfer method for MPEG-4 over IP:– In context with specific IP packets;
– Multiplexed in MPEG-4 Flexmux (carried in IP packets);
– Multiplexed in MPEG-2 TS (carried in IP packets);
MULTIMEDIA SYSTEMS IREK DEFEE
HTTP Transport Protocol
• It is a very simple way to stream media files;
• The wire format is the same as the file-format storage;
• Arbitrary MPEG-4 Intermedia files cannot be streamed directly using HTTP, because of the “loose” in ordering the objects, and possible cross-references within the media file;
• Live streaming for Intermedia files can also be supported using proprietary methods;
MULTIMEDIA SYSTEMS IREK DEFEE
Media Control Protocols
• In order to enable full streaming systems, a media control protocol needs to be defined to support the following features:
1. Seeking:
– Forward;
– Rewind;
– Skip.
2. Bandwidth Scalability
3. Live Streaming
MULTIMEDIA SYSTEMS IREK DEFEE
MPEG-4 Systems• In MPEG-4 Systems, the transport of streams is divided in 4 layers:
– Compression Layer: includes elementary (raw) media streams (audio, video, etc.).
– Synchronization Layer (SL): Adds a header to each unit of an elementary stream, which includes time stamps, reference to a clock elementary stream, and identification of key frames (RandomAccessPoint). This is similar to the task of RTP in IP networks. However, SL does not contain payload type (like RTP), and does not contain the Elementary Stream. In addition, an SL packet does not contain an indication of its length, so it must be framed by a lower-level protocol such as FlexMux or RTP.
– FlexMux Layer: Groups elementary streams according to common attributes, such a QoS requirements. This is very simple multiplexing protocol, but also very low overhead.
– TransxMux Layer: This is the actual transport protocol, such as RTP/UDP, MPEG-2, etc. MPEG-4 does not define its own transport protocol, but assumes the application relies on an existing transport protocol. The FlexMux Layer is optional, but the Synchronization Layer is always present.
MULTIMEDIA SYSTEMS IREK DEFEE
TCP/IP & SL-packet over UDP/IP
• TCP/IP:– Not suitable for real-time transfers (high delays and causes jitter, it was
created for reliable transmission over timely delivery);
– Does not support multicast;
– Congestion control mechanism not suitable for AV media;
• SL-packet over UDP/IP:– SL provides: Timing & Sequence numbering;
– UDP provides: Multiplexing, Length field, Checksum service;
– SL+UDP may be used like a transport protocol for AV media;
MULTIMEDIA SYSTEMS IREK DEFEE
Problems of SL/UDP/IP
• No other media stream can be synchronized with MPEG-4 data carried directly over UDP;
• The dynamic scene and session control concepts cannot be extended to non MPEG-4 data;
• No defined technique for synchronizing MPEG-4 streams from different servers;
• Streams from different servers may collide (their sources may become unresolvable at destination) in a multicast session;
• Mechanisms need to be defined to protect sensitive MPEG-4 data (RTP supports FEC);
• A feedback channel must be defined for quality control;
MULTIMEDIA SYSTEMS IREK DEFEE
RTP & RTCP
• RTP = Real-time Transport Protocol;
• RTCP = Real-time Transport Control Protocol;
• A session consists of an RTP/RTCP pair of channels
• Usually works over UDP/IP (can work with other protocols also);
• RTP supports:– Multicasting;– Payload type identification;– Time stamping;– Sequence numbering;– Delivery monitoring;– From UDP (Multiplexing, Length field, Checksum service);
MULTIMEDIA SYSTEMS IREK DEFEE
RTP
• RTP Problems:– It does not support the timely delivery of data or any other QoS guarantees;– It does not guarantee delivery, so packets may be delivered out of order or get lost
(no mechanism to recover from packet loss);
• Time Stamp (TS):– Place incoming packet in correct timing order– Initial values are picked randomly and independently for each RTP stream;– Increase in time indicated by each packet;
• Sequence Number (SN):– Detect packet loss;– Increase by one for each packet;
• For video frame that is split into multiple RTP packets: they share the same TS but use different SN;
MULTIMEDIA SYSTEMS IREK DEFEE
RTP Characteristics
• RTP can map only one source stream onto RTP session:– Multiplexing causes problems;– For scale coding, each layer must have its own RTP session and multicast group;
• Within each RTP session, source can change its data format over time;
• It allows FEC (Forward Error Correction);
• Synchronize across different media streams;
• Provide feedback on the quality of data using packet counts;
• Identify and keep track of participants;
MULTIMEDIA SYSTEMS IREK DEFEE
MPEG-4 over RTP/UDP/IP
• Easiest is to wrap the MPEG-4 SL packet in an RTP packet:– High overhead: two full headers;– RTCP may not provide enough control for the MPEG-4 stream;
• Several types of MPEG-4 payloads are being defined by the IETF for different ESs;
MULTIMEDIA SYSTEMS IREK DEFEE
RTP ES & SL payload restrictions• RTP packetization is not media aware:
– No unique scheme can be defined, need a payload definition per payload type (not practical);
– This may require the definition of new session and scene description mechanisms to deal with all the flows;
• Common restrictions:– RTP timestamp corresponds to composition time stamp (CTS) of MPEG-4 stream;– Packets should be sent in decoding order;– Streams can be synchronized using RTCP;
• Map SL stream onto RTP session:– SL header is optional;
• Reduced SL headers does not include:– Sequence number (mapped to RTP header);– Composition Time Stamp (mapped to RTP header);
MULTIMEDIA SYSTEMS IREK DEFEE
Streams & Media Control in MPEG-4
• Multiple Strems inMPEG-4:– Port consuming:
• Each AV contains one or more streams;• Each stream needs one RTP session;
– Potential solution:• Selective bundling of ESs-FlexMux -> define a multiplexed MPEG-4 payload (may have
to be defined for multiple payload types);• Generic RTP multiplexing for use with MPEG-4, under development by IETF.
• MPEG-4 Media Control:– Remote interactivity: add or delete a stream, etc.– Media control channel allows renegotiating in time the available network and
processing resources;– Must have an efficient signaling protocol that can handle such messages;
MULTIMEDIA SYSTEMS IREK DEFEE
Media Control Framework
• To allow a client and one or more servers to exchange types of control messages and also allow for peer to peer exchange between two or more clients, the framework requires several components:
– A description of the stored or live presentation;
– A set of protocols that can provide proper services for the back channel message delivery;
– A set of protocols that can allocate resources for the involved hosts and networks;
MULTIMEDIA SYSTEMS IREK DEFEE
Components of media control (1)
• Presentation Description:
– The client needs to refer to the description of a presentation that expresses the temporal and static properties of the presentation itself;
– Must include information about the media, location of the media, etc.
– Should provide multiple description instances of the same presentation so that the client can specify a given combination that fits its needs/capabilities – the client is the orchestrator of the presentation and the server is participating;
MULTIMEDIA SYSTEMS IREK DEFEE
Components of media control (2)
• Client and Server State Representations:
– Out of band signaling is used: the data streams and the control information are carried over separate channels using different protocols, each best suited to their needs and modes of operation;
– As the media streams may be modified by the end user, the server needs to a state if the streams status for each client it is serving;
– The client has to keep track of all the participating streams;
MULTIMEDIA SYSTEMS IREK DEFEE
Components of media control (3)
• Basic Media Control Messages:
– A multimedia system should have access to control messages ranging from remote VCR functions such as stop, play, fast forward and fast reverse, to messages generated in response to user actions to modify the presentation of a given object stream such as add or remove or alter, etc.
– The basic control functionality relates to presentation and stream set-up: play, stop, pause, teardown and recording;
MULTIMEDIA SYSTEMS IREK DEFEE
Real Time Streaming Protocol (RTSP)
• It is an application level protocol that provides an extensible framework to enable controlled delivery of real-time data, such as audio & video;
• It can be transported over UDP, TCP and is designated to work with RTP and HTTP;
• Provide support for bidirectional communications (frame level timing for remote video editing);
• RTSP does:– Control the transmission of media stream;– Use out-of-band signaling;
• RTSP does not:– Define compression schemes;– Define how AV is encapsulated;– Define how to buffer;
MULTIMEDIA SYSTEMS IREK DEFEE
MPEG-4 and RTSP
• From DMIF’s perspective, RTSP is an application alongside MPEG-4 systems;
• The RTSP client and server interact with the MPEG-4 systems;
• The RTSP client and server control the streams through the DAI by an RTSP-DMIF interface;
• The interface is kept very simple by limiting it to field mapping between the RTSP fields and the DAI primitive parameters;
• The RTSP client server interactions are used to control the MPEG-4 elementary streams;
MULTIMEDIA SYSTEMS IREK DEFEE
Session Initiation Protocol (SIP)
• IETF Family of Session Protocols:– Session Description Protocol (SDP);– Session Announcement Protocol (SAP);– Session Initiation Protocol (SIP);
• SIP:– Two basic components: user agent & network server;– Independent of lower layer protocols;– Extensible to be application specific;– Gaining widespread use in IP telephony;
• MPEG-4 and SIP:– Unique ability to control different media types within a single session Multiple
stream transmission in one network session;– User Agent model fits in well with an MPEG-4 Client/Server model (point-to-point
communication);
MULTIMEDIA SYSTEMS IREK DEFEE
Toolboxes for transmitting MPEG-4 over internet
• RTP transport of audio/video/… data, quality-of-service feedback;
• RTSP very simple media control of streams;
• SIP inviting people, media servers to sessions – telephony and streaming audio/video;
• HTTP, SDP retrieve media descriptions;
MULTIMEDIA SYSTEMS IREK DEFEE
Issues
• Encapsulation of MPEG-4 Sync layer packetized stream:– IEC/ISO 14496-8, framework still in revision;– Lots of issues still remain: time axis, buffer management, etc…
• Interactivity between application and End User:– Description of MPEG-4 content;– Initialization of an MPEG-4 session;
• SIP and IPC (Inter Process communication):– How to describe the dynamic process of channel (stream) setup and release?– What control information is necessary and how to transport it?
• Transport and IP QoS:– Must define a mapping mechanism among the different QoS mechanisms:
transport QoS (not available yet), network QoS, etc.
• And more…e.g. DMIF
MULTIMEDIA SYSTEMS IREK DEFEE
DMIF
• Delivery Multimedia Integration Framework
• DMIF is networking part of the MPEG-4 standard
• DMIF specifies the Delivery Layer of MPEG-4 (mostly in generic form)
MULTIMEDIA SYSTEMS IREK DEFEE
MPEG-4 Standard Structure
DAI
ESI
Delivery Aware, media unaware :DMIFFlexMux
Media Aware, delivery unaware :Audio and Visual CompressionScene Description and Object Descriptor Protocol
Media and Delivery unaware :Systems
DeliveryLayer
FlexMuxTool
Sync.Layer
CompressionLayer
Composition
Related to networking
MULTIMEDIA SYSTEMS IREK DEFEE
Why DMIF?
• many different delivery technologies, each with its own peculiarities
• different APIs for different environments (Local Files, Broadcast sources, Interactive servers through a variety of transports)
• no consolidated solution for real time multimedia streaming at certain QoS
• the introduction of QoS support in networks also impacts on charging policies
MULTIMEDIA SYSTEMS IREK DEFEE
DMIF goals
• Separate the delivery aspects from the multimedia application issues
• Guarantee permanence of multimedia application in presence of new delivery technologies
• Allow the concurrent usage of different delivery technologies (e.g. broadcast + local retrieval)
• Allow resource logging to support flexible charging policies
MULTIMEDIA SYSTEMS IREK DEFEE
DMIF
• DMIF addresses the following scenarios:
– multicast/broadcast
– local storage
– remote interactive
Broadcastsource
LocalStorage
Network
MULTIMEDIA SYSTEMS IREK DEFEE
The DMIF reference architecture
• supports the streaming of multimedia content• avoids embedding delivery dependency in a
multimedia application• allows the concurrent usage of multiple delivery
technologies• enables a plug-in approach to add new modules
– to follow technical evolution in content streaming– to allow ad-hoc delivery implementations (e.g. for
charging, admission control, Intelligent Networks …)
MULTIMEDIA SYSTEMS IREK DEFEE
The DMIF reference architecture
Sigmap
TargetDMIF
TargetApp
DNI DAI
Origin.
App
DAI
DM
IF F
ilte
r
Origin. DMIFfor Remote
srv
DNI
Sigmap Network
Origin. DMIFfor Broadcast
TargetDMIF
Target Application
Broadcastsource
Origin. DMIFfor Local Files
TargetDMIF
Target Application
LocalStorage
MULTIMEDIA SYSTEMS IREK DEFEE
MPEG-4 synchronisation, mux & transport
(MPEG-4 own Sync Layer + Flexmux approach)
DAI
ESI
Synchronisation Layer
FlexMux
SL SL SL SL SL SL
FlexMux
SL
HTML page JPG
GIF
VideoAudio
JavaclassOD
BIUBIA
DMIF
DNI
Delivery Layer Control Plane
Applic. Signal.
IP TCP
HTTP UDP
RTP
FlexMux Channels
ES Channels
TransMux Channels
RTCP
QoS Manager
MULTIMEDIA SYSTEMS IREK DEFEE
DNI
MPEG-4 synch., mux & transport (IETF
PROPOSAL, RTP CHANNEL MULTIPLEX)
IP TCP
HTTP UDP
RTP/RTP Mux
Generic MP4 ES (& SL-PDU) to
RTP mapping
HTML page JPG
GIF
Video
AudioJavaclass
ODBIU
BIA
Delivery Layer
DMIF Control Plane
Synchronisation and Protection
TransMux Channels
ES Channels
Applic. Signal.
DAI
ESI
RTCP
MULTIMEDIA SYSTEMS IREK DEFEE
Conclusions
• MPEG-4 is here in a limited fashion;
• We will see a marked growth in MPEG-4 products as chip sets become more readily available;
• Simple basic MPEG-4 service is not a problem;
• There is a lot of ongoing research on how to deliver MPEG-4 content over IP-based networks;
top related