a web services based streaming gateway for heterogeneous a/v
DESCRIPTION
TRANSCRIPT
A Web Services Based Streaming Gateway for Heterogeneous A/V
Collaboration
Hasan Bulut
Computer Science DepartmentIndiana University
Motivation Videoconferencing systems provide a framework
to send/receive audio and video streams. Streaming is a widely used media delivery
technology across Internet to receive Media-On-Demand. It enables users to receive multimedia content and provides some control functionality over the media.
To enable streaming clients to join the real-time videoconferencing sessions and receive multimedia content generated by other clients in the system.
Videoconferencing Systems
Major videoconferencing systems are: H323
H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment.
SIPThe Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences
Access Grid Enhanced Mbone A/V tools ( VIC, RAT ).
Internet 2 network ( Multicast support )
Videoconferencing vs Streaming
A streaming client connects to a streaming server, primarily using RTSP, to establish a session and receive the stream.
Streaming is primarily used for Media-On-Demand. It is also possible to make a live streaming broadcast.
Streaming media aims to improve the quality of service of the stream, by using various streaming codec to improve stream quality and a buffering mechanism to reduce the jitter.
While the delay in conferencing is around 200 msec, the delay in streaming buffer is around 2 – 15 sec.
In this paper we will explain the components of
XGSP Streaming Gateway, their functionality and messages added to the framework to achieve integration. Other components will be briefly explained.
GlobalMMCS, which implements XGSP, uses Web Services technology to integrate heterogeneous conferencing systems. Streaming Gateway has been recently added to this system and it is used internally by other components in the system.
XGSP Collaboration Framework To integrate heterogeneous systems into one
collaboration system, we need to reach the following goals:
Different kinds of application endpoints should join leave in the same collaboration session.
A common A/V signaling protocol has to be designed to support interactions between different A/V collaboration endpoints.
A common user interface should be present for all the collaboration participants using different A/V and data application endpoints.
Different A/V endpoints should collaborate in the same collaboration session.
Global Multimedia Conferencing Systems
(GlobalMMCS)The first prototype of this system includes: A XGSP media server
provides the services of bridging multicast and unicast, video-switching, video-mixing and audio-mixing to H.323, SIP as well as AG endpoints.
H.323, SIP gateways and Real Servers for A/V clients XGSP A/V Session Server
manages real-time A/V sessions, receives messages from gateways and the web server, and performs appropriate actions on the media server.
The web server provides an easy-to-use web interface for users to
join multimedia sessions and for administrators to perform administrative tasks.
GlobalMMCS Architecture
RealStreaming Clients
SIP Gateway
Media Servers;Audio Mixers,Video Mixers,Image Graber
SIP Clients
XGSP Session Server
XGSP Web Server
NaradaBrokeringPublish/Subscribe
Messaging
H.323 Gateway
Access Grid Multicast
Streaming Gateway
Helix ServerH.323
Clients
Two-way RTP-link/linksOne-way RTP-link / linksPushed streamsStreamingCommunication link / links
XGSP Streaming Gateway(XGSP–SG)
Due to the requirements and underlying technology of streaming clients, XGSP–SG demonstrates different functionalities compared to other gateways:
Only uses XGSP message formats, while other gateways only transforms XGSP signal to another form and vice versa.
Performs stream format conversion so that streaming clients can play.
Provides a mechanism to synchronize streams Streaming gateway allows streaming clients
only to receive audio and/or video streams.
XGSP–SG Components and Design
Stream EngineS
tream
C
onvers
ion
Han
dle
rSMIL File Generator
Helix Streaming Server
Stream Engine
Communication linkPushed streamsProcess generation
Streaming Gateway components
XGSP–SG Components I Stream Conversion Handler
performs the communication between Session Server and Streaming Gateway.
maintains an internal database for the streams. Database is updated when the streaming jobs are started or deleted.
initiates/stops Stream Engines for the requested stream
XGSP–SG Components II Stream Engine
converts received audio or video streams into a specified RealStreaming format and pushes the output stream to Helix Streaming Server
composed of two parts, JMF RTP Handler and HXTA Wrapper
JMF RTP Handler: transform the received packets into raw audio or video format
HXTA Wrapper: makes use of HXTA engine to encode streams into RealMedia format
XGSP–SG Components III SMIL File Generator
dynamically constructs a SMIL file using the received audio and video stream IDs
deletes SMIL file if the stream is no longer in the session
SMIL files enables clients to receive multiple streams and synchronize them
Each session in a XGSP session contains only one audio stream, which is the mixed of all audio streams received from clients.
XGSP–SG Messages I InitializeRealGateway - deletes all current
jobs in the session specified JoinStream - starts a job for the stream
specified. JoinStreamReply - reply message for
JoinStream message. OK indicates the job is successfully started, otherwise returns FAIL.
LeaveStream - stops the stream specified. LeaveStreamReply - reply message for
LeaveStream. FAIL indicates that an error occurred during the stop operation, otherwise returns OK.
XGSP–SG Messages II RealStreamEvent - It has two modes,
NewRealStream to indicate a new stream in the session and ByeRealStream to indicate the stream is no longer in the session
RealStreams - requests the list of the RealStreams available in the session
RealStreamsReply - reply message for RealStreams. Includes a list of the RealStreams in the session.
Services added to Session Server
RS Join Service - handles JoinStream messages. If stream job is successfully started, sends RealStreamEvent message (mode = NewRealStream).
RS Leave Service - handles LeaveStream messages. Sends RealStreamEvent (mode = ByeRealStream).
RS List Service - handles RealStreams messages. Replies back with RealStreamsReply listing all the available RealStreams in the session.
RS Gateway Service - handles InitializeRealGateway messages. Sends RealStreamEvent with ByeRealStream mode for each RealStream in that session.
Streaming Interface I Two types of interfaces are developed for
Streaming Gateway. Streaming Admin Interface: provides
administrative features such as starting and stopping streaming jobs.
Streaming Client Interfaces: enables clients to plays streams from an interface
Streaming Admin interface also includes Streaming Client interface functionality to enable administrator with client capabilities.
Streaming Interface II A snapshot from Streaming Admin interface
Streaming Interface III A snapshot from Streaming Client interface
Streaming Interface IV A snapshot of the stream played in RealOne
Player window
Performance Discussion for Streaming Gateway I
Number of
Stream
Frame rates of streams
involved
CPU UsageRange
MemoryUsage
1 23 fps %10 - %25 29 MB
2 23 fps, 25 fps %22- %40 34 MB
3 23 fps, 25 fps, 26 fps
%50 - %75 43 MB
4 23 fps, 25 fps, 26 fps, 16 fps
%65- %95 53 MB
OS Windows XP
CPU Intel® Pentium® 4 CPU 2.26 GHz
Memory 512 MB
JVM version 1.4.2_02
Approximate CPU and memory usage with respect to number of streams
Streaming Gateway machine specifications
Performance Discussion for Streaming Gateway II
In a session (or classroom) usually there is one presenter. Sometimes other clients in the session may be active too.
In a session most of the clients are only listeners.
In such a session providing 3 video streams and a mixed audio stream ( mixed of all audio streams received from clients in the session) is sufficient.
Conclusion and Future Work XGSP framework and, Global-MMCS enable
multiple communities to collaborate with each other.
Streaming Gateway enriches this heterogeneous system by integrating streaming media world into Global-MMCS.
Streaming Gateway is part of Global-MMCS and uses XGSP in order to achieve this integration.
In order to increase the number of streams converted, we also would like to build a streaming job scheduler, which will schedule and coordinate streaming jobs in a distributed environment.
Thank you!