Sudarsun S1

Audio and Video over InternetAudio and Video over Internet

Sudarsun S., M.TechChecktronix India Pvt LtdChennai 600034sudar@burning-glass.com

Sudarsun S2

Introduction Introduction What is Audio/Video over Internet? What are the benefits of AV on Internet?

– Cost of traditional communication– Availability/Mobility with Wireless– Entertainment Industry

On demand Movie, Songs Duplication Protection

– Distance does not matter!– Decentralization

What is and Why Streaming data? Real-time Communication Protocols

Sudarsun S3

AgendaAgenda

Learn the fundamentals of streaming– TCP, UDP and their limitations

– RTP, RTSP to overcome TCP limitations

– Error Correction and Detection Low Quality Piggybacking Interleaving

Understand “On demand” multimedia data– On demand cinema

Video conferencing

Sudarsun S4

Overview Overview

The big picture

Sudarsun S5

Media Data typesMedia Data types Audio : WAV, MP3, RM Video: AVI, MPG, MOV, RM Bandwidth calculation 3GPP, 3GPP2

– 3rd generation, high-speed wireless networks MPEG4

– Professional-quality audio and video streams H264 formats

– MPEG-4 standard (Part 10) used across 3G for mobile devices to iChat AV for video conferencing to HD for broadcast and DVD

Sudarsun S6

TCP & UDPTCP & UDP

Connection Oriented – Is that a Overhead ?

Connectionless – Will that guarantee timely delivery?

Maximum Effort Reliability vs Just In Time with permissible error rate

Packet Sequence – Is that an Advantage ?Summary of general deficiencies

Sudarsun S7

RTP (RFC 3550)RTP (RFC 3550)

Real-time Transport Protocol on UDPUses RTCP (special set of messages) to

exchange periodic reportsOne RTP session per media flow16 bit sequence numbers to detect packet

loss32 bit time stamp to handle jitter due to

network

Sudarsun S8

RTP FormatRTP Format

(V) Version; 2 bits (P) Padding; 1 bit.

(X) Extension; 1 bit. (CC) CSRC Count; 4 bits.

(M) Marker; 1 bit. (PT) Payload Type; 7 bits.

Sequence Number; 16 bits. Time Stamp; 32 bits.

SyncSRCId; 32 bits. ContributingSRCId List;

Sudarsun S9

How RTP Works?How RTP Works?

IP UDP RTP HDR Audio Payload

IP UDP RTP HDR Video Payload

•Uses separate sessions for audio and video

•Uses sequence number to synchronize audio/video

Sudarsun S10

RTP Vs OSIRTP Vs OSI

PCMA audio

MPEG2 video

Application

RTP RTPTransport

UDP UDP

IP IP Network

Ethernet Frame Relay Data Link

Sudarsun S11

RTCPRTCP

Supervise QoSReception reports, Sender reports, Source

descriptionReports: packets sent/lost, inter-arrival

jitterHelps modify sender transmission rate and

for diagnostics purposeRTCP bandwidth = 5% of session BWCommands: SR, RR, SrcDESc, BYE, APP

Sudarsun S12

Continuous MediaContinuous Media

Real-time (Video Conferences)– Interactive– Tight timing relationship– Error rate/Jitter to be near Zero

Streaming– Playback stored media– Error rate need not be very low– Loose timing relationship

Sudarsun S13

RTSP (RFC 2326)RTSP (RFC 2326)

Application-level protocol for control over the delivery of data with real-time properties.

Works on top of HTTP Extensible framework to enable controlled, on-

demand delivery of real-time data. Supports live data feeds and stored clips Controls multiple data delivery sessions (UDP,

multicast UDP, TCP). Delivery mechanisms based on RTP.

Sudarsun S14

Webbrowser

MediaPlayer

Server

Server

RTSP StreamingMedia

HTTP

RTSP

RTP

Audio/video content

Streaming commands

Meta files

Web server

Meta file

Sudarsun S15

RTSP MethodsRTSP Methods

OPTIONS- obtain available methods SETUP- Establish transport ANNOUNCE- alter description of media object. DESCRIBE- get low level description of media object. PLAY- start playback. RECORD- Start recording PAUSE- suspend delivery, keep state SET_PARAMETER- device or encoding control TEARDOWN- remove state.

Sudarsun S16

Error CorrectionError Correction

Piggy backing (FEC)InterleavingCombine both

Sudarsun S17

Low Quality PiggybackingLow Quality Piggybacking

Works for single packet loss, How about multi-packet loss ?

Sudarsun S18

InterleavingInterleaving

Data blackout is recovered with intermittent jitter

Sudarsun S19

Network StrategiesNetwork Strategies

Multicasting reduces amount of redundant data transferred

Streaming media delivery networks allow for multiple, geographically scattered servers (achieved either by splitting or caching)

Dedicated networks guarantee bandwidth Multiple-access transmissions over Internet—

requested info sent from several different locations simultaneously

Sudarsun S20

Streaming MediaStreaming Media

Three modes in which video information can be streamed:– Live broadcasting– On-demand streaming– Conferencing

Sudarsun S21

Live Broadcasting SystemLive Broadcasting System

SLTA: Simulated Live Transfer Agent (Rebroadcasting)

Sudarsun S22

On-demand System StructureOn-demand System Structure

Sudarsun S23

Streaming Media – On demandStreaming Media – On demand

Distribution system similar to live broadcasting system, except:– No direct connection between encoder and

server– Remote proxy servers can cache popular files– Clients allowed to rewind, fast forward, etc.– Server proxy transfers only initiated by client

Sudarsun S24

Where to Get More InformationWhere to Get More Information http://www.mediacollege.com/video/streaming/overview.

html http://www.apple.com/quicktime/technologies/ http://www.teamsolutions.co.uk/streaming.html http://archive.dstc.edu.au/RDU/staff/jane-hunter/video-

streaming.html http://www2.umist.ac.uk/isd/lwt/clickgo/the_guide/

the_guide.htm http://emoney.al.ru/capture-streaming-video-and-audio/ http://www.cit.cornell.edu/atc/itsupport/streaming.shtml http://www.yolinux.com/TUTORIALS/

LinuxTutorialRealVideoStreaming.html http://nms.csail.mit.edu/projects/oxygentv/

Sudarsun S25

LimitationsLimitations

Bandwidth limitationCongestion and Quality of networkBuffer Overhead in Portable devicesCompression and Decompression

overheadSecurity issues

– Eavesdropping– Duplication by Data Augmentation

Sudarsun S26

Some ServersSome Servers

Audio– Winamp’s Shoutcast Server

– Streamsicle Broadcast server

– RealMedia Stream server Video

– Quicktime Server

– Helix universal server

– Macromedia communication server

– Ffmpeg (http://sourceforge.net/projects/ffmpeg/)

Sudarsun S27

ConclusionConclusion

Streaming multimedia poses many challenges to video encoders– Variability of channel performance– High server workloads

Need to adapt coded bit rate to match channel capacities; must be done efficiently and on the fly

Sudarsun S28

Thank YouThank YouSudarsun S., M.TechDirector – Research and DevelopmentChecktronix India Pvt Ltdsudarsun@burning-glass.comNovember 2005