xStreamerModular Multimedia Streaming

Nicolas Staelensnicolas.staelens@intec.ugent.bewww.ibcn.intec.ugent.beINTEC Broadband Communication Networks (IBCN)Department of Information Technology (INTEC)Ghent University - IBBT

xStreamer

In-house developed modular multimedia streamer Alexis Rombaut (alexis.rombaut@intec.ugent.be)

Written in C++ Uses libraries:

libavformat/libavcodec (parsing/encoding/decoding)

live555 (RTSP)

jrtplib (RTP)

Released under General Public License (GPL) Freely available at http://xstreamer.atlantis.ugent.be/

2

3

Modular Multimedia Streaming

Inspired by Click Modular Router & DirectShow

Offers different components Performs basic functions Readers, packetizers, multiplexers, schedulers,

transmitters, receivers, writers, classifiers, analyzers

Streamer is directed graph of components

4

Modular Multimedia Streaming

Supports audio and video

Using RTP packetization: MPEG-1/2/4 Video & Audio

Using MPEG-2 Transport Streams: MPEG-1/2/4 Video & Audio H.264 AVC/SVC

5

Modular Multimedia Streaming

Multitude of supported protocols

RTP/UDP

RTSP/RTP/UDP

UDP

TCP

What can xStreamer do?

Advanced streaming server Own MPEG-2 TS multiplexer SVC streaming Differentiated streaming using classifiers

Proxy/client Proxy: convert differentiated stream into

a single stream Client: save captured stream to file

Video tool No ‘real’ streaming involved Simulate packet loss Collect tracefiles during streaming

6

Create xStreamer configuration

Configuration saved in XML-based file Describes directed graph of components and

connections between components

Graphical User Interface Visualize directed graph Drag components and draw connections Configure components

7

Example: Differentiated SVC streaming

8

Read raw H.264 video

stream

Packetize frames into packets as

defined in RFC3984

Avoid bursts by smoothing packets

over time

Classify NAL units depending

on SVC layer

Stream different layers over different

connections

Example: Proxy/Client

9

Proxy Client

xStreamer as video tool

Offline simulator No ‘real’ streaming Simulate packet loss using Classifier component

Random, Gilbert-Elliott Write resulting packet stream back to file

Tracefile generation Packetizer: video trace Transmitter: sender trace Receiver: receiver trace Classifier: sender & receiver trace

10

xStreamer – HDTV PVS creation

HDTV PVS Creation H.264/AVC Packet loss

Toolchain

1.Encoding

2.Streaming

3.Decoding & syncing

11

HDTV PVS Creation – xStreamer config

12

Packet loss simulator using Gilbert-Elliott model

HDTV PVS Creation – Tracefiles

13

HDTV PVS Creation – Tracefiles

14

Video tracefile generation

0 H 539 1 1.0666670 H 24 1 1.0666670 H 4 1 1.0666670 I 256835 180 1.0666671 P 111970 79 1.1333332 B 52428 37 1.1000003 P 43918 31 1.2000004 B 30967 22 1.1666675 P 62890 45 1.2666676 B 20660 15 1.2333337 P 69342 49 1.3000008 P 65287 46 1.3333339 P 78591 56 1.40000010 B 31715 23 1.36666711 P 78028 55 1.46666712 B 34809 25 1.43333313P 71582 51 1.533333…

<id> <slice_type> <size> <nb_packets> <pts>

HDTV PVS Creation – Tracefiles

15

Sender tracefile

0.00 id 1 udp 240.00 id 2 udp 40.00 id 3 udp 14300.00 id 4 udp 14300.00 id 5 udp 14300.00 id 6 udp 14300.00 id 7 udp 14300.00 id 8 udp 14300.00 id 9 udp 14300.00 id 10 udp 14300.00 id 11 udp 14300.00 id 12 udp 14300.00 id 13 udp 14300.00 id 14 udp 14300.00 id 15 udp 14300.00 id 16 udp 14300.00 id 17 udp 1430…

<tx_time> <pckt_id> <protocol> <size>

HDTV PVS Creation – Tracefiles

16

Receiver tracefile

0.00 id 0 udp 5390.00 id 1 udp 240.00 id 2 udp 40.00 id 3 udp 14300.00 id 4 udp 14300.00 id 5 udp 14300.00 id 6 udp 14300.00 id 7 udp 14300.00 id 8 udp 14300.00 id 9 udp 14300.00 id 10 udp 14300.00 id 11 udp 14300.00 id 12 udp 14300.00 id 15 udp 14300.00 id 16 udp 14300.00 id 17 udp 14300.00 id 18 udp 1430…

<rcv_time> <pckt_id> <protocol> <size>

2 packets lost: id 13 & 14

Combining tracefiles

17

0 H CORRECT

video trace sender trace receiver trace

0 H CORRECT0 H CORRECT0 I DAMAGED…

PVS Creation: decoding & syncing

Decode streamed raw H.264/AVC file avc-decoder based on libavcodec (ffmpeg) Only decodes correctly received frames

Align decoded and original sequence Decoded sequence misses pictures

Lost & damaged Tracefile analysis

Locate missing pictures Conceal missing pictures

Copy previous frame, gray, black, green, …

18

Conclusion

Modular architecture Logical structure Easily extendable

Advanced streaming server SVC Support Differentiated streaming

Video tool Network simulation Tracefile generation

19

Questions ?