nus.soc.cs5248-2010 roger zimmermann project stream h.264 (mpeg-4/avc) via rtp
TRANSCRIPT
NUS.SOC.CS5248-2010Roger Zimmermann
Project
Stream H.264 (MPEG-4/AVC) via RTP
NUS.SOC.CS5248-2010Roger Zimmermann
Goals (1)
Encode video into streamable H.264 format.
Modify the yimasplit utility, which creates data blocks containing pre-computed RTP packets with appropriate RTP headers.
Use the Yima Personal Edition (Yima PE) streaming media server code.
NUS.SOC.CS5248-2010Roger Zimmermann
Goals (2)
Server reads data blocks, schedules and sends out RTP packets.
QuickTime (QT) client decompresses and plays video.
Display video with QT client in SecondLife.
NUS.SOC.CS5248-2010Roger Zimmermann
Project Homepage
DescriptionsYima Personal Edition Source CodeDocumentation (RFCs, etc.) IVLE ForumsTA: Beomjoo Seo
NUS.SOC.CS5248-2010Roger Zimmermann
Advice
Form team (1 or 2 persons).Note: The Yima PE source code is not
very well documented.Start early!
NUS.SOC.CS5248-2010Roger Zimmermann
Introduction to Yima PE
Personal Edition Streaming Media System
NUS.SOC.CS5248-2010Roger Zimmermann
Overview
Command line server
GUI client “Split” utility to
prepare media files
RTSP communication (port 55554)
# ./yimaserver<YimaPE 1.0> begin scheduler <YimaPE 1.0> begin rtsps
NUS.SOC.CS5248-2010Roger Zimmermann
Software Source
DirectoriesServer Server codeClient Client code and GUI librarySplitter Media preparation utilityStreams Sample media (WAV file)
Remove all object files (*.o) before building the executables
NUS.SOC.CS5248-2010Roger Zimmermann
Yima PE Server
RTSP front and backend (one process)Scheduler + FLIB (one process)Qpthread v1.3.1 library for multi-
threading
Must set LP_LIBRARY_PATH to include Qpthread
Server configuration file: config Where are the media files located Name, size [bytes] and duration [sec]
NUS.SOC.CS5248-2010Roger Zimmermann
Splitter
Input: yimaintro.wav (for example)Output: BLOCKS sub-directory
Data block files: yimaintro.wav_1, yimaintro.wav_2, …
Each block is 256,000 bytes and contains 500 RTP packets (of 512 bytes each)
A sample config file is created; must copy contents to the main config file
NUS.SOC.CS5248-2010Roger Zimmermann
Server + Splitter
Server does not care about block contents, i.e., it does not know what kind of media data is stored (MPEG-1/2, WAVE, …)
Server sends RTP packets based on config info: BW = size / duration Packet-level scheduling
Need only modify splitter for MP3 media!
NUS.SOC.CS5248-2010Roger Zimmermann
Client
Operation: [List] button: reads media
entries from local Yima.cfg file [Play], [Pause], [Stop] buttons execute
RTSP commands to serverGUI was built with XForms library; it is
message-driven, with callback functions for buttons, etc.
NUS.SOC.CS5248-2010Roger Zimmermann
Client Structure
3 threadsState
machineGUI“C”
Player“P”
Network“N”
/dev/dsp
Buffer
RTP
RTSP
CommandMessageQueues, e.g.,put_cmd(CtoN, …);
Continuous Media Servers
Introduction Continuous Media Magnetic Disk Drives Display of CM (single disk, multi-disks ) Optimization Techniques Additional Issues Case Study (Yima)
What is a CM Server?
Storage Manager
NetworkNetwork
Memory
Multiple streams of audio and video should be delivered to many users simultaneously.
Some Applications
Video-on-demand News-on-demand News-editing Movie-editing Interactive TV Digital libraries Distance Learning
Medical databases NASA databases
Continuous Display Data should be
transferred from the storage device to the memory (or display) at a pre-specified rate.
Otherwise: frequent disruptions & delays, termed hiccups.
NTSC quality: 270 Mb/s uncompressed; 3-8 Mb/s compressed (MPEG-2).
Memory
Disk
Challenge: Real-Time Media
010
2030
405060
708090
100
Mb/s
Bandwidth requirements for different media types:
1 Mb/s4-6 Mb/s
31 Mb/s
50 Mb/s
20 Mb/s
100 Mb/s
High Bandwidth & Large Size
HDTV quality ~ 1.4 Gb/sUncompressed!Standard: SMPTE 292M
2-hr HDTV ~ 1260 GB
Access Time Transfer Rate Cost / Megabyte
Memory 1 ~ 5 ns > 1 GB/s ~ $0.1
Disk 5 ~ 20 ms < 40 MB/s < $0.005
Optical 100 ~ 300 ms < 5 MB/s < $0.002
Tape sec ~ min < 10 MB/s < $0.001
Streaming Media Servers Streaming media servers require a different “engine”
than traditional databases because of: Real-time retrieval and storage Large media objects
The performance metrics for streaming media servers are: The number of simultaneous displays: throughput N The amount of time that elapses until a display
starts: startup latency L The overall cost of the system: cost per stream, C
Media Types Examples of continuous media are:
Audio Video Haptics
Continuous media are often compressed. There are many different compression algorithms, for example: Motion Picture Experts Group: MPEG-1, MPEG-2, MPEG-4 Joint Photographic Expert Group: Motion-JPEG Digital Video: DV, MiniDV Microsoft Video 9, DivX, … MP3: MPEG-1 layer 3 audio Above codecs are based on
discrete cosine transform (DCT)
Others:– Wavelet-based codecs– Lossless compression
Compression
MPEG-1 180:1 reduction in both size and bandwidth requirement (SMPTE 259M, NTSC 270 Mb/s is reduced to 1.5 Mb/s).
MPEG-2 30:1 to 60:1 reduction.(NTSC ~ 4, DVD ~ 8, HDTV ~ 20 Mb/s)
Problem: loose information(cannot be tolerated by some applications: medical, NASA)
Media Characteristics Data requires a specific bandwidth:
Constant bitrate (CBR) CM
Variable bitrate (VBR) CM
Easier case: CBR
Data is partitioned into equi-sized blocks which represent a certain display time of the media
E.g.: 176,400 bytes represent 1 second of playtime for CD audio (44,100 samples per second, stereo, 16-bits per sample)
Assumed Hardware Platform Multiple magnetic disk
drives: Not too expensive
(as compared to RAM) Not too slow
(as compared to tape) Not too small
(as compared to CD-ROM) And it’s already everywhere!
Memory
Magnetic Disk Drives An electro-mechanical random access storage device Magnetic head(s) read and write data from/to the disk
Disk Drive Internals
Disk Device Comparison
Disk Seek Characteristic
)(
)(
43
21
dcc
dccSeekT
If d < z cylinders
If d >= z cylinders
rpmT ncyAvgRotLate
sec60
2
1
Disk Seek Time Model
Disk Service Time
The disk service time is dependent on several factors:
Seek time Platter diameter (e.g.,
3.5”, 2.5”, 1”) Rotational latency
Spindle speed Data transfer time
Zone-bit recording Read versus write
bandwidth
SeekncyAvgRotLateTransferService TTTT
ServiceEffective T
BBW
– TTransfer: data transfer time [s]
– TAvgRotLatency: average rotational latency [s]
– TService: service time [s]
– B: block size [MB]
– BWEffective: effective bandwidth [MB/s]
MaxTransfer BW
BT
Disk Service Time Model
Data Retrieval Overhead
B 1 KB 10 KB 100 KB 1 MB 10 MB
BWEffective 0.076 MB/s
0.74 MB/s
5.55 MB/s
15.87 MB/s
19.49 MB/s
0.38% 3.7% 27.8% 79.4% 97.5%
• Assumptions:
– TSeek = 10 ms
– BWMax = 20 MB/s
– Spindle speed: 10,000 rpm
SeekMax
Effective
TrpmBW
BB
BW
sec30
Sample Calculations
Summary
Average rotational latency depends on the spindle speed of the disk platters (rpm).
Seek time is a non-linear function of the number of cylinders traversed.
Average rotational latency + seek time = overhead (wasteful).
Average rotational latency and seek time reduce the maximum bandwidth of a disk drive to the effective bandwidth
Traditional production/consumption problem RC = Consumption Rate, e.g., MPEG-1: 1.5 Mb/s. RD = Production Rate, Seagate Cheetah X15: 40-55 MB/s. For now: RC < RD Partition video X into n blocks: X1, X2, ..., Xn
(to reduce the buffer requirement)
X2 X3X1Retrievefrom disk
Displayfrom memory
Display X1 Display X2 Display X3
Time
Continuous Display (1 disk)
Time period: time to display a block (is fixed). System Throughput (N): number of streams. Assuming random assignment of the blocks:
Maximum seek time between block retrievals Waste of disk bandwidth ==> lower throughput Tp=?, N=?, Memory=?, max-latency=?
X2 X3X1
Displayfrom Memory
Display X1 Display X2 Display X3
Retrievefrom Disk
Y3 Y4 Y5
Display Y3 Display Y4 Display Y5
See
k T
ime
Time
Round-robin Display
Using disk scheduling techniques Less seek time ==> Less disk bandwidth waste ==>
Higher throughput
Larger buffer requirement Tp=?, N=?, Memory=?, max-latency=?
X2 X3X1
Displayfrom Memory
Display X1, Y3, Z5
Retrievefrom Disk
Z5Y3 Z6 Y4 Z7Y5
Display X2, Y4, Z6
Time
Cycle-based Display
Group Sweeping Schema (GSS)
Can shuffle order of blocks retrievals within a group Cannot shuffle the order of groups GSS when g=1 is cycle-based GSS when g=N is round-robin Optimal value of g can be determined to minimize memory buffer
requirements Tp=?, N=?, Memory=?, max-latency=?
X2 X3X1
Display X1, W1
Z5Y3 Z6 Y4 Z7 Y5
Display X2, W2
W1 W2 W3
Subcycle 1 Subcycle 2
Group 1 Group 2
System Issues Movie is cut into equi-sized blocks: X0, X1, …, Xn-1. Time required to display one block is called time period Tp. Note: Tp is usually longer than the disk retrieval time of a block;
this allows multiplexing of a disk among different displays.
X0
X0
X0
X1
X1
X1
X2
X2
X0 X1 X2
ServerRetrievalNetwork
Buffer
Display
Time
Time period
Buffer empty
Hiccup
Constrained Data Placement Partition the disk into R
regions. During each time period
only blocks reside in the same region are retrieved.
Maximum seek time is reduced almost by a factor of R.
Introduce startup latency time Tp=?, N=?, Memory=?,
max-latency=?
Hybrid
For the blocks retrieved within a region, use GSS schema. This is the most general approach;
Tp=?, N=?, Memory=?, max-latency=? By varying R and g all the possible display techniques can
be achieved. Round-robin (R=1, g=N). Cycle-based (R=1, g=1). Constrained placement (R>0, g=1), ... A configuration planner calculates the optimal values of R &
g for certain application.
Mix of media types: different RC’s: audio, video;e.g.: Rc(Y) < Rc(X) < Rc(Z)
Different block sizes: Rc(X)/B(X)=Rc(Y)/B(Y)= ...
Display time of a block (time period) is still fixed.
X2 X3X1
Displayfrom Memory
Display X1, Y3, Z5
Retrievefrom Disk
Z5Y3
Display X2, Y4, Z6
Z6 Y4 Z7Y5
Time
Display of Mix of Media
Multiple-disks
Single disk: even in the best case with 0 seek time, 240/1.5 = 160 MPEG-1 streams.
Typical applications (MOD): 1000’s of streams.
Solution: aggregate bandwidth and storage space of multiple disk drives.
How to place a video?
Memory
RAID Striping
All disks take part in the transmission of a block.
Can be conceptualized as a single disk.
Even distribution of display load.
Efficient admission. Is not scalable in
throughput.
d1 d2 d3
X1.1 X1.2 X1.3
X2.1 X2.2 X2.3
X1
Display
Tim
e
d1 d2 d3Retrieval Schedule
X1,Y1,W1,Z1
X2,Y2,W2,Z2
X3,Y3,W3,Z3
d1 d2 d3
X1 X2 X3Y1 Y2 Y3
Z3Z1 Z2
W1W2 W3
Only a single disk takes part in the transmission of each block.
Retrieval schedule Round-robin
retrieval of the blocks.
Even distribution of display load.
Efficient admission. Not scalable in
latency.
Round-robin Retrieval
Hybrid Striping Partition D disks into clusters of d disks. Each block is declustered across the d disks that constitute a
cluster (each cluster is a logical disk drive). RAID striping within a cluster. Round-robin retrieval across the clusters. RAID striping (d=D), Round-robin retrieval (d=1).