nus.soc.cs5248 ooi wei tsang mpeg + rtp. nus.soc.cs5248 ooi wei tsang previously, on cs5248 mpeg...

NUS.SOC.CS5248Ooi Wei Tsang

MPEG + RTP

NUS.SOC.CS5248Ooi Wei Tsang

Previously, on CS5248

MPEG Compression Sequence, GOP, Picture, Slice,

Macroblock, Block, DC/AC Coefficient

I-Frame, P-Frame, B-Frame

NUS.SOC.CS5248Ooi Wei Tsang

Previously, on CS5248

RTP headers SSRC, Media Timestamp, Marker Bit,

Payload Type ..

Application-Level Framing

NUS.SOC.CS5248Ooi Wei Tsang

You are Here

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248Ooi Wei Tsang

Application-Level Framing

NUS.SOC.CS5248Ooi Wei Tsang

How to send/recv?

Let the application decide, not protocol

stacks.

Tennenhouse + Clark

NUS.SOC.CS5248Ooi Wei Tsang

Application Knows Best

How to reorder packetsWhether to ignore lossWhich packet to retransmit

NUS.SOC.CS5248Ooi Wei Tsang

Application Data Unit (ADU)

Can be processed individually, even out-of-order

8-Bit PCM audio: 1 ADU = 1 ByteMPEG1 Video: 1 ADU =

NUS.SOC.CS5248Ooi Wei Tsang

How to chop data into packets?

Every received packet should be useful (even in very lossy environment)

Ideally, 1 ADU in 1 packet

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

RTP HeaderRTP Payload

HeaderRTP Payload

•MPEG-1? 2?•Temporal Reference•I? P? B?•Begin of Slice? End of Slice?

NUS.SOC.CS5248Ooi Wei Tsang

RTP Header

Media Timestamp: 32 bits the instant the first byte in this packet is

captured 90 kHz timestamp (90000 = 1 second)

NUS.SOC.CS5248Ooi Wei Tsang

RTP Header

Marker Bit: 1 if contains the last byte of a frame

NUS.SOC.CS5248Ooi Wei Tsang

RTP Header

Payload Type: 7 bits 32 for MPEG-1

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

MBZ (5 bits) Unused. Must be 0

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

T (1 bit) 1 if there is a MPEG-2 Extension Header

after this header.

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

Temporal Reference (10 bits) The ‘frame number’ of the current frame

within the GOP

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

AN bit and N bit Set to 0 for MPEG-1

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

S (1 bit) Is there a sequence header in this packet?

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

BS (1 bit) and ES (1bit) BS is 1 iff the ‘first’ byte of this payload is a slice

header ES is 1 iff the last byte of this payload is the end of a

slice

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

Picture Type (3 bits) I (1), P (2), B (3), D (4)

NUS.SOC.CS5248Ooi Wei Tsang

RTP Payload Header

Motion Vectors Information Get from most recent picture header

NUS.SOC.CS5248Ooi Wei Tsang

Fragmentation Rules

Sequence header: at the start of payload

GOP header: at the start of a payload (or follows Sequence header)

Picture header: at the start of a payload (or follows Sequence/GOP header)

NUS.SOC.CS5248Ooi Wei Tsang

Fragmentation Rules

A slice must be either First data in the packet, or Follows integral number of slices

A slice may be fragmented if exceeds the size of a packet

NUS.SOC.CS5248Ooi Wei Tsang

Packet Size

1 MTU is 1500 bytes IP HeaderUDP HeaderRTP HeaderRTP Payload HeaderPayload Size =

NUS.SOC.CS5248Ooi Wei Tsang

Project 1

Packetize MPEG-1 Video into RTP Packets

NUS.SOC.CS5248Ooi Wei Tsang

Goal

Read MPEG-1 video, output RTP packets with proper: RTP headers RTP payload headers fragmentation

NUS.SOC.CS5248Ooi Wei Tsang

Mnt (Media Networking Toy)

C++Tcl/OTclUNIX

NUS.SOC.CS5248Ooi Wei Tsang

Overview

C++

OTcl

MntComponents

MntData

NUS.SOC.CS5248Ooi Wei Tsang

MntComponent

recv(MntData *data)push(MntData *data)

NUS.SOC.CS5248Ooi Wei Tsang

MntPump

Subclass of MntComponent

No recv()pump_some( ) {

generate data push(data) pump_timer_.msched(time)

}

NUS.SOC.CS5248Ooi Wei Tsang

Create and Link Components

set a [new MntA]set b [new MntB]

$a add_link_to $b$a instproc on_stop_pumping {} { puts “done!” exit}$a start_pumpingvwait forever

NUS.SOC.CS5248Ooi Wei Tsang

Important Classes

MntComponent

MntRTPFileWriter

MntRTPFileReader

MntPump MntRTPPayloadDumper

MntMPEGFileRTPizer

NUS.SOC.CS5248Ooi Wei Tsang

Important Classes

MntData

MntRTPMPEGPacket

MntRTPHdr MntRTPMPEGHdr

NUS.SOC.CS5248Ooi Wei Tsang

MntRTPMPEGPacket

RTP HeaderRTP Payload

HeaderRTP Payload

MntRTPHdr *rtp_hdr_; MntRTPMPEGHdr *pl_hdr_; unsigned char *pl_data_; int pl_len_;

NUS.SOC.CS5248Ooi Wei Tsang

Example Tcl Scripts

NUS.SOC.CS5248Ooi Wei Tsang

Dealing with Bits

BitStream A buffer

buffer_ endDataPtr_ endBufPtr_

NUS.SOC.CS5248Ooi Wei Tsang

Dealing with Bits

BitParser Read bits from BitStream

BitParser 1011101

NUS.SOC.CS5248Ooi Wei Tsang

Basic Usage

#include “mnt_bit_parser.h”#include “mnt_bit_stream.h”

BitStream *bs = new BitStream(“a.mpg”);BitParser *bp = new BitParser();bp->wrap(bs);

int x;Bp_GetInt(bp, x);

NUS.SOC.CS5248Ooi Wei Tsang

Dealing with MPEG Headers

MntMPEG1SeqHdrMntMPEG1GopHdrMntMPEG1PicHdrMntMPEG1SliceHdr

NUS.SOC.CS5248Ooi Wei Tsang

Basic Operations

hdr->parse(bp)hdr->find(bp)hdr->skip(bp)hdr->dump(bp1,bp2)

NUS.SOC.CS5248Ooi Wei Tsang

Example:MPEG Parsing

NUS.SOC.CS5248Ooi Wei Tsang

Project Homepage

DescriptionsSkeleton CodeMnt Documentation IVLE Forums

NUS.SOC.CS5248Ooi Wei Tsang

Advice

Use the source wiselyStart early