Seamless Switching Seamless Switching of Scalable Video Biof Scalable Video Bitstreams for Efficietstreams for Efficie

nt Streamingnt StreamingXiaoyan Sun, Feng Wu, ShipeXiaoyan Sun, Feng Wu, Shipeng Li, Wen, Gao, and Ya-Qin Zng Li, Wen, Gao, and Ya-Qin Z

hanghang

OutlineOutline• Introduction• Seamless Switching Scheme Among Scalable

Bitstreams• Switching Down Between Two Bitstreams• Switching Up Between Two Bitstreams• Experimental Results• Discussions

IntroductionIntroduction• Why are scalable bitstreams needed? Switchin

g between non-scalable bitstreams– Drifting error

– Key frames

– Large storage requirements• Multiple non-scalable bitstreams• Key frames

Non-scalable bitstream1

Non-scalable bitstream2

drifting error

Non-scalable bitstream1

Non-scalable bitstream2

key frame

IntroductionIntroduction• SP-frames scheme (proposed in H.264)

– Extra bitstreams are used.– The size of S12 is similar to that of I-frame.– When switching down, S12 may deteriorate the net

works.– http://vc.cs.nthu.edu.tw/ezLMS/show.php?id=249

P S2 P

P S1 P

S12

bitstream1

bitstream2

SP frame

extra bitstream

IntroductionIntroduction• Why are multiple scalable bitstreams ne

eded? Single scalable bitstream– FGS

– Low coding efficiency in FGS bitstreams.• Motion prediction is based on the lowest quality

base layer.– Using multiple scalable bitstreams, coding e

fficiency is higher.

Base layer

Enhancement layer 1Enhancement layer 2

Enhancement layer 3

…

Base layer

Enhancement layer 1Enhancement layer 2

Enhancement layer 3

…

Frame n Frame n+1

Seamless Switching Scheme ASeamless Switching Scheme Among Scalable Bitstreamsmong Scalable Bitstreams

• Seamless switching (Definition)– The quality in each scalable bitstream is sm

ooth.– The switching among scalable bitstreams is

drifting-free.– Immediately switching from the current scal

able bitstream to one operated at lower rates without any delay.

Seamless Switching Scheme ASeamless Switching Scheme Among Scalable Bitstreamsmong Scalable Bitstreams

ScalableBitstream 1

ScalableBitstream 2

ScalableBitstream 1

ScalableBitstream 3

Bandwidth

Time

ScalableBitstream 1

ScalableBitstream 2

ScalableBitstream 3

Seamless Switching Scheme ASeamless Switching Scheme Among Scalable Bitstreamsmong Scalable Bitstreams

switching

SF switching frame (switching-up point)

High-bit-rate scalable bitstream

Low-bit-rate scalable bitstream

Switching Down Between Two Switching Down Between Two Bitstreams Bitstreams

• MVs are estimated in SB-H and are applied to both SB-H and SB-L.

• The quantization information of SB-L is coded in SB-H bitstream.– Overhead bits: 3*MB_number+5

• Instead of original frames, base layer frames in SB-H are used to encode base layer frames in SB-L.

First QPRange of difference

☆Switching down at everywhere and with no overhead

Switching Down Between Two Switching Down Between Two BitstreamsBitstreams

• SF Decoder

VLD Q-1 IDCT

MC

MC

DCTQL-1 QLIDCT

ref SB-L refBase

SB-HBase layer

SB-LBase layer

Video

MVs

Switching Down Between Two Switching Down Between Two BitstreamsBitstreams

• Quality loss of SB-L base layer– The reconstructed frames from SB-H base la

yer instead of the original video are used as the input of the SB-L base layer.

– The same set of MVs for SB-H base layer is used to encode the SB-L base layer.

Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams

n-1 n n+1

Switchingpoint

SB-LBase layer

SB-HBase layer

+

=Extra bitstream

☆Considerable amount of overhead bits

Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams

• SF Encoder

IDCT

Qs-1

DCT

Qs

BP VLC

DCT Qs

SB-HBase layer

SB-LBase layer

Extra bitstream

SF framen-1 n+1

n

☆n’ is used instead of n to avoid drifting error

= SB-L Base + Extra bitstream

n’n

Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams

• SF Decoder

VLD Q-1 IDCT

MC

refBase

SB-HBase layer

SB-LBase layer

VideoMVs

DCTQs-1 QsIDCT

BPVLD

Extrabitstream

Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams

• The SF frame is simpler than the SP frame.

• The SF frame gets better quality than the SP frame.

Experimental ResultsExperimental Results• SF scheme is applied on MBPFGS

– MBPFGS is modified from PFGS• Frame base -> Macroblock base•

–

– 10fps• An I-frame is inserted every 1’s in a non-scalable bit

stream • An SF frame is inserted every 1’s in a SF bitstream

32kbps64kbps80kbps112kbps

http://vc.cs.nthu.edu.tw/home/paper/codfiles/hschen/200305141532/pfgs.PPT

MBFGS-H bitstream

MBFGS-L bitstream

Experimental ResultsExperimental Results• Comparisons of extra bitstream

–

– Average PSNR loss caused by SF frames < 0.1 dB

Sequence SF (Qs=3) Lossless

News 32959.11 147878.1

Foreman 34918.33 173569.3

Coastguard 39824.78 158157.7 (Bits)

Experimental ResultsExperimental Results• Rate-distortion (static)

– Non-scalable bitstream (JVT H.26L codec)• Two bitstream with different quality• Switching at I-frame

– FGS and MPFGS• Only one bitstream

Non-scalable

FGS

MPFGS

SF

Inaccurate MVs and input video

Experimental ResultsExperimental Results• PSNR comparisons (dynamic

channel)– Foreman

– Coastguard

72 kbps

152 kbps

72 kbps

152 kbps

DiscussionsDiscussions• The SF scheme can be extended to the case with multi

ple scalable bitstreams.– (N-1) extra bitstreams are required.

• N*(N-1) extra bitstreams are required in SP frames.– High complexity of the decoder in decoding the high-bit-rate

scalable bitstream.• Three times of the MPEG-4 decoder.• Complexity scalability.

– Large size of the overhead bits inside the high bit-rate bitstream.

• The extra bitstream for switching up is large.– if the bitstreams switch up and down once for every second, t

he extra bitstream will cost 30-40 kbps.– An intelligent server should be able to control switching time

s.