Block-based coding

Multimedia Systems and StandardsS2 IF Telkom University

2

Why block-based video coding?

the choice of a suitable video coder depends on the associated application and available resourcesPrimary reasons:

The quality of service they are designed to achieveachieve fairly high compression ratios in real-time scenarios. • Exp: H.263 < 64 kbit/s, 25 f/s and an average PSNR of

30 dBthe scalability of their output bit ratessource code is available

3

Video frame formatbased on the Common Intermediate Format

standardize the horizontal and vertical resolutions in pixels of YCbCr sequences

4

Chrom = ½ Lumthe human eye is less sensitive to the details of the colour information

5

Layering structurevideo frame = k x 16 lines of pixels

k = 1 for sub-QCIF, 9 for QCIF, 18 for CIF, 4CIF and 16CIF

Video frames = Group of Blocks (GOB)6 for sub-QCIF, 9 for QCIF and 18 for CIF, 4CIF and 16CIF

GOB = macroblocks (MB)MB = 4 luminance blocks + 2 spatially corresponding colour difference blocksblock = 8 pixels by 8 lines of Y, U or V

6

QCIF: picture - GOB

7

QCIF: MB - block

8

Methodsvideo frame is divided into a number macroblocks (16x16 blocks of pixels)two coding modes:

INTRA• coded as an independent still image without any

reference to precedent frames spatial redundanciesINTER• exploits the temporal redundancies between

successive frames• higher compression efficiency by employing

predictive coding

9

INTER & INTRA codingcoding only the difference between a frame and its reference INTER frame or predicted frame (P-frame) codingif successive frames are not strongly correlated INTRA frame (I-Frame) codingin INTER frame coding, some MBs could still be INTRA coded mode flag in each MB

10

INTER modemotion search: similarities between the current frame and the reference one the difference image (the residual error frame) is DCT-transformed and quantized converted to a one-dimensional matrix of coefficients using the zigzag-pattern coding

to exploit the long runs of zeros that appear in the picture after quantization.

run-length coder (Huffman coder) assigns variable-length codes to the non-zero levels and the runs of zeros in the resulting one-dimensional matrix.

11

INTER vs qualityan accumulation of INTER coded frames could lead to fuzzy picture quality

the effect of repeated quantizationInsert INTRA frame: to refresh the picture quality after a certain number of INTER frames

12

Motion estimationINTER coding: the block matching (BM) motion estimation process

Each MB in the current frame is compared to MBs of the previous reconstructed frame within a search window of user-defined size

13

The matching criteriononly luminance is usedmay use any error measure e.g. mean square error (MSE), sum of absolute difference (SAD), MAD, SSE, SATD, etc.motion vector (MV): displacement vector between the current MB and its best match

vertical and horizontal components

14

15

INTRA within P-frameIf all SADs corresponding to 16x16 matrices within the search window < a certain motion activity threshold then the current MB is INTRA coded within the P-frame

16

ITU-T H.263

17

OperationFor each MB: the SADs are compared to a motion activity threshold to decide whether INTRA or INTER mode is to be used for a specific MBINTRA mode: the coefficients of the six 8x8 blocks are DCT transformed, quantized, zigzag coded, run-length coded, and then variable-length coded using a Huffman encoder

18

INTER mode: the resulting MV is differentially coded and the same encoding procedure as in INTRA mode is applied on the residual matrix

Operation (cont’d)

19

2-D 8x8 DCT

20

Example of DCT

DC coefficients: assigned an 8-bit length codeword (coded more accurately)AC coefficients: run-length coded

21

Quantizationcontrols the coding efficiency & the quality of the reconstructed video sequencebased on the human visual sensitivityTechniques:

scalar quantization: each sample is quantized independentlyvector quantization: quantization of a group of samples or vectors

22

Quantization: lossymaps the values of the DCT transformed coefficients to a smaller range of valuesthe exact original pixel value cannot be restored

23

H.263 QuantizationQuantisation

INTRADC coefficient: LEVEL = COF/8INTRA AC coefficients: LEVEL = |COF|/(2 x Qp)INTER coefficients: LEVEL = (|COF| - Qp/2)/(2 x Qp)

Inverse quantisationINTRADC coefficient: COF’ = LEVEL x 8INTRA or INTER coefficients:• |COF’| = 0 if LEVEL=0• |COF’| = 2Qp x LEVEL + Qp if LEVEL<> 0, Qp is odd• |COF’| = 2Qp x LEVEL x Qp -1 if LEVEL <> 0, Qp is

even

24

example

25

zigzag scan

Result?

26

Some block-based DCT video coding standards

Standard

Application Bit rates

MPEG-1

Audio/video storage on CD-ROM

1,5 – 2 Mbps

MPEG-2

HDTV/DVB 4 – 9 Mbps

H.261 Video over ISDN

p x 64 kbps

H.263 Video over PSTN

< 64 kbps