Selective Disparity Estimation And Variable Size Motion Estimation Based on

Motion Homogeneity for Multi-View Coding

Liquan Shen, Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An

IEEE Transactions on BroadcastingDec. 2009

2

Outline

Introduction Observations and analysis Proposed algorithm Experimental results

3

Introduction (1/2)

Coding structure proposed by HHI:

View-0View-1Time = t

Time = t-

1

disparity estimation

motion estimation

4

Introduction (2/2)Global Disparity Vector (GDV)

View

Time

GDVahead GDVbehindGDVcur

……

…………

……

Non-anchor frame Anchor frame Anchor frame

Ref. view (view 0)

5

Observations (1/2)

T: Temporal prediction Static BG, homogeneous region Small block size mode for complex motion

V: View prediction Complex motion

6

Observations (2/2)

Block size distribution:

Only the MBs in the region with complex motion need DE and small mode size ME.

7

Goal

Try to decide in advance: the optimal prediction direction (ME/DE) for

MBs the prediction size is 16×16 or not

8

Motion homogeneity determined (1/4)

A uniform motion vector field at 4×4 block level is generated.

MBm,n: a MB located at the mth row, nth column. : the

MVs of its convered 4×4 blocks., , ,{ , }, [4 ,4 3], [4 ,4 3]i j i j i jmv mvx mvy i m m j n n

9

Motion homogeneity determined (2/4)

Neighbor MBs used in calculating the motion homogeneity:

Current MB

4

4 4

4

10

Motion homogeneity determined (3/4)

The motion homogeneities of MBm,n in horizontal and vertical directions are defined as:

The motion homogeneities of MBm,n is defined as:

, ,( , ) ( , )

1 1( , )64 64i j u v

i j Z u v Z

MDy m n mvy mvy

, ,( , ) ( , )

1 1( , )64 64i j u v

i j Z u v Z

MDx m n mvx mvx

( , ) ( ( , ) ( , )) 2MD m n MDx m n MDy m n

11

Motion homogeneity determined (4/4)

If MD(m,n) < Tthen the MB is considered with homogeneous motion.

Otherwise, the MB is considered with complex motion.

The threshold T is fixed for each QP level and different sequences, which is set to 0.1.

12

Selective disparity estimation

MB with homogeneous motion is likely to choose temporal prediction.

If a MB satisfies the criterion of spatially homogenous motion, inter-view prediction can be skipped.

13

Selective variable size motion estimation

When a MB is with homogeneous motion, the best mode size of the MB has a very large probability to be 16×16.

14

Proposed fast DE/ME algorithm

1) Derive MV from left, above, left-above MB, and the corresponding MB in the previously coded view.

2) Compute the motion homogeneity for current MB.3) If a MB is a homogeneous motion, perform 16x16

ME, and go to step 6, otherwise, go to step 4.4) Perform variable size DE and ME.5) Perform intra 4x4 prediction.6) Perform intra 16x16 prediction.7) Determine the best prediction direction and

prediction mode. Go to step 1 and proceed with next MB.

15

Experimental results (1/4)Experimental environment

JMVM 6.0 Test sequences (total of 9):

Downflamence2, Flamencol, Golf1, Golf2, Race1, Exit, Ballroom, Jungle, Uli

Full temporal prediction modes and inter-view prediction (FMD)

3 views are coded QP: 20, 24, 28, 32 CABAC, loop filter are enabled

16

Experimental results (2/4)

Comparison between the proposed method and FMD in JMVM:

17

Experimental results (3/4)

18

Experimental results (4/4)

Compares with other method[11]:

[11] X. Li, D. Zhao, X. Ji, Q. Wang, and W. Gao, “A fast inter frame prediction algorithm for multi-view video coding,” in ICIP, 2007.

Fast Mode Decision Using Global Disparity Vector for Multiview Video

coding

Dong-Hoon Han, Yung-Lyul Lee

2008 Second International Conference on Future Generation Communication and Networking Symposia

20

Outline

Goal Proposed algorithm Experimental results

21

Goal

Using both MB-based region segmentation information and global disparity vector (GDV) among view to reduce encoding time.

Fast mode decision using GDV.

22

Region partition (1/2)

The proposed segmentation of the background and objects block modes for fast mode decision in inter-view prediction:

An MB is decided as background block mode if a derive motion vector is smaller than ¼ in integer pixel unit in case of Direct mode , Inter 16x16, P_SKIP or B_SKIP mode.

23

Region partition (2/2)

Black block: object region

White block: background region

24

Fast mode decision forinter-view prediction

Regions of the vies using inter-view prediction are estimated using MB-based GDV and region segmentation map of reference view.

Region segmentation information of base-view

Region segmentation information of non-base view using GDV and (a)

25

Flow chart

26

Experimental results