Fast disparity motion estimation in Fast disparity motion estimation in MVC based on range predictionMVC based on range prediction

Xiao Zhong Xu, Yun He

ICIP 2008

OutlineOutlineDisparity estimation in MVCDisparity search range decisionSearch range reductionExperimental ResultsConclusion

Analysis(1/3)Analysis(1/3)Disparity of two views is decided by

• distance between objects and cameras• displacement between two cameras

Disparity could be 30~50 pixels for background and up to 100 pixels for foreground

40 pixels

75 pixels

Analysis(2/3)Analysis(2/3)The directions of disparity are identical for

all blocks as long as the relative positions between cameras are not changed.• Global disparity vector

The majority of encoding time is consumed by motion estimation(ME) and disparity estimation(DE).

Over 99% encoding time is spent on ME and DE for full search.

Analysis(3/3)Analysis(3/3)Comparison of corresponding bitrate

reduction using different search ranges

• Search range(SR) = ±16 ， QP=22/27/32/37• The larger search range is, the more bitrate

can be saved.• computation time increases as well

MVC Motion skip mode (1/2)MVC Motion skip mode (1/2)Proposed by LG Co., JVT-W081(April, 2007).Motion information of current MB can be

inferred from the corresponding MB in the picture with the same temporal index of the neighboring view.• mb_type• motion vector (MV)• reference indices

MVC Motion skip mode (2/2)MVC Motion skip mode (2/2)Two main stages

1. Search for corresponding MB• Global disparity vector (by using SAD) is used.

2. Derivation of motion information• Comparing all the MB mode results, and then

choose the best.

Disparity map storageDisparity map storageDisparities of temporal static objects will

be the same of the same view.After encoding inter-view picture, the

disparity map is then refreshed by the disparities of the current picture.

The disparity map of the current picture is recorded as a disparity predictor.

For disparity prediction, the disparity map is stored.

Disparity map

Disparity estimation

Input picture

Local disparity feature decisionLocal disparity feature decisionThree already obtained disparity are

used:• Co-located MB in the disparity map (dis_col)• The disparity of the current 16*16 block (dis_16)• The disparity predicted by spatial neighboring

blocks (dis_pred)In 16*16 mode, dis_16 is set to 0.

Disparity intensityDisparity intensityDisparity intensity:

Categorizing the current block into 3 different modes:• Low disparity mode – background (DV < 2)• High disparity mode – foreground (DV > 3)• Intermediate disparity mode (other cases)

( _ _ _16 )

( _ _ _16 )x x x x

y y y y

DV abs dis col dis pred dis

DV abs dis col dis pred dis

Search range reduction(1/2)Search range reduction(1/2)The directions of disparity are identical

for all blocks• Symmetrical search window is redundant.• Once global disparity direction is detected,

the opposite direction could be paid less attention to.

Global disparity: summing up the already obtained disparity vectors and check its sign.

Search range reduction(2/2)Search range reduction(2/2)Using an unsymmetrical window.

• 4 directional factors, 2 scale factors:• Negative vertical scale (NVS)• Positive vertical scale (PVS)• Negative horizontal scale (NHS)• Positive horizontal scale (PHS)

• Vertical scale (VS)• Horizontal scale (HS)

directional factors

scale factors

16

16

NVS PVS

NHS PNS

Initial search window

Global disparity direction

new search window

Low disp.

Inter. disp.

High disp.

VS=4 VS=2 VS=1

HS=8 HS=4 HS=1

16

16

NVS PVS

NHS PNS

• Global disparity is positive: PVS = VS (PHS=HS)

• Global disparity is negative: NVS=VS (NHS=HS)

±16pixel

±16pixel

• search range：

( , ) ( , )

( , ) ( , )

SR SRSR SR

VNS VPSSR SR

SR SRHNS HPS

16SR

16 16: ( 16,16) ( , )

4 416 16

: ( 16,16) ( , )8 2

vertical

horizontal

2 pixel 8 pixel

4 pixel

4 pixel

PVS

NVS

PHSNHS

4

4, 4

8

8, 2

PVS VS

PVS NVS

PHS HS

PHS NHS

NHS PHS

NVS

PVS

Simulation results(1/3)Simulation results(1/3)Only inter-view prediction is allowed.

I II I

PPP P

Time

View

• Initial search range = ±96• QP=22,27,32,37•50 frames are coded in each view•Sequence resolution: 640*480

Simulation results(2/3)Simulation results(2/3)ISRP: Inter-view search range predictionJMVM uses extended diamond searchDE speedup comparison:

Simulation results(3/3)Simulation results(3/3)Encoding time comparison of DE:

◦The rest parts of encoder are recognized as 1 unit.

ConclusionConclusionCombination of ISRP and JMVM

performs well.Search range of disparity is

properly reduced.retaining certain quality while

significantly improving encoding speed of MVC.