Motion Prediction for Online GamingMotion Prediction for Online Gaming

Rynson W.H. LauRynson W.H. LauMay 14, 2011May 14, 2011

1. Network Latency1. Network Latency According to our earlier experiments, the one-According to our earlier experiments, the one-

way network latency values for different way network latency values for different network connections are shown as follows:network connections are shown as follows:

Network ConnectionNetwork Connection Average LatencyAverage Latency

LAN (2 hosts within the university network)LAN (2 hosts within the university network) 0.64ms0.64ms

Internet (local connection, 2 hosts within HK)Internet (local connection, 2 hosts within HK) 10ms10ms

Internet (overseas connection, between US and HK)Internet (overseas connection, between US and HK) 0.16s0.16s

Internet (overseas connection, between UK and HK)Internet (overseas connection, between UK and HK) 0.2s0.2s

May need to consider other overheads, such as I/O and CPU.May need to consider other overheads, such as I/O and CPU.

Problems of Network Latency in Online GamingProblems of Network Latency in Online Gaming

Motion Synchronization ProblemsMotion Synchronization Problems

Motion discrepancy of shared objects among clients.Motion discrepancy of shared objects among clients.

In a collaborative game, it will be difficult to In a collaborative game, it will be difficult to

synchronize interactions among remote players.synchronize interactions among remote players.

Object Distribution ProblemObject Distribution Problem

In a streaming environment, geometry of visible In a streaming environment, geometry of visible

objects may not be readily available locally at the time objects may not be readily available locally at the time

when they are needed.when they are needed.

Solution to the Network Latency ProblemsSolution to the Network Latency Problems

We accept that prediction is a We accept that prediction is a

prediction. We can never have a prediction. We can never have a

100% accuracy.100% accuracy.

However, we do want a reliable However, we do want a reliable

motion prediction method:motion prediction method:

Given a specific time in the future, Given a specific time in the future,

the prediction method should the prediction method should

return the most likely position of return the most likely position of

the target object.the target object.

2. Related Work2. Related Work

Roughly two types of motion prediction methods:Roughly two types of motion prediction methods:

Short-term predictionShort-term prediction

short prediction length (typically < short prediction length (typically < 1s1s).).

Long-term predictionLong-term prediction

Long prediction length (typically >> Long prediction length (typically >> 1s1s).).

Short-term PredictionShort-term Prediction

General predictors:General predictors: Dead reckoning + polynormial predictor Dead reckoning + polynormial predictor (Singhal & Zyda, 1999)(Singhal & Zyda, 1999)

Modified EWMA scheme Modified EWMA scheme (Chim et al., 1998)(Chim et al., 1998)

Specialized predictors:Specialized predictors: Kalman filter for head motion prediction Kalman filter for head motion prediction (Azuma & Bishop, 1995)(Azuma & Bishop, 1995)

Gauss-Markov process model for head prediction Gauss-Markov process model for head prediction (Liang et al., 1991) (Liang et al., 1991)

and for human motion prediction and for human motion prediction (Capin et al., 1997) (Capin et al., 1997)

Sequential Monte Carlo for motion tracking Sequential Monte Carlo for motion tracking (Isard & Balke, 1998) (Isard & Balke, 1998)

In general, specialized prediction methods are more accurate In general, specialized prediction methods are more accurate but are only suitable for predicting motion of specific objects. but are only suitable for predicting motion of specific objects.

Long-term PredictionLong-term Prediction

Mobility modelling:Mobility modelling:

For mobile handovers -For mobile handovers -

Random process Random process (Thomas et al., 1988; Xie & Goodman, 1993; Broch et al., 1998)(Thomas et al., 1988; Xie & Goodman, 1993; Broch et al., 1998)

Discrete state conditional probability Discrete state conditional probability (Bar-Noy et al., 1995; Choi & Shin, (Bar-Noy et al., 1995; Choi & Shin,

2002)2002)

For bandwidth optimization -For bandwidth optimization -

Zone-based sampling statistics Zone-based sampling statistics (Tabbane, 1995; Bhattacharya & Das, 2002)(Tabbane, 1995; Bhattacharya & Das, 2002)

Zone-based path retrieval Zone-based path retrieval (Liu & Maguire, 1996; Liu et al., 1998; Akyildiz & (Liu & Maguire, 1996; Liu et al., 1998; Akyildiz &

Wang, 2004)Wang, 2004)

Path planning:Path planning:

Collision avoidance using prior knowledge Collision avoidance using prior knowledge (Tado et al., 1995)(Tado et al., 1995) or or

statistics from samples statistics from samples (Tana, 1992)(Tana, 1992)

Moving obstacle avoidance by considering obstacle motion Moving obstacle avoidance by considering obstacle motion

trajectories trajectories (Kruse et al., 1997)(Kruse et al., 1997)

To find an optimal path to move from one place to To find an optimal path to move from one place to

another while satisfying some predefined constraints.another while satisfying some predefined constraints.

Our Streaming-Based DVEOur Streaming-Based DVE

We have developed a DVE, which is based on We have developed a DVE, which is based on

geometry streaming geometry streaming (Chim et al., 1998; Chim et al., 2003)(Chim et al., 1998; Chim et al., 2003). . <demo video>

Objects formatted for progressive transmission.Objects formatted for progressive transmission.

Send initial packet as a user joins the game.Send initial packet as a user joins the game.

Send additional geometry packets as the user moves Send additional geometry packets as the user moves

around in the game.around in the game.

This DVE was later extended into a game engine This DVE was later extended into a game engine

prototype prototype (Li et al., 2004)(Li et al., 2004) . .

Problems:Problems:

Too late if we request objects from the server only Too late if we request objects from the server only when the objects become visible.when the objects become visible.

Need to ask the server for objects before they Need to ask the server for objects before they become visible.become visible.

Need a reliable motion prediction method.Need a reliable motion prediction method.

Predict future visible objects based on the predicted Predict future visible objects based on the predicted future position from the predictor.future position from the predictor.

Request server for the predicted visible objects.Request server for the predicted visible objects.

3. Short-term Motion Prediction3. Short-term Motion Prediction

-100

-50

0

50

100

1 14 27 40 53 66 79 92 105 118 131 144 157 170 183 196

time

We needed a reliable motion prediction method for 3D We needed a reliable motion prediction method for 3D navigation – to address the network latency problem.navigation – to address the network latency problem.

As most desktop DVE systems still use the 2D mouse for As most desktop DVE systems still use the 2D mouse for navigation input, we studied the mouse behavior during navigation input, we studied the mouse behavior during such navigation.such navigation.

We noted that a typical navigation step (from mouse We noted that a typical navigation step (from mouse button down to release) contains a number of pulses.button down to release) contains a number of pulses.

Mouse-based Hybrid Motion PredictionMouse-based Hybrid Motion Prediction When plotting the mouse acceleration against velocity, we When plotting the mouse acceleration against velocity, we

noted that each pulse can be approximated by an ellipse: noted that each pulse can be approximated by an ellipse:

-40

-30

-20

-10

0

10

20

30

0 20 40 60 80 100

velocity

Hence, we model it as:Hence, we model it as:

v = Kv = K11(cos((cos(KK22tt))-1-1) )

where where KK1 1 and and KK2 2 are constants determined by the shape of are constants determined by the shape of the pulse.the pulse.

We proposed an elliptic model to model the We proposed an elliptic model to model the

mouse motion behavior mouse motion behavior (Chan et al., 2005)(Chan et al., 2005)..

Two predictors are needed, one for the motion in Two predictors are needed, one for the motion in

xx axis and the other for motion in axis and the other for motion in yy axis. axis.

The predicted mouse velocity is then mapped to The predicted mouse velocity is then mapped to

the 3D game scene to obtain the player’s future the 3D game scene to obtain the player’s future

position. position.

<Video Demo>

Performance of the PredictorPerformance of the Predictor

0

200

400

600

800

1000

1200

1400

0.1 0.2 0.3 0.4 0.5

Prediction Length (s)

Pred

ictio

n Er

ror (

pixe

ls) Psop

Pgmm

Phm

Hand Motion PredictionHand Motion Prediction

We observed similar motion behavior in finger We observed similar motion behavior in finger motion. motion.

We extended the elliptic model for hand motion We extended the elliptic model for hand motion prediction, by considering constraints within and prediction, by considering constraints within and between fingers between fingers (Chan et al., 2008)(Chan et al., 2008)..

Preformance comparison:Preformance comparison: <Video Demo>

Rock-Paper-Scissor game: sop - Rock-Paper-Scissor game: sop - <Video Demo> , ehm , ehm - - <Video Demo>

4. Long-term Motion Prediction4. Long-term Motion Prediction

We need a reliable motion prediction method for We need a reliable motion prediction method for

object prefetching and resource allocation (such object prefetching and resource allocation (such

as multi-server load-balancing).as multi-server load-balancing).

We are developing a statistical-based method.We are developing a statistical-based method.

The environment is divided into square zones.The environment is divided into square zones.

We collect statistics as users go in and out of We collect statistics as users go in and out of

each zone.each zone.

Given the boundary position that a user enters Given the boundary position that a user enters

the zone, we compute the conditional probability the zone, we compute the conditional probability

that the user will leave the zone at a particular that the user will leave the zone at a particular

boundary position.boundary position.

wherewhere nn is the number of mixture, is the number of mixture,

ffii((yy; ; CCii, , mmii)) is the is the iithth Gaussian distribution function, Gaussian distribution function,

with covariance with covariance CCii and mean and mean mmii, ,

0 <0 < aaii < 1, and < 1, and

n

iiiii mCyfaxyf

1

),;()|(

We model such a distribution function We model such a distribution function ff((yy||xx) as a ) as a

Gaussian mixture:Gaussian mixture:

n

i ia1 1

Performance of the PredictorPerformance of the Predictor

0

5

10

15

20

0 2 4 6 8 10

Prediction Length

Abs

olut

e Err

or (m

)

Psop

Pgmm

[Choi02]

Our Method

[Choi02] computes the probability of the user moving to different neighboring zones based on the user’s previous and current zones.

An interesting feature of this method is that we An interesting feature of this method is that we may consider a zone as a basic block.may consider a zone as a basic block.

We may aggregate this probability values from We may aggregate this probability values from zone to zone to construct a path tree.zone to zone to construct a path tree.

Current Work on Long-term PredictionCurrent Work on Long-term Prediction

How should we measure prediction accuracy How should we measure prediction accuracy with multiple predicted paths, each with a with multiple predicted paths, each with a different probability?different probability?

0

1 2

3 4

5 6

87

18

20

17

19

69 70

71 72

Does prediction at lower Does prediction at lower spatial resolution help reduce spatial resolution help reduce computational cost, given that computational cost, given that the prediction accuracy is the prediction accuracy is high enough?high enough?

5. Combined Motion Prediction5. Combined Motion Prediction

Short-term predictionShort-term prediction

Precise predicted locationPrecise predicted location

Very short prediction lengthVery short prediction length

Prefetching performance can be significantly Prefetching performance can be significantly affected by other factors, such as amount of affected by other factors, such as amount of geometry to be streamed, available network geometry to be streamed, available network bandwidth, CPU processing delay, network latency bandwidth, CPU processing delay, network latency fluctuation, etc.fluctuation, etc.

When applied to our streaming-based DVE, the When applied to our streaming-based DVE, the two types of prediction methods have different two types of prediction methods have different behaviors:behaviors:

Long-term predictionLong-term prediction

Imprecise predicted locationImprecise predicted location

Generally lower prediction accuracyGenerally lower prediction accuracy

Much longer prediction lengthMuch longer prediction length

May start prefetching much earlier in time, but most May start prefetching much earlier in time, but most

of the objects prefetched may not be needed at all, of the objects prefetched may not be needed at all,

due to the imprecise predicted location and the due to the imprecise predicted location and the

lower prediction accuracy.lower prediction accuracy.

Long-term predictor helps prefetch objects in Long-term predictor helps prefetch objects in

low resolution, e.g., base meshes.low resolution, e.g., base meshes.

Short-term predictor helps prefetch refinement Short-term predictor helps prefetch refinement

information, e.g., progressive records, to bring information, e.g., progressive records, to bring

the visible objects to their optimal resolution. the visible objects to their optimal resolution.

One approach that we are considering is to One approach that we are considering is to

combine the two types of predictors to optimize combine the two types of predictors to optimize

the prefetching performance:the prefetching performance:

6. Problems and Future Works6. Problems and Future Works

Can we improve the prediction accuracy by Can we improve the prediction accuracy by

incorporating user interests?incorporating user interests?

Can we improve the prediction accuracy of the Can we improve the prediction accuracy of the

long-term predictor with results from the short-long-term predictor with results from the short-

term predictor (or vise versa)?term predictor (or vise versa)?

Investigation on prediction errors and bounding Investigation on prediction errors and bounding

of prediction errors. of prediction errors. (Lau & Lee, 2010)(Lau & Lee, 2010)

Q & AQ & A