Topographic Products of Topographic Products of Experts applied to a Experts applied to a

Motocross Simulation Motocross Simulation and Simulation and Simulation

StabilisationStabilisation

Benoit Chaperot, Colin FyfeSchool of Computing

The University of PaisleyPaisley, PA1 2BE, SCOTLAND.benoit.chaperot@paisley.ac.uk

colin.fyfe@paisley.ac.uk

Motocross game and advanced Motocross game and advanced artificial intelligence techniquesartificial intelligence techniques

The motocross game offers a good environment to experiment with advanced artificial intelligence techniques.

The motocross game can also strongly benefit from good AI; good AI can enrich the player experience.

TOPOGRAPGHIC PRODUCTS OF TOPOGRAPGHIC PRODUCTS OF EXPERTSEXPERTS 1/3 1/3

A topology preserving mappingA topology preserving mapping, similar to Kohonen’s , similar to Kohonen’s Self Organizing map. Self Organizing map. Points which are mapped close Points which are mapped close to one another share some common feature while to one another share some common feature while points which are mapped far from one another do not points which are mapped far from one another do not share this feature.share this feature.

TOPOGRAPGHIC PRODUCTS OF TOPOGRAPGHIC PRODUCTS OF EXPERTSEXPERTS 2/3 2/3

K (2 dimensional) latent points, tK (2 dimensional) latent points, t11, t, t22, …, t, …, tKK..

To allow local and non-linear modelling, To allow local and non-linear modelling, latent points are mapped through a set of latent points are mapped through a set of M basis functionsM basis functions..

The output of these functions are then The output of these functions are then mapped by a set of weights into data mapped by a set of weights into data spacespace..

Data Space dimension D.Data Space dimension D.

TOPOGRAPGHIC PRODUCTS OF TOPOGRAPGHIC PRODUCTS OF EXPERTSEXPERTS 3/3 3/3

Each latent point takes responsibility:Each latent point takes responsibility:

where: where: is is the Euclidean distance between the data the Euclidean distance between the data point and the projection of the latent pointpoint and the projection of the latent point

Training achieved by modifying only the Training achieved by modifying only the weights:weights:

k ik

ijij d

dr

)exp(

)exp(2

2

qppq mxd

K

k knkd

ndkmmdn rmxw

1

)()( )(

Topographic Products of Experts Topographic Products of Experts and the Motocross Gameand the Motocross Game

Data of dimension D:Data of dimension D: is the number of inputs; the inputs is the number of inputs; the inputs are the state or situation of the bike, i.e. position, are the state or situation of the bike, i.e. position, orientation and velocity of the bike relative to the orientation and velocity of the bike relative to the track, and information about the terrain. In track, and information about the terrain. In previous work, these were the inputs to previous work, these were the inputs to supervised Artificial Neural Networks.supervised Artificial Neural Networks. is the number of outputs; the outputs is the number of outputs; the outputs are the decisions made according to the state or are the decisions made according to the state or situation of the bike, i.e. accelerate or brake, situation of the bike, i.e. accelerate or brake, turn left or right, lean forward or backward.turn left or right, lean forward or backward.

OutputInput DDD

InputD

OutputD

ResultsResults

Size Training Time for 100’000

iterations

Decision Time Lap Time Performance Track Long

MLP 16 KB 10sec 0.1ms 2min 25sec

Kohonen 17KB 1.3sec 0.173021ms 3min 11sec

Kohonen2 49 KB 3.9sec 0.543144ms 3min 27sec

ToPoEK=200,M=26

16 KB 13min 40sec 0.217714 ms 3min 05sec

ToPoE 2K=400,M=37

64 KB 54min40sec

0.368463ms 2min 58sec

Comparison with Kohonen’s SOMComparison with Kohonen’s SOM

ToPoE AI is performing slightly better than ToPoE AI is performing slightly better than the Kohonen AIthe Kohonen AI..

Kohonen SOM is a quantisation method; it Kohonen SOM is a quantisation method; it does not interpolate between projections does not interpolate between projections of latent points into data spaceof latent points into data space. ToPoE . ToPoE does interpolate.does interpolate.

ToPoE AI is more processing intensive ToPoE AI is more processing intensive than Kohonen SOM.than Kohonen SOM.

Comparison with Multi-Layered Comparison with Multi-Layered Perceptron ANNPerceptron ANN

ToPoE AI is not performing as well as the ToPoE AI is not performing as well as the MLP AI.MLP AI.

MLP networks can make the difference MLP networks can make the difference between important and not so important between important and not so important inputs; the two topology-preserving inputs; the two topology-preserving methods cannot do.methods cannot do.

ToPoE AI is more processing intensive ToPoE AI is more processing intensive than MLP AI.than MLP AI.

Change of Development Change of Development EnvironmentEnvironment

Visual Studio 6 to Visual Studio 2005.Visual Studio 6 to Visual Studio 2005. Allows us to be easily up to date and Allows us to be easily up to date and

compatible with other libraries.compatible with other libraries. Easier to share files and projects with other Easier to share files and projects with other

researchers.researchers.

Problems:Problems: Small changes in language.Small changes in language. New compilers and libraries; the recompiled New compilers and libraries; the recompiled

program crashed at first execution.program crashed at first execution.

Stabilisation: Gyroscopic Forces Stabilisation: Gyroscopic Forces 1/21/2

Original ODE implementation:Original ODE implementation:

Some torque is applied each frame, the Some torque is applied each frame, the angular energy increasesangular energy increases, rotating objects gain , rotating objects gain angular velocity and sometimes explode.angular velocity and sometimes explode.

For each body:

1. The inertia tensor is transformed to global space.

2. The angular momentum is computed as IL ,

with I the inertia tensor and the angular velocity.

3. The gyroscopic torque is computed as LT , then added to the body torque accumulator.

Stabilisation: Gyroscopic Forces Stabilisation: Gyroscopic Forces 2/22/2

New New Implementation, Implementation, based on based on conservation of conservation of angular angular momentum:momentum:

Rotating objects Rotating objects do not explode do not explode anymore.anymore.

1. The inertia tensor is transformed to global space.

2. The angular momentum is computed as IL ,

with I the inertia tensor and the angular velocity.

3. The gyroscopic torque is computed as LT .

4. The new velocity of the body, after only the gyroscopic torque has been applied is computed as

TdtI 12

.

5. The new angular momentum is computed as

22 IL

6. The angular momentum of a free rotating body should not increase with time; the new angular velocity of the body is normalised so that its angular momentum is the same as it was before the gyroscopic torque was

applied:

222 L

L .

7. This new velocity is set to the body; no gyroscopic torque or force is added to the accumulators.

Stabilisation: Floating Point Stabilisation: Floating Point NumbersNumbers

Some variables become NAN (Not A Number) every Some variables become NAN (Not A Number) every now and then; game crashes.now and then; game crashes.

Debugging by insertion of macros into the code:Debugging by insertion of macros into the code:

When assertion fails, we look up the flow of execution, When assertion fails, we look up the flow of execution, we add more checks and repeat the process many we add more checks and repeat the process many times to get closer and closer to the source of the times to get closer and closer to the source of the problem until we can identify it.problem until we can identify it.

Usually division by zero or wrong parameters passed to Usually division by zero or wrong parameters passed to Math functions.Math functions.

#define CHECKV(a) ASSERT(_finite(a)!=0)

Stabilisation: MemoryStabilisation: Memory

Game allocates more and more memory Game allocates more and more memory till it finally crashes.till it finally crashes.

Use of tools such as Visual Leak Detector Use of tools such as Visual Leak Detector to identify the problem.to identify the problem.

Problem inside Direct3D library !Problem inside Direct3D library !

ConclusionConclusion

Experiments with ToPoE proved that ToPoE Experiments with ToPoE proved that ToPoE performs better than the Kohonen SOM at performs better than the Kohonen SOM at controlling motorbikes in the motocross controlling motorbikes in the motocross simulation. Both topology preserving mapping simulation. Both topology preserving mapping techniques are outperformed by MLP networks.techniques are outperformed by MLP networks.

The game is a very good platform to experiment The game is a very good platform to experiment with AI and evolution techniques; however it with AI and evolution techniques; however it requires a lot of human work to maintain and requires a lot of human work to maintain and update in relation to other tools and libraries.update in relation to other tools and libraries.

ESANN 2008ESANN 2008

European Symposium on Artificial Neural European Symposium on Artificial Neural Networks. Advances in Computational Networks. Advances in Computational Intelligence and Learning. Bruges Intelligence and Learning. Bruges (Belgium) 23 - 25 April 2008 (Belgium) 23 - 25 April 2008

Special Session: Computational Special Session: Computational Intelligence in Computer GamesIntelligence in Computer Games