pool playing robot -...

The University of Adelaide

Department of Mechanical Engineering

Final Year Project

Pool Playing Robot

Final Report

November 2002

Justin Ghan

Tomas Radzevicius

Will Robertson

Alexandra Thornton

Supervisor: Dr Ben Cazzolato

Executive Summary

This report covers the complete design, construction, commissioning and

preliminary testing of a robot that is capable of competing against a human

opponent in a game of pool.

A number of robots have been designed and built in the past for the pur-

pose of playing pool, or similar games. Upon investigation of these previous

projects, it was found that none of these robots were able to play a real shot

(as would occur in a real game) successfully.

The aim of this project was to achieve what past projects have not — that

is, to design and build a robot which operates independently of human input

and can play pool to a degree approximating a human player.

The process that the robot goes through is functionally similar to that

of a human. A vision system determines the location of the balls on the

table. This information is then processed by a shot selection program that

determines the best shot to play, then outputs a series of commands to a

dedicated mechatronic system that translates these commands into physical

actuation in order to sink a ball.

The vision system consists of a computer controlled camera and image

processing software to determine the ball locations. The scope of the vision

system covers the determination of the location of red, yellow, black and white

balls. This information is then analysed by a shot determination algorithm,

to find the shot with the highest probability of success.

The position and force requirements of the shot are converted by interface

software to a series of commands, which are sent to a dedicated hardware

control card that controls the actuators.

The actuation system for the robot is a modified Cartesian design driven

by stepper motors, with four degrees of freedom to ensure spatial location of

the cue tip over any position on the pool table. The cue tip is actuated by

a long stroke solenoid, modified from a washing machine to operate on an

adjustable power supply.

The aim of the project has been achieved to a certain degree, with a robot

capable of independently playing a game of pool, although the speed and

accuracy of the robot are below initial expectations. The changes necessary

to significantly improve the performace of the robot are outside the scope of

what was achievable in the time frame.

ii

Acknowledgements

We would like to thank first and foremost our supervisor, Dr Ben Cazzo-

lato, for his unending enthusiasm, support, guidance and confidence in our

ability to complete Eddie!

Thank you to Joel, Silvio, Derek and George in Instrumentation for in-

dulging our frequent visits and demands quickly, efficiently, professionally and

humorously!

We would like to acknowledge the assistance of the Mechanical Engineering

workshop staff for their invaluable contribution: Bob, Ron, Malcolm, but

especially Bill!

Thanks to Stephen Farrar for his ideas and assistance for the image pro-

cessing software, and to Pierre Dumuid and Ben Longstaff for helping out

with programming. Thanks to Nicole Carey for editing the draft report.

And finally, thank you to all of our family and friends for supporting us,

feeding us and generally keeping us going throughout the year.

iii

Contents

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

1 Introduction 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Problem definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.3 Aim and significance . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Literature review 3

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 The game of pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.3 The eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.4 The brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.5 Previous pool playing robots . . . . . . . . . . . . . . . . . . . . . . . 5

2.5.1 The Tianjin Normal University model . . . . . . . . . . . . . . 5

2.5.2 The University of Bristol model . . . . . . . . . . . . . . . . . 7

2.5.3 The Massachusetts Institute of Technology model . . . . . . . 8

2.5.4 The University of Waterloo model . . . . . . . . . . . . . . . . 9

2.5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 The game of pool 14

3.1 Rules of 8-Ball . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.2 Physics of pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

iv

4 The eye 17

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Error analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.3.1 Camera types . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.3.2 Selection criteria . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.3.3 Camera options and selection . . . . . . . . . . . . . . . . . . 25

4.3.4 Camera choice . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.3.5 Image processing in hardware . . . . . . . . . . . . . . . . . . 27

4.4 Image representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.5 Template matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.5.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.5.2 Speed considerations . . . . . . . . . . . . . . . . . . . . . . . 34

4.6 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.6.1 Centroid method . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.6.2 Erosion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.7 Distortion filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.8 Locating edges and pockets . . . . . . . . . . . . . . . . . . . . . . . 43

4.9 Final eye software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5 The brain 47

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.2 Listing shots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.3 Possibility analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

v

5.3.1 Calculating ball paths . . . . . . . . . . . . . . . . . . . . . . 50

5.3.2 Trajectory check . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.4 Choosing the best shot . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.4.1 Difficulty of a shot . . . . . . . . . . . . . . . . . . . . . . . . 56

5.4.2 Usefulness of a shot . . . . . . . . . . . . . . . . . . . . . . . . 62

5.4.3 Overall merit function . . . . . . . . . . . . . . . . . . . . . . 63

5.5 Complete simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.6 Final brain software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

6 The arm 67

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

6.2 Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

6.3 Overall function and subfunctions . . . . . . . . . . . . . . . . . . . . 67

6.4 Solution principles to subfunctions . . . . . . . . . . . . . . . . . . . 68

6.5 Concept solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

6.6 Analysis of design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

6.6.1 Energy transfer to cue ball . . . . . . . . . . . . . . . . . . . . 78

6.6.2 Drive mechanism and traverse mechanism . . . . . . . . . . . 80

6.6.3 Control of actuation system . . . . . . . . . . . . . . . . . . . 84

6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

7 Interface 85

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

7.2 Software development . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

vi

7.2.1 Actuator control requirements . . . . . . . . . . . . . . . . . . 86

7.2.2 Command structure development . . . . . . . . . . . . . . . . 86

7.2.3 Pulse train generation . . . . . . . . . . . . . . . . . . . . . . 88

7.2.4 Code algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 92

7.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

7.3.1 Motors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

7.3.2 Solenoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

7.4 User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

8 Commissioning and results 99

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

8.2 Software integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

8.3 Position calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

8.4 Cue ball location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

8.5 Cue tip length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

8.6 Carriage wheels and motor torque . . . . . . . . . . . . . . . . . . . . 104

8.7 Cross traverse shearing . . . . . . . . . . . . . . . . . . . . . . . . . . 106

8.8 Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

8.9 Vision system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

8.10 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

8.10.1 The eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

8.10.2 The brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

8.10.3 The arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

8.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

vii

9 Future work 112

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

9.2 The eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

9.3 The brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

9.4 The arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

10 Conclusion 116

References 117

A Robot code 119

A.1 Main program code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

A.2 Eye software code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

A.3 Brain software code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

A.4 Interface software code . . . . . . . . . . . . . . . . . . . . . . . . . . 143

A.5 Miscellaneous code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

B Visual Basic code 153

C dSPACE code 159

C.1 C code for DS1102 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

C.2 Matlab code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

D Control card circuit 161

E Cost analysis 163

viii

F Solenoid force experiment 165

F.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

F.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

F.3 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

F.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

F.5 Results and analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

F.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

G Technical drawings 168

ix

List of Figures

1.1 Block diagram of the robot subsystems. . . . . . . . . . . . . . . . . . 2

2.1 The University of Bristol robot. . . . . . . . . . . . . . . . . . . . . . 7

2.2 Schematic for the University of Waterloo robot. . . . . . . . . . . . . 10

2.3 The mushroom shaped paddle used in the University of Waterloo robot. 11

2.4 Typical worm gear configuration. . . . . . . . . . . . . . . . . . . . . 12

3.1 Setup for a game of 8-Ball. . . . . . . . . . . . . . . . . . . . . . . . . 15

4.1 Geometry of a shot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.2 Errors in the positions of the cue and target balls. . . . . . . . . . . . 21

4.3 Template of the white ball. . . . . . . . . . . . . . . . . . . . . . . . . 30

4.4 Cross-correlations between an image and the template. . . . . . . . . 31

4.5 Neighbourhood of a peak of the averaged cross-correlation. . . . . . . 31

4.6 Results of the normalised cross-correlation interpolation program. . . 32

4.7 Histogram of the blue components of different coloured pixels. . . . . 37

4.8 A depiction of lens distortion. From ?. . . . . . . . . . . . . . . . . . 41

4.9 A photo of a grid of squares over the pool table, taken by the robot’s

camera. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.1 Region of interference with the path of a ball. . . . . . . . . . . . . . 53

5.2 Diagram for error magnification analysis. . . . . . . . . . . . . . . . . 58

6.1 SCARA manipulator. . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.2 Cartesian manipulator. . . . . . . . . . . . . . . . . . . . . . . . . . . 71

x

6.3 Articulated manipulator. . . . . . . . . . . . . . . . . . . . . . . . . . 71

6.4 Typical stepper motor, and schematic. . . . . . . . . . . . . . . . . . 72

6.5 Solenoid taken from the starter motor of a car. . . . . . . . . . . . . . 78

6.6 Solenoid taken from a washing machine. . . . . . . . . . . . . . . . . 79

6.7 Wheels locating in modified channel section. . . . . . . . . . . . . . . 82

6.8 Tooth belt and pulley. . . . . . . . . . . . . . . . . . . . . . . . . . . 83

7.1 A GUI allowing a user to set various options and trigger the robot to

take its shot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

7.2 A GUI displaying the chosen shot and allowing a user to choose from

other alternatives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

8.1 Robot at the three datum positions. . . . . . . . . . . . . . . . . . . . 101

8.2 The solenoid shaft with the cue tip attached. . . . . . . . . . . . . . . 104

8.3 The solenoid shaft with the cue tip attached. . . . . . . . . . . . . . . 105

8.4 The cables and wires setup. . . . . . . . . . . . . . . . . . . . . . . . 107

8.5 The attachment of the camera to the ceiling. . . . . . . . . . . . . . . 107

F.1 Force experiment setup. . . . . . . . . . . . . . . . . . . . . . . . . . 166

xi

List of Tables

4.1 Tolerance of aim angle for given situations. . . . . . . . . . . . . . . . 22

4.2 Various digital cameras considered. . . . . . . . . . . . . . . . . . . . 25

4.3 Error in centre location using spline interpolation. . . . . . . . . . . . 33

4.4 Error in centre location using bandlimited interpolation. . . . . . . . 34

4.5 Error in centre location using the centroid method. . . . . . . . . . . 39

5.1 Some shot path types. . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.1 Selection critera weightings. . . . . . . . . . . . . . . . . . . . . . . . 77

6.2 Concept design criteria evaluation. . . . . . . . . . . . . . . . . . . . 78

7.1 Actuators command structure. . . . . . . . . . . . . . . . . . . . . . . 88

7.2 Solenoid power levels and commands. . . . . . . . . . . . . . . . . . . 94

E.1 Cost analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

F.1 Washing machine solenoid impulse time results. . . . . . . . . . . . . 167

xii

1 Introduction

1.1 Introduction

Pool is a game with a high degree of complexity, which requires human players to

be adept in concepts of kinematics as well as possessing a high level of hand eye

coordination and strategy.

From a robotic perspective, the game of pool takes on added levels of complexity,

which turn what initially appears to be a frivolous concept into an experiment in

vision systems, artificial intelligence programming, automation and robustness. This

project will address these and other associated issues, culminating in the creation

of an independent pool playing robot.

1.2 Problem definition

The overall goal of this project is to design and build a robot that is capable of

competing against a human opponent in a game of pool (more specifically, a modified

version of 8-Ball).

This will be achieved by utilising a number of different systems and processes. Pri-

mary among these will be a vision system which will determine the position of all of

the balls on the pool table. This information will then be interpreted by a software

program which will use intelligent algorithms to determine the appropriate shot for

the robot to play. This output will be translated into physical actuation through

the utilisation of appropriate robotic and mechatronic systems. A block diagram of

the system is shown in Figure ??.

1

1.3 Aim and significance

Camera

Ball PositionRecognition Shot Selection

Actuator

Command toActuate

The Eye The Brain

Pool Table

The Arm

Software

Hardware

Figure 1.1: Block diagram of the robot subsystems.

1.3 Aim and significance

As detailed above, the main aim for this project is to design and build a robot

which operates independently of human input, and can play pool with speed and

competency comparable to that of a human player.

Unlike most research projects that have open ended targets, the goal of this project

is extremely well defined, with a clear quantifiable target that is easily verifiable.

The significance of this project has its basis not in the application for which this

robot is being constructed, but rather in the development of the systems and struc-

tures behind the application that have the potential for future usage in a wide variety

of industries. These include fields such as automotive, defence, space, manufactur-

ing, and many other areas where there is a reliance on automation.

The technology employed in this project could be transferred to any number of more

practical and relevant industrial applications.

Applying the technology to the game of pool removes the need to research and

understand a complex industrial application for the robot, thereby allowing more

attention to be focused on the design and implementation of the robot, rather than

on the application for which it will be used.

2

2 Literature review

2.1 Introduction

Due to the practical nature of this project, traditional literature review techniques

are not entirely appropriate. While the literature review forms the basis for the

overall design process, this project does not seek to build on previous work, or fill

in gaps in the research, as is the case in most engineering research projects.

This section will look into the physical mechanics of playing pool and how these can

be realised in the robotic arena, as well the different design concepts that have been

used to achieve this in the past.

2.2 The game of pool

To look at how to translate a fairly complex human game into a robotic process,

it was necessary to obtain a detailed understanding of both the kinematics and

physical descriptors associated with the game of pool. It was also necessary to gain

an understanding of the rules governing the play of the game.

? has covered all of the major areas concerning the physics involved in the game

of pool, including basic two-dimensional kinematics, the effects of spin, sliding and

rolling friction, the inelasticity of the collisions, and the effects of different surfaces

in the game. There were a number of important facets of pool that were discussed

in this paper and used during the design phase of this project. For example, the

relationship between cue tip point of contact, spin and the corresponding loss of

translational momentum was a major consideration when designing where to posi-

tion the cue tip.

A more detailed analysis of the game of pool is given in Chapter ??.

The rules governing a pool game differ widely, depending on the location and situ-

ation in which the game is played. There is, however, a core of rules that remain

3

2.3 The eye

unchanged despite localised modifications. The rules used for this project are based

on a set of 8-Ball rules as defined by the Billiard Congress of America (?). Some

modifications or simplifications were made to the rules for the project, in order to

reduce the level of complexity in the design requirements.

For a complete description of the rules of 8-Ball, see Section ??.

2.3 The eye

In order to design the vision system software, a wide range of image processing

techniques were investigated. ? and ? cover many relevant techniques, particularly

those relating to template matching and segmentation. The possible applications of

these techniques to the robot vision system are closely examined in Chapter ??.

Lens distortion correction is a topic that has been widely covered in literature. Most

of the papers written focus on deriving an equation that models the curve of the

distortion surface.

The Open Source Computer Vision Library (?) is a comprehensive collection of

image processing functions. ? has implemented some of these in an extensive

Matlab toolbox that can determine the intrinsic distortion properties of a lens

from a number of calibration photos which consist of a checkerboard photographed

at different angles. From these intrinsic properties, an undistorted image may be

produced from the original using filtering, as mentioned.

Others have derived simpler equations which rely on various assumptions, for ex-

ample ?, ?, ?. These methods rely on a single calibration image, and Harri Ojanen

has written Matlab software that implements the work he has published.

? look at removing the effects of lens distortion in the absence of knowledge relating

to the properties of the lens. In fact, their approach is unique in that is looks

at the fact that “lens distortion introduces specific higher-order correlations in the

frequency domain”, and thus lens distortion can be reduced by minimising these

correlations. However, their results do not provide an exact method of removing

lens distortion.

4

2.4 The brain

2.4 The brain

Many computer programs have been written which simulate the games of pool or

snooker. (The two games are sufficiently similar that most of the game strategies

are common to both, and thus relevant to this project.) Many of these programs not

only allow a human player to interactively take turns in the games but also simulate

computer players who independently decide their own shot and execute them.

One such program is The Snooker Simulation, written by Peter Grogono. His

notes (?) discuss the formulation of automated strategies. The basic strategy imple-

mented in his program is to first generate a number of shots which might be played,

and then to assess the feasibility of the shot and estimate its usefulness. The best

shot is then selected based on these criteria. This method is discussed in more detail

in Chapter ??.

2.5 Previous pool playing robots

There have been numerous previous attempts at designing and constructing a robotic

pool player from as far back as the early 1980’s. This report will analyse four of

these. Due to the limitations of the available technology at the time, all of them

have made concessions to both the method of play and the overall goals of their

robots. All have approached the problem from a different design perspective that

has had an impact on both the design and success of their robots.

2.5.1 The Tianjin Normal University model

The Tianjin Normal University robot (?) utilised a two degrees of freedom mobile

cart and a four degrees of freedom manipulator.

In designing this robot, Qi and Okawa made a number of major simplifications to

the game and the way in which it would be played. This robot was designed to play

a much simplified version of the game of pool: it could only work when there were

two balls on the table and it played a limited number of shots. Qi and Okawa also

appear to have based their concept design on the manner in which a human being

5


would play pool. The robot moves freely around and over the table without being

attached or restricted in any way.

As a result they have created a system that would allow, upon further development,

greater flexibility in the environment that it could be used in. However in doing so,

they have also raised the level of complexity associated with the design. In creating

a free standing, independent, human sized robot, as Qi and Okawa have attempted

to do, it is necessary to make concessions with regard to the level of performance

gained from the system.

The fundamental objective and perspective on which their project was based is

different to those of this project. They decided to design a robot that would imitate

a human, rather than to view the design of the robotic pool player pool as an

industrial application to be modelled and implemented. The final result of this

difference in perspective is manifested in the overall design which added a number

of avoidable levels of complexity that ultimately served to hamper the ability of the

robot to operate effectively.

This occurs as a result of the manner in which human beings play pool. For example,

humans have an ability to judge depth and angle from an oblique location (for

example, at the end of a pool cue) that is difficult to imitate in robotic form. Rather

than taking the view that a single camera from above can locate the balls on the

table, they followed the path of determining where the balls are located by using the

robot itself as a reference point. By placing the camera such that a ball occupies

a certain position in the image, the robot can calculate the position of the ball on

the table from the angles of its joints. As mentioned in the report, this allows for

greater accuracy, but ultimately would require a method of approximately knowing

where on the table the balls are in order to quickly determine where to move the

camera. Additionally, because the camera moves in three-dimensional space, the

algorithms involved are necessarily much more complex.

2.5.2 The University of Bristol model

An example that is more closely associated with this goal is the robot that was built

by the University of Bristol (?), which utilised a modified IBM industrial robot as

6


the actuation mechanism. Pictures of this robot in action can be seen in Figure ??.

Figure 2.1: The University of Bristol robot. From ?.

This robot was developed by the University of Bristol’s Department of Mechanical

Engineering in the mid-1980’s.

It utilised an IBM 7565 gantry assembly robot, an Automatix AV4 vision system,

and an IBM 6150 (RT) supporting an expert system written in SLOP (a Support

LOgic Programming language). A snooker table measuring 4 feet by 6 feet was used.

The IBM 7565 could reach 90% of the playing area of the snooker table.

This robot had a much more advanced vision system than did the Qi and Okawa

version, and could play on a table that was setup as for a standard pool game.

The University of Bristol robot did not have the ability to perform in real time

against a human opponent. This is a property that all of these previous pool playing

robots share.

This robot could not compete against a human, as it did not have the ability to

7


allow human competitors to take their turns. All of the shots had to be taken by

the robot, as it was not programmed to differentiate between turns.

One other obvious restriction on the ability of human competitors to play against

it was the physical construction of the robot.

The traverse of the actuator, as seen in the images above, was located above the

pool table and was bolted to the table via four posts. This would have actively

prohibited the ability of a human to play on this table as they would not have been

able to get as close as necessary to play or to see all of the balls accurately.

Given that the manipulator for this robot was a pre-assembled industrial robot that

was simply modified to hold a cue tip, there was very little that could be done to

improve the accessibility for a human player.

2.5.3 The Massachusetts Institute of Technology model

The MIT model (?) also makes concessions to the overall performance of their robot

in order to reach the design goals.

The report states that a pool playing robot requires up to five degrees of freedom to

be fully functional, but in order to simplify their design, only two degrees of freedom

are present in their final design. This limited the range of the robot to a single fixed

position from which to take a shot.

Their robot only considered a single ball located on half of a pool table. The cue

can only hit from a fixed position — in other words it plays the same shot each time.

The cue is driven by a combination of a stepper motor and spring that provides the

energy to the actuator. The cue rests on a platform with the motor drawing back

the spring, which is then released by a solenoid.

There was no design process outlined in the published paper. It was therefore

difficult to understand why various decisions were made and in what context certain

options were discarded. Without the necessary technical support it was found to be

almost impossible to venture reasons as to why any of the decisions were made, for

example why there was a major difference between the degrees of freedom present

and those laid out in their report. There was no electromechanical control system

8


for the robot, beyond rudimentary computer-based control that only extended as

far as directing the robot when to fire the solenoid.

The focus of their project, as far as image processing is concerned, was not the

determination of the ball locations, but rather the tracking of a moving ball. As

such, the features of the vision system of the MIT model has little in common with

the desired features of the robot to be built for this project.

This project was completed in 1991, and it is interesting to compare the differences

between the technologies used then and now. Rather than processing the image using

computer software, which even today is relatively slow, this project used dedicated

hardware to analyse the image coming from the camera. This allowed the robot to

track the movement of the ball in real time, at approximately 30 frames per second.

The hardware simply used a threshold on the image and averaged the pixels to

estimate the location of the centre of the ball (this method assumes that only one

ball is within view). While this method is not required for accuracy, this shows how

much faster it is to use hardware than software for data processing, even ten years

ago. However, the disadvantage of using dedicated hardware for image processing is

the cost. A typical hardware board for image processing costs about AU$10,000 (?).

Regardless, the image processing techniques used for the MIT project are not useful

to this project, as the type of image processing being performed is different.

2.5.4 The University of Waterloo model

The most promising previous incarnation of the pool playing robot was the robot

designed and constructed by undergraduate students at the University of Water-

loo (?).

A schematic of the initial robot design is shown in Figure ??.

The actuation system for this robot is based on a Cartesian robot traverse that

allows the actuator to move to any position on the table in order to play a shot.

The system is driven by stepper motors which are attached to worm gear shafts that

are controlled by a PC.

The traverse is fixed to the table and the robot moves around on a series of rails.

9


Figure 2.2: Schematic for the University of Waterloo robot. From ?.

The motors are controlled directly through the PC via a CIO-DAS08/Jr-AO I/O

card with two input signals and five output signals.

However, again there were major simplifications made to the manner in which the

robot played pool. The most obvious is the choice of cue. Instead of using a

shaft with a traditional cue tip attached, the designers of this robot used a plastic

mushroom shaped paddle in order to knock the balls around (shown in Figure ??).

Figure 2.3: The mushroom shaped paddle used in the University of Waterloo robot.

From ?.

This causes immediate restrictions on the ability of the robot to play a realistic

game of pool. In any game of pool, situations arise where the white ball is located

10


extremely close, or indeed touching, another ball. In these situations, it is imperative

that the actuator be able to compensate for this, and to play the desired shot without

too much interference with the other balls on the table. The robot designed by

Richmond and Green could not achieve this.

Any balls near to the target ball would have to be moved outside the radius of the

paddle, in order to not inhibit the ability of the robot to play the desired shot.

Clearly this is not a desirable restriction, but it was one which Richmond and Green

chose to concede to in the design of their robot.

Due to the type of actuator chosen for the robot, the angle of rotation for the cue

tip is not relevant. This reduces the level of control programming associated with

the robot.

The time taken for the robot to reach its desired shot position was another factor

that Richmond and Green decided should not be a major consideration in their

robot’s performance criteria. This is evident in their selection of worm gears as the

drive mechanism for their robot.

A schematic of a typical worm gear is shown in Figure ??. Worm gears operate in

areas that require high location accuracy as well as minimal backlash. They achieve

this by having a property that no other type of gear can achieve: the worm can easily

drive the gear, but the gear cannot force the worm to rotate, thereby removing any

potential backlash. This is because the angle of the worm gear teeth relative to the

gear is so small that, when the gear attempts to spin the worm, the friction between

the worm and gear teeth holds the worm in place. For the application of a robotic

pool player this is an important consideration, due to the relatively large impulsive

force that is applied to the traverse at the moment of contact with the cue ball. If

the traverse cannot adequately resist the backlash generated, then this may cause

inaccuracies in future shots due to the errors generated in the calculation of the

position of the actuator.

Worm gears are relatively slow to drive, owing to their unusual geometry, which is a

major drawback. For this project it was felt that there should be a balance between

the necessity for speed and the requirement of accuracy of location. There are other

alternatives for minimizing backlash in the system that do not reduce the overall

speed of the actuator.

11


Figure 2.4: Typical worm gear configuration. From (?).

Digital cameras are substantially cheaper and higher quality now than at the time

of the University of Waterloo project, so the vision system was not advanced as the

one planned for this project. In order to determine the positions of the balls, a mean

of the location of the pixels comprising the image of each ball was used. To isolate

the edges of the table image, a Sobel Operator was used with a threshold, and the

straight lines obtained were located with the Hough Transform.

2.5.5 Summary

As shown above, all of the previous incarnations of a mobile pool playing robot have

made significant reductions to the accuracy of the robot, and its ability to play pool.

None of the systems outlined in the literature review had managed to incorporate

real time play against a human opponent into their designs.

12

3 The game of pool

3.1 Rules of 8-Ball

A thorough knowledge of the rules of pool is crucial in the design of a pool-playing

robot. The rules are, in effect, an idealised complete specification of the requirements

of the robot.

During the design process, it became apparent that constructing a robot which could

strictly obey all of these rules would be infeasible within the time frame. For this

reason, some of the rules were modified in such a way as to preserve as much as

possible the spirit of the game, but to allow a less complex robot design.

There exist a multitude of different versions of pocket billiards (that is, billiards

played on a table with pockets), each of which has different rules of play. The game

of 8-Ball was chosen to form the basis for this project as it is the simplest and most

easily modifiable version. The others, including snooker and American 9-Ball, all

have rigid play structures that add unnecessary complexity to designing a robotic

player, at least in its first iteration.

8-Ball is a game played with a cue ball (the white ball) and fifteen object balls,

which are (in a standard game set) numbered 1 through 15 (?). One player must

pocket the balls numbered 1 through 7 (the solid balls), while the other player must

pocket those numbered 9 through 15 (the striped balls). The first player to pocket

all balls in his group, and then legally pocket the 8-ball, wins the game. A overhead

view of the pool table can be seen in Figure ??, with the balls arrayed as for the

start of a game.

The game is played in an alternating order — one player breaks and continues to

play shots, until such time that a target ball they have hit with the cue ball fails to

be pocketed, or they foul. Then the other player begins their turn. This is repeated

until a player has won.

8-Ball is technically a call shot game, meaning that, before executing a shot, the

player must nominate the pocket into which the target ball selected for the shot will

13

3.1 Rules of 8-Ball

Figure 3.1: Setup for a game of 8-Ball. From (?).

be pocketed. This rule is not generally enforced, and most games are played on a

much more casual basis. However, it is possible, in the instance where the computer

has calculated the desired shot, for the robot to adhere to this rule.

A major modification to the rules was made with regards to the break. The break

is the opening shot of the game, and there is a strict code for what constitutes a

legal break. The rules from the Billiard Congress of America, state that the breaker

must either pocket a ball, or drive at least four balls to the rail. This rule may be

disregarded, as it was felt that this would complicate the overall design process by

enforcing a minimum upper bound for the force supplied by the actuator that may

not be feasible.

The rules concerning what constitutes a foul shot are fairly detailed and play a

major part in the outcome of any game.

A foul shot results in that person who played the foul losing a turn, or equivalently,

giving the other player two shots in a row. Games can be won or lost on the basis

of a foul.

As a result, it was felt that it would be appropriate to ensure that as many of

the rules as possible concerning fouls were incorporated into the game, with the

exception of the foul rule regarding the legality of a shot.

A shot is only legal if the shooter hits one of their group of balls first, and either

14

3.2 Physics of pool

pockets a ball or causes the cue ball or any ball to contact a rail. This is again felt

to increase the level of complexity required of the system beyond a reasonable level

for a first time version of this robot.

This rule does not significantly influence the outcome of a game in the way that

other foul rules do, such as fouling by hitting an opponent’s ball before the player’s.

Other foul shots include: illegally pocketing an opponent’s ball; not making contact

with any of the target balls with the cue ball; pocketing the cue ball; and moving

any of the target balls with anything other than the cue ball.

All of these fouls are crucial to the overall outcome of the game, and should be taken

into account when considering the design of the robot.

3.2 Physics of pool

In order to design and build a robot that can compete against a human being, it is

necessary to ensure that the algorithms governing the control and performance of

the robot take into consideration the physical behaviour that describes a game of

pool.

Pool is a game that, when looked at in any detail, has a remarkable level of com-

plexity governing the behaviour of the balls on the table. These complexities have

the potential to render hopeless any attempt to design a robot to handle these situa-

tions. Therefore it is necessary to make appropriate assumptions in order to obtain

results that are reasonable, but remove some of the more extreme complications

from the calculations associated with shot selection and actuation.

The shot selection software must clearly use principles of kinematics to calculate ball

trajectories. At this point, all analysis has been performed under the assumption

that balls travel atop the table without horizontal spin. This is not the case in

reality. Balls often have a significant amount of horizontal spin, which causes their

paths to curve, as well affecting the forces during collisions with other balls and the

table cushions. However, the degree of complexity in calculating horizontal spin and

its effects is very high, and thus the shot selection algorithm of the robot neglects

these effects. This is explained in more detail in Chapter ??.

15

4 The eye

4.1 Introduction

The function of the vision system, or the “eye” of the robot, is to accurately deter-

mine the positions of the balls on the table.

While there are several possible concepts for physical systems which would be able

to determine the location of the balls on the pool table, the use of an optical camera

was the only one which had the potential to provide accurate results for a reasonable

cost. Thus the hardware component of the eye of the robot consists of a camera

which captures images of the table for software processing.

A table with blue felt measuring 1800 mm × 950 mm was used for the project. In

order to simplify the image analysis, it was decided that the robot would be pro-

grammed to play 8-Ball with a set of “casino” balls. “Casino” balls consist of four

types of monochromatic balls: a white ball (the cue ball), a black ball (the eight

ball), seven red balls (the “bigs”) and seven yellow balls (the “smalls”). There

were two advantages of using these balls. Firstly, the total number of different ball

colours that had to be recognised was reduced to four. Secondly, as these balls are

monochromatic, they have the same appearance irrespective of their position or the

angle from which they are viewed. It should be noted that the use of “casino” balls

does not in any way affect gameplay.

The vision system software must be able to instruct the camera hardware to take

a photo and download that photo. The crux of the software’s role is to determine

the arrangement of balls on the table, identifying each by its colour and estimating

its position. The software must perform these functions with sufficient speed and

accuracy. Determining the positions more accurately requires more computation

and thus more time, so a compromise between speed and accuracy is required.

16

4.2 Error analysis

4.2 Error analysis

A key step in the design of the vision system was the determination of the resolution

required to accurately play a successful shot. Therefore, an analysis of the errors

involved in a shot was performed.

It is important to note that for some sets of pool balls, the radius of the cue ball is

slightly different to the radii of the other balls. While this is taken into account to

some extent here, all three dimensional effects caused by this difference are neglected.

For example, it is assumed that two balls are touching when the horizontal distance

between their centres is equal to the sum of their radii. This is not strictly correct, for

if the two balls have different radii, then their centres will be at different heights from

the table, so they will not be quite touching. The errors caused by this assumption

are very small, so it is always assumed that the centres of all the balls lie on a

single horizontal plane. When coordinates are used, this plane is the x-y plane of

the coordinate system.

Let rc be the radius of the cue ball, and let r be the radius of all other balls.

Denote the initial stationary positions of the cue ball and the target ball by C and

T , respectively, and denote the position of the desired target position (which will

usually be a pocket) by P . Let d, b denote the distances |CT |, |TP | respectively.

Let φ be the angle of−→CT with respect to a fixed direction i.

In order that the target ball moves in a direction towards P , it must be struck by

the cue ball at the position X such that X, T, P lie on a straight line in that order,

and |XT | = rc + r, as shown in Figure ??. (This is true so long as the effects of

spin and friction between the balls is neglected.) Let a denote the distance |CX|,and let α and β denote the angles 6 TCX and 6 CTX respectively.

If the positions C, T and P are given, then d, b and β are known. Then a can be

calculated using the cosine rule:

a2 = d2 + (rc + r)2 − 2d(rc + r) cosβ. (4.1)

Then α can be calculated using the sine rule:

sin α

rc + r=

sin β

a. (4.2)

17

4.2 Error analysis

C

X

T

P

da

b

iφ

α

β

Figure 4.1: Geometry of a shot.

Thus the necessary direction (that is, at an angle θ = φ + α to i) in which the cue

ball should be struck in order to collide with the target ball and propel it towards

P has been determined.

Now suppose that the cue ball is instead struck with an angle error of ∆θ, so that

it is propelled at an angle θ + ∆θ to i.

Assuming that the cue ball still collides with the target ball, denote by X ′ the

position of the cue ball as this collision occurs. |X ′T | = rc + r, as these are the

positions of the balls as they collide. Let α′ = 6 TCX ′ = α + ∆θ. Let a′ denote the

distance |CX ′|, and let β ′ denote the angle 6 CTX ′.

6 CX ′T = 180◦ − α′ − β ′, so the sine rule yields

sin(180◦ − α′ − β ′)

d=

sin α′

rc + r, (4.3)

which can be used to solve for β ′, keeping in mind that 6 CX ′T = 180◦ − α′ − β ′

must be an obtuse angle, from the geometry of the situation.

This collision will cause the target ball to travel in direction−−→X ′T , which gives an

angle error of ∆β = β ′ − β in its trajectory. As the distance to the pocket is

18

4.2 Error analysis

b, the distance, δ, by which the centre of the target ball will miss the point P is

approximately

δ = b sin ∆β. (4.4)

Now suppose that the cue ball and the target ball are at positions C(xC , yC) and

T (xT , yT ), and suppose that the vision system then estimates their positions as

C(xC , yC) and T (xT , yT ).

Assuming that the errors (∆xC , ∆yC) and (∆xT , ∆yT ) are small, the errors in the

estimation of d, b and β are also small. Therefore, equations (??), (??) will produce

little error in the estimation of a and α. A Matlab program was written to check

that the errors in the estimation of the positions of the cue and target balls produced

greater errors in the estimation of φ than in the estimation of α. It demonstrated

that the errors in α were generally around a factor of 10 smaller than those in φ for

a wide range of configurations. The hypothesis was thus verified, so it is reasonable

to not consider the error in α.

Let ∆vC and ∆vT be the components of the errors (∆xC , ∆yC) = (xC−xC , yC−yC)

and (∆xT , ∆yT ) = (xT − xT , yT − yT ) in a direction perpendicular to−→CT , as shown

in Figure ??.

It can be seen that

tan(φ− φ) =∆vC + ∆vT

b, (4.5)

so that, since the error in the angle is small,

φ− φ ≈ ∆vC + ∆vT

b. (4.6)

Let ∆θE denote the component of the error in the trajectory angle of the cue ball

which is due to the vision system. So

∆θE = (φ + α)− (φ + α) ≈ ∆vC + ∆vT

b, (4.7)

since it has been shown that α−α can be approximated as zero. There will also be a

component, ∆θA, of the error in the trajectory angle which is due to the actuation.

Thus the total trajectory error is ∆θ = ∆θE + ∆θA.

19

4.2 Error analysis

C

C∧

T

T∧

d

∆vC

∆vT

iφ

iφ∧

Figure 4.2: Errors in the positions of the cue and target balls.

A Matlab program was written to find δ, given rc, r, d, b, β and ∆θ, using equations

(??), (??), (??) and (??). Assuming rc = r = 25 mm, the allowable tolerances ∆θ

in the trajectory angle were calculated, for various values of d, b and β, which would

ensure δ < 10 mm. From this, the allowable tolerances ∆v in the location of a ball

position in any given direction were calculated (assuming that ∆vC = ∆vT = ∆v).

The results are shown in Table ??.

The results show how accurate the vision system needs to be if the robot is to

be able to consistently successfuly perform shots of each type. As the length of

the table is approximately 1500 mm, shots over distances greater than 1000 mm are

near the difficult end of the scale. Additionally, shots with a high turn angle, β, are

also difficult shots. The vision system should not necessarily be designed to achieve

these shots. However, considering those shots which an average human pool player

might hope to consistently achieve, it can be seen that the vision system should be

able to locate the position of a ball in a given direction (for example, in the x or y

directions) to within approximately 0.5 mm.

20

4.2 Error analysis

Table 4.1: Tolerance of aim angle for given situations.

d β b ∆θ ∆v

100 mm [−5.682◦, 5.682◦] 4.959 mm

100 mm 0◦ 300 mm [−1.908◦, 1.908◦] 1.665 mm

1000 mm [−0.5729◦, 0.5729◦] 0.5000 mm

100 mm [−3.110◦, 2.382◦] 2.078 mm

100 mm 30◦ 300 mm [−0.9513◦, 0.8704◦] 0.7595 mm

1000 mm [−0.2767◦, 0.2695◦] 0.2352 mm

100 mm [−0.1760◦, 0◦] 0 mm

100 mm 60◦ 300 mm [−0.01874◦, 0◦] 0 mm

1000 mm [−0.001664◦, 0◦] 0 mm

100 mm [−1.145◦, 1.145◦] 2.997 mm

300 mm 0◦ 300 mm [−0.3819◦, 0.3819◦] 0.9999 mm

1000 mm [−0.1146◦, 0.1146◦] 0.3000 mm

100 mm [−0.9458◦, 0.8607◦] 2.253 mm

300 mm 30◦ 300 mm [−0.3059◦, 0.2965◦] 0.7761 mm

1000 mm [−0.09078◦, 0.08993◦] 0.2354 mm

100 mm [−0.4248◦, 0.3161◦] 0.8276 mm

300 mm 60◦ 300 mm [−0.1293◦, 0.1172◦] 0.3069 mm

1000 mm [−0.03751◦, 0.03642◦] 0.09536 mm

100 mm [−0.3015◦, 0.3015◦] 2.631 mm

1000 mm 0◦ 300 mm [−0.1005◦, 0.1005◦] 0.8772 mm

1000 mm [−0.03016◦, 0.03016◦] 0.2632 mm

100 mm [−0.2637◦, 0.2467◦] 2.153 mm

1000 mm 30◦ 300 mm [−0.08603◦, 0.08413◦] 0.7342 mm

1000 mm [−0.02561◦, 0.02544◦] 0.2220 mm

100 mm [−0.1491◦, 0.1217◦] 1.062 mm

1000 mm 60◦ 300 mm [−0.04663◦, 0.04360◦] 0.3805 mm

1000 mm [−0.01367◦, 0.01340◦] 0.1169 mm

21

4.3 Hardware

4.3 Hardware

The hardware component of the vision system must be able to take a photo of the

table with sufficient resolution and download the image to the computer so that it

can be interpreted by the software. These two tasks should be fully automated.

4.3.1 Camera types

A typical digital camera consists of an image sensor chip, a lens, and an interface

between the chip and the user. The image sensor chip is a cluster of light sensitive

diodes, each of which converts the light intensity at their location to a voltage. The

set of voltage outputs corresponds to the set of pixels of an image. If the size of the

chip is greater, then so is the number of pixels of the resultant image, and thus the

resolution of the camera is increased. The lens focuses light onto the chip. Methods

of interface between a camera and a computer are dependent upon the application

of the camera.

4.3.1.1 Still cameras

Consumer still cameras have a compact detachable storage medium on which the

captured images are stored. These cameras usually also have a USB or FireWire port

for downloading the images onto a computer. The range of still cameras available

is now almost as broad as that of conventional film cameras.

Some cameras have the ability to be controlled directly through commands sent

from a computer using a USB or FireWire connection. This function is a desirable

feature for this project, since the robot needs to be able to operate autonomously.

Theoretically, a digital still camera could be used, even if it does not have the

capability for remote operation, by building a mechanical switch that physically

pushes the button on the camera to take a photo. However, this option was deemed

unnecessarily complex, so such cameras were discounted for use in this project.

22

4.3 Hardware

4.3.1.2 Video cameras

A wide variety of video cameras exist, but for the purposes of the project, they may

be broadly distinguished by one feature, namely, the nature of their interface with

equipment, which may either be analog or digital. In order to transmit an image

from a video camera to a computer, a frame grabber must be used. (An analysis

of the motion of the balls is beyond the scope of this project, and thus only still

images are required.)

A frame grabber is an input card into a computer that reads the video feed coming in,

and copies one frame of the video as a digital image upon command. A frame grabber

is relatively cheap if the input is digital (for example, in the case of a webcam), but

expensive if the input is analog, due to the inclusion of a high precision analog-to-

digital converter (for example, in the case of a BNC equipped security camera).

4.3.2 Selection criteria

The criteria considered when choosing the camera were connectivity, accuracy and

price. Additionally, the camera should ideally have an interchangable lens so that

different lenses may be fitted. This provides the camera with flexibility, allowing

it to be used for a variety of other applications within the University of Adelaide,

rather than being limited to use in this project, which would not be cost effective.

4.3.2.1 Accuracy

As can be seen from Section ??, the greater the accuracy with which the balls can be

positioned, the wider the range of shots the robot will be able to play successfully.

The accuracy of the camera has a significant impact upon the accuracy of the vision

system as a whole.

The accuracy of the camera is determined by two things: the number of pixels and

the distortion of the lens. Distortion from the lens is related to the width of the

image compared to the distance the camera is away from the object. A webcam, for

example, is designed to be mounted close to a user’s head, while providing a wide

field of vision. This introduces a very high amount of distortion.

23

4.3 Hardware

While it is desirable to minimise distortion, the camera cannot be mounted so high

above the table that the setup becomes impractical. However, the distortion may be

adjusted using a correction algorithm, as is discussed in more detail in Section ??.

Since video cameras are designed to capture image data many times per second,

their image sensors cannot process as much data as a still camera. For this reason,

the resolution of most digital video cameras does not exceed around one megapixel.

Still cameras, on the other hand, have the capacity for much greater resolutions.

4.3.3 Camera options and selection

A variety of cameras were investigated. A sample of those looked at are tabulated

in Table ??.

Camera Number of pixels Price (May 2002)

Webcam 640× 480 $150

pixeLink 1280× 1024 $3500

Canon ≈ 6million $4500

Kodak ≈ 6million >$10k

Table 4.2: Various digital cameras considered.

Webcams were deemed unsuitable for the task. As well as having low resolution,

the large distortion of the lens would prevent the robot from accurately determining

the location of the balls without the use of coarse distortion filtering.

Video cameras with analog outputs such as security cameras do not have much

better resolution than a webcam, although a lens could be used that would give less

distortion. Additionally, an analog frame grabber would be necessary to transfer the

information to a computer, which would increase the cost of the system substantially.

The pixeLink camera provides high resolution with ease of automation. It also has an

interchangeable lens. However, its only interface is through a FireWire connection,

so the camera could only be used for applications in which it was controlled throuh

a computer program. This makes it difficult to use for other applications which

require more flexibility.

24

4.3 Hardware

The Canon and Kodak ranges of professional still cameras were also considered.

These cameras have very high resolution, as well as very high colour sensitivity.

Their lenses are interchangeable, and their software allows remote control of the

camera through a digital connection. They can also be used as regular (“point and

click”) cameras.

The price of the camera (except for a webcam) is far beyond the budget of the

project. The Department of Mechanical Engineering was prepared to contribute

money towards the purchase of a camera, but only so long as it were suitable for use

in applications other than the pool playing robot project, including use as a regular

camera.

4.3.4 Camera choice

The camera that was used was a digital SLR camera, the Canon EOS D60. It was

chosen for two main reasons: it has a high quality CMOS image sensor that has a

resolution of 6.29 megapixels, allowing very accurate analysis and location of the

ball positions; and it could be interfaced with the computer extremely easily.

Third party software for the camera was purchased that allowed remote operation

from within MATLAB. This software, called D30Remote, is available at the Breeze

Systems website (?). This program is functionally similar to the Remote Capture

software that was supplied by Canon with the camera. However, D30Remote also

allows other programs to link to it with a dynamic link library. An example console

program called D30RemoteTest, written in Visual C++, was supplied with the

software. D30RemoteTest was capable of commanding the camera to take a photo,

downloading the photo, and saving the image in the current directory. This program

was modified cosmetically for the project, and renamed TakePhoto.

The program TakePhoto ensured very easy connectivity between the camera and

the computer, because it was able to be called from Matlab. This made it very

simple to obtain a photo image of the table during a Matlab program.

25

4.4 Image representation

4.3.5 Image processing in hardware

Rather than processing the photo from the camera using software on a computer,

it is possible to design dedicated hardware to accomplish the same tasks. Image

processing hardware is used in industrial applications where no interface with a

computer is required. This allows the image processing unit to be much more com-

pact and mobile. However, this is not an advantage for the purposes of this project,

since a computer is required to run the shot selection software in any case.

Image processing hardware has another advantage over computer software. Hard-

ware is much faster than software, as can be seen by its use over ten years ago to

track a moving pool ball at 30 frames per second (?). This processing speed cannot

be achieved with software analysis even on today’s computers.

Image processing hardware, however, is less flexible than its software counterpart.

A given hardware processor has a number of built-in functions, but does not have

the flexibility to perform any others.

Since the University of Adelaide does not own any image processing hardware, the

equipment would have to be purchased. Because of this, the cost is too prohibitive

for use in this project.

4.4 Image representation

A discrete two-dimensional greyscale (black and white) image is represented by a

matrix, f , where the entry fij represents the intensity of the pixel in the ith row

and the jth column. Equivalently, the image can also be represented by a function,

f , with f(i, j) = fij.

A discrete two-dimensional colour image must be represented by three two-

dimensional arrays, fR, fG, fB, where fR(i, j) represents the intensity of the red

component of the pixel in the ith row and the jth column, and fG(i, j) and fB(i, j)

represent the intensities of the green and blue components respectively.

26

4.5 Template matching


It is known what the image of a coloured ball against the felt background will

approximately look like. Thus, a small template of this image can be created. Then

the regions of the image of the table which match this template most closely must

be found. This technique is known as template matching, and is discussed in many

image processing textbooks, for example, ?.

Let f be a greyscale image, and let g be a template. Consider a region R(x, y) of f

which is the same size as g and displaced by (x, y). (For example, if g was a 2× 3

image, then for R(4, 6) = {(5, 7), (5, 8), (5, 9), (6, 7), (6, 8), (6, 9)}.) To compare the

template g to this region of the image f , the mismatch energy is defined as (?)

σ2(x, y) =∑

(f(i, j)− g(i− x, j − y))2, (4.8)

where the sum is taken over the indices (i, j) ∈ R(x, y). Expanding the brackets,

σ2(x, y) =∑

f(i, j)2 +∑

g(i− x, j − y)2 − 2∑

f(i, j) g(i− x, j − y). (4.9)

Therefore, this can be minimised by maximising the cross-correlation,

cfg(x, y) =∑

f(i, j) g(i− x, j − y). (4.10)

This cross-correlation can be computed very quickly in the frequency domain, as

Cfg(u, v) = F (u, v) G?(u, v), (4.11)

where Cfg(u, v), F (u, v) and G(u, v) are the two-dimensional discrete Fourier trans-

forms of cfg(x, y), f(x, y) and g(x, y), and ? denotes complex conjugation.

However, the cross-correlation (??) is not necessarily the best estimate of similarity

between the template g and the region R(x, y) of the image f . In equation (??), it

can be seen that large variations in the average intensity over the region R(x, y) of

the image f will greatly affect the mismatch energy.

A better measure of the similarity is given by the normalised cross-correlation,

γfg(x, y) =

∑

[f(i, j)− fR(x,y)][g(i− x, j − y)− g]√

∑

[f(i, j)− fR(x,y)]2√

∑

[g(i− x, j − y)− g]2, (4.12)

27


where fR(x,y) is the mean value of f over the region R(x, y) of the image, and g is

the mean value of g over the entire template. The computation of the normalised

cross-correlation (??) is significantly slower than that of the (unnormalised) cross-

correlation (??).

These cross-correlation functions only work for greyscale images. In order to cor-

relate two colour images, it is necessary to split each image into their three colour

components, then perform the normalised cross-correlation between corresponding

images. In order to then find the locations at which there is high correlation in all

three colour components, it is necessary to combine the three results in some manner

to obtain a single correlation map. This may be achieved by simply averaging the

three cross-correlations:

γfg(x, y) =γfg,R(x, y) + γfg,G(x, y) + γfg,B(x, y)

3. (4.13)

4.5.1 Interpolation

Once the correlation map between the photo and the template has been found, it is

not difficult to find a peak. The location (m, n) of the maximum value in the cross-

correlation is determined. (If there is more than one position with this maximum

value, then any one is chosen.) Here, m and n are the integer indices of an entry in

the matrix where the maximum value is found.

However, this peak will only provide the location of the centre of a ball in the

photograph to the nearest pixel. For subpixel accuracy, the “true” location of the

maximum of the peak (that is, where the maximum would occur if the photo were

perfect and continuous, rather than noisy and discrete) must be interpolated from

the known values.

To achieve this, a neighbourhood region of (m, n) is examined. A region consisting

of a rows and columns on each side of (m, n) is extracted from the cross-correlation,

resulting in a (2a + 1)× (2a + 1) array of values (m + i, n + j) for i, j = −a,−a +

1,−a + 2, . . . , a− 1, a.

This region is then interpolated k times, each time increasing the number of points

in each direction by a factor of 2. This results in a (2k+1a + 1)× (2k+1a + 1) array

of interpolated values (m + i, n + j) for i, j = −a,−a + 12k ,−a + 2

2k , . . . , a− 12k , a.

28


Figure 4.3: Template of the white ball.

The maximum of the new array of interpolated values of the cross-correlation can

now be found. If the interpolation method approximates the “true” shape of the

cross-correlation peak sufficiently well, then the location of this new maximum

should provide a more accurate approximation of the “true” location of the maxi-

mum of the peak.

The interpolation method was tested on a real photo taken of a portion of the table.

A template of the white ball on the blue background is shown in Figure ??. The

white colour used in the creation of this template were found by taking a similar

photo, and averaging the colours of a set of pixels known to belong to the white ball.

Similarly, the colours of the yellow, red and black balls, as well as the blue colour of

the felt, were found in this fashion.

This template was then cross-correlated with the image of the table, in each colour

component. The three resulting cross-correlations were then averaged. Results are

shown in Figure ??.

The highest peak in the averaged cross-correlation was found, and a neighbourhood

region of this pixel was analysed — in this case, a 3× 3 region (since a = 1). The

values were then interpolated, by performing spline interpolation k = 4 times. This

is shown in Figure ??. Thus the location of the peak could be estimated more

accurately.

This was repeated with templates of the red and yellow balls, until the locations of

all balls were found. The original image of the balls is shown in Figure ??, with

circles plotted over the image where the program estimated the balls to be. The

results seemed very accurate. However, it is difficult to quantify the error, since the

true positions of balls within a photo cannot be found to sufficient accuracy.

29


0

0.2

0.4

0.6

0.8

1red cross−correlation

0

0.2

0.4

0.6

0.8

1green cross−correlation

0

0.2

0.4

0.6

0.8

1blue cross−correlation

0

0.2

0.4

0.6

0.8

1averaged cross−correlation

Figure 4.4: Cross-correlations between an image and the template.

Figure 4.5: Neighbourhood of a peak of the averaged cross-correlation. Initially, a

3 × 3 pixels region around the peak is considered (left). Then spline interpolation

is performed 4 times to produce the interpolated cross-correlation (right).

30


Figure 4.6: Results of the normalised cross-correlation interpolation program.

A Matlab program was written to investigate the accuracy produced by different

methods of interpolation. The program created an ideal image of a coloured ball

against a blue background, with radius r and centred at a random position C. A

pixel with centre P was coloured iff |PC| < r. A template of the coloured ball

was also created in a similar fashion, except that the image of the ball was centred

at a known position. The normalised cross-correlation between the image and the

template was then performed for each colour component, and the results averaged.

The averaged cross-correlation was then interpolated (using various methods), and

the peak of the interpolated cross-correlation C was then found. This was com-

pared to the true centre of the ball, C. The program was repeated for 100 random

placements of the ball, and the root mean squared error |CC|rms was found.

At the time of these tests, a camera for use in the project had not yet been selected,

so a camera of resolution 1920×1440 pixels was assumed. Then, with a ball of radius

approximately 25 mm and a table of length approximately 1600 mm, the maximum

radius of the ball in an image which captures the entire table is 25 mm1600 mm

· 1920 = 30

pixels. This was the value used in the program.

31


Matlab offers two (non-trivial) methods of two-dimensional interpolation, these

being cubic interpolation and spline interpolation. Additionally, a Matlab function

for performing two-dimensional bandlimited interpolation (achieved by zero-padding

in the frequency domain) was written. Preliminary testing showed that the error in

the location of the ball determined using cubic interpolation was, in general, twice

as large as that obtained using spline interpolation, so cubic interpolation was not

considered further.

The values of the parameters a (relating to the size of the region interpolated)

and k (the number of times interpolation is performed) were varied in order to

find a combination which yielded the least error. Results obtained using spline

interpolation and bandlimited interpolation are shown in Table ?? and Table ??

respectively. Note that these results only demonstrate the accuracy of the method

when an ideal image is used.

Table 4.3: Error in centre location using spline interpolation.

a = 1 a = 2 a = 3 a = 5

k = 2 0.0901 pixels 0.1020 pixels 0.0959 pixels —

k = 3 0.0650 pixels 0.0727 pixels 0.0800 pixels 0.0726 pixels



k = 6 0.0573 pixels 0.0684 pixels 0.0699 pixels —

k = 7 0.0570 pixels 0.0707 pixels — —

It was found that using bandlimited interpolation with the parameters a = 4, k = 6

yielded the most accurate results. However, the interpolation was quite slow, since

the amount of computation required increases exponentially with k. Therefore,

as a compromise between accuracy and speed, it was decided that bandlimited

interpolation with the parameters a = 4, k = 5 would be used. This method

estimated the location of the centre of the ball with a root mean squared error of

0.0433 pixels, which is equivalent to an accuracy of 0.0361 mm.

32


Table 4.4: Error in centre location using bandlimited interpolation.

a = 2 a = 4 a = 8

k = 1 0.1518 pixels — —

k = 2 0.0927 pixels — —

k = 3 0.0658 pixels 0.0521 pixels 0.0514 pixels




4.5.2 Speed considerations

It was found that the using cross-correlation and interpolation as described in the

previous section was slow when actual high resolution images were processed. Lo-

cating balls of four different colours requires twelve cross-correlations (three colour

components for each ball colour), which took over a minute, using the normalised

cross-correlation on 640 × 480 images. Since the images to be used in this project

were over twenty times larger than this, the actual time would be many times longer

than this. This length of time was considered to be unacceptably large.

Whether the unnormalised cross-correlation may be used instead of the normalised

cross-correlation has not been determined. The unnormalised cross-correlation is

more difficult to interpret. For example, the red component an image of the yellow

ball produces a higher peak when cross-correlated with the red component of a

template of a red ball, than when with the red component of a template of a yellow

ball. However, it may be possible to interpret these results correctly, using a more

complex method of combining the three cross-correlation results for each colour

component. This could be advantageous, as the unnormalised cross-correlation is

much faster to compute.

The use of the unnormalised cross-correlation would also be advantageous in that

the two-dimensional bandlimited interpolation can be implemented extremely simply

and quickly in the frequency domain. Since the unnormalised cross-correlation is

first computed in the frequency domain, the result can be immediately zero-padded

33


before conversion back into the space domain, thus saving the time which would

have been required to perform two 2-D discrete Fourier transforms. The normalised

cross-correlation, however, cannot be computed completely in the frequency domain,

and therefore the Fourier transform must be calculated before the interpolation can

be performed.

Another possible method considered for speeding up the program was to produce

only one template of a white ball on the black background (instead of having four

templates of different coloured balls on a blue background). This would mean that

only three cross-correlations would need to be performed. Different coloured balls

would then produce peaks of different heights in the cross-correlation, and it was

hypothesised that, by analysing the magnitudes of the peaks at corresponding lo-

cations in the three cross-correlations, the colour of the ball producing the peaks

could be determined.

The problem of speed comes down to the fact that for the large images produced

by the camera, only a small percentage of the image will contain information re-

garding the ball locations. The process of locating, say, the white ball would be

significantly faster if it was known that the white ball was located within a smaller

region. Methods of searching the image quickly to locate the balls approximately

were investigated. Using such a fast algorithm, the ball locations could be deter-

mined approximately, so that only small regions from the photo where the balls were

known to be would be cross-correlated with the templates.

One method of quickly finding the approximate ball locations is to first perform the

cross-correlations using decimated copies of the original image and the template.

Then smaller regions in the neighbourhoods of the peaks found could be searched

using the high resolution image and the template. This method is known as a

coarse-fine search, and is the method that was finally implemented in the program

due to its simplicity. Other methods researched have been included in the following

sections.

34

4.6 Segmentation

4.6 Segmentation

Several techniques for locating the centre of the balls depend upon the ability to

decompose an image into its components. In other words, it is necessary to know

which pixels belong to the image of the white ball, which pixels belong to images

of red balls, which pixels belong to the image of the blue felt, and so on. This is

known as image segmentation.

? lists several image segmentation techniques. The most important of these is

amplitude thresholding. For greyscale images, the pixels of an object are charac-

terised by an amplitude interval. For example, the pixels of a bright image against a

dark background may be identified by finding all pixels with an intensity amplitude

greater than a certain threshold value.

For colour images, instead of a characteristic amplitude interval, pixels would be

characterised by a region in the space where the three dimensions are the red, green

and blue components of the colour. For example, white pixels might be characterised

by the region in which all three components are greater than 0.9. The difficulty

with this method lies in choosing characteristic criteria that are robust, and will not

miscategorise pixels.

One way of choosing criteria is to plot a histogram of each colour component of an

image. For example, if a camera produces values ranging from 0 to 255 for each

colour component, then a histogram of the red component shows how many pixels

have a red component of each value from 0 to 255. Thus if an image of a red ball

and a yellow ball against a blue background is used, then the red, yellow and blue

pixels can be plotted separately on the histogram, and suitable threshold values for

separating these pixels can be chosen graphically. An example of this is shown in

Figure ??.

The problem with this is that there is often significant overlap between, for example,

red ball pixels and yellow ball pixels in the histogram for any one colour component.

Therefore, for the characteristic criteria to be robust, it is not sufficient to simply

find threshold values for one or more colour components.

Another graphical method is to create a three-dimensional histogram, by plotting

the pixels in a three-dimensional space with orthogonal axes corresponding to the

35

4.6 Segmentation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.01

0.02

0.03

0.04Histogram of blue component

intensity of blue component

norm

alis

ed n

umbe

r of p

ixel

s

white pixelsyellow pixelsred pixelsblue pixels

Figure 4.7: Histogram of the blue components of different coloured pixels. This can

be used to determine threshold values for segmentation. For example, all pixels with

a blue component of less than 0.2 could be defined as yellow pixels. It can be clearly

seen that this criterion is not robust.

36

4.6 Segmentation

red, green and blue colour components. Then a three-dimensional region can be

chosen which characterises, for example, yellow ball pixels.

It can be easier to determine such a region if the axes corresponded to colours other

than red, green and blue. For example, pixels could be represented as a sum of

“ball red”, “ball yellow” and “felt blue” components, where “ball red” refers to

the specific shade of red which is the averaged colour of red ball pixels found in

images used. Then by plotting a three-dimensional histogram with these axes, the

pixels belonging to different elements of the image may be more clearly separated.

Through testing in Matlab, it was found that this idea of changing the basis of

the colour space to define regions was no more robust than using the normal RGB

(red, green and blue) colour components.

Ideally, the thresholding would be performed automatically, based on a sample image

of the table. However, this would be difficult to implement, since the techniques for

defining characteristic criteria are largely based on estimating values from graphical

data.

Another method of segmentation is component labeling. Two neighbouring pixels

are said to be connected if they are sufficiently similar. Of course, this must be

rigorously defined. For example, it might be decided that if the three corresponding

colour components of two neighbouring pixels each differed by less than 0.1, then

the two pixels are connected. Then the image can be divided into connected sets.

Again, the difficulty here is defining connectivity criteria which were sufficiently

robust. This method is even more sensitive to poorly defined criteria than thresh-

olding, for if two pixels are found to be connected by the criteria, when in fact

they should belong to different sets, then the two sets are merged into one set of

indistinguishable pixels.

It has been found through experimental Matlab testing that these methods of

segmentation fail in large part due to the texture of the blue felt. Because the

colour distribution is very uneven and widely spread, it is difficult to define criteria

which will not fail at some point due to variations in the felt colour. Filtering out the

texture of the background could make these segmentation methods more suitable.

37

4.6 Segmentation

4.6.1 Centroid method

A correctly segmented image should consist of several sets of pixels, each correspond-

ing to the image of a ball, and the remaining set representing the felt background.

It is now necessary to find the position of the centre of the ball. A simple method

for doing this is to find the centroid of all the pixels in the set. For a set of N pixels

with coordinates (xi, yi), the centroid is found at coordinates

(

1

N

N∑

i=1

xi,1

N

N∑

i=1

yi

)

.

A Matlab program was written to determine the accuracy of this method. The

program created an ideal image of a coloured ball against a blue background, with

radius r and centred at a random position C. A pixel with centre P was coloured iff

|PC| < r. The centroid C of all the coloured pixels was then found and compared

to the true centre of the ball, C. The program was repeated for 1000 random

placements of the ball, and the root mean squared error |CC|rms was found.

Testing was undertaken to determine the accuracy of the ball centre location method

of finding the centroid. Results are shown in Table ??. Note that these results only

demonstrate the accuracy of the method when an ideal image is used.

Table 4.5: Error in centre location using the centroid method.

camera resolution radius of ball |CC|rms

640× 480 pixels 10 pixels 0.0524 pixels = 0.131 mm




Effects on the ball image, such as reflections and edge aliasing, introduce small areas

of colour that do not closely match the actual colour of the ball. If thresholding is

used to determine which pixels are part of the ball and which are not, it will produce

a set of pixels that does not represent the entire ball. This will bias the average and

produce an offset centre location estimate.

38

4.7 Distortion filtering

For this reason, even though the results given by the centroid method are ap-

proximately twice as accurate as those given by the interpolated normalised cross-

correlation method (both methods being applied to ideal images), it was decided

that the latter method was likely to be more robust to image effects such as re-

flections and glare. Initial experiments in adding noise to ideal images created in

Matlab, in order to simulate camera image noise, supported this hypothesis.

However, this method is much faster at providing an inaccurate estimate of the

position of a ball than using cross-correlation.

4.6.2 Erosion

The morphological erosion operation was also considered as a means of finding the

centre of a ball, once the pixels of the image of the ball have been successfully

segmented. The erode operation works by assigning a pixel a value of 1 if that pixel

and the four adjacent pixels are all equal to 1, and assigning a value of 0 otherwise.

An extension to this function is the ultimate erode function, which has the result

of iterating the erode function until only one pixel from each connected set of 1s

remains equal to 1.

Thus, if a set of pixels of an image were determined to be yellow pixels, then the

ultimate erode function could be applied, leaving one pixel remaining from the image

of each yellow ball, which would be at the approximate centre of the ball image.

However, this function, when used with an actual photo, proved to have very high

levels of noise associated with the output. This was due to effects such as reflections

on the balls and non-uniform felt colour. Also, because the function used iteration,

it was not as fast as the centroid method.


In order for a lens to produce a wide angle image, the three dimensional world is

projected onto a curved surface, which is in turn projected onto a plane (?). This

allows a camera to be positioned closer to its subject without losing the breadth of

39


field, and is a property of all lenses. Therefore, for the purpose of this project, this

lens distortion has the undesirable effect of warping the image such that the image

of the pool table that is processed by the computer is not an ideal projection, but

rather one that has a non-linear distortion. See Figure ?? for an depiction of this.

Figure 4.8: A depiction of lens distortion. From ?.

There are many methods published for removing the effects of lens distortion (see

Section ??). All of the methods introduced use various assumptions of the lens to

generate equations that describe lens distortion factors and functions to undistort an

image using these parameters. They share one unfortunate aspect in that they are

all computationally intensive, and thus, if used for this project, would take too long

to perform their task at the beginning of each turn. For example, using a test image

that was one quarter the size of the images used by the robot, it took somewhere in

the order of 30 seconds to undistort this image using Bouguet’s Matlab undistort

toolbox (?). To perform this function (which would take much longer than 30

seconds with the full size images from the camera) at the beginning of every shot

that the robot plays was considered infeasible.

A new method of removing the effects of distortion was required. Undistorting the

image and then finding the ball locations clearly was a technique that took too long

due to the large size of the images. Instead, the balls were located in the distorted

image, and the positions were subsequently adjusting to their actual locations.

In order to achieve this, the lens distortion was modelled by Matlab very simply.

A grid of squares constructed from string was placed across the surface of the pool

table. A photo of the table and the grid was then taken. The photo can be seen in

Figure ??. The pixel locations of each of the vertices of the grid were determined and

40


entered into Matlab. The actual spatial locations of the vertices of the grid were

also entered. A piecewise linear transform was then created to map corresponding

sections of the grid to each other.

Figure 4.9: A photo of a grid of squares over the pool table, taken by the robot’s

camera.

Once the locations of the balls within the distorted image were found, the transform

could then be applied to these ball location coordinates in order to produce the

actual spatial coordinates of these points on the table. While this method definitely

reduces the errors caused by distortion, it is still an inexact method. The main

problem is that locating the balls in the distorted image using cross-correlation will

produce anomalous results, since in the distorted images, the balls often do not

appear to be round. However, the template is of a perfectly round ball, and so

the template matching process will not identify the centres of the balls reliably.

Therefore, a solution which removes the lens distortion before the image processing

occurs would be preferable.

41

4.8 Locating edges and pockets

4.8 Locating edges and pockets

The task of finding the pockets and the edges of the table is one that needs to

be performed only when the camera is moved, and perhaps also periodically for

calibration. Because these operations only have to be performed occasionally, it is

not imperative that they are fast.

Finding the location of the pockets and finding the edges of the table are intercon-

nected. Once the lines corresponding to the edges of the table have been found,

then the gaps in these lines will correspond to pockets.

Only a limited amount of work was spent investigating algorithms to locate the

edges of the table. It was assumed that position of the camera would remain fixed

with respect to the table for long periods, so manually inputting the positions of

the edges of the table area to the software was an adequate solution. Of course,

automated detection is preferable, particularly if the results are more accurate than

those obtained by manual input.

There is a Matlab function which finds edges in an image, using a gradient operator

to determine where the intensity of the image undergoes a sufficient change to qualify

as an edge. Early experiments in using gradient operators produced images full of

noise, thus failing to find connected lines for the edges of the table.

Once the image has been successfully converted to a binary image showing the lines

of the edges of the table, the Radon transform may be used to determine where the

lines are located. The Radon transform is closely related to the Hough transform,

which was used by the team at the University of Waterloo (refer to Section ??) to

detect the edges of their pool table.

The biggest problem involved in the edge detection is due to lens distortion. The

edges of the table are located in the image in the areas that are affected most by the

distortion. Without filtering to remove the distortion, the images of the table edges

will be curved lines, and will therefore not be revealed by the Radon transform. So

there are two ways to find the table edges with this in mind. The first is to apply

a filter to remove the lens distortion from the image, creating an image of the table

with straight line edges, which can be located using the Radon transform. The other

method involves finding specific points on the cushions and constructing the table

42

4.9 Final eye software

edges around these points. These single points can be undistorted after they are

found, using the undistorting transform used to adjust the ball positions.

4.9 Final eye software

The vision system software begins by calling the program TakePhoto. A photo of

the table is taken, and saved to an image file. The software reads in this image

file and crops it, so that the brown outside edges of the table and all extraneous

surroundings are removed.

The image is decimated by a factor of 8, so that it is 64 times smaller. Then, for

each colour of ball (white, black, red, yellow), the positions of the balls of that colour

on the table are found using a coarse-fine search.

First of all, the decimated image is cross-correlated with an appropriate computer-

generated template image of the ball to be found. Three cross-correlations are

performed, one for each of the red, green and blue colour components, and the

results are averaged.

If the white or black ball is being located, then it is known that there is exactly one

ball of this colour on the table. Therefore, the highest peak in the average cross-

correlation is found, and this is taken as the approximate position of the white or

black ball.

If coloured balls, say red, are being located, then it is not known how many balls of

the colour are on the table. The peaks in the average cross-correlation correspond-

ing to the red balls should be higher than the peaks corresponding to differently

coloured balls. The highest peak is found by finding the maximum value of the

average cross-correlation. If the maximum value is higher than 0.65 (the normalised

cross-correlation ranges from -1 to 1), then the software determines that this peak

corresponds to a red ball. If the maximum value is less than 0.65, then the software

determines that there are no red balls on the table.

The software then examines the next highest peak. It achieves this by eliminating

the first peak from the average cross-correlation, setting a circular region (the same

size as the image of a ball) around the maximum value to zero. The maximum value

43

4.10 Summary

of this modified cross-correlation is then found, and is again compared to 0.65 to

determine whether this peak corresponds to a red ball. This process it repeated

until there are no more peaks remaining which are higher than 0.65, so that all of

the red balls have been found.

The value 0.65 was chosen after examination of a large number of average cross-

correlations. The maximum values of the peaks corresponding to the balls being

located were generally 0.8 or higher, while the maximum values of the peaks corre-

sponding to differently coloured balls were generally less than 0.5. Using the thresh-

old value of 0.65 was found through testing to be a robust method of distinguishing

between balls of different colours.

Once the approximate positions of all the balls of a specific colour are found, these

positions can be refined by examining the original (undecimated) image. For each

of the ball positions found, the original image is cropped to a small region around

the position. Then cross-correlation and bandlimited interpolation are carried out,

as in Section ??, to find the positions accurately.

The Matlab code for the program is listed in Appendix ??.

4.10 Summary

A variety of cameras were considered as options for the image system hardware. The

one chosen was the Canon EOD D60 for its high resolution, easy connectivity, and

flexibility. This was interfaced with the computer with third party remote capture

software that provided a DLL that could interface with the program. A console

program, which could be run directly from Matlab that utilised this DLL was

written.

The function used to determine the ball positions was the 2-D cross-correlation. In

order to compute the functions in an appropriate time frame, a coarse-fine search

was used which cross-correlated a decimated image to find the approximate ball

locations. Small regions of the original image where balls were known to be were

then cross-correlated with the template image in order to find the ball location to

the accuracy of one pixel. Bandlimited interpolation was then used to find the ball

44

4.10 Summary

positions to sub-pixel accuracy. Once the ball positions were known, they were

passed through a filter which corrected their positions to account for distortion of

the lens.

45

5 The brain

5.1 Introduction

Algorithms for determining the best shot to take range from exceedingly simple to

tremendously complex. In general, a more complex procedure has the potential to

result in a more successful strategy. In this section, many possible design concepts

will be discussed, not all of which have been implemented. The “brain” of the robot

was programmed to be as intelligent as possible, given the time available to be spent

on its design.

Given the positions of the balls on the table, the program must:

• construct a list of shots,

• decide which shots are possible,

• choose the best shot, based upon some set of criteria.

5.2 Listing shots

At the beginning of any turn, there is a non-empty subset of balls on the table which

the robot should sink. For example, before any balls have been sunk in the game,

the robot can sink any red or yellow balls. Otherwise, the robot should sink either

the remaining red balls on the table, or the remaining yellow balls, or just the black

ball.

Let B0 denote the cue ball. Let n be the number of other balls remaining on the table

at the start of the robot’s turn. Let m be the number of balls on the table which the

robot should sink, so 0 < m ≤ n. Let these balls be denoted by B1,B2, . . . ,Bm. This

set of balls which the robot should sink is always the same as the set of balls which

the cue ball may first collide with, in order to not foul. Let the remaining balls be

denoted by Bm+1,Bm+2, . . . ,Bn. Finally, let the cushions on the four sides of the

46

5.2 Listing shots

table be denoted by C1, C2, C3, C4, and let the six pockets of the table be denoted by

P1,P2,P3,P4,P5,P6.

First consider a shot where the cue ball strikes a ball Bj, causing the ball Bj to

sink in pocket Pi. (It is assumed implicit, in this description and similar following

shot descriptions, that there are no other intermediate collisions with other balls, or

cushions.) Denote this shot path by BjPi. As there are 6 pockets Pi, and m balls Bj

which the robot should sink, there are 6m combinations for this type of shot path.

Next consider a more complex shot path where an extra ball is involved. First,

suppose the cue ball strikes a ball Bj, which then strikes another ball Bk, which is

then sunk in pocket Pi. Denote this shot path by BjBkPi. As there are 6 pockets

Pi, m balls Bk which the robot should sink, and m− 1 other balls Bj which the cue

ball may first hit, there are 6m(m− 1) combinations for this type of shot path.

Alternatively, the cue ball could strike a ball Bj, causing the cue ball to change

direction and strike a second ball Bk, which is then sunk in pocket Pi. Denote this

shot path by [Bj]BkPi. As for the previous case, there are 6m(m− 1) combinations

for this type of shot path.

Another possibility is that the cue ball strikes a ball Bj, which then strikes a ball

Bk, causing the ball Bj to change direction and be sunk in pocket Pi. Denote this

shot path by Bj[Bk]Pi. Here, there are 6 pockets Pi, m balls Bj which the robot

should sink, and n− 1 other balls Bk, so there are 6m(n− 1) combinations for this

type of shot path.

A shot path can also be made more complex by introducing rebounds off the cushion.

Suppose the cue ball first bounces off cushion Ca and then strikes a ball Bj, which

is then sunk in pocket Pi. Denote this shot path by [Ca]BjPi. There are 6 pockets

Pi, m balls Bk which the robot should sink, and 4 cushions Ca, so there are 24m

combinations for this type of shot path.

If instead the cue ball first strikes a ball Bj, which then bounces off cushion Ca and

is sunk in pocket Pi, then denote this shot path by Bj[Ca]Pi. As for the previous

case, there are 24m combinations for this type of shot path.

Clearly more intermediate steps can be introduced in any of the three forms, Bk,

[Bk] or [Ca], ad infinitum. With each additional step, the number of combinations

47

5.2 Listing shots

increases greatly. Table ?? shows a list of several shot path types, and the number

of different shot combinations for each type in terms of m and n. Specific values

are shown for m = 14, n = 15, corresponding to a game situation before any balls

have been sunk, so the shots considered are those where any red or yellow ball is

sunk. It is clear that there exists a systematic method of generating shot paths. The

number of these shots which should be considered is dependent upon the complexity

of calculation required for each shot, the amount of computational power available,

and the usefulness of the inclusion of the shots.

Table 5.1: Some shot path types.

Shot path type Combinations Number Shot path type Combinations Number

BjPi 6m 84 [Ca]BjPi 24m 336

BjBkPi 6m(m− 1) 1092 Bj [Ca]Pi 24m 336

[Bj ]BkPi 6m(m− 1) 1092 [Ca]BjBkPi 24m(m− 1) 4368

Bj [Bk]Pi 6m(n− 1) 1176 Bj [Ca]BkPi 24m(m− 1) 4368

BjBkBlPi 6m(m− 1)(n− 2) 14196 BjBk[Ca]Pi 24m(m− 1) 4368

[Bj ]BkBlPi 6m(m− 1)(n− 2) 14196 [Ca][Bj ]BkPi 24m(m− 1) 4368

Bj [Bk]BlPi 6m(m− 1)(n− 2) 14196 [Bj ][Ca]BkPi 24m(m− 1) 4368

BjBk[Bl]Pi 6m(m− 1)(n− 2) 14196 [Bj ]Bk[Ca]Pi 24m(m− 1) 4368

Bj [Bk][Bl]Pi 6m(n− 1)(n− 2) 15288 [Ca]Bj [Bk]Pi 24m(n− 1) 4704

[Bj ]Bk[Bl]Pi 6m(m− 1)(n− 2) 14196 Bj [Ca][Bk]Pi 24m(n− 1) 4704

[Bj ][Bk]BlPi 6m(m− 1)(n− 2) 14196 Bj [Bk][Ca]Pi 24m(n− 1) 4704

The more complex a shot path is, the lower the probability that it can be successfully

executed. The initial launch of the cue ball will be subject to some random and bias

error due to the inevitable imperfections of the vision and actuation systems. Each

step in the shot path magnifies these errors, so that the executed shot deviates more

and more from the desired path. When a shot path is sufficiently complex that the

magnification of error is large, the probability of successfully executing the shot is

near enough to zero that it should not be considered a viable option.

Some steps in a shot path create a greater magnification of error than others. Specif-

ically, planning a shot involving a collision between two moving balls can be dis-

counted as infeasible. Deviations of the two balls from their paths will cause a

significant error in their point of collision (if they still manage to collide), which will

48

5.3 Possibility analysis

result in much greater deviations in the two balls’ paths after the collision. Addi-

tionally, calculating and executing such shots would require complete simulation of

the physics of the game (see Section ??), as well as extremely accurate calculation

and control of the force imparted to the cue ball. These shot paths will therefore

not be considered further.

The shot path determines the angle at which the cue ball must be struck. To

complete the description of a shot, the force with which the cue ball should be

struck must also be known. A certain minimum force will be required to ensure

that the shot path is carried out to its conclusion (that is, the sinking of a ball). It

may be desirable to consider a variety of shots using the same shot path but with

a different stroke force, as these would result in different configurations of the balls

at the end of the turn, some of which may be more favourable than others. Such

considerations are discussed in Section ??.

Note that in formulating the list of shots, only shots which result in the sinking

of a ball are considered. However, often in a game of pool, a player is faced with

a configuration such that the probability of sinking a ball is very low. In such a

situation it may be more desirable to play a shot which does not sink a ball, but

improves the configuration of the balls on the table, either increasing the potential

of future shots to sink balls or decreasing the potential of the opponent to to sink

balls. Again, this is discussed in Section ??.


A shot is said to be possible if it is geometrically achievable for the balls to interact

in the prescribed manner with the desired outcome. This can be checked by first

calculating all the required ball paths on the table, and then checking to see if the

determined trajectory is feasible.

5.3.1 Calculating ball paths

The process of calculating the paths of the balls is best explained via an example.

49


Say there are six balls other than the cue ball remaining on the table. Let Ti be the

initial position of Bi, for i = 0, 1, 2, 3, 4, 5, 6. The shot path B1[B3]B4[C1]B5B2P5 will

be considered.

First consider the last ball in the path, B2. In order for it to move towards P5, it

must be struck by the ball B5 at a specific position, call this X5. (See Section ??

for details.)

Next consider the previous ball in the path, B5. In order for it to move towards X5,

it must be struck by the ball B4 at a specific position, call this X4.

The previous ball, B4, must arrive at X4 via the cushion C1. If conservation of

translational kinetic energy is assumed so that, when a ball rebounds off a cushion,

the angle of incidence is equal to the angle of reflection, then it is clear that there is

only one direction in which B4 may be struck in order to achieve this. This can be

found mathematically by considering the line parallel to C1 but offset away from it by

a distance equal to the radius of the ball — this line is the rebound line associated

with [C1]. The point obtained by reflecting X4 through this reflection line is the

point towards which B4 should be struck. Denote by V4 the intermediate position

of B4 as it collides with the cushion C1. In order for B4 to move towards V4, it must

be struck by the ball B1 at a specific position, call this X1.

The first ball, B1, must arrive at X1 via a collision with B3. Again, if conservation

of translational kinetic energy is assumed, and it is also assumed that the masses of

the balls are equal, then it can be shown (?) that, after the collision, the velocities

of B1 and B3 are perpendicular. Consider the position V1 of B1 as it collides with

B3, then the velocity of B1 is in the direction−−−→V1X1 and the velocity of B3 is in the

direction−−→V1T3. Thus 6 T3V1X1 is a right angle, and |T3V1| is equal to the sum of the

radii of the balls, so the position V1 can be found. In order for B1 to move towards

V1, it must be struck by the cue ball B0 at a specific position, call this X0.

Hence all the ball trajectories necessary for this shot to succeed have been deter-

mined: B0 must move to X0, then B1 must move to V1 then to X1, then B4 must

move to V4 then to X4, then B5 must move to X5, and finally B2 must move to P5.

All of these are straight line paths. (The effects of ball spin are considered to be

negligible.)

50


Note that, when analysing the path of a ball rebounding off a cushion or another ball,

conservation of translational kinetic energy must be assumed. In practice, kinetic

energy is not perfectly conserved, due to energy losses through heat and sound. More

significantly, translational kinetic energy may be converted to rotational kinetic

energy during a collision, and vice versa.

For this reason, rebounds of these types were physically investigated with pool balls

on the pool table, in order to verify whether the actual behaviour of the balls would

approximate the behaviour predicted by the idealised analysis.

For rebounds off cushions, the prediction that the angle of incidence is equal to

the angle of reflection was found to be valid over a wide range of incident angles

and velocities. Therefore analysis of shots with cushion rebounds could be carried

out reasonably accurately using this prediction. However, in order to have the

robot playing such shots more accurately, it would be necessary to carry out further

investigation into the real world kinematics of the rebound.

The prediction of the behaviour of a ball following a rebound off another ball, namely

that the velocities of the balls following the collision are orthogonal, was found to be

less valid. For most of the trials of these collisions, the deviations from this predicted

behaviour were quite significant. Therefore it was decided that the brain program

would not consider shots involving collisions of this type (for example, [Bj]BkPi and

Bj[Bk]Pi).

Accurate analysis of these shots is possible (for example, see ?), but is very com-

plicated. Considering all the variables required to accurately model the kinematics

of collisions then verges on complete simulation (see Section ??).

5.3.2 Trajectory check

Once all of the ball paths in a shot have been geometrically determined, several

conditions must be checked in order to establish whether or not the shot will be

physically achievable on the table.

For each of the straight line paths of balls in the shot, whether there are any obstacles

blocking the trajectory of the ball must be checked.

51


Say ball 1 of radius r1 at A(xA, yA) is to move (in a straight line path) to B(xB, yB).

The program must check whether there is a clear path for the ball between these

two points. Consider any other ball, call it ball 2, on the table. Say it has radius r2

and is at P (xP , yP ). Then ball 2 will not interfere with the motion of ball 1 if and

only if it is never within a distance r1 + r2 of the path AB. This corresponds to P

lying outside the region shown in Figure ??.

A

B

A1

A2

B1

B2

r1+r

2

Figure 5.1: Region of interference with the path of a ball.

First the program checks whether P lies within the rectangle A1A2B2B1. Let D be

the foot of the perpendicular from P dropped down to AB. Then D lies on the

same side of A as B iff

−→AP · −→AB > 0. (5.1)

Similarly, D lies on the same side of B as A iff

−−→BP · −→BA > 0. (5.2)

These two conditions together are equivalent to D lying between A and B.

To lie within the rectangle, it is also necessary that |PD| < r1 + r2. The perpendic-

ular distance from a point (X, Y ) to a line ax + by + c = 0 is given by

d =|aX + bY + c|√

a2 + b2. (5.3)

The equation of line AB is

(yA − yB)x− (xA − xB)y + (xAyB − yAxB) = 0. (5.4)

52


Thus requiring that PD < r1 + r2 is equivalent to the condition

|(yA − yB)xP − (xA − xB)yP + (xAyB − yAxB)|√

(yA − yB)2 + (xA − xB)2< r1 + r2, (5.5)

which can be rewritten as

|(xP − xB)(yA − yB)− (xA − xB)(yP − yB)|√

(xA − xB)2 + (yA − yB)2< r1 + r2. (5.6)

Thus P lies within the rectangle A1A2B2B1 if and only if inequalities (??), (??) and

(??) all hold.

It is known that P does not lie within the semicircle with centre A (see Figure ??)

as otherwise balls 1 and 2 would initially be intersecting. Thus it remains to check

whether P lies within the semicircle with centre B. It is sufficient to check whether

P lies anywhere within the entire circle, for if it lies within the other half of the

circle, then it lies within the rectangle A1A2B2B1 and thus ball 2 interferes with the

motion of ball 1 in any case.

P lies within the circle with centre B and radius r1 + r2 iff

√

(xP − xB)2 + (yP − yB)2 < r1 + r2. (5.7)

Thus the complete criteria can be stated as follows: ball 2 interferes with the motion

of ball 1 if either inequalities (??), (??) and (??) all hold, or inequality (??) holds.

In cases where balls are extremely close to, but not within, the region of interfer-

ence, a tiny error in the trajectory of the shot may cause an undesirable collision.

Therefore, in the program, the region of interference was enlarged by a small safety

margin to compensate for such errors. Thus in equations (??) and (??), the term

r1 + r2 was replaced by r1 + r2 + ε, where the safety margin, ε, was set to ε = 0.1r.

Continuing the example analysis from the previous section, each of the calculated

trajectories must be checked to determine whether the positions of the other still

stationary balls on the table will interfere with them. That is, it must be checked

whether any of B1,B2, . . . ,B6 block the path of B0 to X0. However, when considering

the path of B5 to X5, only B2 and B6 are checked for interference, because the

positions of B0,B1,B3 and B4 are not known as they have moved.

53


In addition to checking whether there are balls blocking the trajectories, it must be

ensured that the paths are not blocked by the cushions. For example, if a ball, Bk,

is very close to a cushion, and it is to be struck by a ball, Bj, to move in a direction

away from the cushion, then the position, Xj, at which Bj should strike Bk can be

calculated. However, it may be the case that Bj is not be able to reach the position

Xj because it will hit a cushion first. To check whether this will occur, it is sufficient

to find the distance of Xj from the cushion. If the distance is smaller than the r,

the radius of Bj, or else if Xj is on the wrong side of the cushion, then Bj cannot

reach Xj and the shot is impossible.

Another situation where a cushion may block the trajectory of a ball is when the ball

is heading towards a pocket. The four corner pockets of the table may be reached

by a ball at any point on the table, so cushions never block these paths. However,

there is a maximum angle at which balls may enter the two middle pockets. If the

entry angle is wider than this, then the cushion on the side of the pocket will block

the path of the ball. Brief experimentation yielded an estimate for this maximum

entry angle as 60◦. Therefore, when a shot results in a ball being sunk in one of

these two middle pockets, then if the entry angle is greater than 60◦, the shot is

impossible and must be taken off the list.

Now consider a ball, Bj, bouncing off a cushion, Ca, in order to reach a position, Xj.

The intermediate position, Vj, as Bj rebounds off the cushion is calculated geomet-

rically using reflection through the rebound line associated with [Ca]. However, the

cushions are curved in regions near the pockets of the table, so that if the rebound

occurs within these regions, the path of the ball will not reflect through the same

line. This chiefly occurs around the pockets in the middle of the top and bottom

cushions. Therefore, the distance, c, from the pocket that the cushion is curved

must be found. Then for every such rebound off the top and bottom cushions, the

distance of Vj from the pocket along the rebound line must be found. If this distance

is less than c, then Bj will not rebound off Ca to reach Xj as calculated, so the shot

must be eliminated from the list.

A similar effect may occur around the four corner pockets of the table, but these

regions are far smaller, and rebounds very close to these corner pockets are rare and

generally not useful. Therefore, it was unnecessary to perform the same check for

thse pockets.

54

5.4 Choosing the best shot

If the curvature of the cushion near the pockets were modelled in the program,

then it would still be possible to allow rebounds in these regions. However, the

modelling would have to be extremely detailed in order to predict such rebounds

accurately, and the calculations required to determine the intermediate rebound

position would be much more complex. Therefore, this was considered to be an

unnecessary complication.

When the cue ball has been sunk by the opponent and is replaced on the table,

there is an additional rule constraining the allowable shots, namely that the cue ball

must be struck in a “forwards” direction. That is, the component of the cue ball’s

velocity parallel to the longer sides of the table must be positive towards the far end

of the table. Therefore, when such a situation arises, the program must rule out

those shots where the cue ball is not struck forwards.


In order to decide which of the list of possible shots is the “best” shot to attempt,

a numeric value needs to be associated with the shot, estimating how “good” the

shot is. Therefore, a merit function of a shot is defined, based on its difficulty and

its usefulness.

5.4.1 Difficulty of a shot

The difficulty of a shot is the probability that the desired goals of the shot will not

be achieved. Usually this can be considered to depend only upon the probability

that a ball will be sunk. However, in a situation where it is impossible to sink a

ball, the difficulty may be related to the probability that the cue ball will manage

to hit one of the object balls (and thus not foul).

The probability of sinking a ball can be found by calculating the allowable errors in

the paths of each ball in the shot path, working backwards as in Section ??. Using

the example from that section, the allowable deviations in the path of B2 which will

still result in the ball sinking in pocket P5 can be calculated (from the geometry

of the table). From this, the allowable range of positions X5 at which B2 may be

55


struck by B5 can be determined, and thus the allowable deviations in the path of B5

can be calculated, and so on, until the allowable deviations in the path of the cue

ball B0 is determined.

If the random and bias errors inherent in the vision and actuation systems were

known, the exact probability of propelling the cue ball at an angle within the required

tolerance to sink the target ball could be found. This would provide an excellent

indication of the difficulty of the shot.

However, much computation is required to perform these backward error analyses.

Additionally, it is difficult to quantify the errors present in the system. Besides,

all that is required is a indication of the relative probabilities of success for the

shots considered. Therefore, a simple function will be derived which provides an

indication of the probability of success of a shot.

The analysis is initially very similar to that performed in Section ??. Let B1 and

B2 be two balls of radii r1 and r2, respectively, at positions T1 and T2, respectively.

Consider a shot in which B2 is to be struck by B1 in a direction at angle θ2 with

respect to a fixed direction i. Then B1 must strike B2 at a position, X1.

Let a = T1X1 and d = T1T2. Let α = 6 T2T1X1 and β = 6 T1T2X1. Let θ1 be the

angle of−−−→T1X1 with respect to i. All angles are signed, so that they can take on

positive or negative values. A diagram of the setup is shown in Figure ??. Using

the sine rule, it is clear that

sin α

r1 + r2

=sin β

a. (5.8)

Now suppose that B1 is instead struck with an angle error of ∆θ1, so that it is

propelled at an angle θ′1 = θ1 + ∆θ1 to i.

Assuming that B1 still collides with B2, denote by X ′1 the position of B1 as this

collision occurs. Let ∆θ2 be the resulting error in the trajectory angle of B2, so that−−−→X ′

1T2 is at an angle θ′2 = θ2 + ∆θ2 to i.

|X ′1T2| = r1 + r2, as these are the positions of the balls as they collide. Let a′ =

|T1X′1|, α′ = 6 T2T1X

′1 = α + ∆θ1 and β = 6 T1T2X

′1 = β + ∆θ2.

Using the sine rule again,

sin α′

r1 + r2=

sin β ′

a′, (5.9)

56


T1

X1

T2

d

a

r1

r2

i

i

θ1

θ2

α

β

Figure 5.2: Diagram for error magnification analysis.

or, written in terms of ∆θ1 and ∆θ1,

a′ sin(α + ∆θ1) = (r1 + r2) sin(β + ∆θ2). (5.10)

If d � r1 + r2, which is the case in most situations, it is reasonable to use the

approximation a′ ≈ a, so that, using the sine addition formula,

a sin α cos ∆θ1 + a cos α sin ∆θ1

≈ (r1 + r2) sin β cos ∆θ2 + (r1 + r2) cos β sin ∆θ2. (5.11)

Since ∆θ1 and ∆θ2 are small,

sin ∆θ1 ≈ ∆θ1, cos ∆θ1 ≈ 1,

sin ∆θ2 ≈ ∆θ2, cos ∆θ2 ≈ 1.(5.12)

Therefore,

a sin α + (a cos α)∆θ1 ≈ (r1 + r2) sin β + ((r1 + r2) cos β)∆θ2. (5.13)

Using equation ??, the first terms on each side are equal, so

(a cos α)∆θ1 ≈ ((r1 + r2) cos β)∆θ2. (5.14)

57


Finally,

∆θ1

∆θ2

≈ (r1 + r2) cosβ

a cos α. (5.15)

Thus an expression relating to the magnification of the angle error due a collision

between balls has been obtained.

It is interesting to note that the value of ∆θ1

∆θ2

is not always less than 1. If a is

very small, then ∆θ1

∆θ2

can be greater than 1, so that ∆θ1 > ∆θ2. In other words,

the collision is not causing a magnification of angle error at all, but rather it is

diminishing it. This is somewhat counter-intuitive, as one would expect that extra

collisions necessarily cause an increase in error. This formula shows that, if a ball

travels a very small distance in a shot, then the shot can be played more accurately

than the same shot without this ball in the shot path.

Now let a1 be the distance travelled by B1 before it collides with B2, so a1 = a. Let

a2 be the distance travelled by B2 before it reaches a pocket at P . Let δmax be the

maximum distance by which the centre of B2 may miss the point P and still sink in

the pocket.

Then ∆θ2max, the maximum allowable error in the trajectory of B2, is given by

∆θ2max ≈δmax

a2. (5.16)

Then, using (??), ∆θ2max, the maximum allowable error in the trajectory of B2, is

given by

∆θ1max =∆θ1max

∆θ2max

∆θ2max ≈(r1 + r2) cos β

a1a2 cos αδmax. (5.17)

Now to state this result in a more applicable form, consider a shot BjPi. Naming

all variables using the previous conventions, the maximum allowable error in the

trajectory of the cue ball, B0, is given by

∆θ0max ≈(rc + r) cos β0

a0aj cos α0δmax, (5.18)

where δmax is the maximum distance by which the centre of Bj may miss the point

representing Pi and still sink. It will be assumed from this point onwards that δmax is

a constant distance, independent of the pocket and of the initial position of the ball

58


to be sunk. This is not true, since greater or smaller deviations may be permissible

depending on which pocket is targeted, and on the entry angle of the ball. However,

the effect of the variation of δmax on the difficulty function to be defined has not

been investigated.

For shots of the type BjPi, define the dimensionless function ℘(BjPi) to be

℘(BjPi) =(2r)2

a0aj

· cos β0

cos α0, (5.19)

so that (making the valid assumption that rc ≈ r)

∆θ0max ≈ ℘(BjPi)δmax

2r. (5.20)

It can be seen that if ℘(BjPi) is greater, then the maximum allowable error in the

trajectory of the cue ball is greater, so the probability of successfully carrying out

the shot is greater. In other words, the function ℘ provides an indication of the

shot’s probability of success.

Now consider a shot BjBkPi. The maximum allowable error in the trajectory of the

cue ball, B0, is given by

∆θ0max =∆θ0max

∆θjmax

∆θjmax

∆θkmax

∆θkmax ≈(rc + r)(2r) cosβ0 cos βj

a0ajak cos α0 cos αj

δmax. (5.21)

Therefore we define ℘(BjBkPi) to be

℘(BjBkPi) =(2r)3

a0ajak

· cos β0

cos α0· cos βj

cos αj

, (5.22)

so that

∆θ0max ≈ ℘(BjBkPi)δmax

2r. (5.23)

It can be seen that the function ℘ is defined in such a way so as to provide a

comparison of the difficulty of shots, even if they are of different types.

Similarly to the previous example, for a shot BjBkBlPi, the value of the function is

℘(BjBkPi) =(2r)4

a0ajakal

· cos β0

cos α0· cos βj

cos αj

· cos βk

cos αk

. (5.24)

In order to find the value of the function ℘ for shots involving cushion shots, it

is first of all assumed that the angle of incidence is exactly equal to the angle

59


of reflection. Therefore, the shot may be considered equivalent to an imaginary

shot where all of the positions of the balls involved in the shot before the cushion

rebound are reflected through the rebound line. For example, the shot B1[C4]P2

may be considered equivalent to a shot B1P2 where the cue ball and B1 have been

reflected through the rebound line associated with [C4]. The function ℘ can now

be calculated for the imaginary equivalent shot, and this value can be taken as the

value of the function ℘ for the original cushion rebound shot.

However, because the angle of incidence is not exactly equal to the angle of reflection

in practice, there is likely to be magnification of trajectory error due to the cushion

rebound. Therefore, an empirical method of scaling the function ℘ appropriately

is to multiply the value of the function by λn, where n is the number of cushion

rebounds in the shot, and λ < 1 is an appropriate penalty factor.

Apart from the geometry of a shot, another factor affecting the probability of the

success is the force with which the cue ball is struck. Ball spin will cause a ball

to deviate from the path calculated assuming conservation of translational kinetic

energy (which is usually invalid when ball spin is non-zero). However, the greater

the velocity of the ball, the smaller the error introduced by the ball spin. Thus, in

general, striking the cue ball with a greater force increases the probability that the

shot will deviate from the calculated shot path by a lesser extent.

Of course, the cue ball must be hit with enough force so that enough momentum is

transferred to the final ball in the shot to reach the pocket. For a shot BjPi, the

minimum initial velocity, umin, of the cue ball which will result in the target ball

reaching the pocket is given by (?)

u2min =

aj

κ cos2(θj − θ0)+ 2gµra0, (5.25)

where κ = 198g

(

24µs

+ 25µr

)

, g is gravitational acceleration, µr and µs are the rolling

and sliding friction constants of the pool balls on the felt. This can be rearranged

to yield

u2min = 2gµr

(

a0 +1

2gµrκ· aj

cos2(θj − θ0)

)

. (5.26)

Using the values µr = 0.01 and µs = 0.2 (?), it can be shown that u2min is proportional

60


to

a0 +

(

1.87

cos2(θj − θ0)

)

aj. (5.27)

This expression was used to determine how hard the cue ball should be struck.

5.4.2 Usefulness of a shot

The following attributes of a shot increase its usefulness, to varying degrees:

• a target ball (or more than one target ball) is sunk,

• the shot is not a foul,

• the robot’s target balls are positioned in regions from which they are easier to

sink,

• the opponent’s target balls are positioned in regions from which they are more

difficult to sink,

• the next turn belongs to the robot, and the cue ball is left in a good position

for the robot,

• the next turn belongs to the opponent, and the cue ball is left in a bad position

for the opponent.

So far, the only shots considered are those which result in a target ball being sunk,

so they are all equally useful in this respect. In order to assess the other attributes

of a shot, it is necessary to be able to predict, to some degree, the configuration of

the table following the shot.

This can be achieved with a complete simulation of the shot (see Section ??). Failing

this, empirical methods may be used to estimate the approximate positions of balls

following a shot, similarly to how humans would predict the outcome of a shot. For

example, if the shot B1P1 is being considered, then it may be guessed that, following

the shot, the ball B1 will most likely be sunk and therefore no longer on the table.

From the level of force used to strike the cue ball, and the distances involved, the

61


velocity of the cue ball immediately before the collision can be estimated, and there-

fore the velocity after the collision can also be estimated. Finally, the assumption

that the cue ball does not hit any more balls could be made, so that the final resting

position of the cue ball could be estimated.

If the configuration of the table resulting from a shot can be predicted in this way,

then the resulting configuration can be assessed according to the criteria listed above

in order to evaluate how useful it is. For example, if one of the robot’s target balls is

moved very close to the middle of one of the shorter cushions, then this ball is in a

very difficult position to be sunk, and thus this makes the configuration less useful.

Algorithms to carry out an empirical prediction and then evaluation of the table

configuration, as described above, have not been designed in detail.

5.4.3 Overall merit function

The overall merit of a shot is related to both the difficulty and the usefulness of the

shot. If a function were defined to assign a numerical value to the usefulness of a

shot, as has been done for difficulty, then the two functions relating to difficult and

usefulness could be combined to produce a overall merit function, for example, by

multiplying the two functions. In order for this to work, both functions must be

defined in such a way that they take on non-negative values only, a greater value

indicating a “better” (that is, less difficult, or more useful) shot.

Once the overall merit function has been defined, the value of the function for all of

the possible shots found by the brain program should be calculated. Then the shot

with the highest value (that is, the greatest merit) should be chosen as the best shot

to play.

Defining the merit as a function of two numbers associated with the shot, the diffi-

culty and the usefulness, may not be the best approach to achieve the most intelligent

play. A shot may have multiple goals, in which case it is appropriate to consider the

probabilities of each combination of goals being achieved. For example, say shot A

has a very small probability of sinking a ball, but in the event of failure, will leave

the table in a configuration which makes it very difficult for the opponent to sink

his final ball, whereas shot B has a slightly higher probability of sinking a ball, but

62

5.5 Complete simulation

in the event of failure, will leave the opponent with an easy shot to win the game.

Then the value of the merit function for shot A should be higher than that for shot

B, despite the fact that shot B has a lower difficulty than shot A for the singular

goal of sinking a ball. Thus evaluating difficulty and usefulness separately produces

a skewed viewpoint of a shot, under some circumstances.

Clearly the development of a intelligent, robust merit function is a complicated task,

chiefly due to the need to be able to quantify what is usually a vague, subjective

estimation of a shot’s usefulness.

5.5 Complete simulation

In many cases, simply analysing the collisions involved in a shot does not provide

enough information to judge the possibility, difficulty and usefulness of the shot.

To gain all the information about a shot, it would be necessary to simulate it in

its entirety. This involves calculating, at discrete time intervals, the positions of

all balls on the table, as well as their translational and rotational velocities. Each

successive time step is computed by analysing the forces acting on each of the balls

(including frictional forces and collision forces, from other balls or cushions) and

updating all of the information.

This is computationally feasible, as can be seen from the many programs available

which simulate pool and snooker. However, the complexity of such a program would

require a lot of programming time. Even if code from an existing simulation could be

adapted, a great deal of work would need to be done in order to update the program

from an ideal simulation to one incorporating all real world effects. It can be seen in

? and ? that the physics of pool becomes extremely complex if all physical effects

are to be accounted for, which would be necessary in order for the simulation to

provide any useful information to the robot beyond what is already known without

the simulation.

Additionally, it would be necessary to ensure that the kinematics of the simulation

matched the kinematics of the actual table used by the robot extremely closely,

which would require a large amount of time spent in testing. One of the most

difficult aspects would be modelling the variations in smoothness of the felt, and the

63

5.6 Final brain software

unevenness of the surface, both of which have significant effect upon the kinematics

of the balls.

Finally, the extra information which could be provided by such a simulation would

only be very useful in analysing quite complicated shots, and due to the difficulty of

such shots, they should not be often attempted by the robot in any case. Thus, while

the possession of such a simulation would enable the robot to play pool extremely

well, it has been decided that it is far beyond the scope of this project.

5.6 Final brain software

Once the positions of all the balls on the table are found, the shot selection software

begins by compiling a list of shots. Since the robot has no way of detecting when

balls have been sunk, the software must be told if the table is still open, or else

whether its target balls are the red set or the yellow set. Once one of the two

coloured sets is chosen, the software remembers the choice for the remainder of that

game.

The shot path types which the software is programmed to consider are BjPi,

[Ca]BjPi, Bj[Ca]Pi, BjBkPi, [Ca]BjBkPi, Bj[Ca]BkPi, BjBk[Ca]Pi and BjBkBlPi. The

trajectories of the shots on the table are calculated as the list is created.

The software was not programmed to consider the effects of varying the force im-

parted to the cue ball. Therefore, only a single shot was created for each shot path,

rather than a number of shots with different shot strengths.

The overall merit function of the shots was based only upon the difficulty of the

shots. No software was written to judge the usefulness of the shots, so all shots

were regarded as equally useful in that they resulted in a target ball being sunk.

So the merit function implemented was equal to the difficulty function described in

Section ??, with a cushion rebound penalty of λ = 0.5.

Since the calculation of the merit function of a shot is much faster than the possibility

analysis, instead of performing the possibility analysis on all of the shots listed, the

merit function of all the shots is first calculated. The program then sorts the shots

in order of merit, and, starting with the highest scoring shot, performs possibility

64

5.7 Summary

analyses on the shots in this order, until a suitable number of possible shots is found.

This means that the possibility analysis is only carried out on potentially good shots,

saving much time.

Once a shot has been chosen for play, the software must decide the level of power

with which to strike the cue ball. The expression (??) was utilised to determine

which of the three power levels to use.

In the rare case that no possible shots are found, the robot is programmed to hit

the cue ball towards the centre of the table with high power.

The Matlab code for the program is listed in Appendix ??.

5.7 Summary

In order to determine the best shot to play, the shot selection software first sys-

tematically creates a list of shots up to a high level of complexity. A merit score

is assigned to each of these shots, based on the probability that the shot can be

executed successfully, resulting in a target ball being sunk. Finally, possibility anal-

ysis is performed on the shots with the highest merit scores, until a possible shot is

found.

65

6 The arm

6.1 Introduction

The concept design process for this project was based on the process outlined in ?.

By basing the methods employed in fulfilling the design requirements of the project

on a legitimate foundation, the likelihood of redesigning at a later stage was reduced.

This method also encourages the integration of lateral thinking and logic into the

design process in order to ensure that the final design achieves the goals in the most

efficient manner. Another benefit of following the appropriate design process was

to ensure that the solution that was finally derived had been subjected to rigorous

scrutiny that expedited the construction and testing phases.

As outlined in the literature review, there were a number of important facets to take

into consideration in the design of the actuation subsystem, or “arm”, of the robot.

In order to ensure that these considerations are taken into account (so that the final

design reflects the prerequisites of the system) a detailed concept design process is

necessary.

This chapter outlines the processes followed, and details the final design for the arm.

6.2 Specification

The purpose of the robotic actuation system is to convert the theoretical output

from the brain into mechanical action. It is required to play the desired shot as

accurately as possible. In order to achieve these goals, appropriate functions and

subfunctions were defined.

6.3 Overall function and subfunctions

The overall function of the actuation system is defined as follows:

66

6.4 Solution principles to subfunctions

To propel the cue ball in a given direction, at a given speed, from a given point on

the pool table.

The subfunctions utilised to achieve this overall function are:

1. Energy transfer to cue ball — what type of actuator will be utilised to drive

the cue tip into contact with the cue ball whilst imparting a known and desired

force?

2. Postitioning of cue: traverse mechanism and drive mechanism — how will the

cue and actuation system arrive at the correct position in order to transfer

energy to the cue ball?

3. Control of actuation system — how will the actuation system overall be con-

trolled, and how will it interface with the other systems of the project?


The solution principles for the subfunctions were derived from:

• a general investigation of the various options available as solutions for the

required subfunctions,

• previous examples of this technology as outlined in the literature review,

• a general focus on lateral thinking and brainstorming.

At this level of the design process, no potential solution principle is ruled out on

any grounds. A brief description of each solution principle is also provided.

1. Energy transfer to cue ball:

(a) Long stroke/fast response solenoid:

Similar to those found in starter motors in cars, or in any number of

industrial or appliance applications, such as washing machines. Solenoids

actuate a rotor in the center of their coil, that is accelerated when a

67


current is applied to the coils of the solenoid and a magnetic field is

generated.

(b) Pneumatics:

Such as those utilised in the University of Bristol model, with a compres-

sor mounted under the table providing the necessary actuation force to

the cue tip.

(c) Spinning contacts:

Could be constructed as a rotating cylinder that would be lowered onto

the cue ball and impart a rotational momentum to the cue ball, causing

it to “skid” across the table.

(d) Paddle:

Essentially, the actuator would rotate a piece of material like a paddle

until it made contact with the cue ball. The paddle could be oriented to

rotate in either the vertical or horizontal plane.

(e) Mushroom paddle:

A modification of the paddle concept, similar to that utilised in the Uni-

versity of Waterloo model (refer to Section ??).

(f) Combustion mechanism:

A modification of the solenoid actuation system, with the acceleration

provided by a combustion/pressure vessel system instead of an electro-

magnetic method.

(g) Spring/motor trigger model with feedback and force transducers:

An actuator attached to a spring that would impart the kinetic energy to

the actuator, after being tensed through the use of a motor pulling the

spring taut.

2. Traverse mechanism for positioning of cue:

(a) Two link serial robot (SCARA):

SCARAs (Selectively Compliant Assembly Robot Arms) are two-link se-

rial robots consisting of three parallel revolute joints and one prismatic

joint (?). This first joint allows the end-effector to move and orient in

68


a plane, and the fourth joint provides movement normal to the plane.

SCARAs can move very fast, provided the actuators are large, and are

best suited to planar tasks. A configuration of such a robot is shown in

Figure ??.

Figure 6.1: SCARA manipulator. From ?.

(b) Cartesian robot with the option of one or a combination of the following

pulley systems:

i. ball screw,

ii. wire pulley,

iii. rack and pinion,

iv. belt drive,

v. tooth belt.

A Cartesian robot consists simply of three mutually perpendicular pris-

matic joints corresponding to the x, y, and z directions (?). Cartesian

robots have very stiff structures, and hence large robots can be made

using this configuration. A configuration of such a robot is shown in

Figure ??.

(c) Articulated manipulator:

An articulated manipulator consists of two “shoulder” joints and one

“elbow” (?). The “wrist” consists of two or three joints to control the

pitch, yaw and roll of the end-effector. These robots have less stiffness

69


Figure 6.2: Cartesian manipulator. From ?.

than Cartesian robots but provide the least intrusion of the manipulator

structure into the workspace. A configuration of such a robot is shown

in Figure ??.

Figure 6.3: Articulated manipulator. From ?.

Drive mechanism for positioning of cue:

(a) Stepper motor:

Stepper motors are essentially electric motors without commutators. All

of the windings of a stepper motor are part of the stator and the rotor

70


is usually a permanent magnet. A typical stepper motor is shown in

Figure ??.

Figure 6.4: Typical stepper motor, and schematic. From ?.

Stepper motors are controlled through external circuitry that allows for

simple interface with higher level programming such as that found in a

PC. Stepper motors have the ability, due to the physical location of the

stator and rotors, to move in discrete steps. These steps correspond to

an incremental change in the angle of rotation of the rotor. From these

incremental steps it is possible to determine a corresponding translational

change. Stepper motors allow for simple open loop feedback control.

Assuming that the initial rotor orientation (which can be easily deter-

mined by resetting the motor before each actuation) is known and pro-

vided that the stator is not overloaded, causing errors in the step size, all

that is required to determine the final output of the system is to count the

steps taken. If they correspond with the desired input, then the motor

has translated the system the required distance or angle.

To add feedback, all that is necessary is to add a sensor on the stator

that could measure the rotation of the rotor with respect to its initial

orientation.

Stepper motors are extremely useful in robotic applications that are

static, or are being utilised in low load dynamic conditions.

(b) Servo motor:

A servo motor is a DC/AC or brushless DC motor combined with a

position sensor (?). It has a three wire input that controls the operation.

71


The amount of rotation of a stepper motor is controlled by the duration of

the pulse of voltage supplied to the control wire. This method of control is

known as pulse width modulation, with the length of the pulse determined

by the control circuitry. In this case, the length of the pulse determines

the direction in which the servo motor should turn. For example, the

Futaba S-148 motor, operates on a 1.5ms pulse width. The motor expects

a pulse every 20ms. A 1.5ms pulse makes the motor return to its neutral

orientation, but if the pulse is less than 1.5ms the motor will turn the

shaft in one direction, and if the pulse is longer than 1.5ms the shaft will

turn in the operate direction.

The control circuit feedback is given from a potentiometer attached to

the rotor.

Servo motors are extremely good in robotic applications as they are ca-

pable of supplying high power, torque and speed that is necessary in a

large number of applications.

(c) Pneumatics:

Pneumatics run on a compressed air system with forced changes in pres-

sure in the cylinders of the actuation system resulting in a mechanical

output. Pneumatics are commonly used in robotic applications as they

are clean, relatively compact and fairly powerful. The only restriction

with pneumatics, particularly in applications involving a high degree of

motion, is the large amount of piping required. This may be prohibitive

in application for this project, as a relatively large amount of this piping

would need to be added to adequately model the desired output. The

associated cost of purchasing, installing and running an air-compressor

to provide the necessary pressure is also a consideration.

As highlighted earlier, the robot built by students from the University

of Bristol utilised this technology. However it should be noted that in

that instance the industrial robot that was purchased for the project

already had the pneumatic system incorporated into it, hence reducing

the amount of design and testing required.

3. Control of actuation system:

72


(a) Direct PC control:

This would be achieved through an electromechanical interface between

the computer output devices and the control circuits of the motor. This

allows for higher level programming in PC based languages. It also allows

for easier conversion of data from the output of the control algorithms

responsible for shot generation into the required angle of rotation, by

removing the necessity of changing the input requirements on the control

circuits more than once.

Direct PC control is limited only by the effectiveness and reliability of

the computer.

(b) Programmable logic controller:

PLCs operate by downloading an instruction set for a given operation

into a microcontroller and executing the necessary commands from a

machine level. The main problems with PLCs are that they are inflexible

and time consuming to program for complex functions. They have the

added complication of having to first take an output from the shot control

algorithms, then convert it into a format that the PLC can understand,

downloading it to the PLC and converting it again into control signals

for the motors and sensors. This is an inefficient manner in which to

program and removes the flexibility of higher level programming.

(c) Control/driver system:

This system uses a specific hardware drive system for the stepper motors

that allows direct high level control from a PC. It is essentially a “black

box” approach, where a vector signal is input and a series of steps is

output. This improves the flexibility of the control options, which will in

turn allow for greater accuracy. More specifically, these types of control

mechanisms allow input of a vector from the high level control software

that is converted into a sum of a series of steps.

73

6.5 Concept solution


Three concept solutions for the mechanical system were selected by taking different

combinations of the solutions to the subfunctions described above. These three

were chosen on the basis that they were the solutions most likely to meet the design

requirements adequately.

Concept A consisted of

• a long stroke/fast response solenoid as the actuator for the cue tip,

• a two stepper motor drive system operating on a modified Cartesian traverse,

• a C-section extrusion for the rails of the traverse,

• a tooth belt drive with appropriate gear train,

• a stepper motor that controls the angular location of the cue tip,

• driver/PC control of the motors and solenoid.

Concept B consisted of

• a long stroke/fast response solenoid as the actuator for the cue tip,

• a servo motor drive system with feedback control for the traverse,

• a C-section/cylindrical shaft for the rails of the traverse,

• a Cartesian robot with a ball screw drive,

• control/drive system for interface with the brain.

Concept C consisted of

• a pneumatic system for the cue tip actuator,

• a Cartesian robot with a wire pulley drive system actuated by stepper motors,

• PLC control of motors with input vectors obtained from the computer.

74


These were then assessed against a set of weighted criteria in order to determine the

final concept design for the actuation system.

These criteria were:

• Repeatability: How reliable is the actuation going to be? How likely is it that

there will be considerable failings in the construction, testing and operational

phase?

• Accuracy of location: To what degree is the actuation system able to accurately

respond to the desired location? Is the location consistent with the allowable

error margins defined for the project?

• Cost: Is the cost of the actuation system prohibitive?

• Ease of manufacture: Will the manufacturing cost in both time and resources

act to delay the overall progress of the project, and what level of technical

complexity is present in the design?

• Weight: Does the weight of the actuation affect the overall performance of the

project by elongating the response time or affecting the accuracy of location?

• Size: Do the size and physical dimensions impact on the ability of the human

opponent to play the game without interference from the actuation system?

• Speed: Can the actuation system adequately respond in a real time manner

at a human speed in order to take a shot?

• Adaptability: How easy would it be to relocate the robot onto a pool table of

different dimensions?

The weighting for each of these criteria, ranging from 1 (least important) to 10

(most important), were chosen on the basis of ensuring that the overall goals of the

project were achieved as efficiently and accurately as possible.

The weightings can be seen in Table ??

The three concepts were then tabulated, and a rating awarded, depending on how

well each concept met the criteria. These ratings were then multiplied by the in-

dividual criteria weightings, and the resultant products were summed to give an

75

6.6 Analysis of design

Table 6.1: Selection critera weightings.

Criteria Weighting

Repeatability 10

Accuracy of location 10

Cost 7

Safety 5

Ease of manufacture 5

Weight 4

Size 6

Speed 8

Adaptability 3

overall score for each concept. The concept with the highest score would then be

chosen as the concept to be designed more comprehensively.

The results are shown in Table ??.

Based on the results shown in Table ??, Concept A was selected as the concept

design that would be analysed and form the basis for the actuation system.


Once the initial concept design was selected, the next phase in the process was to

more comprehensively design the various components and test the mechanisms to

ensure that the most efficient and accurate system had been selected.

Using the broad concept solutions outlined above, a more detailed description of the

actuation system was devised. A description of the processes undertaken to design

each subfunction is discussed in this section.

76


Table 6.2: Concept design criteria evaluation.

Concept A Concept B Concept C

Criteria Concept Weighted Concept Weighted Concept Weighted

rating subtotal rating subtotal rating subtotal

Repeatability (10) 9 90 9 90 7 70

Accuracy of location (10) 9 90 9 90 7 70

Cost (7) 8 56 6 42 4 28

Safety (5) 8 40 8 40 7 35

Ease of manufacture (5) 8 40 6 30 3 15

Weight (4) 7 28 5 20 5 20

Size (6) 7 42 7 42 7 42

Speed (8) 6 48 3 24 6 48

Adaptability (3) 6 18 4 12 4 12

Total: 452 390 340

6.6.1 Energy transfer to cue ball

The selected component for this was a long-stroke solenoid. Solenoids are available

in any number of different forms. Two different types were investigated for potential

use in the project: a solenoid removed from the starter motor of a car, and one from

a washing machine.

These were chosen for their relatively small dimensions and because the rotors both

allowed modifications to the ends to attach a cue tip. The solenoids are shown in

Figures ?? and ??.

Figure 6.5: Solenoid taken from the starter motor of a car.

77


Figure 6.6: Solenoid taken from a washing machine.

One of the major considerations in selecting which solenoid to use was the ability

to accurately mimic the types of forces that a human being uses when playing pool.

It was therefore necessary to obtain an indication of the range of values of force

found in a typical 8-Ball Game. For a detailed description of the range of values

needed and the method by which they were obtained, refer to Appendix ??. The

range of forces derived was between 80N and 1300N.

A circuit was designed to supply variable power levels to the solenoid, corresponding

to soft, medium and hard shots. This is discussed further in Section ??.

The car starter motor that was used in the design process was from a Holden VN

Commodore. It was designed to be run from a 12V, 20A car battery. It was not

possible to conduct any tests for the starter motor solenoid due to the nature of the

construction and the task for which it was originally designed.

The purpose of a car starter motor is to provide a mechanical action through a

pivot pushing a fly wheel onto the gears. When the solenoid is not actuated, a

spring inside the rotor pushes the rotor out of the stator. When a current is applied

to the solenoid from the battery, the rotor is pulled into the solenoid, with active

resistance from the spring extending the length of the stroke. This action causes the

lever arm of the fly wheel to pivot about a fulcrum, initiating the movement of the

flywheel.

78


Starter motor solenoids are low voltage, high current devices that operate for ex-

tended periods of time. A starter motor solenoid may be activated for a number

of seconds, compared to that of a washing machine that may only be active for

up to half a second. As a result there is no requirement for rapid response, which

manifests as a lower acceleration applied to the solenoid.

In a starter motor, the rapidity of its response is less important than whether the

solenoid can remain fixed in place during the ignition process. For this reason, the

use of a solenoid from a car was deemed to be inappropriate for use in this project, as

it would not be able to supply the required force and, more importantly, acceleration

to the cue tip.

Brief experiments were conducted with the starter motor solenoid, investigating

its general performance. These experiments involved attaching a variable voltage

and current supply to the solenoid, and qualitatively noting the response to high

current. The starter motor solenoid showed measurably slower response rates and

acceleration than the washing machine solenoid. It was deemed unnecessary to

carry out a quantitative experiment, owing to the obvious deficiencies present in the

performance of the starter motor solenoid.

6.6.2 Drive mechanism and traverse mechanism

This section will look at the design structure of the drive and traverse mechanism.

As a game of pool is essentially planar, the choice of a SCARA design seemed

initially obvious. However, a SCARA has not in fact been used, due to the size of

the robot required to reach the entire pool table. Using a robot of this type would

inconvenience a human player making a shot in the same corner that the robot is

mounted. The final design is actually a modified Cartesian robot with two prismatic

joints corresponding to the x and y axes and a third revolute joint to control the z

angle of the cue stick. Cartesian robots have a high level of rigidity, purely due to

their configuration. The final design will use only three stepper motors, whilst still

giving a good range of shots it is able to play. The robot will be able to hit the cue

ball at any (x, y) location in any direction requested by the logic algorithm. Although

this robot is more intrusive over the whole table than a SCARA, it has been designed

79


to minimise the impact on a human player’s game. The use of high-powered stepper

motors provides the potential for swift movement so that no considerable delays will

occur during a game. The robot was designed for ease of assembly, whilst being

moderately cheap to manufacture.

As outlined in Section ??, there are numerous solutions available for the type of

drive and traverse mechanisms. Amongst these possible solutions, two appeared

likely to meet the design criteria more effectively than the others. These were the

C-section rail and the bearing/shaft combination.

C-section rails are commonly found in drafting boards and other low load applica-

tions.

C- or I- section rails have been selected above a shaft/bearing design primarily be-

cause they can be supported mid-span without impairing the motion of the carriage

supporting the cross traverse. This will increase the stiffness of the whole structure

markedly. The same carriage will be mounted inside or around the rail for better

support and rigidity, as then the carriage will not be able to twist around the rail

as could occur with the shaft/bearing configuration.

Small wheels inside or around the rail allow for smooth movement, decreasing the

effects of friction on the motion of the carriage supporting the cross traverse. C- or

I- section rails also allow for the installation of brakes, if required, at a later stage.

As these rails are easily supported from underneath, the carriage (and whole cross

traverse) can easily be removed from the end of the rail if required for modifications

and adjustments.

This configuration has a high level of ease of manufacture and lower cost than the

shaft/bearing configuration considered. A diagram of the configuration is shown in

Figure ??.

The rail/bearing configuration does not allow for mid-span support, as the bearing

surrounds the rail. This could cause the shaft to bow in the middle, since the cross

traverse would potentially be quite heavy. The shaft would have to be supported

at each end, making it difficult to remove the carriage, if required, for modifications

and adjustments of the design.

The effects of friction need to be considered when choosing a bearing/shaft com-

80


Figure 6.7: Wheels locating in modified channel section.

bination, as the bearing would slide along the shaft. It would be difficult to avoid

damaging the bearing when attaching the platform. This design has a low ease of

manufacture and a considerable fabrication manufacturing cost.

It was not feasible to purchase a bearing/rail assembly, as these systems cost up to

thousands of dollars per metre. Commissioning a custom aluminium extrusion was

also considered, but discounted due to the cost involved and lead-time. For these

reasons, a ready made aluminium channel was modified by gluing 8 mm mild steel

rods into the four corners to allow wheels to locate in the middle of the channel.

This avoided large friction forces being generated when the wheels moved axially

and rubbed on the inner walls of the channel. Super strength Araldite was used

to adhere the steel to the aluminium. Figure ?? shows the wheels locating in the

modified channel.

Once the traverse mechanism was selected it was then necessary to select an appro-

priate belt/gear system to translate the carriage of the actuator around the table.

Common belts used in robotic applications include V-belts, flat belts with guide

pulleys, and tooth belts.

The major design consideration for our project was the ability of the traverse to

accurately locate on the rails. The reliability of the whole system depends upon the

81


belts’ ability to accurately locate.

The tooth belt is most accurate due to the tight tolerances of the tooth and gear

system so that slippage of the belt under load conditions is unlikely, or at worst

minimal.

V-belts are prone to distortions and slippage under impact loads that could cause

errors in the response of the system. For example, when the shot is played, there is

a sudden high impulse load applied to the frame and drive system. It is necessary

that this force be absorbed by the system, rather than translated into a motion or

backlash. The carriage and rail system should be able to resist the majority of this

force. However, if the belt system offers minimal resistance, the likelihood of errors

becoming present in the system will increase. Tooth belts, by their nature, add a

level of resistance to motion that cannot be achieved through other belt systems.

A photo of the tooth belt and gear system used in the robot is shown in Figure ??.

Figure 6.8: Tooth belt and pulley.

Pulleys were chosen in such a way that the belt could be clamped to the middle

of the carriage inside the channel and return outside (either above or below). The

basic dimensions of the channel are 25.4× 76.2 mm; thus the pulley ought to be at

least 45 mmin diameter to satisfy the condition described above. The size of the

six pulleys selected for the final design was a diameter of 50 mm. This provided

linear steps in the x and y directions that were sufficiently small to provide the

required position accuracy and also to ensure the torque provided by the motors

would impart a linear force great enough to overcome the large inertia of the robot.

82

6.7 Summary

The belts selected for the final design have a T5 profile and are 10 mm wide; they

have an allowable tensile load of 350N and a maximum linear speed of 80m/s (?).

6.6.3 Control of actuation system

Open loop control was used in the final design with hard stops used to calibrate

each of the stepper motors. Limit switches were not used due to the necessity of an

additional control interface (for example, a PLC).

The control system for the motors and actuators was dependent firstly on the type

of actuator, and then on the programs used to calculate the required shot.

As stepper motors were chosen, the most appropriate method of control for these

motors was to use a pre-existing control driver and to interface this with the PC. A

black box approach to the actual mechanics of the control system was taken. All that

was needed to ensure that the motors were controlled accurately was a knowledge

of the appropriate form of input to the control box for a desired outcome. In this

case, the input is a series of pulses that tell the driver when and how many steps

to increment in the stepper motors. This type of control is readily integrated into

higher level programming languages such as Matlab and allows simple control

programs.

6.7 Summary

The mechanical design phase of the project included a comprehensive design process

to create a mechatronic system that would meet the physical requirements of a game

of pool. The emphasis of the design process was placed on the accessibility and

flexibility of the system. This design philosophy was centred on using industrial

processes to achieve the overall goal.

The chosen robust mechanical design allowed the inevitable modifications that were

required during the commissioning phase to be achieved with a relatively low level

of impact to the overall progress of the project.

83

7 Interface

7.1 Introduction

The goal of the hardware interface is to take the position and force requirements

from the shot selection algorithms and convert these to a series of commands to the

actuators.

The actuation of the motors and solenoid was achieved through direct parallel port

addressing that was decoded by a series of driver boards controlling the stepper

motors and solenoid.

Different options for controlling the output to the actuators in both software and

hardware were investigated. This section discusses the different solutions that were

considered as well as the final method of control used.

7.2 Software development

The final design was chosen to provide an acceptable degree of accuracy whilst still

being feasible in the limited time frame available.

The hardware interface software development process involved four stages:

• identification of the control requirements of the actuators,

• development of a command structure,

• pulse train generation,

• interface code algorithms

84


7.2.1 Actuator control requirements

Before it was possible to design software to control the motors and solenoid it was

necessary to determine the control requirements of each of the actuators.

The stepper motors used in the project, as discussed in Section ??, were attached

to a dedicated driver board. This board could then be connected further to another

control card, or else could be directly driven. Irrespective of the connections to

the driver boards, the motors have two requirements in order to achieve adequate

control: a signal indicating which direction to turn, and a step pulse of certain

duration.

The solenoid merely requires that a DC signal of a variable voltage be applied for a

given amount of time.

7.2.2 Command structure development

Once the requirements of the actuators were determined, it was possible to develop

a software command structure that would allow the motors and actuators to be

accurately controlled.

The command structure was also dependent on the output port used to connect

with the dedicated hardware. There were two ports available for output: the serial

(COM) or parallel port. As mentioned earlier, control code was developed using

both ports as potential final choices in order to determine the simplest and most

effective method of control.

A third factor influencing the design of the command structure was the programming

language used for the coding of the hardware interface software. Certain languages

are better suited to parallel port access as opposed to serial port, and vice versa.

Visual Basic was the original language used for developing the interface software,

as it has a very simple method of input and output via either a serial or parallel

port. It was realised that, since the remainder of the software had been written in

Matlab, the use of a second programming language posed further difficulties to

the overall integration of the code. However, at the time that the coding was begun,

there were no better solutions. Matlab 6.1 only allows I/O through a parallel and

85


serial port to specific types of dedicated hardware. This made it impossible to use

for the purposes of this project, owing to the nature of the hardware developed.

However, when the Visual Basic code was trialed on the dedicated computer for

the robot, it caused fatal exception errors in the operating system. This was a

result of the properties of the dynamic link library (DLL) that was used to access

the parallel port. The DLL was not compatible with the operating system. An

alternate operating system was unable to be used, due to the requirements of the

camera software. Additionally, no other DLL could be found that provided the same

access.

The release of Matlab 6.5, which allows generic parallel and serial port addressing,

meant that there was an alternative available, and it was this version that was

eventually chosen to control the signals sent to the actuators.

Serial port control was discarded on the basis that, firstly, Visual Basic was unable to

directly bit address the serial port, thereby creating complications for the hardware,

but more importantly, the serial port did not have enough pins to allow all of the

actuator control requirements to be met.

The major distinction between the parallel and serial ports is the number of pins that

can be used to transfer data. At least 8 pins needed to be available in order to control

the motion of the motors and solenoid, due to the command input requirements of

these actuators. Serial ports do not have enough available pins. A parallel port has

8 free pins to be used for data transfer and hence was chosen to provide the output

desired.

The command structure developed is shown in Table ??. Bits 0, 2 and 4 control the

direction of the steps in the three stepper motors; setting the bit high corresponds to

clockwise stepping, and setting the bit low corresponds to anti-clockwise stepping.

Bits 1, 3 and 5 control the timing of the steps; a step occurs when the bit is set from

low to high. By controlling the lower six bits appropriately, simultaneous motor

control is possible.

Bits 6 and 7 control the actuation of the solenoid. This is discussed in more detail

in Section ??.

Once the command structure was determined, the algorithms for generating appro-

86


Table 7.1: Actuators command structure.

bit 0 x-position motor direction

bit 1 x-position motor step

bit 2 y-position motor direction

bit 3 y-position motor step

bit 4 angular position motor direction

bit 5 angular position motor step

bits 6 & 7 solenoid power level

priate command sequences for the motors and solenoid could then be determined.

This was conducted in conjunction with the development of the control circuit dis-

cussed in Section ??.

7.2.3 Pulse train generation

The overall function of the hardware interface software was to send sequences of

appropriate commands (as developed in the previous section) to the driver box for

the stepper motors and solenoid, so that the motors would be driven as fast as

possible, without any steps being skipped or any other errors in actuation.

A stepper motor cannot go from being stationary to its maximum speed instantly.

If the code does not specify ramp-up time when the motors start, then the combined

inertia of the load being driven and the internal inertia of the motors would cause

the motor to “slip”. In other words, the currents would be changing through the

phases of the motor, and from the perspective of the software the motors would be

operating correctly. Physically, the stator would be missing steps and be providing

zero torque to the motors during this time. The flow on effect would be to cause

errors in the final location of the carriage and solenoid. Therefore, if a stepper motor

is to be driven at high speeds, ramp-up and ramp-down of speed are required.

Command signal generation was achieved using several methods.

87


7.2.3.1 Visual Basic

Two versions of Visual Basic code were developed. The code provided in Ap-

pendix ?? uses the interrupt timer method. The alternate code using system pauses

is very similar to that used in the final Matlab code, and hence has not been

included in this report. While these programs worked on Windows 98 machines,

the DLL used by Visual Basic to write to the parallel port was incompatible with

Windows 2000, which is the reason these were not used. If necessary this could be

reworked if a different DLL could be found. It is possible to make the step command

duration much shorter using this method, simply by changing the initial timer val-

ues in the code. This code uses sequential, rather than simultaneous, motor control

to allow for simplified testing in the initial phases. The same algorithms as used in

the Matlab code for simultaneous driving could be used.

7.2.3.2 Matlab timer interrupts

Timer interrupts were based on the inbuilt timer in Matlab, which allows variable

interrupt times (allowing for ramp-up and -down). The benefit of using timer in-

terrupts is that it allows other programs to be run while the motors are driving the

actuators. Therefore, the robot was able to move the solenoid over the white ball

as soon as it had located it, and could still silmultaneously run the remaining image

processing and shot selection software while controlling the motors. This reduced

the total time taken to execute a shot significantly. Theoretically, this method of

generating pulses should have been the most efficient.

There were two methods of achieving timer interrupts. The first method was simply

to start a timer, and instruct it to call a specified function periodically, with a

specified period. This function would then send a command signal via the digital

I/O object defined in Matlab. The second method was to set the periodic function

and the timer period as a property of the digital I/O object. Both methods were

essentially equivalent, and gave identical results.

There were a number of problems encountered in implementing this method. The

main problem was that the timers used by Matlab had a maximum resolution of

0.016 s. Therefore, the timer periods were generally rounded to the nearest multiple

88


of 0.016 s. Any specified timer period less than 0.016 s would simply be rounded up

to this minimum value. Generating a pulse train with a period of 0.032 s (0.016 s high

and 0.016 s low) results in 1875 steps per minute, which corresponds to a stepper

motor speed of 4.69 rpm (since there were 400 steps per revolution). The maximum

operating speed for the motors used was specified as approximately 50 rpm. Clearly,

the maximum speed at which Matlab was able to drive the stepper motors was far

less than the maximum speed of the stepper motors.

Another problem with the timer interrupts method was that the period between

interrupts was not regular. The period would jump between 0.016 s and 0.032 s,

resulting in very irregular bumpy control of the stepper motors. If the system pro-

cessor were occupied by further tasks (for example, running the main program of

the robot) then this problem was further exacerbated, with the period between in-

terrupts becoming much more erratic. This behaviour sometimes caused the stepper

motors to skip steps or stall completely.

Therefore, this method could not be used to generate the command signals.

7.2.3.3 Matlab pauses

The other method for generating pulses involved the use of pauses. After each

command signal is sent, the entire system pauses for a specified duration, then

the next command signal is sent, and so on. This method suffered from the same

problem as the timer interrupts method, namely that a time period of less than

0.016 s could not be achieved.

The obvious drawback of this method was that the system could not perform other

tasks whilst the motors were moving. However, the benefit over the timer interrupt

method was that the pulse train was much more regular and contained none of the

random distortions resulting from the previous method. Because the pulse train was

completely smooth and regular, the average speed of this method was faster than

the timer interrupts method.

Ramp-up and -down of speed was achieved by varying the length of the system

pauses during operation. For a certain number of steps at the beginning and end of

every pulse train, the pause delay was doubled to ensure that the motors would not

89


slipping. It was not considered necessary to generate a smooth ramp system based on

a lookup table because of the low speed with which the motors were being driven.

As before, the motors were driven at only 4.69 rpm compared to their maximum

speed of approximately 50 rpm. For this reason the ramping did not need to be

as long or as finely calibrated as it would have needed to be, had the motors been

run at higher speeds. In fact, because the speed was so low, it was found that no

ramping at all was necessary, so it was not used in the final version of the interface

software.

It was clear that the sole benefit of being able to run simultaneously with other

software was not sufficient to warrant the usage of the timer interrupts method. The

pauses method was much more smooth and reliable, and ultimately the difference

in the total time needed to take a shot was not very large.

The final code as incorporated into the robot program can be found in Appendix ??.

7.2.3.4 dSPACE

An alternative to controlling the timing of the command signals from a computer

was to use an external system, such as a microcontroller or a controller board. The

use of the dSPACE board DS1102 to send the command signals to the hardware

driver box was investigated. This board was the only suitable controller available

for use.

The program downloaded to the DS1102 runs continuously. If a sequence of, say, 700

command signals needs to be sent to the driver box, then Matlab sends an array

of 700 eight bit commands to the board. Matlab can then trigger an interrupt to

the program running on the board, which responds by sending the 700 commands

sequentially with a specified period.

The C program code written to run on the DS1102 can be found in Appendix ??.

The code currently only writes to two digital I/O pins, but it would be very simple

to modify the code to write to eight.

A more elegant solution than the one presented here would be to have the command

signal generation algorithms in the C code, so that Matlab would only need to

send four integers (rather than an array of commands) to the board, corresponding

90


to numbers of steps for the x-direction, y-direction and angular stepper motors, as

well as the solenoid power level.

The system was successful in outputting signals at the specified rate. Unfortunately

the dSPACE board was unable to interface with the computer dedicated to the

project. The dSPACE board was only able to interface with an ISA bus, which

has now been superceded by the PCI bus, which was the only type available on the

project computer. However, modifying the code for a compatible controller would be

a simple task, and this would allow fast and smooth control of the stepper motors.

7.2.3.5 Conclusion

While using an external controller to send the command signals to the driver box

would be the best solution to drive the stepper motors at a reasonable speed, this

was not achieved within the time frame of the project. Instead, the Matlab pauses

method was used, since this produced smooth and reliable motion of the robot.

7.2.4 Code algorithms

All three stepper motors have 400 half-steps per revolution, so that one half-step

corresponds to an angular rotation of 0.9◦.

The diameter of the pulleys used in the design is d = 50.93 mm. One half-step

correponds to a linear distance of 1400

πd = 0.400 mm.

The Matlab shot selection algorithm outputs position and angle coordinates rel-

ative to a datum position. These three values are converted to a number of steps

using the ratios above. Then a series of commands is written to the motor driver

boards to invoke the correct number of steps in each motor.

The interface uses the Matlab Data Aquisition Toolbox (DAQ) to convert the

number of steps into a series of commands and hence pulses to the stepper motor

driver boards and solenoid control card.

This is achieved by passing the desired 8-bit output command, represented in the

format shown in Table ??, as a variable to a MEX file which is then written directly

91

7.3 Hardware

to the output memory location of LPT1. The MEX file acts similarly to a dynamic

link library in that it writes directly to the chosen output port. The dedicated

hardware in the driver box then decodes the instructions written to the parallel

port and controls the stepper motors appropriately.

7.3 Hardware

The overall control card circuit can be seen in Appendix ??. The purpose of the

control card is to decode the instructions written to the parallel port memory ad-

dress.

7.3.1 Motors

As mentioned above, the stepper motors have pre-existing driver boards that require

two input commands:

1. the direction of rotation,

2. a step pulse command of some duration.

The parallel port/driver board interface must be able to link directly with the pins in

the parallel port. This was achieved through the creation of a linking card that takes

in the values from the parallel port and translates them into signals that the driver

board can interpret. This was necessary because the driver board was designed to

interface with a PCI card that would have been inserted into the computer. However,

these type of cards are no longer compatible with more recent computer systems and

hence a hybrid interface card was needed.

As the pins’ command structure was designed to correspond to these requirements, it

was fairly simple to then hardwire these pins to the driver boards. Optical isolators

were added to ensure that the computer would be protected if electrical problems

were encountered in the motors.

92

7.3 Hardware

7.3.2 Solenoid

As mentioned in Section ?? above, two of the parallel port pins controlled by the

Matlab interface software commanded the solenoid.

Two different concept solutions were considered for the digital control of the solenoid.

The initial suggestion was to convert the power supply from AC (mains) to 150V

DC. The two pins could be used to determine the level of power supplied to the

solenoid. The first pin is set high when a timer in software is initialised. When

this pin goes high, a variable capacitor is charged for the duration that the timer is

running. When the timer expires, the second pin is set high to discharge the variable

capacitor, causing the solenoid to fire.

This method is the most effective and versatile form of control for the solenoid, as it

makes possible a continuous range of force output. However, there was insufficient

time and resources to realise this design.

The alternative solution was simply to use the four possible output combinations

from the two pins (00, 01, 10, 11) to correspond to a different preset power level

supplied to the solenoid. This allows the power supply to remain AC (thus cir-

cumventing the need for a transformer). Table ?? shows the power levels indicated

by each output combination. The solenoid remains charged until a 00 command is

received.

Table 7.2: Solenoid power levels and commands.

Pin 6 Pin 7 Power level

0 0 no power (0V)

0 1 low power (60V)

1 0 medium power (120V)

1 1 high power (180V)

This is a much coarser version of control, but it is simpler to implement in hardware,

as the use of a variable capacitor is not required. A timer is still required in order

to determine when to reset the pins and turn the solenoid off.

93

7.4 User Interface

This second method was the solution chosen for our hardware. The signals from

the two pins allocated to the solenoid were passed through a decoder which, using

a series of relays, created the required power level in the solenoid.

Interlocking relays were used to ensure that no two power levels could be applied

to the solenoid at the same time. The solenoid circuit was also optically isolated to

prevent damage to the computer. A transformer was incorporated into the design

to supply power to the solenoid and motors, and also to the optical isolators, relays

and the decoder.

7.4 User Interface

A GUI was designed to be displayed on a monitor at the beginning of each of the

robot’s shots. This GUI is shown in Figure ??. The main purpose of this GUI was to

provide a trigger which a user could use to tell the robot to take its shot. However,

it also allowed a user to set a number of options.

The robot can be set to choose and play a shot completely independently. Alterna-

tively, the robot can be asked to display its chosen shot on the screen for verification.

Additionally, the robot can present a list of several alternative shots, and a user can

choose a shot from this list for the robot to play.

A user can also decide the level of complexity of the shots which are to be considered

by the robot. The robot can be ordered to only to consider “simple (direct) shots”,

that is, shots of type BjPi. If the robot is asked to also consider “medium complex-

ity shots”, then the robot will also consider shots of types [Ca]BjPi, Bj[Ca]Pi and

BjBkPi. The robot can further consider “very complex shots” of types [Ca]BjBkPi,

Bj[Ca]BkPi, BjBk[Ca]Pi and BjBkBlPi. Finally, the robot may be set to a “trick shot

mode”, so that it only considers shots from this last set of shot types.

The GUI was programmed so that it would remember the options selected for the

previous shot in the game, and present these as the default options for the next

shot.

The robot may be triggered to take its shot by pressing one of the three buttons,

“Table is still open”, “Go for red” and “Go for yellow”. The first of these buttons

94

7.4 User Interface

Figure 7.1: A GUI allowing a user to set various options and trigger the robot to

take its shot.

95

7.4 User Interface

causes the robot to assume that no balls have yet been sunk, so that it may attempt

to sink any coloured balls. Otherwise, the robot will only play to sink only red

or only yellow balls. Additionally, for the first turn of every game, an additional

button, “Play the break”, is presented.

There is an option for the robot to play forward only. This option should be checked

when the opponent has played a shot resulting in the cue ball being sunk in a pocket.

The rules of 8-Ball then dictate that the cue ball must be struck in a forward direction

in order for the shot to not be deemed a foul.

Finally, there is a button to allow a user to halt the robot software and end the

game.

A second GUI was created to display the shot chosen by the robot software on the

screen, and also to allow a user to choose a shot for the robot to play from a list of

alternatives. This GUI is shown in Figure ??.

Figure 7.2: A GUI displaying the chosen shot and allowing a user to choose from

other alternatives.

96

7.5 Summary

Initially, the GUI will display the best shot found by the shot selection software for

the robot to play. A pull-down menu is also provided so that a user can examine

a number of alternative shots. When a shot is selected, an image of the shot is

displayed, and the score (related to the merit function) of the shot is shown.

While the robot was carrying out various tasks, an update window was displayed to

the screen, containing information about the current activity of the robot.

Additionally, a text-to-speech converter was used to add an element of personality to

the robot. The robot was programmed to randomly speak various statements, some

of which related to its operation, and others which were provided for entertainment.

7.5 Summary

The interface between the software and the dedicated hardware was designed to

meet the requirements of the actuators and the output commands from the shot

determination algorithms. Numerous methods of motor control were developed and

tested, but due to hardware conflicts, only one was able to be integrated with the

system. The success of the interface was evinced by the lack of errors present in the

conversion between the software instructions and the resulting physical actuation.

A graphical user interface was developed to allow user control of several options, and

to communicate the current process being executed by the robot. Voice synthesis

was included to add a human element to the robot.

97

8 Commissioning and results

8.1 Introduction

The commissioning of any prototype system is a time consuming task. After the

construction phase was completed, the integration of the different subsystems of the

robot began. This process involved combining the vision system with the shot selec-

tion algorithm, and then incorporation with the dedicated hardware. This section

outlines the process that was followed during this stage, as well as highlighting the

various problems that were encountered and their respective solutions.

8.2 Software integration

Software integration was reasonably straightforward, owing to the compatibility of

the different software systems and the flexibility of Matlab.

As discussed in Section ??, a DOS console program was used to interface the digital

camera with the computer. This program was written in Visual C++. Through

this console program, Matlab was able to invoke the D30Remote program when it

was necessary to take a photo.

The other software programs were all written in Matlab. As a result, the task of

combining the different systems was relatively simple. The only modifications that

were necessary during the commissioning phase of the project were in response to

run time errors generated by exceptional ball layouts that precluded a normal shot

being played. These modifications added robustness to the system by ensuring that

the robot would always be able to play a shot. More specifically, if the robot was

unable to play a shot to hit one of its target balls, it was programmed to shoot the

cue ball towards the middle of the table on the highest solenoid power level.

As discussed in Section ??, the dedicated hardware and interface program were

developed simultaneously. Owing to this approach, the communication with the

98

8.3 Position calibration

motors and solenoid was extremely simple. The more difficult part of this process

was found to be in the calibration and testing of the system.


It was necessary to compare the number of steps per millimetre to the theoretical

values obtained previously. This was achieved by measuring the distance between

two known points on the table, then counting the number of steps required to drive

the motors between these points. The theoretical values and the actual values

corresponded exactly. This was a direct result of the high quality of the belts and

pulleys selected for the drives, which eliminated the possibility of slipping or lag.

The determination of datum positions was the most crucial aspect of the calibra-

tion of the system. Any errors present in the calibration of the system resulted in

consistent errors in the positioning of the robot during actuation.

As open loop control for the robot was used, there was no feedback from the motors

to check whether the positioning was correct. Hence, it was necessary to ensure that

the starting position of the robot was as accurate as possible. This was achieved to

varying degrees of success.

The location of the datum position in the x-direction was located by using a hard

stop placed in the channel at the location where the cross traverse had just moved

out of the field of vision of the camera, allowing an uninterrupted view of the table.

The position of this datum with respect to the table was at the end from which

the break is played. The y-direction datum position was found by driving the cross

traverse carriage to the right hand side of the table with reference to the break

position. The robot in the datum position can be seen in Figure ??.

The angular datum position was the most difficult of the positions to accurately

define due to the lack of feedback in the motors. The angular datum position was

chosen to be in line with the x-axis of the table as shown in Figure ??. However,

great difficulty was experienced in determining whether the motor was in its initial

position. The angular position of the cue was also found to be the most critical

in terms of its affect on the ability of the robot to successfully complete its shot.

99


Figure 8.1: Robot at the three datum positions.

100

8.4 Cue ball location

A significant amount of time was spent attempting to successfully calibrate this

position.

The first calibration method attempted was to turn on the motors and use the cue

as a visual guide, by sighting the cue against the rail to test whether they were

parallel. The angular motor was stepped until it was judged that the solenoid was

aligned with the x-axis. This method was not robust, and as a result, a different

method was found.

The solenoid bracket was driven to the x- and y-direction datum positions, and then

rotated until the solenoid bracket just touched the upright. The bracket was then

rotated a specific number of steps in the anti-clockwise direction to provide a datum

for the angular position. The number of steps was determined by trial and error.

The number of steps to align the cue tip with the x-axis in the datum position was

found to be between 19 and 20 steps anti-clockwise from the upright. This was the

maximum resolution obtainable. If it were possible to increase the resolution of the

motors, say from 0.9◦ per step to 0.09◦ per step, the accuracy of the datum would

be greatly improved.

The next step in calibration process was to determine the stand-off distance between

the cue tip and the cue ball. This was calculated using trial and error, judged by

how well the cue hit the cue ball.

Finally, the relative distance between the datum of the cue and the zero position in

the photo of the table.

These calibration values were then placed into the main control program and used

throughout the commissioning process.

8.4 Cue ball location

Once calibrated, the next phase of commissioning involved the testing of the posi-

tional accuracy of the vision system with regard to accurately locating the actuator

near the cue ball. The solenoid actuator was programmed to position the shaft of

the angular position motor directly over the center of the cue ball. This allowed an

easily verifiable method of checking (x,y) calibration.

101

8.5 Cue tip length

The optimum position for the solenoid before actuation was determined to be the lo-

cation at which the shaft would make contact with the cue ball just before maximum

extension. This allowed the maximum velocity of the solenoid and hence maximum

force to be applied as well as reducing the amount of spin transferred to the cue

ball. The number of steps required from the center of the ball to the optimum firing

position was determined empirically through trial and error. The use of a video

camera that allowed easy measurement of the point of contact with respect to the

centre of the ball was considered, however due to time constraints was unable to be

implemented.

8.5 Cue tip length

One of the problems encountered in the commissioning of the robot was finding the

optimum length of the cue tip. It was a requirement of the system that the cue

tip have the largest possible stroke length, both to improve power and to move the

point of contact closer to the center of the ball. The cue tip also had to be high

enough off the table when fully retracted, so as not to interfere with the placement

of the balls on the table.

The pool table frame was designed to allow the table surface to be adjusted to

obtain the most level playing surface. However, owing to the quality of the table

used, there was still some degree of bowing in the table. This resulted in the height

of the cue tip changing from location to location across the table. As a result, a

fixed position cue was inappropriate as it did not allow the necessary adjustments

to be made during the commissioning phase.

Initially the cue tip, which was made of wood, had a hole bored, and a 10UNF grub

screw glued into the cue. This was then attached to the solenoid. This did not

allow any adjustments of the length of the cue tip and hence was replaced. Instead,

a 5mm threaded rod was glued into the cue tip. Two lock nuts were used to allow

variable positioning of the cue tip, by placing one 8mm washer in between two

5mm washers. The 8mm washer was larger in diameter then the hole into which

the solenoid retracted. Hence, by adjusting the position of the lock nuts on the

threaded rod, it was possible to vary the distance that the solenoid retracted. A

102

8.6 Carriage wheels and motor torque

photo of the solenoid shaft with the cue tip attached is shown in Figure ??.

Figure 8.2: The solenoid shaft with the cue tip attached.

Once this was completed, a series of height measurements were made over the table

to determine the profile of heights and deduce the worst corner. The cue was ad-

justed so that, in the highest corner of the table, the retracted cue tip cleared the

height of the balls in any orientation.


The initial design used a four wheel system in all carriages to assist in locating the

carriages in the rails. When the robot was assembled the wheels running in the rails

caused too much friction and as a result the motors stalled. This was particularly

the case in the x-direction. This would not necessarily have required the changes

that were eventually made had the motors driving the carriages been more powerful.

The two potential solutions to this problem were to remove the top two wheels in

all carriages and use the weight of the uprights and traverse to locate the wheels

103


in the rails, or to find a way to increase the power and hence torque output of the

motors.

There were two possible solutions for boosting the power output. The first was to

use a gear box; this was not used as it was too time and cost intensive to buy and

retrofit. The second method, which was implemented, was to increase the torque

output by increasing the power supplied to the motors. The stepper motors were

designed to be operated at 2.5A; this was increased to 3A. Whilst this is not a

recommended solution for future problems encountered with motor power, it was

the only feasible solution available in the time frame given, and coupled with the

changes to the wheel locations, provided an acceptable, if temporary, solution to the

problem.

This solution worked extremely well in the x-direction carriages, however, there

were problems found with the y-direction. When the wheels were removed, the

carriage rotated about the C-section rail, as a result of the center of mass of the

structure being offset from the geometric center. This caused the cue tip to shift in

its position, depending on the orientation of the solenoid bracket. It also caused the

axles of the wheels in the lower rail to rub severely against the inner channel wall,

further causing the motor to stall. It was therefore necessary to place one wheel

back in the top rail. However, the position (rather than being adjustable as it was

initially) was made fixed to ensure that the wheel would not slide around during

operation. The final configuration of the carriage wheels can be seen in Figure ??.

Figure 8.3: The solenoid shaft with the cue tip attached.

104

8.7 Cross traverse shearing

8.7 Cross traverse shearing

It was found that during prolonged operation, the carriages in the x-direction became

misaligned. It was determined that the problem stemmed from the use of grub

screws, used to attach the pulleys to the motor shafts, and the shaft from the motor

onto the coupling. The unequal friction levels in the rails also contributed to the

problem by increasing the level of torque required to move the shaft. The grub

screws were located on flats machined into the shaft of the motors, as well as in the

shaft attached to the coupling. The main difficulty with grub screws is that if there

is any real resistance to motion, they have a tendency to grab and tear the metal

of the shafts. This caused slippage in the shaft attached to the coupling, and as a

result, the relative position of the carriage on that side would slip. The problem

was minimised by regularly monitoring the status of the grub screws and tightening

when necessary. Teflon lubricant was applied to all of the rails to reduce the effects

of friction and this was seen to have a significant effect on the frequency of the cross

traverse shear.

8.8 Cables

The power cables for the stepper motots, solenoid and camera, and the USB cable

for the camera all required to be suspended above the table. The cables needed to

be arranged in such a way so that they did not obscure the image of the table when

a photo was being taken. Also, the cables from the solenoid and angular position

stepper motor needed to be long enough to extend to the far side of the table, as

well as not interfering with the balls, when the carriage was moving over the table.

The solution was to suspend a wire from the camera mount, which followed the

camera cables and met the other cables close to where they joined the solenoid

mount. As the robot needed to be out of the frame of the photo when the camera

was operating, the cable needed to be fairly taught so as to not droop into the image.

The final configuration of the cables and wires setup can be seen in Figure ??.

Because the cable was attached to the camera mount, any significant tension applied

to the cable would cause the mount to shift position, causing errors in the captured

105

8.9 Vision system

Figure 8.4: The cables and wires setup.

image. This problem was reduced by driving the carriages to the datum positions

to take a photo, while still leaving enough slack in the wire so as not to shift the

camera position. However, it was not possible to take a photo with the traverse off

the other end of the table, as this caused a small shift in the camera position.

8.9 Vision system

The attachment of the camera to the ceiling above the pool table is shown in Fig-

ure ??.

Figure 8.5: The attachment of the camera to the ceiling.

106

8.9 Vision system

It was necessary to position the camera as accurately as possible such that the plane

of the camera lens was parallel to the plane of the table. This was achieved by testing

the alignment of two parallel lines marked out on the floor beneath the camera. The

orientation of the camera was adjusted until the the images of the parallel lines were

parallel in the photo taken from that camera position. It was difficult to adjust the

camera mount, so the final position may not necessarily have been correct. Image

distortion made this calibration difficult because the lines that were used to judge

the position were distorted in the calibration image.

The vision system required calibration once the camera was mounted above the table

before it could be used and interfaced with the arm of the robot. This calibration

involved two variables: the horizontal movement between the table and the camera,

and the zoom of the lens. Changes in these variables result in changes in the cropping

limits of the photo, and the radius (measured in pixels) of the images of the balls

on the table respectively. This is because the camera and the robot are not linked,

so that any relative movement between the two creates a need for recalibration. If

the camera was mounted on a stiff frame that was physically attached to the pool

table, these issues would not be a problem.

Calibration of the vision system involved setting the template colours to correspond

to the approximate average colours of the balls (determined by the lighting in the

room).

In order to align the table image with the actual tabletop, a fixed point was used

to define the zero pixel of the image. The image was cropped with this point as

pixel (0,0), and the number of steps in the x- and y-directions required to drive the

solenoid bracket to this point from the datum were measured. This point could then

be referenced in both coordinate systems, allowing the ball positions in the image

to be mapped accurately to physical positions on the table.

An auto-calibration program has been written to determine the cropping points

of the image, but the program has not been tested. It is believed that the auto-

calibration will prove not robust enough to be relied upon to yield sufficiently accu-

rate calibration values.

107

8.10 Results

8.10 Results

Quantitative results were not investigated as comprehensively as would have been

desirable. However, it has been possible to qualitatively comment on the perfor-

mance of the robot.

This section discusses in more detail the overall performance of the three different

systems of the robot.

8.10.1 The eye

The vision system of the robot was proved successful in locating the balls in the

image of the table. With minor adjustments, and assuming that there was noth-

ing impeding in the photo, the balls were located in the image and their colour

determined 100% of the time.

Small errors came about trying to reconcile the positions of the balls in the image

(with reference to the zero pixel) and the position of the balls on the table (with

reference to the robot datum). Issues regarding this were solved with trial and error

of the cropping values of the image.

The only further error in the vision system related to the lens distortion. The

accuracy of the distortion filter used to correct this error was unable to be quanti-

fied; however, it must be assumed that it was reasonably accurate since the robot

frequently was able to sink balls.

8.10.2 The brain

The brain generally selected shots which were consistent with shots that would be

chosen by a human player.

A problem arose with the points used to define the positions of the pockets. The

software assumed that, in order to sink a ball in a pocket from any point on the

table, there was a fixed point towards which the ball must aim. However, there is

no such fixed point. At extreme angles, or when other balls are very close to the

108

8.11 Summary

pockets, the points defined in the program may not necessarily be the optimal points

at which to aim. This caused the shot selection software to sometimes ignore shots

which were quite easy.

8.10.3 The arm

The primary goal of the arm was to take the output from the shot determination

program and accurately position the robot over the desired location on the table.

As mentioned above, owing to the restrictions in time after commissioning, the

results from the arm positioning systems were extremely preliminary.

Qualitatively, the arm system achieved its goal of positioning the cue in the desired

location.

As highlighted in Section ??, there were a number of changes made to the cali-

bration systems that attempted to improve the overall accuracy of the positioning

mechanism. This improved the accuracy of the system considerably, especially in

terms of angular positioning.

Recommendations for future modifications are contained in Section ?? and stem

from the results of the initial qualitative testing of the robot.

8.11 Summary

The commissioning process overall was relatively smooth. There were of course

errors present in the system, which have been discussed. As with any prototype

development process, it is extremely difficult to predict absolutely all of the potential

errors in a system during the design phase. Even with as comprehensive and detailed

a design process as was followed in this project, there are always errors that cannot

be foreseen. However, it is possible is to predict general areas where problems could

be encountered, and this was achieved in this project. The effect of this was to allow

easy determination of errors as there was already a place to look.

Comparatively, the number and severity of the errors encountered in the commis-

sioning phase were low. Considering that the commissioning phase was conducted

109

8.11 Summary

over a period of a little more than two weeks, and the errors still present in the

project at the end of this period were identified as being primarily as a result of

the limitations in the hardware systems used and not as a result of the design or

construction, it can be concluded that the commissioning and design phases were

overall a success.

110

9 Future work

9.1 Introduction

There are several areas that could be developed and modified to increase the accu-

racy and level of strategy of the robot. As this was the first version of this robot,

some of these issues were considered infeasible in the allowable time frame to design

and build a complete robot. Other problems encountered had more effect on the fi-

nal product than first predicted. These ideas are discussed in relation to the eye, the

brain and the arm. The following discussion is intended as suggestions that could

be developed by future students to address problems and shortcomings encountered

by this year’s project team.

9.2 The eye

At present, the vision system of the robot is exemplary in its operation of locating

the balls in the image of the table. There are few aspects of its operation that could

be made more robust. However, there are two problems that could still be worked

upon to make the eye more capable of accurately locating the balls on the table.

The first of these is image distortion. Image distortion is present in almost all lenses

to a certain degree and has been investigated to a point as discussed in Section ??.

In the context of the real-time play, implementing an inverse filter on an image of the

table and balls was infeasible. This was due to the size of the original images, and

hence the filter size and processor time dedicated to this task. For this incarnation

of the robot, it was sufficient to adjust the coordinates of the ball centres, depending

on where in the image of the table the balls were found to be located. It is realistic

to say that the digital camera and lens used for this project will be used by the

Department of Mechanical Engineering in the future, possibly in an application

requiring image distortion correction. It is suggested that a more accurate method

for correcting image distortion be investigated and implemented.

111

9.3 The brain

The second aspect of the vision system that requires future development is detection

of the cushions and pockets of the table. This would make the robot more robust as

without edge detection it cannot be determined if the table and camera have moved

in relation to each other. This would make the robot more flexible to changes in

location and remove the need for careful and time-consuming calibration. Edge

detection is discussed in Section ??.

The use of “bigs” and “smalls” is also suggested as an area for further research. The

use of a casino set of balls does not hinder the ability of the robot nor a human to

enjoy a game of pool. This suggestion could be seen as purely academic, but would

also increase the scope of the robot to play games such as 9-ball.

9.3 The brain

Further development of the brain of the robot includes modelling of ball collisions so

as to predict the final position of the cue ball for the common strategy of leaving the

cue ball in a difficult position for the opponent to play a shot. This would require

intimate knowledge of the table’s friction coefficients and the physics of pool.

It is suggested that multiple shot strategy be incorporated into the brain code. This

could include the robot detecting when its opponent has fouled, allowing the robot

to have two shots and determining that it has potted an object ball, earning another

shot. Further to this, the brain code could determine the optimum position for the

cue ball behind the foul line or within the “D” when its opponent has sunk the white

ball.

Ultimately, it is desired that the robot adopt a self learning algorithm such that the

code is modified to increase the accuracy of play based on the result of the intended

shot compared to the actual result (and possibly the outcome of shot modelling).

Once the robot has achieved sufficient accuracy to be dubbed a “pool shark”, code

should be implemented to modify the skill level of the robot so that less talented

players may be competitive with the robot.

112

9.4 The arm

9.4 The arm

The speed with which the Matlab data acquisition toolbox could write to the

parallel port was believed to be much faster than was actually achieved, thus greatly

increasing the time for the robot to take a shot. Methods to combat this problem

were developed, including controlling the motors in Visual Basic or with a dSPACE

board. Tests have indicated that both these methods could be successful in speeding

up play but had to be discarded due to conflicts with either software (in the case

of VB coding) or hardware (as encountered when attempting to implement the

dSPACE board).

In order for the robot to play shots with forward and back spin, two more degrees

of freedom are required: one to control the pitch of the cue and another to control

the height the cue sits above the table. This would require the redesign of the cue

assemble and the use of up to two more motors. The effects of spin would need to

be investigated further and the shot selection algorithm modified accordingly.

Using a different electric circuit could increase the resolution of the force supplied by

the solenoid. This idea was investigated and not pursued only due to the timeframe

of this project. Alternatively, other methods of cue actuation, pneumatics in par-

ticular, could be investigated to see if another method provided a similar resolution

and maximum force to that of the washing machine solenoid currently used in the

robot.

Position and angular feedback from the motors is recommended to determine if

slippage of the motors has occurred and for calibration purposes. Problems with

calibration have been discussed in Section ??. Feedback control could then be

implemented to increase the repeatability and accuracy of the mechanical design.

An error of up to 0.45 degrees exists due to the step size of the motor controlling

angular location of the cue. The use of a 10:1 reduction gearbox is recommended to

reduce this error by 90 percent. This modification could be incorporated into the

possible redesign of the cue assembly.

Presently, as discussed in Section ??, the system commands the stepper motors and

solenoid by changing commands to the dedicated driver boards through the parallel

port. In other words, the software tells the stepper motors and solenoid when to

113

9.4 The arm

pulse and when to fire. Unfortunately, the software system was unable to emulate

the necessary command speed required to drive the motors at a faster speed. In the

future as an alternate or combined solution using the dSPACE board, the construc-

tion of a more complex piece of dedicated hardware could be implemented. The

solution would still involve the use of either the parallel port or the converted PCI

card, but the method of output to the motors would be different. More specifically,

instead of the software determining the step duration, the hardware would interpret

a string value that was passed to it from the software (for example a total number of

x, y and angular steps and solenoid power setting) from which the hardware would

decode and implement the actuation. This has the obvious benefit of circumventing

the maximum resolution of the command speed present in the software; consequently

allowing finer step size resolution, thereby increasing the speed of the motors. This

could be achieved through purchasing an off-the shelf micro-controller, or if it was

deemed more efficient, to construct a specific one.

114

10 Conclusion

The aim of this project was to design and build a robot capable of playing pool

with speed and accuracy comparable to that of a human player. This aim has

been achieved to a certain degree, with a robot capable of independently playing

a game of pool, although the speed and accuracy of the robot are below initial

expectations. The changes necessary to significantly improve the performace of the

robot are outside the scope of what was achievable in the time frame.

A comprehensive literature review was conducted, covering previous published at-

tempts at constructing pool playing robots, the game of pool itself, and other tech-

nical areas relevant to the design. None of the previous published attempts have

been successful in creating an autonomous robot, mostly due to the limitations of

the technology at the time.

An industrial process design approach was used to create the most efficient mechan-

sims for achieving the goal of the project. The component subsystems of the robot

were designed concurrently. The robot goes through the same overall processes as

a human: locating the balls, selecting a shot to play and executing the shot. The

method of implementation of each task differs from that of a human. When design-

ing the robot the most efficient and accurate methods available were used rather

than the most “human-like”.

The robot comprises of a vision system, shot selection algorithms, hardware interface

and a mechatronic actuation system. The integration, commissioning and prelim-

inary testing of the robot has been completed. The areas of weakness have been

investigated and documented for future reference.

115

References

Allen-Bradley. Servo motors. http://www.ab.com/motion/servo/, 2002.

BCA. Billiard Congress of America. http://www.bca-pool.com/, 2002.

Jean-Yves Bouguet. Camera calibration toolbox for Matlab. http://www.vision.

caltech.edu/bouguetj/calib_doc/, 2002.

Breeze. Breeze Systems. http://www.breezesys.com/, 2002.

Christian Bruer-Burchardt and Klaus Voss. Automatic lens distortion calibration

using single views. Mustererkennung, 2000.

John J. Craig. Introduction to Robotics: Mechanics and Control. Addison-Wesley,

second edition, 1989.

George E. Dieter. Engineering Design. McGraw-Hill International Editions, third

edition, 2000.

Farid and Popescu. Blind removal of lens distortion. Journal of the Optical Society

of America, 2001.

Jim Green and Josh Richmond. Robotic image processing pool player. http://

ocho.uwaterloo.ca/~workshop/RIPPP/, 1998.

Peter Grogono. Mathematics for snooker simulation. Private notes, January 1996.

Kenneth H. L. Ho. A snooker playing robot. http://www.bmc.riken.go.jp/

sensor/Ho/chicago/Robotics/Snooker/snooker.html, 1998.

HSW. How stuff works. http://www.howstuffworks.com/, 2002.

Wesley H. Huang. A simple robot pool player. Project report, Massachusetts Insti-

tute of Technology, 1991.

Anil K. Jain. Fundamentals of Digital Image Processing. Prentice-Hall, New Jersey,

1989.

116

10 CONCLUSION

Douglas W. Jones. Control of stepping motors, a tutorial. http://www.cs.uiowa.

edu/~jones/step/, 1998.

MG. Melles griot. http://www.mellesgriot.com/, 2002.

Harri Ojanen. Automatic correction of lens distortion by using digital image pro-

cessing. http://www.math.rutgers.edu/~ojanen/undistort/undistort.pdf,

1999.

OpenCV. Intel open source computer vision library. http://www.intel.com/

research/mrl/research/opencv/index.htm, 2002.

J. Pers and S. Kovacic. Nonparametric, model-based radial lens distortion correction

using tilted camera assumption. In H. Wildenauer and W. Kropatsch, editors,

Proceedings of the Computer Vision Winter Workshop, pages 286–295, February

2002.

Bingchen Qi and Yoshikuni Okawa. Building an intelligent mobile robot for billiards.

In 30th International Symposium on Robotics, pages 605–612, 1999.

John C. Russ. The Image Processing Handbook. CRC Press, second edition, 1995.

SciTech. SciTech. http://www.scitech.com.au/, 2002.

Ron Shepard. Amateur physics for the amateur pool player. http://www.playpool.

com/apapp/, 1997.

Transdev. Transdev 2002 catalogue, 2002.

117

Appendix A Robot code

A.1 Main program code

%PLAY Play pool.

% Invoke a virtual virtuoso pool player.

clear

close all

% set up parallel port

parport = digitalio(’parallel’, ’LPT1’);

hwlines = addline(parport,0:7,’out’);

motorsbusy = 0;

timedelay = 0.001;

% define the colours, as previously sampled from photos

feltblue = [0.21 0.39 0.76];

ballwhite = [0.95 0.94 0.87];

ballblack = [0.02 0.04 0.05];

ballyellow = [0.98 0.76 0.10];

ballred = [0.96 0.21 0.20];

% game identification

gamedatetime = datestr(now);

gamedatetime(find(gamedatetime == ’:’)) = ’.’;

logfiledir = [pwd filesep ’gamelogs’ filesep gamedatetime];

pwdnow = ’’;

turn = 1;

wantexit = 0;

thoughtsoutput = 1; % default output thoughts to file

tgt = 1; % default first target is a break shot

shotcomplexity = [1 1 0]; % default consider all shot complexities

hmshots = 2; % default automatic shot selection by robot head

hmshotsvalues = [3 6 10 20 100];

hmshotsstring = num2str(hmshotsvalues(1));

for ii = 2:length(hmshotsvalues)

hmshotsstring = [hmshotsstring ’|’ num2str(hmshotsvalues(ii))];

end

hmshotslast = 1; % default first number on list

% boundaries of the table in the photo

tabletop = 211;

tablebottom = 1760;

tableleft = 24;

tableright = 3011;

mmperpixel = 1746/(tableright - tableleft);

% distortion correction

load(’cp.mat’) % load control points as ’cpphoto’ and ’cpsteps’

tform = cp2tform(cpsteps,cpphoto,’piecewise linear’);

% position/angle conversion

stepspermm = 2.5;

stepsperradian = 400/(2*pi);

zeropos = [-233 345]; % location of position (0,0)

balltosolenoid = 117*stepspermm; % in steps, how many

118


% initialise positions

Xpos = 0;

Ypos = 2952;

Apos = -100;

doXstep = 0;

doYstep = 0;

doAstep = 0;

% speech files

speech.on = 1; % if we want the robot to talk

speech.sync = ’async’;

speech.ad = wavread(’ad.wav’);

speech.allday = wavread(’allday.wav’);

speech.benisgreat = wavread(’benisgreat.wav’);

speech.bill = wavread(’bill.wav’);

speech.boyfriend = wavread(’boyfriend.wav’);

speech.chalk = wavread(’chalk.wav’);

speech.daddy = wavread(’daddy.wav’);

speech.doozy = wavread(’doozy.wav’);

speech.drben = wavread(’drben.wav’);

speech.fact19 = wavread(’fact19.wav’);





speech.faster = wavread(’faster.wav’);

speech.hello = wavread(’hello.wav’);

speech.icantalk = wavread(’icantalk.wav’);

speech.letsplay = wavread(’letsplay.wav’);

speech.look = wavread(’look.wav’);

speech.pints = wavread(’pints.wav’);

speech.playthisone = wavread(’playthisone.wav’);

speech.red = wavread(’red.wav’);

speech.shotlooksgood = wavread(’shotlooksgood.wav’);

speech.silvioetal = wavread(’silvioetal.wav’);

speech.thinking = wavread(’thinking.wav’);

speech.yellow = wavread(’yellow.wav’);

% set up GUI sizes

screensize = [1 1 1024 768]; % get(0,’screensize’);

% fonts

uifont.name = ’Tahoma Bold’;

uifont.smallsize = 10;

uifont.bigsize = 14;

% colours

backcolour = [0.4 0.4 0.4];

buttoncolour = [0.3 0.3 0.6];

frontcolour = [0.4 0.4 0.7];

popupcolour = [1 1 1];

textcolour = [1 1 1];

% for the trigger GUI

window.width = 450;

frame.spacing = 12;

frame.left = frame.spacing + 1;

frame.width = window.width - 2*frame.spacing;

button.spacing = 12;

button.left = frame.spacing + button.spacing + 1;

button.width = window.width - 2*(frame.spacing + button.spacing);

pushheight = 36;

radioheight = 24;

popupheight = 20;

popupwidth = 50;

editheight = 20;

editwidth = 200;

putfilewidth = 20;

119


titleheight = 40;

% for the choose shot GUI

popupheight2 = 30;

popupwidth2 = 130;

pushwidth2 = 60;

textwidth2 = 120;

% for the update GUI

updatewindow.width = 600;

updatewindow.textheight = 30;

updatewindow.bigtextheight = 120;

updatewindow.height = 4*updatewindow.textheight + updatewindow.bigtextheight;

updatewindow.left = screensize(1) + (screensize(3) - updatewindow.width)/2;

updatewindow.bottom = screensize(2) + (screensize(4) - updatewindow.height)/2;

while 1

logfilenamedefault = [’turn’ num2str(turn) ’thoughts.txt’];

logfilename = logfilenamedefault;

% display the GUI, wait for a button to be pressed

triggergui

if turn == 1

if speech.on == 1

if rand < 1/5

wavplay(speech.hello,speech.sync)

elseif rand < 1/4

wavplay(speech.allday,speech.sync)

elseif rand < 1/3

wavplay(speech.chalk,speech.sync)

elseif rand < 1/2

wavplay(speech.letsplay,speech.sync)

else

wavplay(speech.pints,speech.sync)

end

end

end

uiwait

close all

% exit the program

if wantexit == 1

break

end

if thoughtsoutput == 0

% do not output

fid = 0;

elseif thoughtsoutput == 1

% output to screen

fid = 1;

elseif thoughtsoutput == 2

% open file to write thoughts to

if not(exist(logfiledir,’dir’))

filesepindices = find(logfiledir == filesep);

mkdir(logfiledir(1:filesepindices(1)),logfiledir(filesepindices(1)+1:end));

end

fid = fopen(fullfile(logfiledir,logfilename),’w’);

end

fprintf(fid,[’Turn #’ num2str(turn) ’.\r\n\r\n’]);

tic

updatewindowhandle = figure(’Name’,’Progress update’,...

’Position’,[updatewindow.left updatewindow.bottom ...

updatewindow.width updatewindow.height],...

120


’Color’,backcolour,...

’Resize’,’off’,...

’MenuBar’,’none’,...

’NumberTitle’,’off’);

messagehandle = uicontrol(’Style’,’text’,...

’Position’,[1 2*updatewindow.textheight+updatewindow.bigtextheight+1 ...

updatewindow.width updatewindow.textheight],...

’BackgroundColor’,backcolour,...

’String’,’Taking and downloading photo...’,...

’FontName’,uifont.name,...

’FontSize’,uifont.bigsize,...

’ForegroundColor’,textcolour);

adhandle = uicontrol(’Style’,’text’,...

’Position’,[1 updatewindow.textheight+1 ...

updatewindow.width updatewindow.bigtextheight],...


’String’,{’ORDER ROBOPOOLPLAYER NOW!’,’Only $14,999!’,...

’Ring 1900-ROBO-POOL’,’While stocks last.’},...




% tell the camera to take a photo

if speech.on == 1

if rand < 1/2

wavplay(speech.look,speech.sync)

end

end

dos(’TakePhoto’);

% read in the image

set(messagehandle,’String’,’Reading image file...’)

rawphoto = imread(’photo.jpg’,’jpeg’);

rawphoto = rawphoto(tabletop+1:tablebottom,tableleft+1:tableright,:);

% compress it by decimating

decimation = 8;

decphoto = (1/256)*double(rawphoto(decimation:decimation:end,decimation:decimation:end,:));

% ball radii

rb = 25.32/mmperpixel; % radius of the other balls

rw = 25.32/mmperpixel; % radius of the white ball

% what number is the black ball ("eight ball")?

bb = 8;

% choose a method of interpolation by uncommenting one line

% interpmethod = ’none’;

% interpmethod = ’spline’;

interpmethod = ’bandlimited’;

balls = zeros(2*bb-1,2);

% find the white ball

set(messagehandle,’String’,’Finding white ball...’)

[whi,dummy] = findball(rawphoto,decphoto,decimation,rw,ballwhite,feltblue,1,1,interpmethod);

fprintf(fid,[’The white ball is at ’ coord2str(whi) ’.\r\n’]);

% convert to steps

whi = fliplr(tforminv([tableleft+whi(2) tabletop+whi(1)],tform));

if tgt == 1

% tell the arm to play the break

set(messagehandle,’String’,’Playing break...’)

while motorsbusy

121


pause(1)

end

Xposnew = round(zeropos(1) - whi(2) + balltosolenoid);

Yposnew = round(zeropos(2) + whi(1));

Aposnew = -100;

solenoidpower = 3;

TasksExecuted = 0;

while (Xpos ~= Xposnew) | (Ypos ~= Yposnew) | (Apos ~= Aposnew)

movearm

pause(timedelay)

end

else

% % for testing only

% % tell the arm to move over the white ball

% set(messagehandle,’String’,’Moving arm into position...’)

%

% Xposnew = round(zeropos(1) - whi(2));

% Yposnew = round(zeropos(2) + whi(1));

% Aposnew = Apos;

% solenoidpower = 0;

%

% TasksExecuted = 0;

% while (Xpos ~= Xposnew) | (Ypos ~= Yposnew) | (Apos ~= Aposnew)

% movearm

% pause(timedelay)

% end

%

% pause

% break % for testing

% find the black ball

set(messagehandle,’String’,’Finding black ball...’)

[balls(bb,:),dummy] = ...

findball(rawphoto,decphoto,decimation,rb,ballblack,feltblue,1,1,interpmethod);

b = bb:bb; % black ball number

fprintf(fid,[’The black ball is at ’ coord2str(balls(b,:)) ’.\r\n’]);

clear dummy

% determine how many red balls and find them

set(messagehandle,’String’,’Finding red balls...’)

[balls(1:bb-1,:),nr] = ...

findball(rawphoto,decphoto,decimation,rb,ballred,feltblue,-1,bb-1,interpmethod);

r = 1:nr; % red ball numbers

if nr == 0

fprintf(fid,’There are no red balls.\r\n’);

elseif nr == 1

fprintf(fid,[’The red ball is at ’ coord2str(balls(r,:)) ’.\r\n’]);

else

fprintf(fid,[’The red balls are at ’ coord2str(balls(r,:)) ’.\r\n’]);

end

% determine how many yellow balls and find them

set(messagehandle,’String’,’Finding yellow balls...’)

[balls(bb+1:2*bb-1,:),ny] = ...

findball(rawphoto,decphoto,decimation,rb,ballyellow,feltblue,-1,bb-1,interpmethod);

y = bb+1:bb+ny; % yellow ball numbers

if ny == 0

fprintf(fid,’There are no yellow balls.\r\n’);

elseif ny == 1

fprintf(fid,[’The yellow ball is at ’ coord2str(balls(y,:)) ’.\r\n’]);

else

122


fprintf(fid,[’The yellow balls are at ’ coord2str(balls(y,:)) ’.\r\n’]);

end

% convert to steps

balls = fliplr(tforminv([tableleft+balls(:,2) tabletop+balls(:,1)],tform));

ry = [r y];

ryb = [r b y];

% table dimensions

Xtotal = cpsteps(end,2);

Ytotal = cpsteps(end,1);

bndry = 48*stepspermm;

X1 = bndry;

X2 = Xtotal + 1 - bndry;

Xmid = (X1 + X2) / 2;

Y1 = bndry;

Y2 = Ytotal + 1 - bndry;

Ymid = (Y1 + Y2) / 2;

% ball radii

rb = 25.32*stepspermm; % radius of the other balls

rw = 25.32*stepspermm; % radius of the white ball

% pocket locations

rp = 1.4*rb; % pocket radius

epd = 0.3*rb; % edge pocket displacement

% these are where the robot aims to sink a ball

pockets = [X1+rb Y1+rb;

X1-epd Ymid;

X1+rb Y2-rb;

X2-rb Y1+rb;

X2+epd Ymid;

X2-rb Y2-rb];

% these are where the holes are physically

pocketcentres = [X1-sqrt(0.5)*rp Y1-sqrt(0.5)*rp;

X1-(rp+epd) Ymid;

X1-sqrt(0.5)*rp Y2+sqrt(0.5)*rp;

X2+sqrt(0.5)*rp Y1-sqrt(0.5)*rp;

X2+(rp+epd) Ymid;

X2+sqrt(0.5)*rp Y2+sqrt(0.5)*rp];

np = size(pockets,1);

fprintf(fid,[’The pockets are at ’ coord2str(pockets) ’.\r\n\r\n’]);

% which colour(s) are we trying to sink?

if speech.on == 1

if rand < 1/3

if tgt == 3

wavplay(speech.red,speech.sync)

elseif tgt == 4

wavplay(speech.yellow,speech.sync)

end

elseif rand < 1/2

wavplay(speech.thinking,speech.sync)

end

end

if tgt == 2

targets = ry;

fprintf(fid,’I’’m trying to sink red or yellow.\r\n’);

elseif tgt == 3

targets = r;

fprintf(fid,’I’’m trying to sink red.\r\n’);

elseif tgt == 4

targets = y;

fprintf(fid,’I’’m trying to sink yellow.\r\n’);

end

123


if length(targets) == 0

targets = b;

fprintf(fid,’Since there are none left, I’’ll try to sink black.\r\n’);

end

fprintf(fid,’\r\n’);

% create structure ’shotlist’ with fields

% pathtype, pathnos, coords, goodness

makeshotlist

% try to find the ’hmshots’ best shots

% put their indices into vector ’bestshots’

bestshots = [];

findagoodshot

if length(bestshots) == 0

fprintf(fid,’Shit, I’’m screwed.\r\n\r\n’);

bestshots = [1];

end

fprintf(fid,’Total thinking time was %.2f seconds.\r\n\r\n’,toc);

% display the GUI, wait for a button to be pressed

close all

chooseshotgui

if speech.on == 1

if shotlist(bestshots(1)).goodness > 0.003

if rand > 1/2

wavplay(speech.playthisone,speech.sync)

else

wavplay(speech.shotlooksgood,speech.sync)

end

else

wavplay(speech.doozy,speech.sync)

end

end

uiwait

close all

fprintf(fid,’Looks like I’’m playing shot #%g.\r\n\r\n’,bestshots(playshot));

updatewindowhandle = figure(’Name’,’Progress update’,...

’Position’,[updatewindow.left updatewindow.bottom ...

updatewindow.width updatewindow.height],...




’NumberTitle’,’off’);

messagehandle = uicontrol(’Style’,’text’,...

’Position’,[1 2*updatewindow.textheight+updatewindow.bigtextheight+1 ...

updatewindow.width updatewindow.textheight],...


’String’,’Playing the shot...’,...




adhandle = uicontrol(’Style’,’text’,...

’Position’,[1 updatewindow.textheight+1 updatewindow.width ...

updatewindow.bigtextheight],...


’String’,{’ORDER ROBOPOOLPLAYER NOW!’,’Only $14,999!’,...

’Ring 1900-ROBO-POOL’,’While stocks last.’},...




% tell the arm to play the shot

124


if speech.on == 1

if rand < 1/6

wavplay(speech.faster,speech.sync)

elseif rand < 1/5

wavplay(speech.fact19,speech.sync)

elseif rand < 1/4


elseif rand < 1/3


elseif rand < 1/2


else


end

end

if turn == 1

Xposnew = -250;

Yposnew = 2700;

Aposnew = -100;

solenoidpower = 0;

TasksExecuted = 0;


movearm

pause(timedelay)

end

end

shot = shotlist(bestshots(playshot));

shootangle = - atan2(shot.coords(2,2)-shot.coords(1,2),shot.coords(2,1)-shot.coords(1,1));

Xposnew = round(zeropos(1) - whi(2) - balltosolenoid*sin(shootangle));

Yposnew = round(zeropos(2) + whi(1) - balltosolenoid*cos(shootangle));

Aposnew = round(stepsperradian*shootangle);

if length(shot.pathtype) > 2

solenoidpower = 3;

elseif length(shot.pathtype) == 2

vec1 = shot.coords(2,:) - shot.coords(1,:);

vec2 = shot.coords(4,:) - shot.coords(3,:);

if norm(vec1) + (1.87 / (cosang(vec1,vec2))^2) * norm(vec2) < 1000*stepspermm

solenoidpower = 1;

elseif norm(vec1) + (1.87 / (cosang(vec1,vec2))^2) * norm(vec2) < 2000*stepspermm

solenoidpower = 2;

else

solenoidpower = 3;

end

else

solenoidpower = 1;

end

TasksExecuted = 0;


movearm

pause(timedelay)

end

end

% tell the arm to move to the end of the table

set(messagehandle,’String’,’Returning arm to home position...’)

if speech.on == 1

if rand < 1/6

wavplay(speech.ad,speech.sync)

elseif rand < 1/5

125


wavplay(speech.icantalk,speech.sync)

elseif rand < 1/4

wavplay(speech.benisgreat,speech.sync)

elseif rand < 1/3

wavplay(speech.drben,speech.sync)

elseif rand < 1/2

wavplay(speech.silvioetal,speech.sync)

else

wavplay(speech.bill,speech.sync)

end

end

Xposnew = 0;

Yposnew = 2700;

Aposnew = -100;

solenoidpower = 0;

TasksExecuted = 0;


movearm

pause(timedelay)

end

close all

fprintf(fid,’Well folks, that’’s the end of turn #%g.\r\n\r\n’,turn);

if thoughtsoutput == 2

fclose(fid);

save([logfiledir ’\wkspace.mat’],...

’X1’,’X2’,’Xtotal’,’Y1’,’Ymid’,’Y2’,’Ytotal’,...

’bb’,’r’,’b’,’y’,’ry’,’ryb’,’targets’,’nr’,’ny’,’np’,...

’balls’,’whi’,’pockets’,...

’shotlist’,’hmshots’,’bestshots’,...

’gamedatetime’,’turn’)

end

turn = turn + 1;

end

A.2 Eye software code

function [pos,nb] = findball(rawphoto,decphoto,decimation,r,ballcolour,backcolour,nb,maxnb,method)

%FINDBALL Find a ball using a coarse-fine search.

% The image table is decimated and the ball located

% approximately. The region around the determined location

% is then examined at the original resolution.

%

% If nb = -1 then the number of balls is determined.

rup = ceil(r);

[posapprox,nb] = findballxcorr(decphoto,r/decimation,ballcolour,backcolour,nb,maxnb,’none’);

pos = zeros(maxnb,2);

for ii = 1:nb

Xcoords = (posapprox(ii,1)-1)*decimation-rup:(posapprox(ii,1)+1)*decimation+rup;

Xcoords = intersect(Xcoords,1:size(rawphoto,1));

Ycoords = (posapprox(ii,2)-1)*decimation-rup:(posapprox(ii,2)+1)*decimation+rup;

Ycoords = intersect(Ycoords,1:size(rawphoto,2));

126


photosection = (1/256)*double(rawphoto(Xcoords,Ycoords,:));

postemp = findballxcorr(photosection,r,ballcolour,backcolour,1,maxnb,method);

pos(ii,1) = interp1(Xcoords,postemp(1));

pos(ii,2) = interp1(Ycoords,postemp(2));

end

function [pos,nb] = findballxcorr(table,r,ballcolour,backcolour,nb,maxnb,method)

%FINDBALLXCORR Find a ball using cross-correlation.

% The table image is cross-correlated with a template of a

% ball of radius r of colour ballcolour against a backcolour

% background, and nb positions of the ball are returned.

%

% If nb = -1 then the number of balls is determined.

if method(1) == ’n’ % using no interpolation

rup = ceil(r);

% create a template of the ball

template = ballimage(2*rup+1,2*rup+1,rup+1,rup+1,r,ballcolour,backcolour);

% cross correlate the image with the template

XCR = normxcorr2(template(:,:,1),table(:,:,1));

XCR = XCR(rup+1:size(XCR,1)-rup,rup+1:size(XCR,2)-rup,:);

XCG = normxcorr2(template(:,:,2),table(:,:,2));

XCG = XCG(rup+1:size(XCG,1)-rup,rup+1:size(XCG,2)-rup,:);

XCB = normxcorr2(template(:,:,3),table(:,:,3));

XCB = XCB(rup+1:size(XCB,1)-rup,rup+1:size(XCB,2)-rup,:);

% average the red, green and blue correlations

XCtemp = (XCR + XCG + XCB)/3;

% get rid of the edges

XC = zeros(size(XCtemp));

edgerid = 0; % round(0.06*size(XC,1));

XC(edgerid+1:size(XC,1)-edgerid,edgerid+1:size(XC,2)-edgerid) = ...

XCtemp(edgerid+1:size(XC,1)-edgerid,edgerid+1:size(XC,2)-edgerid);

% % have a look at the cross correlation results

% clf

% subplot(2,2,1)

% surface(flipdim(XCR,1),flipdim(table,1),...

% ’FaceColor’,’texturemap’,’EdgeColor’,’none’,’CDataMapping’,’direct’)

% view(-35,45)

% subplot(2,2,2)

% surface(flipdim(XCG,1),flipdim(table,1),...


% view(-35,45)

% subplot(2,2,3)

% surface(flipdim(XCB,1),flipdim(table,1),...


% view(-35,45)

% subplot(2,2,4)

% surface(flipdim(XC,1),flipdim(table,1),...


% view(-35,45)

% pause

% clf

% surface(flipdim(XC,1),flipdim(table,1),...


% view(-35,45)

% pause

127


if nb == -1 % don’t know how many balls there are

firstmax = max(max(XC));

pos = [];

if firstmax > 0.65 % otherwise we can assume that there are no balls

while (max(max(XC)) > 0.68*firstmax) & (size(pos,1) < maxnb)

% (otherwise we can assume that there are no more balls)

% locate the ball approximately by finding a maximum

% disp(num2str(max(max(XC))))

[xmax ymax] = find(XC == max(max(XC)));

pos = [pos; xmax(1) ymax(1)];

% delete the surrounding area

XC = drawcircle(XC,pos(end,:),r,0,1);

end


end

nb = size(pos,1);

else % do know how many balls there are

pos = zeros(nb,2);

for ii = 1:nb




pos(ii,:) = [xmax(1) ymax(1)];


% if ii < nb

XC = drawcircle(XC,pos(ii,:),r,0,1);

% end

end


end

elseif method(1) == ’s’ % using spline interpolation

rup = ceil(r);



% how big a region do we want to interpolate? (2a+1 by 2a+1)

a = 1;

% how many times are we interpolating (by 2)?

k = 4;

xm = a*(2^k) + 1;

ym = a*(2^k) + 1;









XC = (XCR + XCG + XCB)/3;

128


pos = zeros(nb,2);

for ii = 1:nb



posest = [xmax(1) ymax(1)];

% let’s look more closely at the region around ’posest1’

XCzoom = XC(posest(1)-a:posest(1)+a,posest(2)-a:posest(2)+a);

% interpolate!

XCzoomint = interp2(XCzoom,k,’spline’);

[xx yy] = find(XCzoomint == max(max(XCzoomint)));

dx = (mean(xx)-xm) * 2^(-k);

dy = (mean(yy)-ym) * 2^(-k);

pos(ii,:) = posest + [dx dy];


if ii < nb


end

end

elseif method(1) == ’b’ % using bandlimited interpolation

rup = ceil(r);



% how big a region do we want to interpolate? (2a by 2a)

a = 4;

% by how much are we interpolating?

k = 32;

xm = a*k + 1;

ym = a*k + 1;









XC = (XCR + XCG + XCB)/3;

pos = zeros(nb,2);

for ii = 1:nb



posest = [xmax(1) ymax(1)];

% let’s look more closely at the region around ’posest1’

XCzoom = XC(posest(1)-a:posest(1)+a-1,posest(2)-a:posest(2)+a-1);

% interpolate!

129


XCzoomint = bandltdinterp2(XCzoom,k);

[xx yy] = find(XCzoomint == max(max(XCzoomint)));

dx = (mean(xx)-xm) / k;

dy = (mean(yy)-ym) / k;

pos(ii,:) = posest + [dx dy];


if ii < nb


end

end

else

error([method ’ is an invalid method.’])

end

function ballimage = ballimage(Lx,Ly,x,y,r,ballcolour,backcolour)

%BALLIMAGE Create an image of a ball.

% ballimage(Lx,Ly,x,y,r,ballcolour,backcolour) is a colour

% image of size Lx by Ly, with a ball centred at (x,y) with

% radius r, of colour ballcolour against a backcolour

% background.

ballimage = zeros(Lx,Ly,3);

% background

ballimage(:,:,1) = backcolour(1)*ones(size(ballimage(:,:,1)));



% ball

for xx = floor(x-r):ceil(x+r)

for yy = floor(y-r):ceil(y+r)

if (xx-x)^2 + (yy-y)^2 <= r^2

ballimage(xx,yy,:) = ballcolour;

end

end

end

function AA = bandltdinterp2(A,k)

%BANDLTDINTERP2 Two-dimensional bandlimited interpolation.

% The matrix A is expanded to a matrix AA by zero-padding in

% the frequency domain. AA is k times bigger than A in both

% directions. Both dimensions of A must be even.

halfX = size(A,1)/2;

halfY = size(A,2)/2;

B = fftshift(fft2(A));

BB = zeros(2*halfX*k,2*halfY*k);

BB((k-1)*halfX+2:(k+1)*halfX,(k-1)*halfY+2:(k+1)*halfY) = B(2:2*halfX,2:2*halfY);

BB((k-1)*halfX+1,(k-1)*halfY+2:(k+1)*halfY) = B(1,2:2*halfY)/2;

BB((k+1)*halfX+1,(k-1)*halfY+2:(k+1)*halfY) = B(1,2:2*halfY)/2;

BB((k-1)*halfX+2:(k+1)*halfX,(k-1)*halfY+1) = B(2:2*halfX,1)/2;

BB((k-1)*halfX+2:(k+1)*halfX,(k+1)*halfY+1) = B(2:2*halfX,1)/2;

BB((k-1)*halfX+1,(k-1)*halfY+1) = B(1,1)/4;

BB((k-1)*halfX+1,(k+1)*halfY+1) = B(1,1)/4;

130


BB((k+1)*halfX+1,(k-1)*halfY+1) = B(1,1)/4;

BB((k+1)*halfX+1,(k+1)*halfY+1) = B(1,1)/4;

AA = ifft2(fftshift(BB))*k^2;

A.3 Brain software code

%MAKESHOTLIST Construct a list of shots and rank them.

% goodness penalty for cushion rebounds

penalty = 0.5;

% which cushions can a ball bounce off to get into pocket i?

pock(1).cush = [2 4];

pock(2).cush = [2 3 4];

pock(3).cush = [2 3];

pock(4).cush = [1 4];

pock(5).cush = [1 3 4];

pock(6).cush = [1 3];

% reinitialise shotlist

shotlist = [];

% create an emergency shot in case nothing else is found

shotlist(1).pathtype = ’’;

shotlist(1).pathnos = [];

shotlist(1).coords(1,:) = whi;

shotlist(1).coords(2,:) = [Xmid Ymid];

shotlist(1).goodness = -1;

if shotcomplexity(1)

% examine shots of the type ’BjPi’

set(messagehandle,’String’,’Considering ball-pocket shots...’)

fprintf(fid,’Considering ball-pocket shots...\r\n’);

for ii = 1:np

for jj = targets

shotlist(end+1).pathtype = ’BP’;

shotlist(end).pathnos = [jj ii];

shotlist(end).coords(4,:) = pockets(ii,:);

shotlist(end).coords(3,:) = balls(jj,:);

shotlist(end).coords(2,:) = balls(jj,:) - (rw+rb) * ...

(pockets(ii,:)-balls(jj,:))/norm(pockets(ii,:)-balls(jj,:));

shotlist(end).coords(1,:) = whi;

vec1 = shotlist(end).coords(2,:) - shotlist(end).coords(1,:);

vec1d = shotlist(end).coords(3,:) - shotlist(end).coords(1,:);


if cosang(vec1,vec2) > 0

shotlist(end).goodness = (2*rb)^2 * cosang(vec1d,vec2) / ...

(norm(vec1) * norm(vec2) * cosang(vec1,vec1d));

else

shotlist(end).goodness = 0;

end

end

end

end


% examine shots of the type ’CaBjPi’

131


set(messagehandle,’String’,’Considering cushion-ball-pocket shots...’)

fprintf(fid,’Considering cushion-ball-pocket shots...\r\n’);

for ii = 1:np

for jj = targets

for aa = 1:4

shotlist(end+1).pathtype = ’CBP’;

shotlist(end).pathnos = [aa jj ii];




(pockets(ii,:)-balls(jj,:))/norm(pockets(ii,:)-balls(jj,:));

if aa == 1 % top cushion

% reflect whi through x = X1+rw to get [2*(X1+rw)-whi(1),whi(2)]

shotlist(end).coords(3,:) = ...

[X1+rw interp1([2*(X1+rw)-whi(1) shotlist(end).coords(4,1)],...

[whi(2) shotlist(end).coords(4,2)],X1+rw)];

elseif aa == 2 % bottom cushion

% reflect whi through x = X2-rw to get [2*(X2-rw)-whi(1),whi(2)]


[X2-rw interp1([2*(X2-rw)-whi(1) shotlist(end).coords(4,1)],...

[whi(2) shotlist(end).coords(4,2)],X2-rw)];

elseif aa == 3 % left cushion

% reflect whi through y = Y1+rw to get [whi(1),2*(Y1+rw)-whi(2)]


[interp1([2*(Y1+rw)-whi(2) shotlist(end).coords(4,2)],...

[whi(1) shotlist(end).coords(4,1)],Y1+rw) Y1+rw];

elseif aa == 4 % right cushion

% reflect whi through y = Y2-rw to get [whi(1),2*(Y2-rw)-whi(2)]


[interp1([2*(Y2-rw)-whi(2) shotlist(end).coords(4,2)],...

[whi(1) shotlist(end).coords(4,1)],Y2-rw) Y2-rw];

end

shotlist(end).coords(2,:) = shotlist(end).coords(3,:);






shotlist(end).goodness = penalty * (2*rb)^2 * cosang(vec2,vec3) / ...

((norm(vec1)+norm(vec2)) * norm(vec3));

else


end

end

end

end

% examine shots of the type ’BjCaPi’

set(messagehandle,’String’,’Considering ball-cushion-pocket shots...’)

fprintf(fid,’Considering ball-cushion-pocket shots...\r\n’);

for ii = 1:np

for jj = targets

for aa = pock(ii).cush

shotlist(end+1).pathtype = ’BCP’;

shotlist(end).pathnos = [jj aa ii];



% reflect balls(jj,:) through x = X1+rb to get [2*(X1+rb)-balls(jj,1),balls(jj,2)]


[X1+rb interp1([2*(X1+rb)-balls(jj,1) pockets(ii,1)],...

132


[balls(jj,2) pockets(ii,2)],X1+rb)];


% reflect balls(jj,:) through x = X2-rb to get [2*(X2-rb)-balls(jj,1),balls(jj,2)]


[X2-rb interp1([2*(X2-rb)-balls(jj,1) pockets(ii,1)],...

[balls(jj,2) pockets(ii,2)],X2-rb)];


% reflect balls(jj,:) through y = Y1+rb to get [balls(jj,1),2*(Y1+rb)-balls(jj,2)]


[interp1([2*(Y1+rb)-balls(jj,2) pockets(ii,2)],...

[balls(jj,1) pockets(ii,1)],Y1+rb) Y1+rb];


% reflect balls(jj,:) through y = Y2-rb to get [balls(jj,1),2*(Y2-rb)-balls(jj,2)]


[interp1([2*(Y2-rb)-balls(jj,2) pockets(ii,2)],...

[balls(jj,1) pockets(ii,1)],Y2-rb) Y2-rb];

end




(shotlist(end).coords(4,:)-balls(jj,:))/norm(shotlist(end).coords(4,:)-balls(jj,:));







shotlist(end).goodness = penalty *(2*rb)^2 * cosang(vec1d,vec2) / ...

(norm(vec1) * (norm(vec2)+norm(vec3)) * cosang(vec1,vec1d));

else


end

end

end

end

% examine shots of the type ’BjBkPi’

set(messagehandle,’String’,’Considering ball-ball-pocket shots...’)

fprintf(fid,’Considering ball-ball-pocket shots...\r\n’);

for ii = 1:np

for kk = targets

for jj = targets

if jj ~= kk

shotlist(end+1).pathtype = ’BBP’;

shotlist(end).pathnos = [jj kk ii];


shotlist(end).coords(5,:) = balls(kk,:);

shotlist(end).coords(4,:) = balls(kk,:) - (2*rb) * ...

(pockets(ii,:)-balls(kk,:))/norm(pockets(ii,:)-balls(kk,:));










if (cosang(vec1,vec2) > 0 & cosang(vec2,vec3) > 0)

shotlist(end).goodness = (2*rb)^3 * cosang(vec1d,vec2) * cosang(vec2d,vec3) / ...

(norm(vec1) * norm(vec2) * norm(vec3) * cosang(vec1,vec1d) * cosang(vec2,vec2d));

133


else


end

end

end

end

end

end


% examine shots of the type ’CaBjBkPi’

set(messagehandle,’String’,’Considering cushion-ball-ball-pocket shots...’)

fprintf(fid,’Considering cushion-ball-ball-pocket shots...\r\n’);

for ii = 1:np

for kk = targets

for jj = targets

if jj ~= kk

for aa = 1:4

shotlist(end+1).pathtype = ’CBBP’;

shotlist(end).pathnos = [aa jj kk ii];




(pockets(ii,:)-balls(kk,:))/norm(pockets(ii,:)-balls(kk,:));





% reflect whi through x = X1+rw to get [2*(X1+rw)-whi(1),whi(2)]


[X1+rw interp1([2*(X1+rw)-whi(1) shotlist(end).coords(4,1)],...

[whi(2) shotlist(end).coords(4,2)],X1+rw)];


% reflect whi through x = X2-rw to get [2*(X2-rw)-whi(1),whi(2)]


[X2-rw interp1([2*(X2-rw)-whi(1) shotlist(end).coords(4,1)],...

[whi(2) shotlist(end).coords(4,2)],X2-rw)];


% reflect whi through y = Y1+rw to get [whi(1),2*(Y1+rw)-whi(2)]


[interp1([2*(Y1+rw)-whi(2) shotlist(end).coords(4,2)],...

[whi(1) shotlist(end).coords(4,1)],Y1+rw) Y1+rw];


% reflect whi through y = Y2-rw to get [whi(1),2*(Y2-rw)-whi(2)]


[interp1([2*(Y2-rw)-whi(2) shotlist(end).coords(4,2)],...

[whi(1) shotlist(end).coords(4,1)],Y2-rw) Y2-rw];

end









shotlist(end).goodness = penalty * (2*rb)^3 * cosang(vec2,vec3) * cosang(vec3d,vec4) / ...

((norm(vec1)+norm(vec2)) * norm(vec3) * norm(vec4) * cosang(vec3,vec3d));

else


end

134


end

end

end

end

end

% examine shots of the type ’BjCaBkPi’

set(messagehandle,’String’,’Considering ball-cushion-ball-pocket shots...’)

fprintf(fid,’Considering ball-cushion-ball-pocket shots...\r\n’);

for ii = 1:np

for kk = targets

for jj = targets

if jj ~= kk

for aa = 1:4

shotlist(end+1).pathtype = ’BCBP’;

shotlist(end).pathnos = [jj aa kk ii];



shotlist(end).coords(6,:) = balls(kk,:) - ...

(2*rb)*(pockets(ii,:)-balls(kk,:))/norm(pockets(ii,:)-balls(kk,:));


% reflect balls(jj,:) through x = X1+rb to get [2*(X1+rb)-balls(jj,1),balls(jj,2)]


[X1+rb interp1([2*(X1+rb)-balls(jj,1) shotlist(end).coords(6,1)],...

[balls(jj,2) shotlist(end).coords(6,2)],X1+rb)];


% reflect balls(jj,:) through x = X2-rb to get [2*(X2-rb)-balls(jj,1),balls(jj,2)]


[X2-rb interp1([2*(X2-rb)-balls(jj,1) shotlist(end).coords(6,1)],...

[balls(jj,2) shotlist(end).coords(6,2)],X2-rb)];


% reflect balls(jj,:) through y = Y1+rb to get [balls(jj,1),2*(Y1+rb)-balls(jj,2)]


[interp1([2*(Y1+rb)-balls(jj,2) shotlist(end).coords(6,2)],...

[balls(jj,1) shotlist(end).coords(6,1)],Y1+rb) Y1+rb];


% reflect balls(jj,:) through y = Y2-rb to get [balls(jj,1),2*(Y2-rb)-balls(jj,2)]


[interp1([2*(Y2-rb)-balls(jj,2) shotlist(end).coords(6,2)],...

[balls(jj,1) shotlist(end).coords(6,1)],Y2-rb) Y2-rb];

end












shotlist(end).goodness = penalty * (2*rb)^3 * cosang(vec1d,vec2) * cosang(vec3,vec4) / ...

(norm(vec1) * (norm(vec2)+norm(vec3)) * norm(vec4) * cosang(vec1,vec1d));

else


end

end

end

end

end

end

135


% examine shots of the type ’BjBkCaPi’

set(messagehandle,’String’,’Considering ball-ball-cushion-pocket shots...’)

fprintf(fid,’Considering ball-ball-cushion-pocket shots...\r\n’);

for ii = 1:np

for kk = targets

for jj = targets

if jj ~= kk

for aa = pock(ii).cush

shotlist(end+1).pathtype = ’BBCP’;

shotlist(end).pathnos = [jj kk aa ii];



% reflect balls(kk,:) through x = X1+rb to get [2*(X1+rb)-balls(kk,1),balls(kk,2)]


[X1+rb interp1([2*(X1+rb)-balls(kk,1) pockets(ii,1)],...

[balls(kk,2) pockets(ii,2)],X1+rb)];


% reflect balls(kk,:) through x = X2-rb to get [2*(X2-rb)-balls(kk,1),balls(kk,2)]


[X2-rb interp1([2*(X2-rb)-balls(kk,1) pockets(ii,1)],...

[balls(kk,2) pockets(ii,2)],X2-rb)];


% reflect balls(kk,:) through y = Y1+rb to get [balls(kk,1),2*(Y1+rb)-balls(kk,2)]


[interp1([2*(Y1+rb)-balls(kk,2) pockets(ii,2)],...

[balls(kk,1) pockets(ii,1)],Y1+rb) Y1+rb];


% reflect balls(kk,:) through y = Y2-rb to get [balls(kk,1),2*(Y2-rb)-balls(kk,2)]


[interp1([2*(Y2-rb)-balls(kk,2) pockets(ii,2)],...

[balls(kk,1) pockets(ii,1)],Y2-rb) Y2-rb];

end




(shotlist(end).coords(6,:)-balls(kk,:))/norm(shotlist(end).coords(6,:)-balls(kk,:));












shotlist(end).goodness = penalty * (2*rb)^3 * cosang(vec1d,vec2) * cosang(vec2d,vec3) / ...

(norm(vec1) * norm(vec2) * (norm(vec3)+norm(vec4)) * ...

cosang(vec1,vec1d) * cosang(vec2,vec2d));

else


end

end

end

end

end

end

% examine shots of the type ’BjBkBlPi’

set(messagehandle,’String’,’Considering ball-ball-ball-pocket shots...’)

136


fprintf(fid,’Considering ball-ball-ball-pocket shots...\r\n’);

for ii = 1:np

for ll = targets

for kk = ryb

if kk ~= ll

for jj = targets

if (jj ~= ll & jj ~= kk)

shotlist(end+1).pathtype = ’BBBP’;

shotlist(end).pathnos = [jj kk ll ii];


shotlist(end).coords(7,:) = balls(ll,:);

shotlist(end).coords(6,:) = balls(ll,:) - (2*rb) * ...

(pockets(ii,:)-balls(ll,:))/norm(pockets(ii,:)-balls(ll,:));



(shotlist(end).coords(6,:)-balls(kk,:))/norm(shotlist(end).coords(6,:)-balls(kk,:));












if (cosang(vec1,vec2) > 0 & cosang(vec2,vec3) > 0 & cosang(vec3,vec4) > 0)

shotlist(end).goodness = (2*rb)^4 * ...

cosang(vec1d,vec2) * cosang(vec2d,vec3) * cosang(vec3d,vec4) / ...

(norm(vec1) * norm(vec2) * norm(vec3) * norm(vec4) * ...

cosang(vec1,vec1d) * cosang(vec2,vec2d) * cosang(vec3,vec3d));

else


end

end

end

end

end

end

end

end


clear pock

clear vec1 vec1d vec2 vec2d vec3 vec3d vec4

%FINDAGOODSHOT Find a good shot.

% Looks through the shot list in order of descending

% goodness and finds a possible shot.

set(messagehandle,’String’,’Checking and ranking shots...’)

% create an index of the shots in order of goodness

[dummy,best] = sort(-[shotlist.goodness]);

clear dummy

% tolerance in pixels

eps = rb/10;

137


for ii = best(1:end-1)

shot = shotlist(ii);

if shot.goodness == 0

break

end

if ii == best(1)

fprintf(fid,’The best shot is shot #%g: white ball’,ii);

else

fprintf(fid,’The next best shot is shot #%g: white ball’,ii);

end

for jj = 1:length(shot.pathtype)

if shot.pathtype(jj) == ’B’

if shot.pathnos(jj) < bb

fprintf(fid,’ to red ball %g’,shot.pathnos(jj));

elseif shot.pathnos(jj) > bb

fprintf(fid,’ to yellow ball %g’,shot.pathnos(jj));

else

fprintf(fid,’ to black ball’);

end

elseif shot.pathtype(jj) == ’C’

if shot.pathnos(jj) == 1

fprintf(fid,’ off the top cushion’);

elseif shot.pathnos(jj) == 2

fprintf(fid,’ off the bottom cushion’);


fprintf(fid,’ off the left cushion’);


fprintf(fid,’ off the right cushion’);

end

elseif shot.pathtype(jj) == ’P’

fprintf(fid,’ into pocket %g.\r\n’,shot.pathnos(jj));

end

end

thisshotworks = 1; % prove me wrong

rr = rw; % radius of the moving ball

ballsmoved = []; % coloured balls which have moved from their original position

for jj = 1:length(shot.pathtype)

% is a ball in the way?

for ib = ryb

% has this ball already moved?

if ismember(ib,ballsmoved)

% then don’t need to check if this ball is in the way

continue

end

% is it within (rr+rb+eps) of the path line?

cond1a = (abs((balls(ib,1)-shot.coords(2*jj,1)) * ...

(shot.coords(2*jj-1,2)-shot.coords(2*jj,2)) - ...

(shot.coords(2*jj-1,1)-shot.coords(2*jj,1)) * ...

(balls(ib,2)-shot.coords(2*jj,2))) < ...

(rr+rb+eps) * norm(shot.coords(2*jj,:)-shot.coords(2*jj-1,:)));

% is it after the beginning of the path?

cond1b = (dot(balls(ib,:)-shot.coords(2*jj-1,:),...

shot.coords(2*jj,:)-shot.coords(2*jj-1,:)) > 0);

% is it before the end of the path?

cond1c = (dot(balls(ib,:)-shot.coords(2*jj,:),...

shot.coords(2*jj-1,:)-shot.coords(2*jj,:)) > 0);

% is the ball in the shot path?

cond2a = not(ismember(’B’,shot.pathtype(find(shot.pathnos == ib))));

% is it within (rr+rb+eps) of the end of the path?

138


cond2b = (norm(shot.coords(2*jj,:)-balls(ib,:)) < rr+rb+eps);

if (cond1a & cond1b & cond1c) | (cond2a & cond2b)

fprintf(fid,’Ball %g is in the way.\r\n’,ib);

thisshotworks = 0;

end

end

% is a cushion in the way?


if shot.coords(2*jj,1) < X1+rr

fprintf(fid,’The top cushion is in the way.\r\n’,ib);

thisshotworks = 0;

elseif shot.coords(2*jj,1) > X2-rr

fprintf(fid,’The bottom cushion is in the way.\r\n’,ib);

thisshotworks = 0;

elseif shot.coords(2*jj,2) < Y1+rr

fprintf(fid,’The left cushion is in the way.\r\n’,ib);

thisshotworks = 0;

elseif shot.coords(2*jj,2) > Y2-rr

fprintf(fid,’The right cushion is in the way.\r\n’,ib);

thisshotworks = 0;

end

end

% is the ball is bouncing off the gap in the top or bottom cushion (where pocket 2/5 is)?

if shot.pathtype(jj) == ’C’ & (shot.pathnos(jj) == 1 | shot.pathnos(jj) == 2)

if shot.coords(2*jj,2) > Ymid-rp-sqrt(0.5)*(rp+epd)-eps & shot.coords(2*jj,2) < ...

Ymid+rp+sqrt(0.5)*(rp+epd)+eps

fprintf(fid,’Can’’t bounce off a pocket, ya doof.\r\n’);

thisshotworks = 0;

end

end

% update the list of balls which have moved


rr = rb;

ballsmoved = [ballsmoved shot.pathnos(jj)];

end

end

if playforwards == 1

% is the shot going forwards?

if shot.coords(1,2) <= shot.coords(2,2)

fprintf(fid,’You’’re not allowed to play backwards.\r\n’);

thisshotworks = 0;

end

end

if shot.pathnos(end) == 2 | shot.pathnos(end) == 5

% is the target ball at too wide an angle to make it into the middle pockets?

% (cannot approach from an angle less than pi/6)

if abs(shot.coords(end-1,2)-shot.coords(end,2))*tan(pi/6) > ...

abs(shot.coords(end-1,1)-shot.coords(end,1))

fprintf(fid,’That ball’’s not going to make it into the pocket at that angle.\r\n’);

thisshotworks = 0;

end

end

if thisshotworks == 1

fprintf(fid,’What a fantasterrific shot!\r\n’);

bestshots = [bestshots ii];

if length(bestshots) >= hmshots


break

end

139


end


end

clear shot thisshotworks

clear cond1a cond1b cond1c cond2a cond2b

function table = ...

tablepic(Xtotal,Ytotal,X1,X2,Y1,Ymid,Y2,np,rp,epd,whi,rw,bb,ryb,balls,rb,pocketcentres,dd)

%TABLEPIC Create and display a table image.

% Dimensions of the table are determined from

% Xtotal, Ytotal, X1, X2, Y1, Ymid, Y2.

% Pocket positions are determined from

% np, rp, epd.

% Ball positions are determined from

% whi, rw, bb, ryb, balls, rb.

Xtotal = round(Xtotal/dd);

Ytotal = round(Ytotal/dd);

X1 = round(X1/dd);

X2 = round(X2/dd);

Y1 = round(Y1/dd);

Ymid = round(Ymid/dd);

Y2 = round(Y2/dd);

rp = rp/dd;

epd = epd/dd;

whi = whi/dd;

rw = rw/dd;

balls = balls/dd;

rb = rb/dd;

pocketcentres = pocketcentres/dd;

table = zeros(Xtotal,Ytotal,3);

% make a polygon mask for the edge shape

edgepolyX = [X1+sqrt(2)*rp X1 X1-sqrt(2)*rp X1 X1 X1-(rp+epd)];

edgepolyX = [edgepolyX fliplr(edgepolyX)];

edgepolyX = [edgepolyX Xtotal+1-edgepolyX];

edgepolyY = [Y1 Y1-sqrt(2)*rp Y1 Y1+sqrt(2)*rp Ymid-rp-(rp+epd) Ymid-rp];

edgepolyY = [edgepolyY Ytotal+1-fliplr(edgepolyY)];

edgepolyY = [edgepolyY fliplr(edgepolyY)];

edgemask = double(roipoly(table(:,:,1),edgepolyY,edgepolyX));

% table outline (brown and blue)

table(:,:,1) = 0; % 0.5 - 0.5*edgemask;

table(:,:,2) = 0; % 0.2 - 0.2*edgemask;

table(:,:,3) = 0.3 + 0.2*edgemask;

% black pockets

for ip = 1:np

table = drawcircle(table,pocketcentres(ip,:),rp,[0 0 0],1);

end

% white ball

table = drawcircle(table,whi,rw,[1 1 1],1);

% red/yellow/black balls

for ib = ryb

if ib < bb

table = drawcircle(table,balls(ib,:),rb,[1 0 0],1);

elseif ib > bb


else


140


end

end

function [] = shotpic(tableonly,np,whi,rw,bb,ryb,balls,rb,pocketcentres,dd,shot)

%SHOTPIC Create and display an image of a shot, with labels.

whi = whi/dd;

rw = rw/dd;

balls = balls/dd;

rb = rb/dd;

pocketcentres = pocketcentres/dd;

shot.coords = shot.coords/dd;

table = tableonly;

howopaque = 0.3;

ballno = 0;

for ii = 1:length(shot.pathtype)-1

if ballno == 0

table = drawcircle(table,shot.coords(2*ii,:),rw,[1 1 1],howopaque);

elseif ballno < bb

table = drawcircle(table,shot.coords(2*ii,:),rb,[1 0 0],howopaque);

elseif ballno > bb


else


end

if shot.pathtype(ii) == ’B’

ballno = shot.pathnos(ii);

end

end

imshow(table)

% path lines

ballno = 0;

thickness = 2;

for ii = 1:length(shot.pathtype)

if ballno == 0

line(’YData’,shot.coords(2*ii-1:2*ii,1),...

’XData’,shot.coords(2*ii-1:2*ii,2),...

’Color’,[0.99 0.99 0.99],...

’LineWidth’,thickness)

elseif ballno < bb



’Color’,[1 0 0],...


elseif ballno > bb



’Color’,[1 1 0],...


else



’Color’,[0 0 0],...


end

if shot.pathtype(ii) == ’B’

ballno = shot.pathnos(ii);

end

end

141


pocketcentres(2,1) = pocketcentres(1,1);

pocketcentres(5,1) = pocketcentres(4,1);

% pocket labels

for ip = 1:np

text(round(pocketcentres(ip,2)),round(pocketcentres(ip,1)),num2str(ip),...

’HorizontalAlignment’,’center’,...

’FontSize’,9,...

’Color’,[0.99 0.99 0.99])

end

% red/yellow/black ball labels

for ib = ryb

if ib < bb

text(round(balls(ib,2)),round(balls(ib,1)),num2str(ib),...



’Color’,’k’)

elseif ib > bb

text(round(balls(ib,2)),round(balls(ib,1)),num2str(ib),...



’Color’,’k’)

else

text(round(balls(ib,2)),round(balls(ib,1)),[num2str(ib)],...



’FontWeight’,’Bold’,...

’Color’,[0.99 0.99 0.99])

end

end

function cosalpha = cosang(vec1,vec2)

%COSANG Return the cosine of the angle between two vectors.

cosalpha = dot(vec1,vec2)/(norm(vec1)*norm(vec2));

A.4 Interface software code

%MOVEARM Send commands to a driver box to control stepper motors.

command = zeros(1,8);

if TasksExecuted == 0

if Xpos < Xposnew

command(1) = 1;

elseif Xpos > Xposnew

command(1) = 0;

end

if Ypos < Yposnew

command(3) = 1;

elseif Ypos > Yposnew

command(3) = 0;

end

if Apos < Aposnew

command(5) = 1;

elseif Apos > Aposnew

command(5) = 0;

142


end

else

if Xpos < Xposnew

command(1) = 1;

Xpos = Xpos + doXstep;

doXstep = not(doXstep);

command(2) = doXstep;

elseif Xpos > Xposnew

command(1) = 0;

Xpos = Xpos - doXstep;

doXstep = not(doXstep);

command(2) = doXstep;

end

if Ypos < Yposnew

command(3) = 1;

Ypos = Ypos + doYstep;

doYstep = not(doYstep);

command(4) = doYstep;

elseif Ypos > Yposnew

command(3) = 0;

Ypos = Ypos - doYstep;

doYstep = not(doYstep);

command(4) = doYstep;

end

if Apos < Aposnew

command(5) = 1;

Apos = Apos + doAstep;

doAstep = not(doAstep);

command(6) = doAstep;

elseif Apos > Aposnew

command(5) = 0;

Apos = Apos - doAstep;

doAstep = not(doAstep);

command(6) = doAstep;

end

end

putvalue(parport,command)

TasksExecuted = TasksExecuted + 1;

if [Xpos Ypos Apos] == [Xposnew Yposnew Aposnew]

pause(0.5);

if solenoidpower == 1

putvalue(parport,[0 0 0 0 0 0 0 1])

elseif solenoidpower == 2


elseif solenoidpower == 3


end

pause(0.2);


motorsbusy = 0;

end

143


A.5 Miscellaneous code

%TRIGGERGUI Create a GUI to trigger the robot’s shot.

triggerwindow = figure(’Name’,[’Robot turn #’ num2str(turn)],...

’Position’,[1 1 5 5],...




’NumberTitle’,’off’,...

’Visible’,’off’);

% -------------------------------------------------------------------------

% exit button

fpos = frame.spacing + 1;

bpos = fpos + button.spacing;

frame5 = uicontrol(’Style’,’frame’,...

’Position’,[frame.left fpos frame.width pushheight+2*button.spacing],...

’BackgroundColor’,frontcolour);

fpos = fpos + pushheight + 2*button.spacing + frame.spacing;

uicontrol(’Style’,’pushbutton’,...

’Position’,[button.left bpos button.width pushheight],...

’BackgroundColor’,buttoncolour,...

’String’,’Exit program’,...



’ForegroundColor’,textcolour,...

’Callback’,’wantexit = 1; uiresume’);

% -------------------------------------------------------------------------

% choose targets


if turn == 1


’Position’,[frame.left fpos frame.width 3*pushheight+radioheight+5*button.spacing],...


fpos = fpos + 3*pushheight + radioheight + 5*button.spacing + frame.spacing;

elseif tgt == 1 | tgt == 2


’Position’,[frame.left fpos frame.width 2*pushheight+radioheight+4*button.spacing],...


fpos = fpos + 2*pushheight + radioheight + 4*button.spacing + frame.spacing;

else


’Position’,[frame.left fpos frame.width pushheight+radioheight+3*button.spacing],...


fpos = fpos + pushheight + radioheight + 3*button.spacing + frame.spacing;

end

pf = uicontrol(’Style’,’checkbox’,...

’Position’,[button.left bpos button.width radioheight],...

’BackgroundColor’,frontcolour,...

’String’,’Cue ball replaced on table; play forward only.’,...


’FontSize’,uifont.smallsize,...


bpos = bpos + radioheight + button.spacing;

if tgt == 1 | tgt == 2


144


’Position’,[button.left bpos (button.width-button.spacing)/2 pushheight],...


’String’,’Go for red’,...



’ForegroundColor’,[0.9 0.1 0.1],...

’Callback’,’tgt=3; playforwards = get(pf,’’Value’’); uiresume’);


’Position’,[button.left+(button.width+button.spacing)/2 bpos ...

(button.width-button.spacing)/2 pushheight],...


’String’,’Go for yellow’,...



’ForegroundColor’,[1 1 0],...


bpos = bpos + pushheight + button.spacing;




’String’,’Table is still open’,...





bpos = bpos + pushheight + button.spacing;

if turn == 1




’String’,’Play the break’,...





end

end

if tgt == 3




’String’,’Go for red’,...





end

if tgt == 4




’String’,’Go for yellow’,...





end

% -------------------------------------------------------------------------

% shot complexity

145




’Position’,[frame.left fpos frame.width 4*radioheight+2*button.spacing],...


fpos = fpos + 4*radioheight + 2*button.spacing + frame.spacing;

complexitytrick = uicontrol(’Style’,’radiobutton’,...



’String’,’Trick shot mode: only consider very complex shots’,...




’Value’,0,...

’Callback’,[’set([complexitylow complexitymedium complexityhigh],’’Value’’,0);’...

’shotcomplexity = [0 0 1];’]);

bpos = bpos + radioheight;

complexityhigh = uicontrol(’Style’,’radiobutton’,...



’String’,’Also consider very complex shots’,...




’Value’,0,...

’Callback’,[’set([complexitylow complexitymedium complexitytrick],’’Value’’,0);’...



complexitymedium = uicontrol(’Style’,’radiobutton’,...



’String’,’Also consider medium complexity shots’,...




’Value’,0,...

’Callback’,[’set([complexitylow complexityhigh complexitytrick],’’Value’’,0);’...



complexitylow = uicontrol(’Style’,’radiobutton’,...



’String’,’Consider only simple (direct) shots’,...




’Value’,0,...

’Callback’,[’set([complexitymedium complexityhigh complexitytrick],’’Value’’,0);’...


if shotcomplexity == [1 0 0]

set(complexitylow,’Value’,1)

elseif shotcomplexity == [1 1 0]

set(complexitymedium,’Value’,1)


set(complexityhigh,’Value’,1)


set(complexitytrick,’Value’,1)

end

% -------------------------------------------------------------------------

% how many shots to find

146


% hmshotslist = 0 means that the robot automatically chooses a shot

% otherwsie, the robot presents a user with hmshotslist choices



’Position’,[frame.left fpos frame.width 2*radioheight+popupheight+2*button.spacing],...


fpos = fpos + 2*radioheight + popupheight + 2*button.spacing + frame.spacing;

uicontrol(’Style’,’text’,...

’Position’,[button.left+1 bpos button.width-popupwidth-button.spacing popupheight-4],...


’String’,’Number of alternatives:’,...




’HorizontalAlignment’,’right’);

% The -4 is so that the text aligns with the number in the popupmenu

hmshotslist = uicontrol(’Style’,’popupmenu’,...

’Position’,[window.width-frame.spacing-button.spacing-popupwidth+1 bpos ...

popupwidth popupheight],...

’BackgroundColor’,popupcolour,...

’Enable’,’off’,...

’String’,hmshotsstring,...



’Value’,hmshotslast,...

’Callback’,[’hmshots = hmshotsvalues(get(hmshotslist,’’Value’’));’...

’hmshotslast = get(hmshotslist,’’Value’’);’]);

bpos = bpos + popupheight;

manualchoose = uicontrol(’Style’,’radiobutton’,...



’String’,’Manually choose a shot from given alternatives’,...




’Value’,0,...

’Callback’,[’set(autochoose,’’Value’’,0);’...

’set(hmshotslist,’’Enable’’,’’on’’);’...

’hmshots = hmshotsvalues(get(hmshotslist,’’Value’’));’]);


autochoose = uicontrol(’Style’,’radiobutton’,...



’String’,’Let the robot decide upon a shot automatically’,...




’Value’,0,...

’Callback’,[’set(manualchoose,’’Value’’,0);’...

’set(hmshotslist,’’Enable’’,’’off’’);’...

’hmshots = 0;’]);

if hmshots == 0

set(autochoose,’Value’,1);

else

set(manualchoose,’Value’,1);

set(hmshotslist,’Enable’,’on’);

end

147


% % -------------------------------------------------------------------------

% % where to output thoughts

%

% % thoughtsoutput = 0 means thoughts are not output

% % thoughtsoutput = 1 means thoughts are output to screen

% % thoughtsoutput = 2 means thoughts are output to fullfile(logfiledir,logfilename)

%

% bpos = fpos + button.spacing;

%

% frame1 = uicontrol(’Style’,’frame’,...

% ’Position’,[frame.left fpos frame.width 3*radioheight+editheight+2*button.spacing],...

% ’BackgroundColor’,frontcolour);

% window.height = fpos + 3*radioheight + editheight + 2*button.spacing + frame.spacing;

%

% uicontrol(’Style’,’text’,...

% ’Position’,[button.left+1 bpos ...

% button.width-editwidth-putfilewidth-2*button.spacing editheight-3],...

% ’BackgroundColor’,frontcolour,...

% ’String’,’Log file:’,...

% ’FontName’,uifont.name,...

% ’FontSize’,uifont.smallsize,...

% ’ForegroundColor’,textcolour,...

% ’HorizontalAlignment’,’right’);

% % The -3 is so that the text aligns with the number in the edit box

%

% logfiledisp = uicontrol(’Style’,’edit’,...

% ’Position’,[window.width-frame.spacing-2*button.spacing-putfilewidth-editwidth+1 bpos ...

% editwidth editheight],...

% ’BackgroundColor’,’white’,...

% ’Enable’,’off’,...

% ’String’,fullfile(logfiledir,logfilenamedefault),...


% ’HorizontalAlignment’,’left’,...

% ’Callback’,[’[logfiledir logfilename ext] = fileparts(get(logfiledisp,’’String’’));’...

% ’logfilename = [logfilename ’’.’’ ext];’]);

%

% logfilegetdir = uicontrol(’Style’,’pushbutton’,...

% ’Position’,[window.width-frame.spacing-button.spacing-putfilewidth+1 bpos ...

% putfilewidth editheight],...

% ’Enable’,’off’,...

% ’String’,’...’,...

% ’Callback’,[’pwdnow = pwd;’...

% ’if exist(logfiledir,’’dir’’),’...

% ’cd(logfiledir);’...

% ’[tempname,tempdir] = uiputfile(logfilename,’’Logfile name’’);’...

% ’cd(pwdnow);’...

% ’else,’...

% ’[tempname,tempdir] = uiputfile(logfilename,’’Logfile name’’);’...

% ’end,’...

% ’if not(tempname == 0),’...

% ’logfiledir = tempdir;’...

% ’logfilename = tempname;’...

% ’set(logfiledisp,’’String’’,fullfile(logfiledir,logfilename));’...

% ’end’]);

% bpos = bpos + popupheight;

%

% texttofile = uicontrol(’Style’,’radiobutton’,...

% ’Position’,[button.left bpos button.width radioheight],...


% ’String’,’Save thoughts in log file’,...




% ’Value’,0,...

% ’Callback’,[’set([textnowhere texttoscreen],’’Value’’,0);’...

% ’set([logfiledisp logfilegetdir],’’Enable’’,’’on’’);’...

148


% ’thoughtsoutput = 2;’]);

% bpos = bpos + radioheight;

%

% texttoscreen = uicontrol(’Style’,’radiobutton’,...



% ’String’,’Output thoughts to screen’,...




% ’Value’,0,...

% ’Callback’,[’set([textnowhere texttofile],’’Value’’,0);’...

% ’set([logfiledisp logfilegetdir],’’Enable’’,’’off’’);’...


% bpos = bpos + radioheight;

%

% textnowhere = uicontrol(’Style’,’radiobutton’,...



% ’String’,’Do not output thoughts anywhere’,...




% ’Value’,0,...

% ’Callback’,[’set([texttoscreen texttofile],’’Value’’,0);’...

% ’set([logfiledisp logfilegetdir],’’Enable’’,’’off’’);’...


%

% if thoughtsoutput == 0

% set(textnowhere,’Value’,1);

% elseif thoughtsoutput == 1

% set(texttoscreen,’Value’,1);

% else

% set(texttofile,’Value’,1);

% set([logfiledisp logfilegetdir],’Enable’,’on’);

% end

% -------------------------------------------------------------------------

% title



’Position’,[frame.left fpos frame.width titleheight+2*button.spacing],...


window.height = fpos + titleheight + 2*button.spacing + frame.spacing;


’Position’,[button.left+1 bpos button.width titleheight],...


’String’,’ROBOPOOLPLAYER’,...


’FontSize’,titleheight/2,...


% -------------------------------------------------------------------------

window.left = screensize(1) + (screensize(3) - window.width)/2;

window.bottom = screensize(2) + (screensize(4) - window.height)/2;

set(triggerwindow,’Position’,[window.left window.bottom window.width window.height]);

%CHOOSESHOTGUI Create a GUI to choose the robot’s shot.

dd = 4; % decimation of displayed images

149


choosewindow = figure(’Name’,[’Robot turn #’ num2str(turn)],...

’Position’,[1 1 5 5],...




’NumberTitle’,’off’,...

’Visible’,’off’);

imwidth = length([dd:dd:size(rawphoto,2)]);

imheight = length([dd:dd:size(rawphoto,1)]);

imaxes = axes(’Units’,’pixels’,...

’Position’,[frame.spacing+1 popupheight2+frame.spacing+button.spacing imwidth imheight]);

tableonly = ...

tablepic(Xtotal,Ytotal,X1,X2,Y1,Ymid,Y2,np,rp,epd,whi,rw,bb,ryb,balls,rb,pocketcentres,dd);

shotpic(tableonly,np,whi,rw,bb,ryb,balls,rb,pocketcentres,dd,shotlist(bestshots(1)))

bestshotspopvalues = ’’;

for ii = bestshots

bestshotspopvalues = [bestshotspopvalues ’Shot #’ num2str(ii) ’|’];

end

bestshotspopvalues = [bestshotspopvalues ’Table photo’];

playshot = 1;

bestshotspop = uicontrol(’Style’,’popupmenu’,...

’Position’,[frame.spacing+floor(imwidth/2)-popupwidth2+1 frame.spacing+1 ...

popupwidth2 popupheight2],...

’BackgroundColor’,popupcolour,...

’String’,bestshotspopvalues,...



’Callback’,[’playshot = get(bestshotspop,’’Value’’);’...

’if playshot <= length(bestshots),’...

’shotpic(tableonly,np,whi,rw,bb,ryb,balls,rb,pocketcentres,’...

’dd,shotlist(bestshots(playshot))),’...

’set(chooseshot,’’Enable’’,’’on’’);’...

’set(showgoodness,’’String’’,[’’Goodness: ’’ ’...

’num2str(round(100+10*log([shotlist(bestshots(playshot)).goodness])))]);’...

’else,’...

’imshow(rawphoto(dd:dd:end,dd:dd:end,:)),’...

’set(chooseshot,’’Enable’’,’’off’’);’...

’set(showgoodness,’’String’’,’’’’);’...

’end’]);

chooseshot = uicontrol(’Style’,’pushbutton’,...

’Position’,[frame.spacing+floor(imwidth/2)+button.spacing+1 frame.spacing+1 ...

pushwidth2 popupheight2],...


’String’,’Play’,...




’Callback’,’uiresume’);

showgoodness = uicontrol(’Style’,’text’,...

’Position’,[frame.spacing+floor(imwidth/2)+pushwidth2+2*button.spacing+1 frame.spacing+1 ...

textwidth2 popupheight2-4],...


’String’,[’Shot score: ’ num2str(round(100+10*log([shotlist(bestshots(1)).goodness])))],...




% The -4 is so that the text aligns with the number in the popupmenu


150


’Position’,[frame.spacing+1 popupheight2+imheight+2*frame.spacing+button.spacing ...

imwidth titleheight],...


’String’,’ROBOPOOLPLAYER’’S SHOT!’,...


’FontSize’,titleheight/2,...


window.width2 = imwidth + 2*frame.spacing;

window.height = imheight + popupheight2 + titleheight + 3*frame.spacing + button.spacing;

window.left = screensize(1) + (screensize(3) - window.width2)/2;

window.bottom = screensize(2) + (screensize(4) - window.height)/2;

set(choosewindow,’Position’,[window.left window.bottom window.width2 window.height]);

function pic = drawcircle(pic,coord,r,colour,howopaque)

%DRAWCIRCLE Draw a circle on a given image.

for xx = intersect(floor(coord(1)-r):ceil(coord(1)+r),1:size(pic,1))

for yy = intersect(floor(coord(2)-r):ceil(coord(2)+r),1:size(pic,2))

if (xx-coord(1))^2 + (yy-coord(2))^2 <= r^2

pic(xx,yy,:) = howopaque*colour’ + (1-howopaque)*squeeze(pic(xx,yy,:));

% dodgy stuff must be done in order to have the weird sized

% matrices add properly

end

end

end

%COORD2STR Convert coordinate pairs to a string.

function str = coord2str(vecs,ls)

x = vecs(:,1);

y = vecs(:,2);

numofvecs = size(vecs,1);

str = [’’];

for ii = 1:numofvecs

str = [str ’(’ num2str(round(x(ii))) ’,’ num2str(round(y(ii))) ’)’];

if ii < numofvecs

str = [str ’,’];

end

end

151

Appendix B Visual Basic code

Dim a As Boolean

Dim Xsteps As Integer

Dim Ysteps As Integer

Dim AngSteps As Integer

Dim Solenoid_Power As Integer

Dim Port_Location As Integer

Dim Command_1 As Long

Dim command_1a As Long



Dim command_3a As Long







Dim command_10 As Long

Private Sub Form_Load()

’ This section will load the values outputted from the MATLAB program

’ and convert them to steps and output them to the motors

Open App.Path & "xdir.txt" For Input As #1 ’ Reads in inputs

Open App.Path & "ydir.txt" For Input As #2 ’ from vision

Open App.Path & "ang.txt" For Input As #3 ’ system program

Open App.Path & "sol.txt" For Input As #4

Dim OutputArray(4) As Integer

Dim i As Integer

Input #1, OutputArray(1)




’ Diagnostic Check: Directly Input Steps if necessary

’ OutputArray(1) = 100 ’ X direction

’ OutputArray(2) = 0 ’ Y direction

’ OutputArray(3) = 0 ’ Angle Direction

’ OutputArray(4) = 0 ’ Solenoid Power Level

For OutputArray(i) = 1 To 4

If OutputArray(i) = vbEmpty Then

’ If the files do not exist then exit the subroutine and input manually

intPress = MsgBox("File Not Found", vbCritical)

Exit Sub

End If

Next OutputArray(i)

Xsteps = OutputArray(1)

Ysteps = OutputArray(2)

AngSteps = OutputArray(3)

Solenoid_Power = OutputArray(4)

’ Move to the initial position:

Call Motor_Output(Xsteps, Ysteps, AngSteps, Solenoid_Power)

Xsteps = -Xsteps ’ Invert all values.

Ysteps = -Ysteps

152


AngSteps = 0

Solenoid_Power = 0

’ Now return carriage to original position:

Call Motor_Output(Xsteps, Ysteps, AngSteps, Solenoid_Power)

’ Once returned from the output subroutine, delete the confirm file

’ to allow the program to indicate that it is finished:

Kill "C:\My Documents\tom\tom\Uni\Project\vbstuff\File Loader\Done.txt"

End

End Sub

Private Sub Motor_Output(Xsteps, Ysteps, AngSteps, Solenoid_Power)

’ System Initialisation:

Timer1.Interval = 20 ’ 20 mS

Timer2.Interval = 20 ’ 10 mS

Timer3.Interval = 500 ’ 0.5 S

Timer1.Enabled = False



Command_1 = &H1 ’ Binary: 0000 0001 Positive x-direction bit

command_1a = &H2 ’ Binary: 0000 0010 Step in negative x-direction

Command_2 = &H3 ’ Binary: 0000 0011 Step in positive x-direction

Command_3 = &H4 ’ Binary: 0000 0100 Positive y-direction bit

command_3a = &H8 ’ Binary: 0000 1000 Step in negative y-direction

Command_4 = &HC ’ Binary: 0000 1100 Step in positive y-direction

Command_5 = &H10 ’ Binary: 0001 0000

Command_6 = &H30 ’ Binary: 0011 0000

Command_7 = &H80 ’ Binary: 1000 0000

Command_8 = &H40 ’ Binary: 0100 0000

Command_9 = &HC0 ’ Binary: 1100 0000

command_10 = &H0 ’ Binary: 0000 0000 negative and y-direction bit

a = False

Port_Location = &H378

’ Sets the initial location of the Parallel Port in Memory

’ This may differ between computers

’ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

’ %%%% X - DIRECTION CONTROL %%%%

’ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

If Xsteps > 0

Then Timer1.Enabled = True

Do While a = False

DoEvents

Out Port_Location, Command_1 ’ Set Pin 1 High (Binary: 0000 0001)

Loop

a = False

For Pulses = 0 To Xsteps

If Pulses < Xsteps / 4 Or Pulses > (3 * Xsteps) / 4 Then

’ Set timer to half max power timer and make system wait (set initiallly to 20ms)

Timer1.Enabled = True ’ Delay for 20mS then send new command

Do While a = False

’ Send step pulse and then wait for timer to expire

DoEvents

Out Port_Location, Command_2

’ Sets step Pin high and Sets the Direction as Positive (Binary: 0000 0011)

Loop

a = False

Else

Timer2.Enabled = True ’Delay for 10mS then send new command

Do While a = False

DoEvents

153



Loop

a = False

End If

Timer2.Enabled = True

Do While a = False

DoEvents


Loop

a = False

Next Pulses

End If

If Xsteps < 0 Then

a = False


Do While a = False

DoEvents

Out Port_Location, command_10 ’ Set inital 20ms delay

Loop

a = False

Pulses = Xsteps

’ Sets bit 1 Low but still allows stepping in negative direction

For Pulses = Xsteps To 0

If Pulses > Xsteps / 4 Or Pulses < (3 * Xsteps) / 4 Then



Do While a = False

DoEvents

Out Port_Location, command_1a ’ Delay for 20mS then send new command

Loop

a = False

Else

’ Set timer to full speed timer (set initially to 10ms)


Do While a = False

’ Set set negative x direction pin

DoEvents

Out Port_Location, command_1a

Loop

a = False

End If


Do While a = False

DoEvents

Out Port_Location, command_10

Loop

a = False

Next Pulses

End If

If Xsteps = 0 Then


End If

’ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%

’ %%%% Y-DIRECTION CONTROL %%%%

’ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%

If Ysteps > 0 Then


a = False

Do While a = False

DoEvents


Loop

154


a = False

For Pulses = 0 To Ysteps

If Pulses < Ysteps / 4 Or Pulses > (3 * Ysteps) / 4 Then

’ Set timer to half max power timer and make system wait (set initially to 20ms)


Do While a = False


DoEvents


’ Sets step Pin high and Sets the Direction as Positive (Binary: 0000 1100)

Loop

a = False

Else


Do While a = False

DoEvents


Loop

a = False

End If


Do While a = False

DoEvents


Loop

a = False

Next Pulses

End If

If Ysteps < 0 Then

a = False


Do While a = False

DoEvents

Out Port_Location, command_10 ’ Set inital 20ms delay

Loop

a = False

Pulses = Ysteps

’ Sets bit 1 Low but still allows stepping in negative direction

For Pulses = Ysteps To 0

If Pulses > Ysteps / 4 Or Pulses < (3 * Ysteps) / 4 Then

’ Set timer to half max power timer and make system wait (set initially to 20ms)


Do While a = False

DoEvents


’ Delay for 20mS then send new command

Loop

a = False

Else

’ Set timer to full speed timer (set initially to 10ms)


Do While a = False

’ Set set negative y direction step pin

DoEvents


Loop

a = False

End If


Do While a = False

DoEvents

Out Port_Location, command_10 ’ wait a further 20ms and step again

Loop

a = False

Next Pulses

155


End If

If Ysteps = 0 Then


End If

’ %%%%%%%%%%%%%%%%%%%%%%%

’ %%%% ANGLE CONTROL %%%%

’ %%%%%%%%%%%%%%%%%%%%%%%

If AngSteps > 0 Then


Do While a = False

DoEvents

Out Port_Location, Command_5 ’ Set Pin 1 High Binary: 0001 0000

Loop

a = False

For Pulses = 0 To AngSteps

If Pulses < AngSteps / 4 Or Pulses > (3 * AngSteps) / 4 Then


Timer1.Enabled = True ’Delay for 20mS then send new command

Do While a = False


DoEvents


’ Sets step Pin high and Sets the Direction as Positive Binary: 0011 0000

Loop

a = False

Else


Do While a = False

DoEvents


Loop

a = False

End If


Do While a = False

DoEvents

Out Port_Location, Command_5 ’ Set Pin 1 High Binary: 0001 0000

Loop

a = False

Next Pulses

End If

’ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

’ %%%% SOLENOID POWER LEVELS %%%%

’ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

If Solenoid_Power = 0 Or vbEmpty Then

’ Enable the 0.5s timer

a = False


Do While a = False

DoEvents

Out Port_Location, command_10 ’ If no force required do nothing

Loop

’ Timer just to allow the hardware to register the signal

End If

If Solenoid_Power = 1 Then

a = False


Do While a = False

DoEvents

Out Port_Location, Command_7 ’ Low force setting

156


Loop

Out Port_Location, command_10 ’ Reset the solenoid

End If


a = False


Do While a = False

DoEvents

Out Port_Location, Command_8 ’ Med Force Setting

Loop


End If


a = False


Do While a = False

DoEvents

Out Port_Location, Command_9 ’ High Force Setting

Loop


End If

End Sub

Private Sub Timer1_Timer() ’ 20 mS

a = True

txtAck.Text = txtAck.Text & "1 "

Timer1.Enabled = False ’ reset the timer

End Sub

Private Sub Timer2_Timer() ’ 10 mS

a = True

txtAck2.Text = txtAck2.Text & "2 "


End Sub

Private Sub Timer3_Timer() ’ 0.5 seconds

a = True

txtAck3.Text = txtAck3.Text & "3 "


End Sub

157

Appendix C dSPACE code

C.1 C code for DS1102

#include <brtenv.h> /* basic real-time environment */

#define INT_TABLE (0x000000) /* address of interrupt table */

#define INT_ACK (0x800000) /* acknowledge interrupts */

/* global variables for the communication with the MATLAB workspace */

float mydata[2][1000];

int motorsbusy = 0;

float timeinterval = 0.01;

long *int_tbl=(long*)INT_TABLE;

volatile long *ioctl=(long*)IOCTL;

void c_int04(void)

{

int i;

float timeprev;

tic0_init();

motorsbusy = 1;

for(i=1; i<=1000; i++)

{

timeprev = tic0_read();

while(tic0_read() < timeprev + timeinterval)

{

master_cmd_server();

host_service(0,0);

}

ds1102_xf0_out(mydata[1][i]);

ds1102_xf1_out(mydata[2][i]);

}

motorsbusy = 0;

*ioctl=(*ioctl)&(0x8000)|INT_ACK; /* acknowledge */

asm(" nop"); asm(" nop"); /* nop’s are necessary ! */

asm(" andn 0008h,IF"); /* clear bit in interrupt flag register */

}

void int_enable(void)

{

int_tbl[4]=(long)c_int04;

asm(" or 0008h,IE"); /* set bit in interrupt enable register */

asm(" or 2000h,ST"); /* and in status register */

}

main()

{

ds1102_init_xf0(1); /* configure XF0 for output */

ds1102_init_xf1(1); /* configure XF1 for output */

int_enable(); /* enable DSP interrupts */

while(1)

{

while(msg_last_error_number() == DS1102_NO_ERROR)

158

Appendix C dSPACE code

{


host_service(0,0);

};

msg_info_set(MSG_SM_USER, msg_last_error_no, "Error occurred.");

while(msg_last_error_number() != DS1102_NO_ERROR)

{


host_service(0,0);

};

msg_info_set(MSG_SM_USER, 0, "Error released.");

};

}

C.2 Matlab code

mlib(’SelectBoard’,’ds1102’)

channel_desc = mlib(’GetMapVar’,’mydata’,’Type’,’FloatDSP32’,’Length’,length(mydata));

mlib(’Write’,channel_desc,’Data’,mydata);

mlib(’Intrpt’)

159

Appendix D Control card circuit

160

Appendix D Control card circuit

This page is to be substituted with the control card circuit.

161

Appendix E Cost analysis

A summary of the costs of the project are shown in Table ??. The theoretical cost

indicates what the items would cost to buy. The actual cost indicates the actual cost

to the project budget. Where the actual cost is less than the theoretical cost, the

items have been generously supplied by the Department of Mechanical Engineering

for use in the project, with the exception of the pool balls, which were donated by

The Ballroom.

The extras refer to costs incurred to the project budget, but these items are not

necessary for the actual construction of the robot. (All figures are in Australian

dollars.)

162

Appendix E Cost analysis

Table E.1: Cost analysis.

Item Price Quantity Theoretical cost Actual cost

ActuationStepper motor: high powered $300 2 $600 $0

Stepper motor: regular $100 1 $100 $0Stepper motor control box $3000 1 $3000 $0Washing machine solenoid $80 1 $80 $0Transformer (for solenoid) $100 1 $100 $60

Table baseFrame: mild steel 35× 18 RHS $5/m 18m $90 $90

Channel section (for rails) $55 1 $55 $55

Vision systemCamera $4500 1 $4500 $0

Camera lens $480 1 $480 $0D30Remote software $100 1 $100 $100

MiscellaneousElectrical (plugs, boxes, loom, etc.) $200 — $200 $200Mechanical (belts, pulleys, clamps) $830 — $830 $830

Pool table $325 1 $325 $325Pool balls $90 1 set $90 $0Computer $4000 1 $4000 $0

Matlab license (incl. toolboxes) $2660 1 $2660 $0Labour $50/hour 10 hours $500 $0

Total: $17710 $1660

ExtrasCar starter motor solenoid $88 1 $88 $88

MIT thesis $120 1 $120 $120Total: $17918 $1868

163

Appendix F Solenoid force

experiment

F.1 Introduction

This experiment was conducted in order to design a robotic mechanism that can play

a range of billiard shots comparable to that of a human, and that can withstand

forces exerted by the actuation system. During a game of billiards, many types

of shots are made ranging from short, soft shots to breaks and what is commonly

known as the “six pocket theory” shot. This experiment was conducted to determine

the range of forces used in a typical pool game on the table acquired for this project.

F.2 Aim

To determine the range of forces used during a typical game of pool.

F.3 Apparatus

• Tektronix Digital CRO

• Bruel & Kjaer charge amplifier

• Graphite insulated cables (to reduce noise)

• 8200 Force Transducer (reference sensitivity at 159.0Hz, 23◦C is 3.80 pC/N)

The apparatus is set up as shown in Figure ??.

F.4 Method

The voltage outputs corresponding to the forces used for various pool shots (from

short, soft shots to breaks to “six pocket theory” shots) were recorded for each

164

Appendix F Solenoid force experiment

Figure F.1: Force experiment setup.

member of the group. This experiment was conducted on the table acquired for the

project as well as on other pool tables, so as to leave scope for transferring the robot

to a different or larger table.

F.5 Results and analysis

The voltage output read from the digital CRO was converted into Newtons using

the conversion factor 100N/V. The results are shown in Table ?? below.

F.6 Conclusion

It was determined from the results that an amateur player uses forces ranging ap-

proximately from 80N to 1300N during the course of a game. The actuation system

must therefore be able to produce a similar force range to these values.

165

Appendix F Solenoid force experiment

Table F.1: Washing machine solenoid impulse time results.

Voltage Force Voltage Force12.0V 1200N 1.8V 180N12.0V 1200N 2.2V 220N11.0V 1100N 3.1V 310N13.0V 1300N ← Largest force 8.0V 800N7.0V 700N 1.6V 160N7.0V 700N 5.5V 550N10.0V 1000N 5.0V 500N9.0V 900N 9.5V 950N3.0V 300N 2.4V 240N2.0V 200N 2.4V 240N5.5V 550N 5.0V 500N0.8V 80N ← Smallest force 7.6V 760N13.0V 1300N 7.0V 700N1.7V 170N 8.2V 820N1.8V 180N 4.2V 420N1.0V 100N 10.0V 1000N

166

Appendix G Technical drawings

167

Appendix G Technical drawings

This page is to be substituted with drawings.

168