Registration number 100178897

2019

Raspberry Pi Open Project - DraughtsRobot

Supervised by Dr Edwin Ren

University of East Anglia

Faculty of Science

School of Computing Sciences

Abstract

Consumer grade robotics has seen a large increase in popularity in the recent years al-

lowing people to begin developing relatively cheap robotic projects for needs that suit

them. This could lead to a fast advancement in affordable robot tech. This report docu-

ments the design and implementation of a robotic arm that is capable of playing check-

ers against a human player. This project incorporates fundamental robotic technologies

including computer vision, kinematics and controlling actuators. A fully functioning

robotic arm is achieved that can play a human in a game of checkers.

CMP-6013Y

Contents

1 Introduction 6

1.1 Aims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 Areas of Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4 Road Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Research 10

2.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Arm Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.1 SCARA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.2 Cartesian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.3 Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3.1 Servo Control and Programming . . . . . . . . . . . . . . . . . 14

2.4 Computer Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4.1 Object Tracking/Board recognition . . . . . . . . . . . . . . . 15

2.4.2 Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4.3 Programming language . . . . . . . . . . . . . . . . . . . . . . 15

3 Identification of Issues 15

3.1 Issues Identified In Planning . . . . . . . . . . . . . . . . . . . . . . . 16

4 Design & Implementation 17

4.1 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Software Components . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.2.1 Software Architecture . . . . . . . . . . . . . . . . . . . . . . 18

4.2.2 Language and Libraries . . . . . . . . . . . . . . . . . . . . . . 18

4.2.3 Game Engine Design . . . . . . . . . . . . . . . . . . . . . . . 19

4.2.4 Game Engine Implementation . . . . . . . . . . . . . . . . . . 20

4.2.5 Servo and Arm control Design . . . . . . . . . . . . . . . . . . 22

4.2.6 Servo and Arm control Implementation . . . . . . . . . . . . . 23

Reg: 100178897 iii

CMP-6013Y

4.2.7 Computer Vision Design . . . . . . . . . . . . . . . . . . . . . 27

4.2.8 Computer Vision Implementation & Camera Calibration . . . . 29

4.2.9 Kinematics Design and Implementation . . . . . . . . . . . . . 34

4.3 Hardware Components Design and Implementation . . . . . . . . . . . 36

4.3.1 Parts List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.3.2 Raspberry Pi + Raspbian . . . . . . . . . . . . . . . . . . . . . 37

4.3.3 Power Supply & Servo interface . . . . . . . . . . . . . . . . . 37

4.3.4 Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.3.5 Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5 Testing and Full system 42

5.1 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.1.1 Game Engine Testing . . . . . . . . . . . . . . . . . . . . . . . 42

5.1.2 Computer vision testing . . . . . . . . . . . . . . . . . . . . . 43

5.1.3 Arm Testing/Servo . . . . . . . . . . . . . . . . . . . . . . . . 44

5.2 Integrated system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6 Evaluation 46

6.1 Limitations of the design & Addressed issues . . . . . . . . . . . . . . 47

6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

7 Gantt Chart 49

References 51

Reg: 100178897 iv

CMP-6013Y

List of Figures

1 SCARA Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2 Cartesian/Gantry Configuration . . . . . . . . . . . . . . . . . . . . . . 13

3 Arm Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5 File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6 UML for important Classes . . . . . . . . . . . . . . . . . . . . . . . . 19

7 Flow of control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

8 Sending coordinates to arm . . . . . . . . . . . . . . . . . . . . . . . . 21

9 MG996r PWM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

10 Initialising arm object and binding servos . . . . . . . . . . . . . . . . 25

11 Example of perspective transform applied warped image . . . . . . . . 29

12 Checking for movement on board . . . . . . . . . . . . . . . . . . . . 33

13 Left: Arm with rotational constraints. Right: Planar manipulator . . . . 34

14 Birds eye view of arm over board and ZMap . . . . . . . . . . . . . . . 35

15 Ground and power leads to servo . . . . . . . . . . . . . . . . . . . . . 38

16 Measurements of the arm . . . . . . . . . . . . . . . . . . . . . . . . . 39

17 Remounted grabber . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

18 Boards tested . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

19 Grayscale red and green board . . . . . . . . . . . . . . . . . . . . . . 41

20 Board with Black/Red pieces . . . . . . . . . . . . . . . . . . . . . . . 42

21 Game engine passing and receiving values . . . . . . . . . . . . . . . . 43

22 Testing Digital board . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

23 Testing Pi camera image of board . . . . . . . . . . . . . . . . . . . . 44

24 Testing arm and grabber . . . . . . . . . . . . . . . . . . . . . . . . . 45

25 Testing servo accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . 45

26 Record of time spent on tasks . . . . . . . . . . . . . . . . . . . . . . . 47

27 Final Project Gantt chart . . . . . . . . . . . . . . . . . . . . . . . . . 50

Reg: 100178897 v

CMP-6013Y

1 Introduction

Robotics is used in a vast number of industries to perform roles that humans once

occupied, with applications ranging from the mass scale production of cars to

extremely precise open surgery- A robotic arm even sits on the face of another planet-

Mars Curiosity Rover (JPL 2019).

For these robots to autonomously perform tasks it is usually necessary for them to be

able to understand their environment, and then interact with it. This requires mapping,

using sensors to gain a representation of the surroundings and determine an objects

position and orientation (pose). To interact with an object the robot must have a

reliable coordinate system accompanied by an equally reliable way to translate

movements of servos/actuators, which move the arm, to the real world coordinates- this

translation of movement is known as kinematics. A robotic arm consists of a chain of

links moved by motors and actuators, with an end-effector to interact with the

environment, higher degrees of freedom are achieved by adding more joints or adding

planes of movement to current joints.

Many of the computer vision algorithms still used today date back to the 1960’s -

1980’s. Including the Canny edge detector (Canny 1986) and the Harris corner edge

detector (Lowe 1999) used in object detection. Although the introduction of robotics in

manufacturing occurred in the 1980’s, this technology has only recently, 2000’s, been

accurate and reliable enough to be used for safety critical systems such as surgery, with

advancements happening constantly- see Asimo Honda’s humanoid robot (Honda

2019) and MIT’s Cheetah (MIT 2019).

Historically, robotics is expensive, with hobbyist dissuaded from joining the industry

unless backed by a sizeable wallet. However, with the prevalence of low cost

computing hardware, as well as the declining cost of consumer robotics equipment this

field is becoming more accessible now than ever. By developing an arm with enough

accuracy to play checkers, whilst constrained by a budget, a view into the progress of

this field will be visible. Demonstrating the trajectory of this field, and the capability of

low cost hardware/software.

Reg: 100178897 6

CMP-6013Y

1.1 Aims

The aim of this project is to have a game of checkers/draughts between a human and a

robot arm. A program running on a Raspberry Pi, will be used to control the arm and

gather movement data on the human player. This could be further extended to allow

the robot to play against itself, as well as adapting the arm movement for other

applications such as integration with different games.

By taking the final aim of this product and decomposing it, several milestones can be

generated each of which must be met to complete the project:

• Selecting a robot arm with a sufficient range of movement and accuracy to repeat-

edly lift and place a draughts piece.

• Use inverse kinematics to derive the rotations for the servo motors so that the

end-effector will be placed over the desired tile on the game board.

• Use computer vision to recognise the board, differentiate team pieces and track

where pieces are, as well as determine if the human has taken their turn.

• Alter an open source checkers game to send movements to the arm and control

the state of play.

• Link the working arm, camera, and an open source Draughts game (complete with

AI), together to play a game.

Reg: 100178897 7

CMP-6013Y

1.2 Motivation

Robotics is a rapidly expanding market, with its adoption present in almost all

industries in some shape or form. Kinematics, known as "the geometry of motion", is

one discipline that underpins robotic advancement as it allows us to control of complex

robotics capable of many degrees of movement.

Computer Vision(CV) is another vital aspect of robotics, as it allows a system to react

in real time based on what it sees. CV algorithms have many applications, including

self driving cars, facial recognition, object tracking in sports and are even being

implemented into medicine to identify tumours (TheDuckCow 2019).

Because of the emerging prevalence of these two technologies I am interested in

learning more; This is where my personal motivation is derived. I desire to learn about

robotics and computer vision as I believe these fields would be a great way for my

career to progress, therefore am interested in involving these as part of my degree and

gaining practical experience. This kind of technology has the capability of improving

many peoples, for example those with disabilities, quality of life and can also remove

the need for humans to work jobs that can be carried out by robots.

The advancement of CV and robotics have the potential to change many lives, possibly

even preventing people from working poorly paid, monotonous or dangerous labour

jobs. If robotics can improve the world in which i live, i would love to help improve

this field advance; I believe in my lifetime these areas will rapidly change at a rate we

may not currently comprehend.

I have no previous experience in either fields so my approach will be utilising

fundamental techniques in attempt to gain an understanding of two critical aspects of

robotics. This will allow me to grow my knowledge base and understand which aspects

of robotics are important to me.

Reg: 100178897 8

CMP-6013Y

1.3 Areas of Understanding

To complete this project a good level of understanding of python programming is

required, as well as a firm grasp of available computer vision libraries, in this instance

openCV is required coupled with knowledge of feature detection is to recognise and

extract key objects from an image. Knowledge of data structures will be needed to

understand how to store camera calibration settings and board configurations. An

understanding of servo control and programming is vital, to allow manipulation of the

robotic arm. A brief understanding of electronics is required to understand how servos

receive control signals, and to ensure they are correctly powered and driven by the

Raspberry Pi’s hardware interface. A firm understanding of trigonometry is key as the

kinematics algorithms are based on complex trigonometry formulae.

1.4 Road Map

This section introduced the project and explained its motivation and application.

Section 2 will discuss relevant findings related to this project- Servo control,

kinematics algorithms, computer vision, hardware interfaces and programming; then

briefly describe two similar projects. Section 3 discusses issues identified during

development, which later ties in with section 5 showing which issues occurred and

relevant solutions. Section 4 documents the design decisions and the planning process,

as well as gives pseudo-code representations of algorithms used and describes the

software used. Section 4 also describes the implementation of each component of the

system. Section 5 describes any testing carried out on each component, then the overall

state of the complete system. The outcome is examined in section 6 including an

evaluation, then describes limitations to the design along with issues as well as future

development. Here the success of the project is evaluated. The final section, 7, shows a

gantt chart that represents the order in which tasks were completed, and their duration.

Reg: 100178897 9

CMP-6013Y

2 Research

This section explores research into similar topics and details any relevant findings. The

areas researched include: Servo control, arm design, hardware/software interfaces

(GPIO), error calculation and Limitation, computer vision and object tracking, camera

calibration and Programming language/Environment.

Firstly, related work is discussed focusing on other projects that have completed

similar tasks. Using this, research will be conducted into the relevant fields, and any

important information needed for design considerations will be reported. The research

discussed is as follows: Arm design and servo control; this research identifies different

forms of suitable arms and various servo hardware interfaces. Then computer vision

and object tracking methods are reported, identifying any necessary algorithms needed,

as well as providing support for camera calibration. Finally, research will show which

programming languages may be suitable for this project.

2.1 Related Work

Many projects exist that utilise a robotic arm along with a computer vision algorithm to

complete some task. More precisely there are several examples of projects that have

used an arm to play checkers/chess. Some of the ways in which these projects were

carried out will be identified and discussed in this section, giving me some indication

on what can be learned, and identifying areas of research. Two projects - Raspberry

Turk (), and Buttercup chess robot() utilise arms to play a game of chess, using

computer vision for piece detection.

Raspberry Turk uses a SCARA arm to play chess, the arm is controlled by a python

program, linked to an Arduino controlling 3 servos. The pieces are picked up by an

electromagnet on the end of a rod, that is lowered down to the pieces. This project uses

openCV for the vision module to detect the board and A neural network is trained to

recognise the pieces on the board.

Buttercup also uses a SCARA type arm to play chess. OpenCV allows the detection of

the board squares as well as pieces on the board. The pieces are picked up by a claw

Reg: 100178897 10

CMP-6013Y

like grabber controlled by a servo motor. The game engine tracks the movement of

players pieces, meaning no neural network is needed. The implementation details are

described fully in the paper.

Other examples of a robot playing a human exist, but the two cases talked about are

relevant to this project as they are completed on a low budget and use similar hardware

to what is used in this project. Most examples found online contain arms that cost

upwards of £1,000. This is clearly not feasible. Some other examples exist on

YouTube, however the documentation for these is harder to locate, therefore they dont

provide helpful as the work cannot be studied and used to help this project.

2.2 Arm Design

A robotic arms main objective is to translate linear or rotational servo movement into a

real world position. By using a coordinate system for the real world servo rotations can

be calculated that map movement between two desired points. In robotics there are

several design methodologies that can be utilised for positioning an end effector (The

point at the end of an arm that interacts with the environment) at a desired coordinate

in 3D space. The most common are outlined in this section.

The position of the end effector can be determined through either inverse or forward

kinematics, this is used in almost all robotic systems and uses trigonometry to calculate

the correct angles of movement at each joint(Richard 1981).

Due to the project budget some design methodologies may be inadequate. One

consideration with regards to arm choice is that- This is not an engineering project,

thus building an arm from scratch is out of scope. This makes the Arm choice and

research biased towards purchasable build kits and affordable hardware.

2.2.1 SCARA

Selective Compliance Assembly Robotic Arm, is a common methodology used in mass

production. This type of arm is used to interact with items laying flat beneath the arm.

As demonstrated in Figure 4 this arm allows slight compliance (flexibility) in the X and

Reg: 100178897 11

CMP-6013Y

Y planes and rigid movement in the Z. The end effector is placed above the target and

lowered into position. This type of arm is commonly used, for example, to lower pins

into holes with high tolerances, and thus this methodology is a large contender for use

in this project. The reach of the arm is limited by the lengths of the inner and outer arm.

Figure 1: SCARA Configuration

2.2.2 Cartesian

Cartesian robots have many uses, including that of 3D Printers. The end effector has

rigid movement in X,Y,Z planes. As demonstrated in Figure 2 the arm sits above the

desired workspace as a gantry. This design requires a large frame however is extremely

accurate. Research shows that build kits exist for this type of robot, however they can

be costly, with simple models costing £300 - RobotShop 2019.

This arm is mechanically the best candidate for the project, however due to price

constraints is not the correct choice.

2.2.3 Arm

This design is likely what most people think of when hearing the term robotic arm. The

Robotic arm comes in many shapes and forms. These types of arms are preferred by

hobbyists, and have a large amount of forum support, as well as many available build

Reg: 100178897 12

CMP-6013Y

Figure 2: Cartesian/Gantry Configuration

kits. As seen in Figure 3 this arm has 4 servos, allowing 4 points of articulation. This

allows control in the Z direction with one servo, and two-three servos control the X,Y

direction, depending on the need of the system. The robot is modular and easily

changeable, meaning any required modifications can be made as well as more versatile

than the other methodologies as the end effector is not always adjacent to the floor.

This approach does not confine the robot to just checkers, and allows the manipulator

to be used for other applications if desired.

Figure 3: Arm Configuration

Reg: 100178897 13

CMP-6013Y

2.3 Hardware

2.3.1 Servo Control and Programming

To control a robotic arm servos are required to manipulate the position at each joint in

the arm. There exist many servos capable of relatively high precision that would be

suitable for Arm control.

Servos are generally controlled by Pulse Width Modulation (PWM). The signal pin on

the servo receives a 20ms pulse signal. The position of the motor is changed by varying

the length of the pulse by 15ms, shortening the pulse moves the servo clockwise and

lengthening moves counter clockwise, depending on the model. (MG669r manual)

Research through various hobbyist websites, and arm build kits pointed towards the

MG996r servo being a suitable candidate(Arduino.com 2016). The motors are

reasonably priced and capable of a high enough resolution for this application.

There are libraries available for servo control in various different languages, including

python, C, C++ and java. These allow signal control from either an Arduino or

Raspberry Pi. Since a Raspberry Pi contains a processor it is advantageous to the

Arduino. The Pi’s GPI/O board is only capable of running one pin on PWM, however

the library PIGPIO provides a software based PWM, therefore makes control of

various motors from any of the 30 pins possible (abz.me 2019).

2.4 Computer Vision

For a robot to be able to interact with the real world, it must fist be able to see the

world in which it is placed. Therefore it is vital to use some form of video analysis.

This analysis allows detection of objects in the environment, allowing code to react to

events in view of the camera. OpenCV is a Library available that provides functions

aimed at real time image processing- Full documentation available at www.opencv.org.

There exist algorithms that can detect various features in an image, for example

contour detection and edge detection.

Reg: 100178897 14

CMP-6013Y

2.4.1 Object Tracking/Board recognition

Canny edge detection(Canny 1986) is an algorithm applied to an image to detect all

straight dominant edges. This algorithm generates a thresholded image containing only

the edges, as well as a list containing all edge corners.

Chessboard detection is used in camera calibration, this could prove useful for the

project as it is a function that can recognise if a board is in view. This is a possible

route to follow when designing the computer vision module.

Hough Circle transforms provide a consistent and reliable way to detect circular shapes

of varying size in an image, this could be useful when detecting board pieces and their

position within the grid.

2.4.2 Camera Calibration

A chessboard detection algorithm is a standard way to calibrate a camera and check for

distortion. Camera calibration can allow for images to be re-sized or a change of

perspective allows a consistent image to be captured. A standard method for camera

calibration is described by OpenCV.org(OpenCV 2019). Camera calibration allows a

consistent image is captured for a program to use.

2.4.3 Programming language

Since the systems examined use python this is a good language to aim to use. Many

other languages exist for servo control, like c++ or java. However, Python is easy to use,

lightweight and allows for use with openCV, and PI.GPIO. It is the obvious candidate

for use in this project especially due to many open source games existing.

3 Identification of Issues

This section discusses any problems/issues identified whilst planning, It gives an

insight into the issues that may affect the success of the project, these are identified in

the planning/research stage. Section 5 assesses any issues that occurred in the time

between planning and completing the project.

Reg: 100178897 15

CMP-6013Y

3.1 Issues Identified In Planning

• Accuracy of Arm: Since a build kit will be used an arm with sufficient articula-

tion, reach and accuracy will need to be used. There are many build kits available

for purchase, some of which will be documented in section 2.3 of the report

• Look-up Table vs On the Fly calculation: Determining the angle at which each

motor should rotate to map all positions onto a chess board could be hard-coded

or worked out on the fly. Each approach would work, but I believe a look-up table

would introduce less error. The angles could be calculated using Inverse Kine-

matics, however the real world values may not align with the calculated values

due to servo error.

• Calibration: Will the error introduced into the system need to be mitigated using

calibration points, or will the it not amount to enough to make a noticeable differ-

ence. If the error is unworkable a calibration function may be needed to track the

location of the arm using CV.

• Grabber: There are two ways in which pieces can be lifted, electromagnet or

pincer. The lifting technique will likely be a magnet, since this is more simple to

control using GPIO and also has no moving parts to calibrate. However, a claw

aligns more directly with a robotic arm and allows a greater degree of freedom

for future applications and adaptations.

• Computer vision accuracy: Determining when the human player has taken their

turn, and when the human takes a piece from the AI will require the use of a

camera. There needs to exist a sufficient method of determining whose turn is

next, if a piece is taken, moved and when a game ends.

• Expense: Since the project is based largely on hardware this can become expen-

sive. This is further discussed in the Planning section. However the budget for the

project cannot exceed £150, therefore lower budget hardware will likely be used,

this may affect the accuracy of parts of the system.

Reg: 100178897 16

CMP-6013Y

4 Design & Implementation

This section first shows a system overview then details the Design and implementation

of the required components. Software components are described with pseudo-code and

hardware components are shown with diagrams. The design and implementation of

each separate system, once tested and functioning, can be joined together to create a

single working system.

4.1 System Overview

Figure 4: System Architecture

• A. The Raspberry Pi is the main control system, it handles the game engine

(checkers game), as well as servo control and computer vision. The main sys-

tem is a checkers game running in python.

• B. The camera is used to detect the current state of the chess board, If a piece

moves the game state is updated and gets the movement for the computers turn.

Reg: 100178897 17

CMP-6013Y

• C. The arm receives controls from the Raspberry Pi, over the GPIO interface.

• D. The human and arm both play using this chess board.

• E. The human player moves their piece, playing against a min-max algorithm.

4.2 Software Components

4.2.1 Software Architecture

The project uses a modular, and simple file structure keeping functionality separated to

files containing individual modules. Each file contains several python files that are

used by the system at some point. This section did not necessarily incorporate any

design, however it was an emergent outcome from creating each module separately.

Figure 5 shows the hierarchy view of files used by the software element of this project.

ChessRobot is the root file. A Virtual environment is also included in this structure

with the required dependencies however is not shown here as it is contained within the

code. For the whole system to be started the file checkers.py must be ran.

Figure 5: File Structure

4.2.2 Language and Libraries

Python is the most suited language for this project to succeed. I have used python

previously and am familiar with how the language works. As shown in the UML this

program will be written using OOP design, thus creating objects for the Arm, Camera,

players and game.

Reg: 100178897 18

CMP-6013Y

Libraries/Dependencies used in this projects are as follows:

• OpenCV: is used to process the images taken by the camera.

• PI.GPIO: used for serial pin control on the Raspberry Pi, allowing python to

communicate over hardware interfaces with the servo motors.

• NumPY: used for storing and manipulating complex data types, and saving data

to files.

• Python 3: language used, must be installed to run the project.

The image shown in Figure 6 shows the basic UML for the game engine and its

interaction with the Arm class and CV module. The checkers game creates two

abstract classes, player and robot. The robot sends moves to the arm, and the player

pulls its moves from the CV module. This is modified from the original code, where

the player inputs their moves from the keyboard and the robot gets its moves from the

AI. The robot still gets its move from here, however the move is returned to the engine

to be sent to the Arm class.

Figure 6: UML for important Classes

4.2.3 Game Engine Design

To make each of the elements in the system work together they need to be controlled

by a central system, this is the purpose of the game engine. It determines who takes the

Reg: 100178897 19

CMP-6013Y

next turn, the scoring and keeps track of the position of pieces. An open source

application needs to be used so that i can freely modify code and use it in my project

without breaching licences. The following engines may be suitable:

• Carson Checkers game and AI : Simple checkers game with min max AI. This

program is a python 2.7 program and thus not suitable for this project. Source at

https://github.com/codeofcarson/Checkers

• Raven Checkers: More complex checkers game, ports exist in python 3. However

this uses a GUI, this may not be suitable for the project as a GUI is not required.

Source at : https://github.com/bcorfman/raven-checkers

• Checkers Game with Agents: This game uses a learning agent to play against a

human, is available in python 3.6 and has a command line interface. Source found

at: https://github.com/VarunRaval48/checkers-AI

The game chosen for this project is the Checkers game with Agents as both the player

and agent are derived from the same superclass meaning their methods and attributes

are the same, this makes integration easier as next moves, and best moves can be called

with the same code. Also this is lightweight and available in python 3 without

modification and also requires no external libraries.

4.2.4 Game Engine Implementation

The engine chosen is modified so that it can send commands to the arm, and calculate

the human players move using the vision class, the flow of control during a game is

shown in Figure 7. A copy of the source code can be obtained from:

"https://github.com/VarunRaval48/checkers-AI", working as of April 2019. The robots

agent code is modified to send coordinates to the arm class before sending them to the

checkers class to update the board. This means the arm will move the piece, but if it

fails the game will not be updated until the piece is moved by the arm. The agent class

contains code for human and robot players getMove method. The input for the human

player getMove is redirected to the computer vision class to detect pieces, once the

moves have been detected the game engine is updated with the correct moves.

Reg: 100178897 20

CMP-6013Y

Figure 7: Flow of control

To move the arm using the game engine only a small modification is required. When

the AI calculates its best move, a list containing this move is created, before the list is

returned to game engine, it is passed to the arm, if the arm move is successful the move

is then passed to the game engine. The code snippet below shows the code required for

this. The check in the first 2 lines sees if the move is a double take, if it is the end

position is set to the last position. This prevents the arm missing double moves whilst

the game engine registers them, thus placing the physical game out of sync.

Figure 8: Sending coordinates to arm

Reg: 100178897 21

CMP-6013Y

The game engine handles the input of moves from the CV algorithm in a similar way,

usually the input is taken from the keyboard for the human. This function is changed to

take the input from the CV algorithm, the CV module reports back the changed board,

the method for determining where the player has moved is explained in the CV section.

A large change from the initial design, shown in the project notebook term 2 week 7, is

with the role of the game engine, originally the computer vision algorithm would track

each piece and determine its colour and position updating the engine with the piece of

every counter and its position, however since the game engine tracks all possible

moves, and where each piece is on the board, the computer vision just has to track

which pieces move so the game engine knows which move the player made. This

simplifies the role of the CV algorithm and removes redundant functionality.

4.2.5 Servo and Arm control Design

The arm uses 4 servos to manipulate the position of the end-effector and one servo to

open and close the claw. As shown in the section 4.2.3, the game engine sends

coordinates to the arm class. For the arm to move there needs to exist some way for the

coordinates to be translated into a set of instructions to be sent to the servos. Since the

arm and the board are of a defined size these numbers will only need to b calculated

once, a dictionary would be the likely solution to storing these values. The positions

can be calculated with a kinematics algorithm then stored. Because the servos may not

be perfectly accurate, storing the values allows them to be adjusted, rather than

working out the rotations on the fly, which is much harder to account for error. If erro

needs to be accounted for this can be done for each board position.

Servos are controlled using PWM (pulse width modulation). By sending a signal of

varying length at regular intervals to the signal wire on the servo, the position of the

motor can be changed. The documentation for the MG996r servos used states - 20ms

pulse cycle with 5/15 ms duty with a minimum duty change of 0.1ms, this is

demonstrated in Figure 9

Reg: 100178897 22

CMP-6013Y

Figure 9: MG996r PWM

The library Pi.GPIO allows control of the duty cycle and the generation of specified

pulse widths, thus allowing motor control by varying the duty cycle. The method in

which the servo rotations are calculated is described in Section 4.2.9.

4.2.6 Servo and Arm control Implementation

As discussed in the previous section a look-up table will be used for servo values, so

they can be calculated once. A csv file is used to store the positions then these are read

into the Arm class.

The Arm class is used to control the servos. The methods init() and startPins() handle

the setup and initialisation of servo pints and stopPins() de-allocates and cleans the

pins up for the next use. The following process is a high level explanation of how the

servos are controlled with Pi.GPIO:

1. A pin object is assigned to a physical GPIO board pin according to the boardcom

numbering.

2. The Pins mode it set to output a signal.

3. Assign the pin a pulse width in Hz. According to the specification of mg996r -

1000ms/20ms duty = 50Hz.

Reg: 100178897 23

CMP-6013Y

4. Start the pulse on the pin.

5. The duty cycle is changed by passing a new duty value to the pin object.

6. Stop the pin, Cleanup GPIO registers for a clean exit.

The arm receives coordinates from the game engine, these must be converted into a

value in range of the pulse cycle. Firstly, the duty cycle setting the servos to the min (0

degrees), mid (90 degrees) and max (180 degrees) must be calculated. According to the

specification (ServoDatabase 2019) these values are 5ms, 10ms and 15ms respectively,

however after testing these values are incorrect probably due to using cheap motors

(see project notebook - Term 2 week 3).

By attaching a servo horn and using a protractor the duty value ranges were calculated

manually with: Min = 3.1ms Mid = 7.15ms Max = 11.2ms.

Using the max and min values the range and resolution of movement can be calculated.

Since the minimum change in duty cycle is 0.1ms the motors range is equal to Formula

(1). The range calculated shows the number of unique positions the motor can achieve

between 0 and 180 degrees, the resolution is the number of degrees each unique

position represents and is calculated in Formula (2). In an ideal circumstance the

motors would map one degree of rotation per unique position between 0 and 180

degrees, this would make the accuracy within 1 degree. To have a 1.1 degree resolution

the motors would need to accept a minimum duty cycle change of 0.045ms, which they

do not.

Range =11.2−3.1

0.1= 81 (1)

Formula (2) calculates the resolution over 180 degrees using output from (1).

Resolution =18081

= 2.2 (2)

Formula (3) calculates minimum change in duty cycle to get 180 degrees of revolution.

Range =8.1180

= 0.045 (3)

Reg: 100178897 24

CMP-6013Y

Using the value in Formula (3) a formula can be derived to convert a given degree, to a

duty cycle Formula (4).However, this formula assumes there are 180 unique positions,

when we know there are 81. Since the minimum increment of the duty cycle is 0.1 ms

the values will be rounded to the nearest 0.1ms. The offset is the duty cycle value

needed to set the servo to 0 degrees, thus offsets the formula to work with the mg996r

servos.

Duty cycle from Degree = degree×0.045+o f f set (4)

Each motor was tested to check min/max values. 3/6 complied, the remaining ones

were returned to the retailer. A batch of 5 more were ordered and the ones which

complied were kept.

The Formula in (4) allows angles to be calculated by a kinematic algorithm, and passed

directly to the arm.

The arm class has to complete several tasks: Move a piece from start to end position

and remove a piece from the board. The algorithms to complete these tasks are detailed

below: Figure 10 shows each servo being initialised. To change the duty cycle of a pin

the function .ChangeDucyCycle(ms) is used.

Figure 10: Initialising arm object and binding servos

Algorithm 1 gets the servo rotations for each motor and passes them to the moveArm

function. If both moves return true then the arm successfully moved a piece.

Reg: 100178897 25

CMP-6013Y

Algorithm 1 moveAtoB(Arm startCoord endCoord)1: Start← servoMoveDictionary(startCoord)

2: End← servoMoveDictionary(endCoord)

3: movedA← moveArm(Start[0],Start[1],Start[2],Start[3],1)

4: movedB← moveArm(End[0],End[1],End[2],End[3],0)

5: if movedA and movedB equal True then6: returns True

7: else8: returns False

9: end if

Algorithm 2 is responsible for picking up or putting down a piece depending on the

grabberState variable- 1 picks a piece up, 0 places it down. The code for this algorithm

is found in the Arm class. For the arm to pick a piece up correctly, the base must be

angled to the correct heading, then joint 2 must be set first, stopping the arm hitting the

board when joint 1 and 3 move into position. If joint 2 is set after, joint 1 and 3 may

place the arm below the level of the board, thus moving the end effector into a position

below the board- This is obviously impossible.

Reg: 100178897 26

CMP-6013Y

Algorithm 2 moveArm(Arm base j1 j2 j3 grabberState)1: base, j1, j2, j3← degreesToDuty(base, j1, j2, j3)

2: if grabberState equal to 0 then3: Arm.j1.ChangeDuctCycle(90 degrees)

4: Arm.j2.ChangeDuctCycle(90 degrees)

5: end if6: Arm.base.ChangeDuctCycle(base)

7: Arm.j1.ChangeDuctCycle(j2)

8: if grabberState equal to 1 then9: Arm.grabber.Open()

10: else11: Arm.grabber.Close()

12: end if13: Arm.j2.ChangeDuctCycle(j1)

14: Arm.j3.ChangeDuctCycle(j3)

15: if grabberState equal to 1 then16: Arm.grabber.Close()

17: else18: Arm.grabber.Open()

19: Arm.j1.ChangeDuctCycle(90 degrees)

20: Arm.j2.ChangeDuctCycle(90 degrees)

21: Arm.ResetArm()

22: end if

4.2.7 Computer Vision Design

This element of the project utilises OpenCv and various widely used computer vision

algorithms to determine what moves are made by the human player. There are a several

objectives for this section to be successful:

• Recognise the checkers board and calibrate the camera successfully so it only sees

the board. Formally- get the board perspective transform matrix.

Reg: 100178897 27

CMP-6013Y

• Extrapolate the outer corners of the board from the inner corners.

• Detect which squares on the board contain a piece.

• Detect when a piece is moved, start and end point.

A camera mounted above the board will be used to take the images. There are two

methods for capturing images - 1) using the Pi camera, this captures a low resolution

image that can be directly accessed by openCV as a web-cam 2) iPhone camera using a

RTSP (real time streaming protocol) application (Japan 2019), this approach is less

reliable as it requires a constant connection but the images are a much higher

resolution.

After comparing images (see project notebook- Term 2 week 8) the Raspberry Pi

camera was found to be more suitable. This is mainly due to the image file size, ones

taken on iPhone are roughly 4MB compared to the 100Kb images taken by the Pi

camera. This large increase in file size leads to a delay in image processing, as the

image contains a larger search space. Also the higher resolution image picks up more

noise, thus making the corners look more distorted. This is recorded in the project

notebook.

The purpose of calibrating the camera is to ensure the piece detection algorithm works

on images taken at different angles or heights, meaning if the project is taken apart and

rebuilt, as long as the calibration function succeeds the computer vision algorithm will

succeed. This also results in a consistent image to be used for the game engine, with

each corner of the chess board in the same place each time. For the camera to be

calibrated a picture is taken, then a perspective warp is calibrated, this changes all the

squares on the board to the same dimensions. This warp can later be applied to all

images, resulting in the board looking symmetrical, and giving a birds eye view.

Using corner detection, and camera calibration the board will be selected from an

image and checked for which tiles contain a square.

Reg: 100178897 28

CMP-6013Y

4.2.8 Computer Vision Implementation & Camera Calibration

The implementation of camera calibration is shown in Algorithm 3 and returns an

array of inner corners found using openCV’s findChessboardCorners, and a

transformation matrix using getPerspectiveTransform. The input and output is shown

in Figure 11, notice the output is a birds eye view with indicated inner corners whereas

the input is of a board from an off centre view. The transformation matrix returned by

Algorithm 3 can be applied to all images with the same perspective as Input and yield

the same perspective Output, without having to recalculate the transform again thus

improving performance. The Transformation matrix is saved to a file, to be used by the

camera class. The camera class will apply this warp to every image taken, meaning the

vision engine does not have to handle this transform each time, the input it receives is a

birds eye view of a board.

Figure 11: Example of perspective transform applied warped image

The algorithm is easiest to understand with an informal description alongside the

pseudo-code. Firstly the image is converted into gray scale, then the canny image

algorithm finds all edges within the image. This edged image is now scanned for

contours, which are sets of closed geometrical shapes, with only the largest contour by

area saved. If this contour has 4 edges then it must be a square and thus the board. A

new image size is calculated using the corners of the largest square, and the width and

height are calculated. Line 15 uses an openCV function to return a matrix given the

corners of the square and the desired new image size. Line 15 Takes the

Transformation matrix and returns a new image, with the new corners as the

chessboard corners. Finally the inner corners of the board are found on line 17.

Reg: 100178897 29

CMP-6013Y

Algorithm 3 getWarpedCorners(inputImg)1: grayImg← inputImg.convertToGrayscale

2: cannyImg← grayImg.findAllEdges

3: contours← cannyImg.findAllContours

4: largestContour← maxContour in contours by area

5: if largestContour has length equal to 4 then6: boardEdges← largestContour

7: end if8: topLeftCorner← boardEdge with smallest sum

9: bottomRightCorner← boardEdge with largest sum

10: topRightCorner← boardEdge with smallest difference

11: bottomLeftCorner← boardEdge with largest difference

12: square← topLeftCorner, topRightCorner, bottomRightCorner, bottomLeftCorner

13: Width ← maxDistance between topLeftCorner topRightCorner and bottomLeft-

Corner bottomRightCorner xcoord

14: Height ← maxDistance between topLeftCorner bottomLeftCorner and topRight-

Corner bottomRightCorner ycoord

15: transformationMatrix ← openCV.getPerspectiveTransform(square, newImg-

Corners)

16: warpedImg ← openCV.warpPerspective(inputImg, transformationMatrix, (Width,

Height))

17: listInnerCorners← openCV.findChessboardCorners(warpedImg, (7,7 ))

18: returns listInnerCorners, warpedImg

This function only returns the inner corner coordinates, for an 8x8 board there are 7x7

inner corners, next the outer corners coordinates must be calculated to give a full 8x8

board/9x9 list of corners. This is fairly straight forward, and relies on the board having

regularly sized squares, this is why the perspective warp is needed. The average x and

y distance between each inner corner is calculated. Using simple arithmetic the new

corners can be found. The 7x7 list is extended to a 9x7 list by duplicating the first and

last elements of each list in place. The y coordinates of these duplicated elements will

Reg: 100178897 30

CMP-6013Y

be correct as they are on the same row, however the first elements are decremented by

the average x distance, and the last elements are incremented by the average x distance

thus generating correct x,y coordinates. The same is done in the y dimension, the first

list and last lists are duplicated in place to create a 9x9 list, decrementing the first lists

y coordinates by the average y distance and incrementing the last lists y coordinates by

the average y distance. This method generates all the corners for an 8x8 board.

Once a complete set of board corners have been extrapolated, an algorithm to detect

which squares contain checkers pieces can be implemented. This algorithm uses a

widely used method, known as a Hough circle transform (OpenCV 2013). Algorithm 4

documents the implementation used in this project. Hough transform is used to

identify circles in the image with a max radius 10% more than half the size of a square,

this method aims to eradicate any false positives by ensuring only circles smaller than

the squares are detected. The algorithm returns a list of circles, which are then sorted

from top left to bottom right row by row. The list of circles contains 3 values x

coordinate, y coordinate and radius.

Algorithm 4 findCircles(cameraImg)1: height, width, channels← cameraImg.shape

2: maxCircleRadius← half average(height+width) + 10%

3: gray← cameraImg.ConvertToGray

4: blur← 5pxBlur(gray)

5: circles← HoughCircles(blur,minimum radius=1, max radius = maxCircleRadius )

6: orderedCircles← circles ordered by x then y

7: returns8: orderedCircles

The location of pieces on the board can now be determined. Using the calculated 8x8

corners and the detected circles, an algorithm detectPieceLocation is implemented to

return a list showing which squares have a piece inside. Squares not containing pieces

are ignored, as they are not useful.

Reg: 100178897 31

CMP-6013Y

Algorithm 5 detectPieceLocation(orderedCircles, cornerList)1: Dictionary populatedSquares

2: for each corner in cornerList do3: tlx← top left x coordinate of corner

4: brx← bottom right x coordinate of corner

5: tly← top left y coordinate of corner

6: bry← bottom right y coordinate of corner

7: for each circle in orderedCircles do8: if tlx < circle x < brx and tly < circle y < bry then9: corner← this corner contains a circle

10: populatedSquares.add(corner)

11: else12: continue13: end if14: end for15: end for16: returns populatedSquares

Now that the computer vision can detect the board, calculate the corners, and

determine which squares contain pieces, this information can be used by the game

engine to determine where a user moves. Checking if a user has taken a move is as

easy as running the detectPieceLocation algorithm every second (or n seconds), as

when a user moves a piece the new location of pieces will be updated. In line with the

official competition rules, players should press a timer indicating the end of their move,

so this could be another approach, as this would ensure the game engine knows exactly

when to look for a humans move, as their move is explicitly stated. However the

approach described in the paragraph means this is not helpful, as only moves that are

valid will be considered as a players move. So if an illegal move is made the game

engine will not be updated, and will continue to wait for a valid move to be made.

The way in which a players move is detected is one of the benefits for the chosen game

engine. The engine provides a list of all possible start and end locations for the human

Reg: 100178897 32

CMP-6013Y

player in a 2xn list, each row in this list is called a move. The moves, rows, are

formatted as follows :[[startx,starty][endx,endy]]. To find the move a player made the

xy coordinate of every player piece on the board is calculated using Algorithm 5 (after

the player has moved), the piece that has moved can only have moved to a location

within the list of moves corresponding to an [endx][endy] value, so these values are

compared to the populatedSquares list returned by the detectPieceLocation algorithm.

Using the known end position of the moved piece the list of total moves possible can

be narrowed down to ones only containing the correct end location. There may now be

one or more moves left, to check which move is correct the board squares

corresponding to [startx][starty] are checked for piece occupancy, if they do not

contain a piece then this is the correct start point, otherwise it contains a piece so the

move must not have originated here- look at the next start values in the list. Figure 12

demonstrates this in operation. The value returned is the players start and end move.

Figure 12: Checking for movement on board

Reg: 100178897 33

CMP-6013Y

4.2.9 Kinematics Design and Implementation

Kinematics is defined as "the geometry of motion" and is the system used to calculate

the angles for each link in the arm chain. Kinematics solves the problem of mapping

the end-effector (EE), claw , to a desired point on the board, using known arm link

lengths and known distance to pieces. The kinematics algorithm used to calculate the

servo angles is adapted from an open source 3DOF planar arm project (Qrobotics

2018). Planar arm means the arm works in one plane, in this case the Cartesian plane.

Figure 13 Demonstrates the arm used in this project, the angles in the right image are

the ones that need to be calculated to correctly position the arm over a given coordinate

in the Cartesian plane and the left image shows the degrees of freedom of each link.

The arm uses two algorithms to calculate the servo rotations needed to pick a piece up.

The first algorithm is designed by myself, and the second one takes in the lengths of

the links, along with a desired x,y coordinate and EE angle and returns the value in

degrees in which each motor must be set to in order to reach the x,y coordinate relating

to a square on the board.

Figure 13: Left: Arm with rotational constraints. Right: Planar manipulator

The first algorithm is used to point the arm in the correct direction to a given square,

thus eliminating the 3rd dimension and creating a problem that can be solved on a

Reg: 100178897 34

CMP-6013Y

single plane. Figure 14 shows D as the desired end direction, and angle A is the angle

required to move the arm from 90 degrees to A. The function returns a map, see Figure

14, of all the A angles required to face the centre of each square on the board. The

angle A can be calculated with trigonometry and finding the hypotenuse is easy since

the opposite and adjacent sizes are known as each square is the same size. The values

for x and y list used in Algorithm 6 are known as each square is the same size, so to

calculate the adjacent side the column number is incremented by 3cm (distance

between two square centres) per column, the same is done with rows to find the

opposite side length.

Once the correct angle for A has been calculated, the problem can be solved using the

3DOF planar kinematic function. The function needs an x and y coordinate with the

origin at the base of the first joint, this is demonstrated in 13, the length of the

hypotenuse is used for the x value as it sits this distance from the base. The y value is

consistent, and is the height of the first joint from the board, meaning it is always

negative. The algorithm returns a list of 3 servo values that are stored to a csv, joint 1, 2

and 3. These values contain the rotations to reach each square on the board. Due to

error in accuracy of the servos, these values do not correctly map to the real world ones,

therefore many values have been manually tested and adjusted so the correct angles can

be passed to the arms movement function. This is a huge benefit of working out values

and storing them, as opposed to calculating on the fly. The inverseKinematics function

in the movement class shows a formal implementation along with the output values.

Figure 14: Birds eye view of arm over board and ZMap

Reg: 100178897 35

CMP-6013Y

These two functions provide the values required for each of the 4 servos to move the

end effector to a square on the board. The kinematics function is not described as it is

adapted from an open source project, therefore this code was not written by me due to

the complexity of the problem.

Algorithm 6 getZmap()1: x← list of x distances

2: y← list of y distances

3: ZCoordinateMap

4: for each square in board do5: opposite← x[square]

6: adjacent← y[square]

7: hypotenuse← squareRoot(opposite2,ad jacent2)

8: A← inverseCos(adjacent/hypotenuse)

9: ZCoordinateMap.add(A) at current square

10: end for11: returns ZCoordinateMap

4.3 Hardware Components Design and Implementation

4.3.1 Parts List

This project set out an initial budget of £150 maximum, this list details all parts

purchased and their price. Links to the pieces purchased are contained within the

notebook resources section.

• 6DOF Arm = £50

• Raspberry Pi = £34

• 5 Mg996r servo motors = £6 x 5

• Pi Camera + Cable = £10

• Wires + Breadboad + Power Supply = £30

Reg: 100178897 36

CMP-6013Y

• Player pieces = £6

Therefore the total cost of all parts is £160 placing it slightly over budget.

4.3.2 Raspberry Pi + Raspbian

The whole system needs some way for the elements to be connected together, and for a

central controller. Although the game engine handles the software side, there needs to

be an operating system running the python application and interfacing with the servos

and camera. The two main considerations for this were Raspberry Pi and Arduino.

An Arduino can be connected to any computer via USB and runs on any operating

system acting as a hardware controller for servos, however this must be controlled

using C ++. This is not convenient as the Game engine, and computer vision modules

are written is Python. Because of this the Arduino is not helpful, For this reason a

Raspberry Pi is the best choice of controller.

A Raspberry Pi is a small modular computer with a GPIO board and other ports. The

Raspberry Pi is chosen as it has ample processing power and can be moved around

easily, as well as being capable of running a fully fledged operating system. Raspbian

is the chosen operating system for this project and is a lightweight Linux distribution

running a terminal based interface. Python and openCV can be installed using a

package manager (PIP). A Virtual environment is is created for the project which

allows dependencies to be installed separate to site packages for the OS.

4.3.3 Power Supply & Servo interface

Each servo requires 4.8v to operate. A Raspberry Pi has a power pin, however the

current supplied is not enough to power each servo. Using the Pi the servos jitter,

because they are under-powered, this is not acceptable as it creates unreliable PWM

pulses.

Therefore to supply the 4.8v to each servo an Elego MB102 power supply is used,

information about this is in the notebook resources tab. This can output 3.3v or 5v and

Reg: 100178897 37

CMP-6013Y

has three separate outputs, each acting as a separate source, this is helpful if the servos

draw too much current and this effects the operation.

The servos are wired as seen in Figure 15. The left image shows the power supply

attached in series. The yellow cable is attached to the ground pin on the Raspberry Pi

for use with PWM. The middle image shows the wiring for the servos, red wires are

power, green are ground and the other colour is the signal wire, connected to the

Raspberry Pi GPIO board. The right image shows the servo signal pins connected to

the Raspberry Pi, 5 in total. This setup provides a stable current to each motor allowing

for consistent motor control.

Figure 15: Ground and power leads to servo

4.3.4 Arm

As discussed in the research section, there are several robot designs to be considered,

however the most cost effective solution is the robotic arm. This also is expandable for

use in other projects after the completion of this and fits in with my motivation for this

project. The arms dimensions are shown in Figure 16.

Reg: 100178897 38

CMP-6013Y

Figure 16: Measurements of the arm

The arm used for this project was purchased from an online retailer, AliExpress. The

arm comes disassembled, and without servos, these were purchased separately. Once

assembled, the arm measures 36cm, and when accounting for the base has a max reach

of 31.5cm. This information is useful for the board design discussed in the following

subsection. The original arm was modified, a servo was removed, as there was too

much strain placed on joint one and two resulting in unreliable movement. Figure 17

shows the original arm, then the arm with the extra link removed. The removed link

allowed rotation of the claw, however this was not applicable to this project, thus

determined to be a waste of weight. This removal was straight forward since all the

parts contain pre-drilled holes, the claw was simply turned around and remounted, in

line with the centre of the arm. The only difference is the orientation of the servo, in the

altered design the servo sits atop the grabber, however this does not impede function, it

also allows the arm to reach closer to itself, meaning it can sit closer to the board.

Reg: 100178897 39

CMP-6013Y

Figure 17: Remounted grabber

4.3.5 Board

The board is designed to fit the arm specification. The max reach of the arm is 31.5cm,

therefore the board must not exceed this in any dimensions. The board size decided

upon is 24cmx24cm, leaving a maximum of 7.5cm spare reach for the arm.

Using a 24cm board each tile is 3cm wide, which is conveniently slightly larger than

the counters purchased at the start of the project.

One important consideration is the colour of the board, since the computer vision

aspect of the game relies on detecting the chessboard pattern, to calibrate the camera,

as well as the player counters during play. Three boards were considered and these are

shown in Figure 18.

Figure 18: Boards tested

All of the three boards function correctly when tested with digital copies, however

when tested on printed boards only the right two images work. This is because the

algorithm used to detect the board first converts the image to grayscale, the grays on

Reg: 100178897 40

CMP-6013Y

the left board are too similar, thus making the algorithm fail. Figure 20 demonstrates

this on a printed board taken by the Raspberry Pi camera. The border around the

boards is used in Algorithm 3, and is detected as the largest contour. This helps with

camera calibration. Therefore the black and white with a red border seems like the best

fit. When placing the arm this border also has to be taken into account, for this reason

the arm must not cover the border, this is why there is a space between the base and the

board.

Figure 19: Grayscale red and green board

The board design must also take into consideration the colour of the players pieces.

The pieces purchased at the start of this project are Black and Red, this sometimes

causes some error in detection, as the black pieces do not contrast with the board well

enough depending on the light. For this reason white and red pieces may be chosen for

the final demonstration, as it allows sufficient contrast between the board tiles and the

pieces. Other colours could also be used, for example brown/white.

Figure shows the completed board design. This image is taken by the Pi camera, it is

clear there is some difficulty in recognising the pieces on the right of the board. This

demonstrates why more contrasting colours are needed.

Reg: 100178897 41

CMP-6013Y

Figure 20: Board with Black/Red pieces

5 Testing and Full system

5.1 Testing

Due to the modular nature of this system testing was carried out whilst developing

each module. This allowed for any problems to be detected early, and before the whole

system was integrated together. Each section of the system had one point of interaction

with each other part, thus allowing errors to be detected and resolved more easily.

5.1.1 Game Engine Testing

The game engine itself was a working project when downloaded, so the functionality

of the game was not thoroughly tested. Several games were played to make sure the

game was working correctly. The testing for the game engine was mainly to ensure the

correct values were being passed to the arm when the player and the computer took

their turn.

Figure 21 shows the values passed from the game engine to the arm, and then the CV

module to the game engine. Both work as intended.

Reg: 100178897 42

CMP-6013Y

Figure 21: Game engine passing and receiving values

5.1.2 Computer vision testing

Some brief testing was described in the board section of this report, however further

testing is required to ensure the camera can recognise the board repeatedly, and detect

moved pieces. This testing was carried out by running the Computer vision algorithm

separately, then testing for movement detection on the board. There were few problems

with this since the environment tested in is well lit, however errors occur when the light

level is low, or too much glare from the board confuses the contour detection algorithm.

To test the accuracy of the Vision, it was first tested on digital images, shown in Figure

22, then images taken by the pi camera, shown in Figure 23. The sets of images below

show the testing of several images, the annotation describes what has been done to the

image. The corners found by the vision algorithm are plotted to give a visual

representation. Further testing is shown in the project notebook, term 2 week 8,

including testing of the piece recognition algorithm. This testing was limited, as it

works consistently as long as the calibration succeeds, the pieces are recognised as

long as the lighting conditions are good.

Reg: 100178897 43

CMP-6013Y

Figure 22: Testing Digital board

Figure 23: Testing Pi camera image of board

5.1.3 Arm Testing/Servo

Each servo was initially tested by placing it in the centre of a protractor with a pencil

attached to find the max and min the range, then to determine the accuracy. To test the

arm, each position on the board was moved to using the move arm function and

iterating through all pieces on the board. Some positions further from the base are less

reliable, this makes sense as the error is accumulated the further from the base the arm

moves. This testing led to a slight change in the position of the arm, originally the arm

would sit between the human and robot players, now it sits behind the robot. This also

Reg: 100178897 44

CMP-6013Y

improves accuracy as the claw collides less with adjacent pieces. More testing images

of the arm are shown in the project notebook. The Arm was also tested by picking a

piece up from each location, shown in Figure 24.

Figure 24: Testing arm and grabber

To test the servo accuracy a protractor was used, this method is shown in Figure 25.

This method was used to ensure all servos had the same tolerances and min/max

values. This meant each servo sets to the same degree based on the duty cycle written,

this means less error is introduced and attempts to make the kinematics algorithm work

first time, without too much adjustment.

Figure 25: Testing servo accuracy

5.2 Integrated system

All elements of the system are connected to the Raspberry Pi and tested to make sure

they work together. The arm and computer vision functions correctly, however a

change in light can affect the accuracy of the detection of the pieces, this is not

common though. To test the system working together the game is played a number of

times, making sure to take different starting moves. Testing of the complete system

Reg: 100178897 45

CMP-6013Y

shows the software works reliably, however the hardware lets the project down, as the

motors are low accuracy. Overall the integration of the system went smoothly as each

element was engineered to fit together, with as little information passed between each

module as possible. By doing this points of failures can be identified and controlled,

making bug fixing easy and passing control around the system a straight forward

process.

6 Evaluation

To gauge the success of this project the initial aims can be compared to the achieved

outcome.

This project met all aims defined in Section 1.1:

• A robot arm can pick up and move checkers pieces.

• Inverse kinematics was used to build a dictionary of rotations for each servo.

• A computer vision algorithm recognises the board and tracks players moves.

• A chess engine sends movements to the arm to control it, as well as the running

of the game.

• All components are linked together allowing a human to play a robot at a game of

checkers.

Each of the criteria stated at the start of the project were met, therefore this can be

considered successful.

The project took roughly 170 hours to complete. I believe this could have been cut

down. The project initially set out to play Scrabble, then Chess, before settling on

checkers, therefore time was spent researching these before starting the build of the

checkers robot. The chart in Figure 26 shows a rough record of time spent each week

and a rough guide to the type of work completed that week.

Reg: 100178897 46

CMP-6013Y

Figure 26: Record of time spent on tasks

Throughout the project time was allocated to three main tasks, Research, Report

writing, and building or coding. The first section of the project focused more heavily

on researching current systems and ways in which to implement each section, this

meant when the design and implementation phase started there would be less errors.

Progress was delayed due to the arm taking a long time to arrive, however this left time

to start the computer vision side of the project. This led up till Christmas. After

Christmas the arm arrived so progress started on this, whilst spending some time

implementing the vision. The arm and the vision module were completed at almost the

same time, and because each element had to be continuously tested as it was being

designed the complete modules had very few issues. Well the arm had accuracy issues,

but these couldn’t be solved by software. The game engine took the shortest amount of

time to modify, but finding a suitable one to use was a lengthy task, this is since one

with specific elements was needed. After everything had been completed the individual

modules were bought together and combined, they worked fairly reliably with a few

hiccups. I believe progress in the first term may have been slower due to the lack of

communication with my advisor, however in the second term i had regular meetings

which helped solve issues and allowed me to ask questions about progress.

6.1 Limitations of the design & Addressed issues

The completed system contains some issues and limitations. Appropriate action to

resolve these could be implemented with further development. However, due to time

and budget these cannot be resolved in the final version of this project.

Reg: 100178897 47

CMP-6013Y

• The low accuracy of the arm sometimes results in a piece not being correctly

lifted. This could be solved using more accurate servos, however the budget for

this project would not cover the cost.

• Grabber consistency, due to the grabber being made from metal it sometimes

knocks adjacent pieces, this is rare however still a limitation of the design. The

best strategy to mitigate this is to use smaller pieces, or a more precise claw. Fu-

ture development would include a claw that has smaller tolerances, and a smaller

footprint. Meaning it would be less clumsy.

• During low light, or high glare lighting the computer vision accuracy is dimin-

ished. However as long as the camera is pre-calibrated this is usually not an issue.

This is detrimental when the lighting conditions are not ideal for calibration, one

recommendation is possibly using a manual calibration function.

• An issue with the pieces is their size relative to the board squares, when the grab-

ber reaches down to lift a piece, it sometimes collides with the adjacent pieces.

To solve this smaller pieces could be used. The pieces used in the demonstration

may be different to ones pictured in this report.

• Servo jitter caused by high strain on the arm, causing the potentiometers to be

overloaded and the rotational position to slip from a desired location.

6.2 Future Work

With more time, or budget this project could be adapted in many ways. Some of the

future developments are stated in this section:

• Adapt to play other games. By using the same piece detection method this project

could be adapted to play chess fairly simply, as long as a chess game works in a

similar way (by listing best moves). This may require an improvement of accuracy

though.

• Improve motor accuracy. By getting more accurate servos this project could be

adapted to carry out more high precision tasks.

Reg: 100178897 48

CMP-6013Y

• Change computer vision method. The method works with optimal lighting, how-

ever by using calibration points on the board this accuracy could be improves.

Similar techniques to QR codes can be used to orient and calibrate the board, this

removing the reliance on the detect chessboard algorithm.

• Robot vs robot. By using another arm, or the same one the robot could play

against itself, this could be an entertaining item to have on a coffee table and

would be an interesting development.

• Change in arm position. By moving the arm above the board the strain on motors

may be improved, this would remove much of the servo jitter caused by strain on

the arm.

6.3 Conclusion

In conclusion, the project achieved its end goals. Some issues were identified and there

were a few setbacks, but these did not drastically change the project outcome. The

project allowed me to gain experience in the areas of robotics, kinematics and

computer vision. The overall project is something that can easily be adapted to play

other games or provide other functionality in the future.

The motivation for this project stemmed from personal interest and also wanting to see

what current consumer grade robotics is capable of. This project proved that for as

small as £150 a robot can be designed and constructed to play a game of checkers. This

makes me wonder what could be possible in the near future and shows the capability of

low cost electronics. Could the average person have some sort of robot in their home,

in the coming years, to complete trivial tasks and if so this could greatly impact certain

people, like ones with disabilities. This field is yet to be explored in much depth and

I’m excited to see what comes of it in my lifetime.

7 Gantt Chart

Reg: 100178897 49

CMP-6013Y

Proj

ects

ched

ule

show

nfo

re-v

isio

nw

eek

num

bers

and

sem

este

rwee

knu

mbe

rs

89

1011

1213

1415

1617

1819

2021

2223

2425

2627

2829

3031

3233

3435

3637

3839

4041

12

34

56

78

910

1112

CB

12

34

56

78

9E

B10

1112

1314

Proj

ectp

ropo

sal

Lite

ratu

rere

view

Plan

ning

/Res

earc

h

Cod

eC

VM

odul

e

Cod

eA

rmC

lass

Cod

eG

ame

engi

ne

Test

ing

Pro

ject

deliv

ery

Dem

opr

epar

atio

n

Figure 27: Final Project Gantt chart

Reg: 100178897 50

CMP-6013Y References

References

abz.me (2019). The pigpio library. URL: http:// abyz.me.uk/ rpi/ pigpio/ .

Arduino.com (2016). Recommendation for High Torque Servo on Robotic Arm. URL:

https:// forum.arduino.cc/ index.php?topic=394272.0.

Banerjee, Nandan (2019). Buttercup Chess Robot. URL: http: // www.nandanbanerjee.

com/ projects/ buttercup-chess-robot .

Canny, J (1986). “A Computational Approach to Edge Detection”. In: IEEE Trans. Pat-

tern Anal. Mach. Intell. 8.6, pp. 679–698. ISSN: 0162-8828. DOI: 10.1109/ TPAMI.

1986.4767851. URL: https:// doi.org/ 10.1109/ TPAMI.1986.4767851.

Honda (2019). the world’s most advanced humanoid robot2019. URL: https: / / asimo.

honda.com.

Japan, Kazuki Yoshida (2019). Live-Reporter Security Camera. URL: https : / / itunes.

apple.com/ us/ app/ live-reporter-security-camera/ id996017825?mt=8.

JPL, NASA (2019). rovers arm 2019. URL: https:// mars.nasa.gov/ mer/ mission/ rover/

arm/ .

Lowe, David G (1999). “Object recognition from local scale-invariant features”. In:

Object Recognition.

Meyer, Joey (2019). Raspberry Turk. URL: http:// www.raspberryturk.com.

MIT (2019). MIT biomimetics robotics Lab. URL: https:// biomimetics.mit.edu.

OpenCV (2013). Hough Circle Transform. URL: https : / / opencv - python - tutroals .

readthedocs.io/ en/ latest/ py_tutorials/ py_imgproc/ py_houghcircles/ py_houghcircles.

html .

Reg: 100178897 51

CMP-6013Y References

OpenCV (2019). OpenCV Python camera calibration. URL: https:// opencv-python/ py-

tutorials/ py_calibration.html .

Paul, Richard P (1981). Robot Manipulators: Mathematics, programming, and control;

the computer control of robot manipulators. MIT.

Qrobotics (2018). Forward and Inverse Kinematics with 3 DOF Planar Robot. URL:

https:// www.youtube.com/ watch?v=VohqXRskHgs.

RobotShop (2019). Cartesian Robot. URL: https:// www.robotshop.com/ uk/ makeblock-

xy-plotter-robot-kit-v20-electronics.

ServoDatabase (2019). TowerPro MG996R Servo. URL: https: / / servodatabase.com/

servo/ towerpro/ mg996r .

Szeliski, Richard (2011). Computer Vision. Springer-Verlag London Limited.

TheDuckCow (2019). Tumour processing with openCV. URL: https:// theduckcow.com/

dev/ opencv/ cancer-tumor-processing/ .

Reg: 100178897 52