networking coroware robot - florida gulf coast...

1

Networking CoroWare Robot

Emily Blatz

Carly Hessler

Nowele Rechka

Draft 4

December 1, 2014

CNT 4104 Software Project in Computer Networks

Instructor: Dr. Janusz Zalewski

Software Engineering Program

Florida Gulf Coast University

Ft. Myers, FL 33965

2

1. Introduction

The CoroWare Robot (Fig. 1 and 2), which is further referred to as the CoroBot, is an

educational robotic device. Previously, work (Fig. 3 and 4) was done to control the CoroBot via

website and voice commands via the Kinect. The current project on this device is a continuation

of four known previous projects [1]-[4].The primary technologies we aim to use to this purpose

are web-based technologies as well as the drivers behind the XBox 360 controller (XInput [3]).

The purpose of extending the previous work is to provide more mobile capabilities to control the

CoroBot with. By mapping its controls to the XBox controller, using a computer (monitor,

mouse, and keyboard) or website will not be as necessary, reducing most of the hardware

components to options.

The benefits of this project include learning ways to extend and maintain a project with legacy

behind it, which is something that the members of this project have not experienced much before

this. This is a core part of being a Software Engineer so it is imperative that the team learn how

to do this.

Figure 1 - CoroWare CoroBot as given by the manufacturers

3

Figure 2 - CoroWare Robot at start of project (October 2014)

4

Figure 3 - GUI from previous project [2]

5

Figure 4 - CoroWare Robot with Kinect

6

2. Software Requirements Specification

The goal of this project is to repair and improve the relevant portions of the previous projects

[1]-[4] and then expand it with additional control capabilities. The original website is outdated

and the Kinect functions do not work consistently or correctly. Our aim for the current project is

to get the original website up and running again, debug and upgrade the Kinect functions, and

introduce controlling it via the XBox 360 controller.

2.1. Physical Diagram

As shown in Figure 5, the CoroBot is currently connected to a Kinect, monitor, keyboard,

and mouse that a user can use to control the CoroBot. Speakers can be attached to hear

CoroBot response sounds. These devices can be connected physically to the CoroBot or

wirelessly via the website. The CoroBot is already supplied with Power and Reset

buttons, battery power type switch, and a battery supply outlet in its standard design. For

this project, an XBox 360 controller is being added to enhance the capabilities to control

the CoroBot. The point of this extension is to provide a form of control that is mutually

exclusive to having the CoroBot connected to a monitor, mouse, and keyboard or through

the website. The user will be able to choose to incorporate almost any combination of

controls to operate the CoroBot, an example of this would be:

● voice commands via Kinect to engage the CoroBot, tilt the camera, and move the

CoroBot in a certain direction

● motion commands via the XBox controller to move the CoroBot in a certain

direction

● monitor the path the CoroBot is taking via the Kinect camera (that can be

connected directly to the CoroBot or streaming from the website).

7

Figure 5 - Physical Diagram of Networking CoroWare Robot

8

2.2 Context Diagram

As shown in Figure 6, which follows the physical diagram in Figure 5, the CoroBot

Software can receive commands from the XBox controller, mouse, keyboard, and Kinect

(via voice command) to change the CoroBot’s directional motion and Kinect camera tilt.

As shown on a Use Case Diagram in Figure 7, a user can:

● move the joysticks and press the buttons on the XBox controller

● use the mouse and keyboard to select and type options within the GUI

● issue voice commands via Kinect.

The CoroBot receives video and depth from the Kinect camera which can be viewed on a

monitor that is either physically connected to the CoroBot or streaming from the website.

The CoroBot outputs response sounds via speakers when a command is received.

9

Figure 6 - Context Diagram of Networking CoroWare Robot

10

Figure 7 - Use Case Diagram for CoroBot inputs

This use case diagram shows the interactions between the user and the system,

which affect the CoroBot. The user has the choice of inputting commands in three

ways: opening the GUI and using the mouse and keyboard, entering an action

through the Xbox controller, or giving a voice command via Kinect. Any one of

these three actions will give instructions for the CoroBot to execute.

11

2.3. Software Requirements

Following the CoroBot Software interactions with external devices, as shown in Figure 6,

software requirements can be formulated as stated below. It is assumed that the CoroBot

Software will boot when the CoroBot is running on battery power and the power button is

pressed. It is assumed that the CoroBot has a built-in capabilities for audio.

2.3.1. Input Requirements

2.3.1.1. The CoroBot Software shall handle and store a Direction and Distance

received via the mouse, keyboard, XBox controller, or Kinect voice command for

its next motion step.

2.3.1.2. The CoroBot Software shall handle and store an Angle received via the

mouse, keyboard, XBox controller, or Kinect voice command for its next tilt

move.

2.3.1.3. The CoroBot Software shall handle video and depth received from the

Kinect camera to display to the user.

2.3.1.4. The CoroBot Software shall handle a component’s power setting received

via the mouse, keyboard, XBox controller, or Kinect voice command.

2.3.2. Output Requirements

2.3.2.1. The CoroBot Software shall acknowledge that a Kinect voice command

was received with a response sound.

2.3.2.2. The CoroBot Software shall display to the Monitor the command it will

execute next before executing that command and until the next command is

acknowledged and displayed. As shown in Figure 3, the format of the displayed

command is: “Commands: [current command]”.

2.3.2.3. The CoroBot Software shall move the CoroBot in a Direction for a

Distance stored as per Requirement 2.3.1.1.

2.3.2.4. The CoroBot Software shall tilt the Kinect camera at an Angle stored as

per Requirement 2.3.1.2.

2.3.2.5. The CoroBot Software shall stream video and depth from the Kinect

12

camera to the Monitor or website.

2.3.2.6. The CoroBot Software shall engage or disengage a component’s power as

per Requirement 2.3.1.4.

13

3. Design Description

3.1. Software Architecture

As shown in Figure 8, which is consistent with the Context Diagram in Figure 6, there are

four core modules for the CoroBot Software, three of which handle the possible inputs

for the CoroBot to receive commands. The GUI allows users to input commands via

mouse and keyboard that move the wheels or Kinect (in the previous projects [1-4] the

interface also moved the arm, but this no longer functions). The Xbox Controller Module

allows users to press buttons and move the joysticks to move the wheels or Kinect, but

not with specific numerical value inputs (unlike the Kinect and GUI). The Kinect Module

allows for voice commands to control the wheels, as well as modify where its own

camera is facing.

14

Figure 8 - Software Architecture of the CoroBot Software

15

3.2. Detailed Design

3.2.1. Static Perspective

The static perspective of the design is shown in Figure 9. The CoroBot Software

is dependent on the use of several different control systems, including the Xbox

controller, voice commands, and a GUI. The GUI is dependent on inputs from the

mouse and keyboard.

Figure 9 - Class Diagram of CoroBot Software

16

3.2.2. Dynamic Perspective

The dynamic perspective of the design is shown in Figure 10. The CoroBot

receives input from four sources: the mouse, the keyboard, the Xbox controller,

and voice commands. Each input prompts the CoroBot to perform an action.

Figure 10 - Sequence Diagram of CoroBot inputs

17

4. Implementation and Testing

The CoroBot has been tested and proven to turn on for a short amount of battery power, respond

to a limited number of voice commands via the Kinect, as well as successfully move its wheels.

The video streaming functions properly and responds to tilt commands. Unfortunately, the

robotic arm does not respond to any currently working programs. As such, we will not focus on

the robotic arm as maintenance appears to be too complicated for a single semester. Being that

all of what is desired to function in the CoroBot does so, work has begun on mapping CoroBot

controls to the XBox 360 controller and expanding the existing set of Kinect voice commands.

CoroBot’s functionality can be proven with the following tests:

4.1. Motion commands and component power setting commands provided to CoroBot are

displayed to the monitor before execution.

4.1.1. Per requirement 2.3.2.2.

4.2. Using the XBox controller directional pad (Dpad) buttons to make the CoroBot move

forward (up button), backward (down button), left (left button), or right (right button).




4.3. Using a voice command and ensuring the CoroBot performs a response sound.





4.4. Pressing an XBox controller button (Dpad buttons, A, B, X, Y, or triggers) and

ensuring the CoroBot performs a response sound.





4.5. Pressing an XBox controller letter button (A, B, X, or Y) and ensuring the

corresponding CoroBot component is enabled/disabled.

18


4.5.2. Per Requirement 2.3.1.4.

4.6. Using the XBox controller triggers to move the Kinect camera up (right trigger) or

down (left trigger).



19

5. Conclusion

With the completion of the project, the CoroBot is now controllable via mouse, keyboard, XBox

controller, and voice command. Each control system can work exclusively of the others or in

conjunction with the others. Currently, the XBox controller allows for components like the

motors, arm function, camera, and depth sensor to be engaged or disengaged via the letter

buttons A, B, X, and Y respectively; the DPad provides motion control at a standard distance per

direction; and the camera can be tilted up or down via the right and left triggers respectively. The

CoroBot now responds to all received commands with a response sound and displays all received

commands to the monitor. The CoroBot video can be viewed on an attached monitor via the GUI

or remotely via its Twitch account.

The biggest difficulty in implementing the XBox controller was developing a timer to check on

the controller button states continuously without interfering with the existing CoroBot software.

A manual, non-static timer had to be developed that could check various buttons for press,

pressure, or direction. Another issue we faced was finding the previous projects’ website. Due to

the fact that the website that was supposed to provide another way to control the robot did not

exist any longer, it was not possible to upgrade it. In order to be able complete the main portion

of the project, the XBox control system, and still provide remote streaming via website, we chose

to re-create the web-streaming service that existed in previous projects by implementing a

Twitch web-streaming account on the CoroBot. A constant problem in this project was the

battery life of CoroBot. With the provided hardware, CoroBot charges on tether and then runs on

battery for only 2-3 hours. This is a difficult way to develop and test code, as there is no battery

status or warning of how long it will last.

Further work that can be done on this project include repairing the arm functions, which may

only be a hardware issue rather than a software issue. Within our working XBox controller code

we have provided some skeleton code to eventually integrate arm functions with the XBox

controller capabilities. An expansion suggestion would be to allow for multiple XBox controllers

to power certain sections of the robot; for example, one controller would be for motion and

camera controls while another would be for arm controls. Each section would be dedicated to a

particular controller index, so only player one could control motors for example. Another route

20

would be to focus on a new website to control CoroBot remotely, rather than attempting to

upgrade an expired website. Future projects would also greatly benefit from changing the power

supply, as the primary issues faced during this project stemmed from the lack of a power tether.

21

6. References

[1] A. Fernandez, V. Fernandez and T. Nguyen, "Robotic Control with Kinect Vision

Comprehensive Report," Florida Gulf Coast University, 2013. URL:

http://itech.fgcu.edu/faculty/zalewski/projects/files/FernandezCorowareWithKinectDraft2

.pdf

[2] V. Fernandez, “CoroWare CoroBot Wireless Connectivity (Draft #8),” Florida Gulf Coast

University, 2012. URL:

http://itech.fgcu.edu/faculty/zalewski/projects/files/Victor_Corobot_V8.pdf

[3] S. Caguana, A. Meyers, “Networking CoroWare Robot (Draft 4),” Florida Gulf Coast

University, 2013.

[4] S. Caguana, V. Giannone, “CoroWare Robot: Networking and Remote Upload,” Florida Gulf

Coast University, 2014.

[5] Microsoft, “XInput Game Controller APIs (Windows),” Microsoft, 2014. URL:

http://msdn.microsoft.com/en-us/library/windows/desktop/hh405053(v=vs.85).aspx

networking coroware robot - florida gulf coast...

Documents