design, analysis, and construction of the rpuav …larry.aamodt/beckner/alexander_beckner... · 5.1...
TRANSCRIPT
1
Rosario ROV
Rosario ROV
Alexander Beckner
Advisor(s):
Dr. Larry Aamodt
Dr. Delvin Peterson
Dr. Kirt Onthank
2
Rosario ROV
3
Rosario ROV
Abstract
This paper contains the designs of the electrical portions of the underwater remotely operated
vehicle (ROV) as well as the processes used. The electrical portions include the ROV’s onboard
computer, code running on the ROV’s computer, communication with the peripheral sensors,
communication between the ROV and the surface, the camera, the code and software used for
operation of the camera, the camera feed, and the development of the GUI and the software used.
This paper is to be used as a summary of my work and a reference for future projects on the
ROV.
4
Rosario ROV
Table of Contents
ABSTRACT ................................................................................................................................................ 3
TABLE OF CONTENTS ........................................................................................................................... 4
TABLE OF FIGURES ............................................................................................................................... 8
LIST OF TABLES ...................................................................................................................................... 9
ACKNOWLEDGEMENTS ...................................................................................................................... 10
1 INTRODUCTION AND OBJECTIVES ........................................................................................ 11
1.1 Introduction ............................................................................................................................................... 11
1.2 Project Objectives and Requirements ......................................................................................................... 11
2 ROSARIO ROV COMPUTER ....................................................................................................... 12
2.1 Onboard Computer ..................................................................................................................................... 12
2.1.1 Microcontroller ......................................................................................................................................... 13
2.2 Beaglebone Black (BBB) .............................................................................................................................. 14
2.2.1 BBB Hardware ........................................................................................................................................... 14
2.3 Development on the BBB ............................................................................................................................ 15
2.3.1 Setting Up the BBB ................................................................................................................................... 15
2.3.2 Secure Shell into BBB ................................................................................................................................ 16
2.3.3 BBB Internet Access via USB ..................................................................................................................... 17
2.3.4 Compiling on the BBB ............................................................................................................................... 17
2.3.5 Cross-Compiling on the BBB ..................................................................................................................... 18
3 SOFTWARE .................................................................................................................................... 20
3.1 Communications ......................................................................................................................................... 20
3.2 TCP Communication ................................................................................................................................... 21
3.2.1 The Client Server Model ........................................................................................................................... 21
5
Rosario ROV
3.3 ROV-Surface Network ................................................................................................................................. 22
3.3.1 Communications from ROV ...................................................................................................................... 23
3.4 ROV Client Code - client.py ......................................................................................................................... 24
3.4.1 I2C vs. SPI .................................................................................................................................................. 24
3.4.2 Reading and Preparing Data ..................................................................................................................... 24
3.4.3 Recording Data ......................................................................................................................................... 24
3.4.4 Transmitting Data ..................................................................................................................................... 25
3.4.5 Python versus C++ .................................................................................................................................... 25
3.4.6 Problems in Reading Data ........................................................................................................................ 26
3.5 Surface Server Code – SurfaceServer.cpp.................................................................................................... 27
3.5.1 Setting up a C++ TCP Server...................................................................................................................... 27
3.5.2 Receiving Connections and Data .............................................................................................................. 29
3.5.3 Recording the Packet ................................................................................................................................ 29
3.6 ROV Server and Surface Client .................................................................................................................... 30
3.7 Problems in Writing Data............................................................................................................................ 31
4 VIDEO FEED .................................................................................................................................. 32
4.1 Preliminary Designs .................................................................................................................................... 32
4.2 USB Webcam .............................................................................................................................................. 32
4.3 FFMPEG and Bitrates .................................................................................................................................. 33
4.3.1 OpenCV and FFMPEG with Logitech C270 on Angstrom .......................................................................... 34
4.3.2 MJPG-Streamer with C270 on Ubuntu ..................................................................................................... 34
4.3.3 OpenCV and FFMPEG with Logitech C920 on Angstrom .......................................................................... 34
4.4 Video Streaming Methods .......................................................................................................................... 35
4.5 Continued Work on Video Streaming ......................................................................................................... 36
4.6 Miscellaneous Notes on Video Streaming ................................................................................................... 36
5 GRAPHICAL USER INTERFACE (GUI) .................................................................................... 37
5.1 Initial Design Considerations ...................................................................................................................... 37
6
Rosario ROV
5.1.1 List of Controls and Data .......................................................................................................................... 37
5.1.2 Deciding on the Best Interface ................................................................................................................. 38
5.1.3 Video Feed and First GUI Draft ................................................................................................................. 39
5.2 QT ............................................................................................................................................................... 40
5.2.1 VLC QT ...................................................................................................................................................... 40
5.3 Developing the GUI..................................................................................................................................... 40
5.3.1 Controls .................................................................................................................................................... 40
5.3.2 TCP Client .................................................................................................................................................. 41
5.3.3 Displaying Data ......................................................................................................................................... 41
5.4 GUI Results ................................................................................................................................................. 42
5.5 GUI Problems and Continued Work ............................................................................................................ 43
6 OTHER CONSIDERATIONS ....................................................................................................... 44
6.1 Social .......................................................................................................................................................... 44
6.2 Sustainability .............................................................................................................................................. 44
7 CONCLUSIONS .............................................................................................................................. 45
REFERENCES .......................................................................................................................................... 46
APPENDIX A – BBB PIN LAYOUTS .................................................................................................. 47
APPENDIX B – STARTUSBNETWORKBBB & STARTUSBNETWORKLX ............................. 50
APPENDIX C – CLIENT.PY .................................................................................................................. 51
APPENDIX D – SURFACESERVER.CPP ........................................................................................... 53
APPENDIX E – BEAGLESERVER.CPP ............................................................................................. 55
APPENDIX F – CONTROLS.PY ........................................................................................................... 59
APPENDIX G – MAINWINDOW.CPP ................................................................................................ 60
APPENDIX H – MAINWINDOW.H .................................................................................................... 65
7
Rosario ROV
8
Rosario ROV
Table of Figures
Figure 1: Depiction of Project Objective. ..................................................................................... 11
Figure 2: Beaglebone Black and Pin Layout ................................................................................ 14
Figure 3: Beaglebone Black “User Boot” button location ............................................................ 15
Figure 4: ROV-Surface Network Diagram ................................................................................... 22
Figure 5: ROV to Surface Packet Proposed Design ..................................................................... 23
Figure 6: Data path from ROV to surface ..................................................................................... 25
Figure 7: Communications: Packet to ROV ................................................................................. 30
Figure 8: Surface to ROV Packet Proposed Design ..................................................................... 30
Figure 9: Logitech C920 USB Webcam ....................................................................................... 34
Figure 10: Real Time Protocol Data Flow .................................................................................... 35
Figure 11: User Datagram Protocol Data Flow ........................................................................... 35
Figure 12: First GUI Draft ............................................................................................................ 39
Figure 13: GUI Main Window ...................................................................................................... 42
9
Rosario ROV
List of Tables
Table 1: Comparison of Microcontroller Options ........................................................................ 13
Table 2: Comparison of Video Bitrates and Filesizes .................................................................. 33
Table 3: List of GUI Requirements and Abilities ......................................................................... 37
Table 4: Best Options to For Control of ROV and Display of Data in GUI................................. 38
10
Rosario ROV
Acknowledgements
I would like to thank the team advisor, Dr. Delvin Peterson, who has helped and coordinated this
team throughout the past two years, and Dr. Larry Aamodt, who has helped lead the focus for
technical issues, both in overall scope and on the electrical side. From answering design
questions using his technical expertise to simply being there so we could rehearse our
presentations, he has been readily available and very helpful. I would also like to thank the
project client, Dr. Kirt Onthank who approached the Engineering Department with this project,
being an enthusiastic and helpful supporter from the beginning. In addition to this advisor and
sponsor we would like to thank other engineering professors who have assisted us in our
research. We would like to thank Dr. Larry Aamondt who assisted with the electrical side of this
project, Professor Don Riley who helped answer questions relating to the CAD model, Dr. Qin
Ma who assisted with questions relating to materials, and Dr. Brian Roth who assisted us during
the optimization portion of the project.
11
Rosario ROV
1 Introduction and Objectives At a depth of 1000 ft the pressure exhibited on a submersed object by seawater can exceed 30
atm (440 psi). In order to get an idea of what this kind of pressure is like, creating a similar
pressure on the screen of a typical smartphone (such as the iPhone 4) would take a weight of
about 2570 lbs equally distributed over the surface of the screen. In such an extreme environment
pressure is just one of the many considerations that have to be taken into account when designing
a vehicle to explore such areas, and although the mechanical challenges stand out, an equal
number of electrical challenges existed.
1.1 Introduction
The ROV project began when Dr. Kirt Onthank, a professor from the Walla Walla University
biology department, approached the engineering department with the idea of having our
department design an underwater ROV for them.
1.2 Project Objectives and Requirements
This ROV operates at a depth of 1000 ft, which introduces problems that would not be present in
a shallower operating range. Fortunately, several key electrical systems are the same for either
range, including the onboard electrical systems for the sensors and the systems on the surface.
This project focuses on the development of these systems, namely sending and receiving data
between the two systems, establishing a camera feed, and developing a Graphical User Interface
(GUI) to make control of these systems easier. The problems that stem from operating at the
1000 ft depth, namely power, are not part of the scope of this project, with the focus being the
development of a platform to design the ROV around.
Figure 1: Depiction of Project Objective.
12
Rosario ROV
2 Rosario ROV Computer The ROV needed an onboard system capable of several things. First, it must be able to
interface with sensors, motors, and any other instrument that needs to be controlled or read data
from. A prototype board, such as an Arduino or mbed board, would be ideal for such a project.
Second, the system must be powerful enough to read data from and send controls to all of the
peripherals in as real-time an environment as possible. Included in this section is the ability to
read and send data to and from the ROV, and to be able to communicate over a network ideally
with simple IP commands. Third, the ROV’s onboard computer must be able to handle the
sending of video feed to the surface, as well as the recording of video on the ROV itself.
2.1 Onboard Computer
The ROV would require an onboard computer for interfacing with the microcontroller, storing
data and video, and interfacing with the surface computer. Initially, the idea of a barebones mini-
pc was considered. High definition video takes up a large amount of hard disk space, and aside
from managing the video feed, the processes on the ROV are not very demanding. Of course,
this notion was not well-based, for the reason that experience with video feed was limited at the
project’s beginning. High processing power was not required, so computers by Foxconn and the
Beagleboard XM were seriously considered. This option was not chosen, and it will be described
in the section on the Beaglebone Black why an onboard PC was not necessarily required.
13
Rosario ROV
2.1.1 Microcontroller
Regardless of what computer would eventually be on the ROV, a microcontroller of some sort is
required. The options considered were the mbed LPC11U24, the Beaglebone Black (BBB), the
Arduino Mega, and the Intel Galileo. The options considered were chosen simply because they
were what was available and what was most familiar. The figure below lists and compares
several of the attributes compared from each board.
MBED BBB ARDUINO GALILEO
CHIP ARM® Cortex™-M0 Core
AM335x 1GHz ARM® Cortex-A8
ATmega1280 Intel® Quark SoC X1000
CPU FREQUENCY
48 MHz 300 MHz (Changeable) 16 MHz 400 MHz
MEMORY 8 KB RAM 32 KB FLASH
512MB DDR3 RAM, 4GB 8-bit eMMC on-board flash storage
8 KB of SRAM, 4 KB of EEPROM, 128 KB FLASH
512 KB SRAM, 8 MB FLASH
CONNECTIVITY USB 2xSPI, I2C , UART, 6xADC, GPIO
USB client, USB host, Ethernet, HDMI, 2x46 pin headers
54 digital IO pins, 16 analog inputs, 4 UARTIO, USB, ICSP header
13 Digital Pins, 5 Analog Inputs, ICSP, UART
SOFTWARE COMPATIBILITY
Mbed Developer Debian, Android, Ubuntu, Cloud9 IDE, plus much more
Arduino Development Arduino Development
OPERATING VOLTAGE
5V 5V, 3.3V pins 5V 3.3V
PRICE $45.00 $54.00 $48.50 $55.00
Table 1: Comparison of Microcontroller Options
The Beaglebone Black was the top candidate for the microcontroller, as it is the strongest
performer in the majority of the criteria. Aside from this, the other microcontrollers were
discovered to have some flaws. The mbed does not have strong capabilities in connecting to
other computers. Communication was limited to simple RX and TX pins. The Arduino, although
a capable piece of equipment and often the go-to choice for hobbyists, does not have a robust
development environment. Arduino can only be programmed with Arduino’s C-like
programming language, which often provides everything a simple project needs. In this project
however, several different systems need to be integrated together, and similar to how Apple
computers often do not work with non-Apple products, Arduino is not readily able to connect to
non-Arduino equipment. For this same reason the Galileo board was ignored, along with the fact
that the amount of experience and support for this new board is limited.
14
Rosario ROV
2.2 Beaglebone Black (BBB)
The BBB is a low-cost, community-supported development platform for developers and
hobbyists. For a very low price, the consumer gets a basic computer on a microcontroller, able to
run a full-blown Linux distribution. The BBB can be programmed in nearly any language,
making it easy to develop on. For its power and capabilities, it is also very small, ideal for a
small, watertight chamber. As noted in the section earlier, an onboard computer of some sort is
required, and because of the power of the BBB, it is able to do the jobs of both the
microcontroller and onboard PC. Examples online showed the BBB able to do many of the tasks
required of our ROV. The OpenROV community makes use of the BBB as its primary computer
on its vessel, and was a key influence in considering the BBB. Of course, this needed to be tested
and verified, primarily with the most demanding task of streaming video.
2.2.1 BBB Hardware
The BBB comes with two headers, with two rows each as shown in the figure below, of a total of
92 pins. Many of these pins are general purpose input/output pins (GPIO), of which 65 are
possible digital I/O pins, 8 PWMs, 4 timers, 7 analog inputs at 1.8 volts, 4 universal
asynchronous receiver/transmitters (UART), 2 I2C ports, and 2 SPI ports. Refer to appendix A
for more detailed diagrams of the locations of these pins.
For communication between other computers, the BBB has available to it both a USB 2.0 mini
female port, which served as the primary source of power and initial communication between the
board and the development tool, and a USB 2.0 female port, which is used for communicating
with peripherals such as a camera or memory expansion, as well as an Ethernet port. The BBB
can be powered by the USB mini port alone for undemanding tasks such as running simple
programs and communicating with the development environment over USB, but for more taxing
applications, such as powering a camera and communicating with peripherals through the pins,
the 5V DC power jack must be employed. The BBB also has available to it a micro SD card slot,
which was used for installing firmware and potentially as added memory, as well as a micro
HDMI port, which was not employed.
Figure 2: Beaglebone Black and Pin Layout
15
Rosario ROV
2.3 Development on the BBB
Initially, the BBB takes more time to setup than other microcontroller boards, but the process is
straightforward if time consuming.
2.3.1 Setting Up the BBB
To begin developing on the BBB, first it must be flashed with the appropriate firmware.
Beagleboard, the manufacturers of the BBB support Angstrom and Debian distributions of
Linux, although unsupported images of Ubuntu and other distributions are available. More
reference material at the time of beginning development was available for Angstrom, and so that
system was chosen, although it has been shown that the software developed on the Angstrom
distribution works on Debian.
There are two distribution versions to choose from: a version that runs directly from the micro-
SD Card or a version that writes to the BBB’s on-board eMMC, called a “flasher” image. The
advantage of the eMMC version is that once it is written to the eMMC the BBB will boot in
about 10 seconds and the micro-SD card slot will remain available. For the remaining steps, the
eMMC flasher image is used and Windows is used as the desktop operating system.
• Using a web browser, download the image file to the desktop computer. You should
end up with a file of the form BBB-eMMC-flasher-distro-YYYY.MM.DD.img.xz.
• Under Windows, install “Win32 Disk Imager” from the Sourceforge.net
website:sourceforge.net/projects/win32diskimager/files/latest/download. Extract the
download to a folder that you can find again in the future, and execute
Win32DiskImager.exe directly from within that folder.
• Place a micro-SD card that has sufficient capacity into your computer’s SD card slot
and then start the Disk Imager tool. Insert the SD card first.
• Using Win32 Disk Imager, write the .img file to your SD card
Now that the firmware is loaded onto the
microSD card, it must be flashed onto the
on-board flash memory of the BBB. Make
sure the BBB is powered down, and
unplugged from the power source. Insert
the microSD card into the microSD slot on
the back of the BBB.
Hold the "User Boot" button down, shown
in the figure, and then plug in the power,
either through USB or the 5V adaptor.
Keep holding down the button until you
see the bank of 4 LED's light up for a few
seconds. Now release the button. It will
take anywhere from 30-45 minutes to flash
the image onto the on-board chip. Once it's
done, the bank of 4 LED's to the right of the
Ethernet will all stay lit up at the same time. The BBB is ready to use.
Figure 3: Beaglebone Black “User Boot” button location
16
Rosario ROV
2.3.2 Secure Shell into BBB
Secure Shell (SSH) is a UNIX-based command interface and protocol for securely getting access
to a remote computer. It is widely used by network administrators to control Web and other
kinds of servers remotely, and is a perfect and preferred way of easily accessing the BBB and its
files. SSHing into the BBB was the first step whenever developing on the BBB, and although
many guides are present online of how to use SSH, the steps and code will be shown here. From
this point on, Linux was chosen as the development environment for the BBB as SSH over
Windows is more of a hassle, as well as more support being available for Linux.
Plug in the BBB to computer through a USB connection
Open a terminal window in Linux and type the following command:
ssh 192.168.7.2 -l root
The command above indicates the local IP address of the BBB is 192.168.7.2 and you are
attempting to login with the user root
The first time that you log in, you will receive a warning. Type yes, to continue.
Press enter again when prompted for a password, or try root.
You now have access to the BBB
Once inside the BBB, the board can be treated as any other Linux system. Commands such as
creating and removing files or directories are the same, as well as getting software and updates.
17
Rosario ROV
2.3.3 BBB Internet Access via USB
In order to get updates and software such as compilers to place code on the BBB, access to the
internet is required. Many guides are available online, though many are faulty. Several
commands were required, some on both sides of the USB connection, to make internet access
possible, and are contained in the file StartUSBNetworkBBB and StartUSBNetworkLX. The
terminal commands on the BBB side once SSHed are
/sbin/route add default gw 192.168.7.1 echo "nameserver 8.8.8.8" >> /etc/resolv.conf
and the terminal commands on PC side, assuming the use of Ubuntu, are
sudo iptables –APOSTROUTING –t nat –j MASQUERADE sudo echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward > /dev/null
These commands were paramount to development on the BBB as they allowed for essential
software and updates to be downloaded onto the BBB. For ease of use, these commands were
placed in files, made executable with chmod +x, and run every time the BBB is powered on.
2.3.4 Compiling on the BBB
Two methods were used for developing the various scripts used on the BBB: onboard compiling
and cross-compiling. Onboard compiling allowed for the smaller scripts to be written and
compiled on the BBB. Larger scripts that would be more difficult for the BBB to compile were
done with cross-compiling where the program would be compiled on a larger computer to be
placed onto the BBB. It should be noted that either method could create whatever script is
required on the BBB, so the use of either is a matter of preference.
In order to compile programs on the BBB, appropriate software, such as a compiler, needs to be
installed. The majority of the scripts on the BBB are written in C++ due to the familiarity of the
language. To install a compiler, SSH into the BBB, make sure internet connection is established,
and use the following command:
apt-get install g++
The supported Linux distributions for the BBB include the text editor nano for creating scripts.
In order to compile a .cpp script, say HelloWorld.cpp, use the following command to compile:
g++ HelloWorld.cpp –o HelloWorld
where the parameter after –o is the name of the executable file. In order to run any compiled
executable file, use “./” followed by the name of the file.
18
Rosario ROV
2.3.5 Cross-Compiling on the BBB
Cross-Compiling on the BBB was the preferred method for development on the BBB. Although
the setup is much more tedious and time consuming, the robustness of having a proper integrated
development environment makes troubleshooting, compiling, using outside references, and
overall development much faster. The Eclipse IDE was chosen since it is free, available on
Linux, and contains many tools and reference materials for development on the BBB. The steps
for setup are summarized in the video titled “Beaglebone: C/C++ Programming Introduction for
ARM Embedded Linux Development using Eclipse CDT” by Derek Molloy. This video, along
with several others of his focusing on the BBB, very well summarizes the steps for setting up
Eclipse and developing on the BBB. The following steps summarize the setup of Eclipse on
Ubuntu for the BBB.
• Install Eclipse, either through the Ubuntu Software Center or through the command
sudo apt-get install eclipse
• Once installed, the first time will ask for a workspace, or directory for where Eclipse
will keep the projects. Any directory will suffice and the choice is up to preference.
• Start Eclipse, then select Help > Install New Software. In the drop-down menu under
“Work with:” select the Eclipse-Version Update Site. The list will populate after a
few seconds.
• Select the “C/C++ Development Tools”, “Remote System Explorer (RSE)”, and the
“C/C++ Library API Documentation Hover Help…” packages. Click Next, Next,
accept the terms of agreement, and Finish.
• Restart Eclipse once the software has finished installing. When in eclipse, you should
see a C/C++ Development option for projects. Select this and once a project has
opened, select Window>Show View>Other. Select Remote Systems>Remote
Systems.
• On the bottom of the screen, select the “Remote” tab next to the “Tasks” tab. In this
tab, right click “Local” and select New>Connection in the menu.
• On the new menu, select Linux, and click Next.
• Change the Host name to 192.168.7.2, which should also change the connection name
to 192.168.7.2, and click Next.
• Select ssh.files, and click Next.
• Select processes.shell.linux, and click Next.
• Select ssh.shells, and click Next, and then Finish. At this point, you should be able to
access the BBB via SSH inside of Eclipse. If prompted for a password, the default
password is root. You should be able to see any files or directories in the BBB.
The previous steps only need to be performed once, while the following steps must be done
whenever creating a new C++ project for the BBB through Eclipse.
• Start a new C++ project by selecting File>New>Project.
• In the page that opens, select C/C++>C++ Project, and click next
• Determine a name for the Project, select “Hello World C++ Project”, and click Next.
• Fill in the information for Author, and click Finish. If prompted with an “Open
Associated Perspective?” window, select yes.
19
Rosario ROV
At this point, the project is working, but the compiler is set to compile for the architecture of
the platform running Eclipse. Eclipse must be changed to development on the BBB. Use the
following steps to set up Eclipse.
• Open a terminal, outside of Eclipse, and use the following commands to install a
compiler of arm architecture.
sudo apt-get install g++ arm-linux-gnueabi sudo apt-get install g++ arm-linux-gnueabihf
• Return to Eclipse, and select Project>Properties.
• On the left-hand side, select C/C++ Build>Settings. Be the Tool Settings tab is
selected and GCC C++ Compiler is selected. In the Command blank, replace
“g++” with “arm-linux-gnueabi-g++”.
• Next select GCC C Compiler and similarly replace “gcc” with “arm-linux-
gnueabi-gcc”.
• Next select GCC C++ Linker and similarly replace “g++” with “arm-linux-
gnueabi-g++”.
• Next select GCC Assembler and similarly replace “as” with “arm-linux-gnueabi-
as”.
• On the far left side, select C/C++ General>Paths and Symbols. Make sure the
Includes tab is selected, and select GNU C. Click Add…, and then in the opened
window click File Systems. Navigate to /usr/arm-linux-gnueabi/include and click
ok. In the Add directory path window, check “Add to all configurations”, and
click OK.
• Select GNU C++. Click Add…, and then in the opened window click File
Systems. Navigate to /usr/arm-linux-gnueabi/include/c++/4.6.3/ and click ok. In
the Add directory path window, check “Add to all configurations”, and click OK.
• Select the Library Paths tabs. Click Add…, and then in the opened window click
File Systems. Navigate to /usr/arm-linux-gnueabi/lib and click ok. In the Add
directory path window, check “Add to all configurations”, and click OK.
• Click apply, and cross-compiling is ready to go.
Eclipse is ready to compile for the BBB, and the only thing left is to place the compiled
application on the BBB. After building a project, under the Project’s Debug Folder, copy the file
with name of the project followed by [arm/le], and simply paste into the directory of choice on
the BBB through the Remote tab. Now the application is on the BBB, and ready to be executed.
20
Rosario ROV
3 Software Several pieces of software are running at the same time, both on the BBB and on the surface
computer. Rather than have one, single piece of software on either end running, several scripts
were preferred due to the need for asynchronous receiving and sending of data, and, quite
honestly, due to the lack of experience with higher end communications between computers. The
software can be separated by communications, sensor reading, and GUI display.
3.1 Communications
When presented with the problem of what method to use for communication between the two
systems, a few things needed to be considered. First, the protocol must have a series of checks,
and handshakes on either side to ensure complete transfer of packets. Second, these protocols
must be able to transfer over long distances, a 1000 foot tether, and although this aspect is mostly
affected by the electrical properties of the tether’s material, some protocols were ignored because
of this criterion. Third, ease of use and familiarity of the protocol are important in order for
future groups to work on this effectively. There were four options seriously considered:
User Datagram Protocol (UDP)
Real-time Transport Protocol (RTP)
Hypertext Transfer Protocol (HTTP)
Transmission Control Protocol (TCP)
Ultimately, the last option TCP was chosen. UDP, although similar to TCP and effective at
sending information between systems was not chosen for the reason that there is no set of checks
and balances to ensure the total transfer of the packet. UDP is similar to a radio broadcast, where
a signal transmits into the void and listeners tune it to the correct frequency, but at the cost of an
imperfect signal. RTP is similar to TCP as well, but was more than this project required and
added features that were unfamiliar. HTTP would admittedly have been a good choice for
communication between the systems, as it is familiar to many and used for just this type of
problem. HTTP was only passed over because of the lesser amount of experience with HTTP
than with TCP.
21
Rosario ROV
3.2 TCP Communication
TCP is a connection-oriented protocol, which means a connection is established and maintained
until the applications at each end have finished exchanging messages. It determines how to break
application data into packets that networks can deliver, sends packets to and accepts packets
from the network layer, manages flow control, and, because it is meant to provide error-free data
transmission, handles retransmission of dropped packets as well as acknowledgement of all
packets that arrive.
3.2.1 The Client Server Model
Most interprocess communication uses the client server model, referring to the two processes
which will be communicating with each other. One of the processes, the client, connects to the
other process, the server, usually to make a request for information. The client needs to know of
the existence of and the address of the server, but the server does not need to know the address of
the client prior to the connection being established. Once a connection is established, both sides
can send and receive information.
The system calls and parameters for establishing a connection are somewhat different for the
client and the server, but both involve the basic construct of a socket. A socket is one end of an
interprocess communication channel. The two processes each establish their own socket.
The steps involved in establishing a socket on the client side are
1. Create a socket with the socket() command
2. Connect the socket to the address of the server using the connect() command
3. Send and receive data. There is more than way to do this, but the simplest is to use
the read() and write() commands
The steps involved in establishing a socket on the server side are
1. Create a socket with the socket() command
2. Bind the socket to an address using the bind() command. For a server socket on the
internet, an address includes the port number on the host machine
3. Listen for connections with the listen() command
4. Accept a connection with the accept() command. This typically blocks until a client
connects with the server and is one of the key reasons to have this code to be a separate
process on either side of the tether
4. Send and receive data with the read() and write() commands
22
Rosario ROV
3.3 ROV-Surface Network
The ROV-Surface network is rather simple, and is summarized in the figure below. From the
surface, controls are sent from the GUI, written in C++ utilizing the QT libraries, down through
the tether via a client server connection. The server in this connection is located on the BBB on
the ROV, also written in C++ but not utilizing QT. Data from the ROV is handled in another
separate client server connection. Here the client is on the ROV, written in Python, and sends all
ROV data, including sensor readings, ROV status, and emergency alerts, up the tether to the
server on the surface, written in C++.
The surface server is a separate process running alone from the GUI. Remember that certain
commands utilized in receiving data on a server include blocks and pauses. If this were inside of
the GUI, then these pauses would not allow real-time control of the ROV. Instead, the surface
server writes the data received to a text file which is then read by the GUI and displayed.
Figure 4: ROV-Surface Network Diagram
23
Rosario ROV
3.3.1 Communications from ROV
There are several things the operator on the surface needs to know from the ROV when it is
below the surface. First, in normal operation the ROV needs to be able to read the sensors and
send those readings to the surface. This sensor data also needs to be recorded, and in order to
prevent any sort of loss of data in the event of an emergency both the ROV and the surface
record this sensor data. Aside from sensor data, the ROV sends various status updates, including
orientation, heading, and onboard power. In case of emergency, such as a leak, the ROV must
inform the operator.
Each piece of data is typically an int type or the string conversion of the int, usually a single byte
or close to. Even though every piece of data compounded together increases the size of the
packet, the size of this packet will not be a problem. A standard Category 5 ethernet cable has
typical speeds of 10 Mbps and a maximum speed of 100 Mbps, leaving plenty of bandwidth for
any foreseeable amount of data to be sent. Most of the used bandwidth will be the video,
discussed in the camera section.
When designing the packet, it was clear early on that the best design would simply be a delimited
list. In the case of this list, the delimiter is a semicolon. This delimited list is a string format, as
the conversion of all types of data could be easily expressed and transmitted as a string as long as
the surface and the ROV were on the same page as to what format each piece of data was.
The server decodes this data structure, aware of where each piece of data is expected to be ahead
of time. In the early stages of packet design, the timestamp was the first item in the list. This is
entirely arbitrary, and was chosen because packet design occurred before the development of
peripheral sensors to draw data from. Note, the order of these pieces of data is completely
arbitrary, and as long the server parses the correct piece of the package, the data can be placed in
any order.
If the early packet design were to be expanded upon, which at this time includes the timestamp,
horizontal acceleration, temperature, and a leak detector, the order of the data will not be
organized. It is imperative and primary that the ROV be able to send all necessary data first, but
it should be noted that in order for development to be easy, the data should be grouped together
in the list, with like data together. Below is a proposed order of the package, also mirrored in the
surface server and parsing script.
𝑇𝑖𝑚𝑒𝑠𝑡𝑎𝑚𝑝; 𝑋 𝐴𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛; 𝑌 𝐴𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛; 𝑍 𝐴𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛; 𝑅𝑜𝑙𝑙; 𝑃𝑖𝑡𝑐ℎ; 𝑌𝑎𝑤; →
𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒; 𝑃𝑟𝑒𝑠𝑠𝑢𝑟𝑒; 𝑆𝑎𝑙𝑖𝑛𝑖𝑡𝑦; 𝐿𝑖𝑔ℎ𝑡 𝑆𝑡𝑎𝑡𝑢𝑠; 𝑆𝑡𝑜𝑟𝑎𝑔𝑒 𝐴𝑣𝑎𝑖𝑙. ; →
𝐿𝑒𝑎𝑘 𝐷𝑒𝑡𝑒𝑐𝑡𝑜𝑟 1; 𝐿𝑒𝑎𝑘 𝐷𝑒𝑡𝑒𝑐𝑡𝑜𝑟 2; … ; 𝐿𝑒𝑎𝑘 𝐷𝑒𝑡𝑒𝑐𝑡𝑜𝑟 𝑛; 𝐵𝑎𝑡𝑡𝑒𝑟𝑦 𝑆𝑡𝑎𝑡𝑢𝑠;
Figure 5: ROV to Surface Packet Proposed Design
24
Rosario ROV
3.4 ROV Client Code - client.py
Refer to Appendix C for the full client.py code.
The client.py code has several jobs. First, it reads the sensors, including scientific and ROV
status monitors. Second, it records the data on the ROV. Third, it connects to the surface server
and sends the most recent data recorded to be viewed by the operator.
3.4.1 I2C vs. SPI
Early in the design process, it was decided that I2C would be the best way to communicate
between the BBB and peripherals, with SPI being the second choice. With two pinouts, I2C
allows the communication to all the devices needed. Compared to SPI, I2C needs fewer
connections. Only 2 signal wires are needed, against three or more for SPI depending on the
number of slave devices. Since the number of SPI’s available on the BBB is limited, it was not a
good choice for this project.
3.4.2 Reading and Preparing Data
When attempting to read a device in I2C, two pieces of information are needed: the slave’s
address and the slave’s register containing the data. When coding the slave devices, the addresses
were arbitrarily assigned, but the register is fixed due to the architecture of the chip on the slave
devices. This created some problems, discussed in section 3.5.1. Depending on the slave device,
the type of data read would require a different command, usually the difference being a signed or
unsigned byte. Refer to the code for examples. The data is read from each slave device and then
promptly converted to string type data. Once all data is acquired, it is appended in the order the
server expects, using “;” as a delimiter. This appended string is the package, ready to be
transmitted and recorded.
3.4.3 Recording Data
Right now, the client.py script does not record data on the ROV. This task is menial, and was
overlooked since this script was written near the end of the project when time was short. Effort
was focused more on reading from the slave devices. The following commands inserted to the
client.py script should allow for recording of the information, though it hasn’t been tested. This
code should be placed at the end of the while loop, or before transmission.
text_file = open("data.txt", "w") text_file.write(packet) text_file.close()
25
Rosario ROV
3.4.4 Transmitting Data
In TCP communications, it is best to connect to a server, send the data, and then close the
connection, only to re-establish it when the next piece of data is to be sent. In development, this
makes sure the socket doesn’t use an already busy port or leave a port unavailable from failing to
close the socket. The client.py script connects to the server at 192.168.7.1 on port 50007, an
arbitrary available port. Some ports are assigned to system functions, so when choosing a port,
around 50000 and up are usually available. When choosing a port though, the internet was
referred to for a list of available ports, since the assignments are seemingly random.
After sending the packet to the server, the code waits for a confirmation of the packet delivery.
This is in the code simply for debugging, so it can be seen on both sides that the packet arrived.
After the confirmation data is received, the socket closes, and is established again at the start of
the while loop.
3.4.5 Python versus C++
It should be noted that this script is written in Python rather than C++. The reasoning for this has
to deal with how easy it is to read or write in C++ on the BBB. If C++ is used, it involves writing
through device trees, a subject that was researched but found lacking on material in order to
develop a strong understanding of the method. Rather, hobbyists strongly prefer using a python
library originally intended for a Raspberry PI that has been ported over to the BBB. This library
contains commands for reading and writing through I2C, and because TCP is standard, a python
client script can communicate with a C++ server. The arrangement is shown below.
Figure 6: Data path from ROV to surface
26
Rosario ROV
3.4.6 Problems in Reading Data
3.4.6.1 Major Problems
The jobs of developing the BBB and developing the peripherals were both handled by last year’s
team. Originally, it was understood to be one person, would work on the BBB, the ROV’s central
computer, and the communications as a continuation of the previous year’s work. The other team
member would focus on the development of the ROV’s power systems, determining how to
manage power as well as transfer it to the ROV. The job of sensor development was split among
the two, since it fell under both focuses. The development of the peripherals ended up being
carried out by one person, with power management coming secondary. This was seemingly not a
problem, since the peripherals, including leak detection circuits and accelerometers, were
developed and used in developing communications.
Through lack of communication, the peripherals were developed on the Arduino Uno project
board. Arduino, as discussed before, is great for simple projects for hobbyists, but encounters
problems when trying to work with non-Arduino systems. Remember that the slave’s address
and the slave’s register containing the data are needed by I2C in order to fetch data. Arduino uses
a pseudo-C script whose I2C commands only request a slave’s address. Arduino is expecting the
user to communicate with other Arduino devices, whose register address is standard.
Unfortunately, Arduino hasn’t revealed the register address, or it has yet to be discovered. This
creates a problem, and leaves the system without the ability to communicate with all devices.
It is important that this problem be resolved. Either the Arduino’s secret register must be found,
although the group was unable to find it after several hours, or redo the hardware on another
architecture, preferably mbed which uses standard C++ libraries and I2C commands.
3.4.6.2 Minor Problems
Smaller problems to be tackled exist as well. The first problem occurs in the event the socket
used in communications is not properly closed. If the socket is not properly closed, than the port
is not freed up. Because the port is a hard-coded address, when the client.py script attempts to
connect to the server, the connection is refused. The client has to wait for several minutes for the
system to automatically free up the port. This could be a problem in the event of an emergency
where either the ROV must restart all processes or the surface computer crashes, and having the
ROV unavailable for several minutes is not a viable option.
In order to fix this problem, explore the following command:
socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
This command will set the socket to reuse the port address if it is busy. This seems like an easy
fix, but it hasn’t been tested. This would allow the script to continue unabated and allow for
smooth operation of the ROV.
27
Rosario ROV
3.5 Surface Server Code – SurfaceServer.cpp
Refer to Appendix D for the full SurfaceServer.cpp code.
The SurfaceServer.cpp code has three jobs. First, it sets up a server to receive transmission from
the ROV. Second, it receives and the packet. Third, it records the packet in two places. The
server’s jobs are very similar to the client.py script, and can be thought of as the mirroring of the
client’s job in some cases.
3.5.1 Setting up a C++ TCP Server
Unlike a client, servers in TCP only need to know the port number where the client is going to
try to connect. The server is not trying to connect to any device, and therefore doesn’t need an IP
address at initialization. The server receives the IP address information when a client connects on
the designated port. To re-iterate, the port used in SurfaceServer.cpp is 50007 and is entirely
arbitrary.
Setting up a robust server requires more commands than a client, and this is where much time
and effort was spent learning how to make a strong server script. In setting up the TCP server,
there are a few lines of code that need to be understood.
sockfd = socket(AF_INET, SOCK_STREAM, 0);
The socket() system call creates a new socket. It takes three arguments. The first is the
address domain of the socket. There are two possible address domains, the unix domain for two
processes operating on the same system, and the Internet domain. The symbol
constant AF_UNIX is used for the former, and AF_INET for the latter.
The second argument is the type of socket. There are two choices here. The first, a stream socket
in which characters are read in a continuous stream as if from a file or pipe. The second, a
datagram socket, in which messages are read in chunks. The two symbolic constants
are SOCK_STREAM and SOCK_DGRAM. The third argument is the protocol. If this argument is
zero, which it usually is, the operating system will choose the most appropriate protocol, which
is TCP for stream sockets and UDP for datagram sockets.
if (sockfd < 0)
error("ERROR opening socket");
The socket system call returns an entry into the file descriptor table, usually a very small integer.
This value is used for all references to this socket. If the socket call fails, it returns -1. In this
case the program displays and error message and exits.
bzero((char *) &serv_addr, sizeof(serv_addr));
The function bzero() sets all values in a buffer to zero. It takes two arguments, the first is a
pointer to the buffer and the second is the size of the buffer.
28
Rosario ROV
serv_addr.sin_family = AF_INET;
The variable serv_addr is a structure of type struct sockaddr_in. This structure has
four fields. The first field is short sin_family, which contains a code for the address
family. It should always be set to the symbolic constant AF_INET.
serv_addr.sin_port = htons(portno);
The second field of serv_addr is unsigned short sin_port , which contain the port
number. However, instead of simply copying the port number to this field, it is necessary to
convert this with the function htons(), which converts a port number in host byte order to a
port number in network byte order.
serv_addr.sin_addr.s_addr = INADDR_ANY;
The third field of sockaddr_in is a structure of type struct in_addr which contains
only a single field unsigned long s_addr. This field contains the IP address of the host.
For server code, this will always be the IP address of the machine on which the server is running,
and there is a symbolic constant INADDR_ANY which gets this address.
if (bind(sockfd, (struct sockaddr *) &serv_addr,
sizeof(serv_addr)) < 0)
error("ERROR on binding");
The bind() system call binds a socket to an address, in this case the address of the current host
and port number on which the server will run. It takes three arguments, the socket file descriptor,
the address to which it is bound, and the size of the address to which it is bound. The second
argument is a pointer to a structure of type sockaddr, but what is passed in is a structure of
type sockaddr_in, and so this must be cast to the correct type. This can fail for a number of
reasons, the most obvious being that this socket is already in use on this machine. This is the
problem spoken of earlier, where a script would crash leaving the hard coded port still in use.
listen(sockfd,5);
Once these lines of code are executed, the server simply waits for connections.
The listen system call allows the process to listen on the socket for connections. The first
argument is the socket file descriptor, and the second is the number of connections that can be
waiting while the process is handling a particular connection. This should be set to 5, the
maximum size permitted by most systems.
29
Rosario ROV
3.5.2 Receiving Connections and Data
When a client connects to the server, a few things happen. Follow along in the code for clarity.
clilen = sizeof(cli_addr);
newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
if (newsockfd < 0)
error("ERROR on accept");
The accept() system call causes the process to block until a client connects to the server.
Thus, it wakes up the process when a connection from a client has been successfully established.
This is the main reason for having this script separate from other scripts, since we don’t want
other operations such as controls to be blocked. The second argument is a reference pointer to
the address of the client on the other end of the connection, and the third argument is the size of
this structure.
The remaining pieces of code are much easier to follow and require less explanation.
bzero(buffer,1024);
n = read(newsockfd,buffer,1023);
if (n < 0)
error("ERROR reading from socket");
printf ("%s\n",buffer);
The code initializes the buffer using the bzero() function, and then reads from the socket.
The read()command will block until there is something for it to read in the socket, after the
client has executed a write(). It will read either the total number of characters in the socket or
1023, whichever is less, and return the number of characters read.
3.5.3 Recording the Packet
The packet is not parsed by the server. In order to keep the information flowing, certain
operations are handled by other processes to alleviate any sort of unnecessary strain on the
server. Rather, the parsing is done by the GUI. The server writes the packet in the format it’s
received. This is one reason for having the packet in string format, in order to have it readable at
any time, even without the use of a program to decode it.
The packet is written to two text files, datafile.txt and controlfile.txt, for a simple reason. The
data needs to be read and displayed by the GUI as well as stored whole for later analysis. The
GUI can read from a text file very easily, but it is much easier to read from the first line of a text
file than the last line. The first line will always be on line one of the code, but because data is
constantly being received and recorded, the last line is not fixed. For this simple reason, the data
is written to a file where one line is constantly being overwritten, controlfile.txt, and the data is
written to a file where each piece of data is written to a new line, datafile.txt.
The location of these files is arbitrary, and was placed on the desktop for convenience. It is
strongly recommended that these files be placed with the server’s source code or another place
that will not change depending on the username of the computer.
30
Rosario ROV
3.6 ROV Server and Surface Client
The ROV server, BeagleServer.cpp in Appendix E, is identical to the SurfaceServer.cpp code in
operation, but utilizes a python script for I2C Communication, controls.py in Appendix F. The
surface client is located within the GUI and is discussed in section 5.3.2. It connects to the server
on the ROV, which in turn communicates the controls to the peripherals through I2C. The I2C
communications are once again carried out in python. The flow of data is represented in the
figure below.
Figure 7: Communications: Packet to ROV
In this case, controls are the data sent down to the ROV. Once again, the preliminary packet
design is not complete, and only includes light control and the horizontal pan of the camera.
Below is a proposed packet to send controls to the ROV. The control packet is smaller than the
data packet as the proposed design only needs control of ROV movement and camera movement.
𝑙𝑖𝑔ℎ𝑡 𝑖𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦; 𝐶𝑎𝑚𝑒𝑟𝑎 𝐻 − 𝑃𝑎𝑛; 𝐶𝑎𝑚𝑒𝑟𝑎 𝑉 − 𝑃𝑎𝑛; 𝐶𝑎𝑚𝑒𝑟𝑎 𝑍𝑜𝑜𝑚; →
𝑇ℎ𝑟𝑢𝑠𝑡𝑒𝑟 1; 𝑇ℎ𝑟𝑢𝑠𝑡𝑒𝑟 2; … ; 𝑇ℎ𝑟𝑢𝑠𝑡𝑒𝑟 𝑛; 𝐵𝑎𝑡𝑡𝑒𝑟𝑦 𝑆𝑡𝑎𝑡𝑢𝑠;
Figure 8: Surface to ROV Packet Proposed Design
31
Rosario ROV
3.7 Problems in Writing Data
Alone, the C++ server script operates well. The server is able to read from the surface, but
problems arise in writing with the Python script controls.py. Python is used again because it
makes it easy to communicate with the pins. The problems of I2C communication with Arduino
devices persist, but the fatal flaw comes from executing a python script from within a C++ script.
This seems to slow down the server, putting either a block or a hold in the execution of the
server. This is exactly what was avoided on the surface’s server. Frequently, the executing of the
python script keeps the GUI client from connecting to the ROV, causing the GUI script to crash.
This problem must be addressed for smooth operation of the ROV. Several solutions exist. The
first would be to do away with the python script and figure out how to use I2C in C++. This
option was pursued near the end of the project, but with little resources, it was not successful. A
second option would be to determine how to setup code to deal with errors within the GUI.
Literature and references exist, but because the software used for the GUI itself is unique and
requires a learning curve, it may be the more difficult option. The third option would be to
rewrite the ROV’s server script in python, combining the two previous scripts into one. This
option may work, but because of the familiarity with TCP in C++ and not knowing how to make
as robust a server in python, this option was not pursued.
32
Rosario ROV
4 Video Feed Video feed was a necessity in this project as it allows the operator to see what the ROV sees and
is the primary feedback for controlling the ROV. Creating a video feed was the most difficult
part of this project. The most time and effort was placed in this portion of the project, and
although the video feed works, the mysteries of video streaming are still elusive. There were
several attempts at establishing video feed, and most involved different steps.
4.1 Preliminary Designs
The video feed not only allows for control of the ROV, but is another source of data for analysis.
Because of this, the video feed needs to be clear and have enough definition to perform good
analysis when reviewed. Acceptable performance was determined by the client, Dr. Onthank.
The starting point was to look at the limitations of the ROV. Size would be the largest restraint,
as the camera would need a housing unit for protection from the ocean and space is limited on
the ROV. The camera would need to either be small enough to fit within the ROV, whose
dimensions were not initially known, or come with a pre-built, separate housing. A camera in a
separate housing proved to be very expensive. Waterproof cameras were not a feasible choice,
primarily because of the price. Not only would the camera need to be purchased, but other
hardware and peripherals to interface it with the ROV’s system would be needed as well. It
would be best to find a camera that could interface with the BBB.
Initially, the concern of not having enough available bandwidth in the tether or enough
processing power on the ROV for HD video weighed heavily. In order to combat this, the idea of
having two cameras, a low definition camera and a high definition camera, was considered. The
low definition feed would be sent to the surface while the high definition would record on the
ROV itself. Of course, this would theoretically take up twice as much space and twice as much
power. Though this idea was researched thoroughly, it proved needless.
4.2 USB Webcam
The final choice came after viewing websites that had provided most of the aid in learning how
to use the BBB. The website sought to teach three things
1. Capture video from a USB webcam under Linux
2. Capture image frames from a USB webcam under Linux
3. Use OpenCV to capture and process image frames
A USB webcam seems to be the best choice for developing the ROV, and perhaps ultimately as
the camera onboard the ROV. For initial development, they are low priced, readily available, and
generic enough to allow for testing the performance of the camera on the ROV end and on the
surface end. The only concern about using a USB webcam was that they tend to require more
light. 1000 feet underwater is very dark. The camera was tested in a dark environment, provided
with lighting to simulate the ROV’s lighting, and the performance was deemed acceptable.
33
Rosario ROV
4.3 FFMPEG and Bitrates
FFMPEG is a complete, cross-platform solution to record, convert and stream audio and video,
and allows the conversion of raw, or any other video format, into the format desired, such as
MPEG4. Raw files are much larger than necessary, and take up a large amount of space, with
more data than we need and more than the BBB could handle. Each conversion requires a codec,
to encode and later decode the video, and FFMPEG comes with those codecs.
To determine the codec to use, the bitrates were compared for different acceptable resolutions.
These bitrates were calculated with the following formula, the Kush gauge:
𝑝𝑖𝑥𝑒𝑙 𝑐𝑜𝑢𝑛𝑡 × 𝑓𝑟𝑎𝑚𝑒𝑠 𝑝𝑒𝑟 𝑠𝑒𝑐𝑜𝑛𝑑 × 𝑚𝑜𝑡𝑖𝑜𝑛 𝑓𝑎𝑐𝑡𝑜𝑟 × 0.07
1000= 𝑏𝑖𝑡 𝑟𝑎𝑡𝑒 𝑖𝑛 𝑘𝑏𝑝𝑠
The pixel count is simply the number of pixels per frame, the product of the frame’s width
multiplied by its height. The frames per second of a camera can vary widely, but for the ROV
smooth video is required not only for analysis but for good control, and so an appropriate frames
per second (fps) would be 24 or 30. The motion factor is a number 1, 2, or 4 that is based on the
expected motion in the image. The highest motion factor, 4, was used for the calculations,
assuming there is much excitement underwater. The table below is a comparison of bitrates.
Frame Width
Frame Height
Pixel Count
FPS Motion Factor
Bitrate (kbps)
Filesize (MB/minute)
320 240 76800 24 4 516.10 3.87
320 240 76800 30 4 645.12 4.84
480 270 129600 24 4 870.91 6.53
480 270 129600 30 4 1088.64 8.16
1024 576 589824 24 4 3963.62 29.73
1024 576 589824 30 4 4954.52 37.16
1280 720 921600 24 4 6193.15 46.45
1280 720 921600 30 4 7741.44 58.06
1920 1080 2073600 24 4 13934.59 104.51
1920 1080 2073600 30 4 17418.24 130.64
Table 2: Comparison of Video Bitrates and Filesizes
The largest bitrate here would be around 17 Mbps, and assuming a standard Category 5 Ethernet
cable which has typical speeds of 10 Mbps and a maximum speed of 100 Mbps, a codec must
decrease the bitrate to below 10 Mbps, lest the resolution be lowered. Although most of the
resolutions above are below 10 Mbps, the highest definition is easily achievable with an
appropriate codec. The most common encoded format, MPEG4, was tried first.
34
Rosario ROV
4.3.1 OpenCV and FFMPEG with Logitech C270 on Angstrom
The first attempt at establishing a camera feed was using a Logitech C270 USB webcam with
OpenCV and FFMPEG. The C270 was chosen for the sole reason that it was purchasable at a
local retail store. Using the sample code provided by FFMPEG and edited by Derek Molloy of
the University of Dublin, the first attempt at video compression was a mild success. Using the
C270, video was recorded, saved on the BBB, and played on the host computer. Unfortunately,
streaming of MPEG4 video was not accomplished.
4.3.2 MJPG-Streamer with C270 on Ubuntu
Through further research, options other than FFMPEG were discovered. MJPG-Streamer is an
encoder to allow for video streaming in MPEG4 format, but the condition is it only runs on
Ubuntu. Ubuntu, although not officially supported by the makers of the BBB, was ported onto
the BBB by hobbyists and enthusiasts. Ubuntu was loaded onto the BBB, removing Angstrom,
and video streaming was attempted again. This time, video was streamed from the BBB to the
host computer. An exhilarating success at first glance, but under further examination, it was
shown that the pins were not accessible. The Ubuntu option was scrapped.
4.3.3 OpenCV and FFMPEG with Logitech C920 on Angstrom
Because of time constraints and little to no progress, a new camera was chosen. The sample code
used provided a tutorial on how to use the code with a specific USB webcam, one different from
the one first used. The tutorial’s camera, the Logitech C920, is a slightly more expensive camera
able to use a codec that the cheaper C270 was not capable of. The H.264 codec is a superior, and
newer, codec than MPEG4, and is actually an evolved version of the MPEG4 codec. H.264 is
also efficient enough to encode the highest resolution from the table into an acceptable bitrate.
The Logitech C920, shown below, was purchased, and immediately video streaming was
accomplished.
Figure 9: Logitech C920 USB Webcam
35
Rosario ROV
4.4 Video Streaming Methods
With the video streaming, the decision of how to stream the video needed to be decided. This
decision was similar to deciding what method of communication to use between the ROV and
the surface. The first option was Real-Time Protocol (RTP). Like TCP, RTP requires the
destination address and port number to work. The figure below describes the flow of data in
encoding and streaming through RTP.
Figure 10: Real Time Protocol Data Flow
The streaming of video is different from communication though. Here, a large amount of
information, a .sdp file describing the properties of the streams origin, is required to view the
video with RTP. This .sdp file needs to be loaded every time the video is viewed, and if the file
is not available the stream is not viewable. Instead, the option of User Datagram Protocol (UDP),
also available for video, is a better choice. In UDP, the system just needs to know what port to be
looking on, and the address of the ROV. This information will not change between users, and is a
better choice for the video. . The figure below describes the flow of data in encoding and
streaming through UDP.
Figure 11: User Datagram Protocol Data Flow
Capture video with script
utilizing OpenCVLibraries
Read input and skip
transcoding of the video
Assign appropriate header to
stream
Output to address of
specific port of the specific PC
specified
View video stream with appropriate
.sdp file
Capture video with script
utilizing OpenCVLibraries
Read input and skip
transcoding of the video
Assign appropriate header to
stream
Output to specific port
on BBB
View video stream from port address
36
Rosario ROV
4.5 Continued Work on Video Streaming
At this point in the project, the streaming and recording of high definition is possible. They have
not been tried at the same time, which was not attempted due to time constraints on the project.
A simple editing of the code would allow for both in unison.
Video is recorded on the BBB’s small onboard storage, around 8 GB. High Definition video will
quickly use this up, and with estimated operation times of nearly 6 hours, video must be recorded
on external storage. This could either be through the microSD card, or through the use of a USB
dongle and external hard drive, neither of which have yet been attempted.
4.6 Miscellaneous Notes on Video Streaming
All code is available from https://github.com/derekmolloy/boneCV/ with tutorial videos
explaining the inner workings of the code. Although the code was tested and changed in
numerous ways, the unchanged code is definitely the best place to continue work on video
streaming.
When streaming video with the Logitech C920 through the BBB, be sure that the camera is
supplied enough power. The BBB powers itself through a USB connection, but the current
supplied through the host computer’s USB may not be enough. Simply use the 5V power supply
to ensure proper operation of the camera.
37
Rosario ROV
5 Graphical User Interface (GUI) A graphical user interface, or GUI, is a type of interface that allows users to interact with
electronic devices through a set of graphics and icons as opposed to text. Most consumer
programs are GUIs, as they are more user friendly and have a smaller learning curve. Because
the operator of the ROV will not necessarily be someone well-versed in computer commands and
lower level programming, a biology student for example, a GUI is a must have on this project.
Unfortunately, electrical engineering students are not usually taught how to make GUIs, and so
the construction of robust GUIs needed to be learned.
5.1 Initial Design Considerations
Before actually coding the GUI, it was prudent to list the necessities the GUI would have, decide
what the best icon, such as a button, slider, or other interface, would serve each necessity best,
and sketch a rough idea of what the GUI would look like.
5.1.1 List of Controls and Data
Below is a table of the necessary controls, data, and other amenities the GUI would need to have.
Controls Data Other Amenities
Move Forward Pressure Video Feed
Move Backward Depth Event Log
Rotate Left Temperature
Rotate Right Salinity
Dive Acceleration in X Direction
Surface Acceleration in Y Direction
Light Power Acceleration in Z Direction
Light Intensity Speed
Open Video Feed Battery Status
Pan Camera Left Heading
Pan Camera Right Orientation
Pan Camera Up Leak Alert
Pan Camera Down
Zoom Camera In
Zoom Camera Out
Table 3: List of GUI Requirements and Abilities
These items are only a preliminary list of what would be needed. Of course, other options and
useful tools would be discovered and implemented. This meant the GUI needed to either have
real estate available for these additions from the beginning or the GUI needed to be easily
changeable in order to move the different pieces of the GUI around and make more space.
38
Rosario ROV
5.1.2 Deciding on the Best Interface
With the preliminary needs of the GUI determined, the tool to best implement these controls and
to display the data needed to be decided. Below is a continuation of the previous table with the
best tool for each option,
Controls Option Data Option Other Amenities
Option
Move Forward Button Pressure Text Box Video Feed
Viewer
Move Backward Button Depth Text Box Event Log Text Box
Rotate Left Button Temperature Text Box
Rotate Right Button Salinity Text Box
Dive Button Acceleration in X Direction Text Box
Surface Button Acceleration in Y Direction Text Box
Light Power Button Acceleration in Z Direction Text Box
Light Intensity Slider or Dial Speed Text Box
Open Video Feed Button Battery Status Bar or Text Box
Pan Camera Left Button or Slider Heading Text Box or Compass
Pan Camera Right Button or Slider Orientation Text Box or Altimeter
Pan Camera Up Button or Slider Leak Alert Alert Symbol
Pan Camera Down Button or Slider
Zoom Camera In Button or Slider
Zoom Camera Out Button or Slider
Table 4: Best Options to For Control of ROV and Display of Data in GUI
These options are readily available in most GUIs, and the assumption was made that any sort of
GUI library or builder would have these options available to build the ROV GUI. In deciding on
what option would best service each control, one aspect stood out above any other. Was the
range of a particular control limited or limitless? An example of this would be the rotating of the
ROV left and panning the camera left, two seeming similar controls. The ROV can theoretically
turn left forever, as there is seemingly no restraint, but the camera, located on a gimbal, only
views the forward viewpoint, and has a limited range of rotation. For limitless range, a button
would suffice better as it can simply be held down by the operator for any length of time. A
limited range could be controlled with a button as well, but a slider would show the range and
where the system currently was within that range.
For data, a textbox was the obvious choice. This textbox would hopefully be able to display the
text in different colors to express either danger or an alert. For example, if the orientation of the
over read that it was upside down, the text or widget should display concern.
39
Rosario ROV
5.1.3 Video Feed and First GUI Draft
Though the controls and display of data were the most numerous pieces, arguably the most
important piece of the GUI would show what the ROV sees and is recording. A video viewer
would also require the most amount of space, large enough so that the video feed’s quality would
not be reduced by a smaller screen. Whatever GUI library or builder that was used would need
either a built-in video player or the ability to port in a video player.
With this list of preliminary concerns, the GUI below was sketched. On the left side are controls
for the ROV, as well as alerts for power and loss of connection. Bars also show the ROV’s speed
and battery or power level. On the left, readings from the sensors are displayed with controls for
the camera and lights placed below. In the center is the video feed, with the space above and
below the feed left open for additions.
The early GUI draft brought to light a few things. First, every control either needs to be clear in
its purpose or, at the very best, properly labeled. Second, though the primary controls would be
available at all times in the main GUI, other less commonly used functions might be required. A
good way to include these functions would be through drop down menus. Third, though alerts
may be expressed with a relevant icon, perhaps a pop-up warning should be employed, as this
would force the problem to be addressed. Fourth, though data is displayed on the side of the
video, it would be prudent to have data such as the timestamp overlayed onto the video feed.
This way, when analysis is performed after operation, there will only be one timestamp for the
data and the video feed rather than two.
Figure 12: First GUI Draft
40
Rosario ROV
5.2 QT
At this point, the GUI needed to begin construction. It was desired that the GUI be written in
C++, as this would allow quick learning for future project members. Microsoft Visual Studio
was the first consideration, as it is familiar to students and contains good GUI building utilities.
Unfortunately, it isn’t free, which would make development difficult. A few quick searches on
the internet on the subject of making C++ GUIs revealed that the crowd favorite was a
development environment called QT. QT writes GUIs in C++ utilizing special QT libraries. The
GUI can also be developed either through code or with a graphical interface. QT is open source,
free, and contains a large archive of examples, tutorials, and explanations on using QT. QT was
decided on as the development tool.
QT is available on both Windows and Linux, but because all development so far was in a Linux
environment, QT was installed on Ubuntu. QT can be installed either through the Ubuntu
Software Center or by visiting QT’s webpage and downloading it from there. The version used in
development was 5.5.1 for Linux 64-Bit.
5.2.1 VLC QT
QT comes with a built in media player library, QTMultimedia. This library is implemented in a
QT example media player GUI, but unfortunately, it wouldn’t work. The amount of libraries and
subsequent understanding of these libraries required was too much. An alternative method was
available. When video feed was first implemented, VLC was used to open and view the video
stream. In VLC, this is done with Media>Open Network Stream… and then type in the network
URL box “udp://@192.168.7.2”. This will open the stream and play it in VLC in the same way a
video file would be opened. VLC can actually be ported into QT. To port VLC into QT, install
the VLCQT libraries by visiting https://vlc-qt.tano.si/ and following the instruction for
downloading VLCQT for Ubuntu.
5.3 Developing the GUI
To begin developing the GUI, the best place to start was the example video player that came with
the VLCQT libraries. Since the video feed is the center of the GUI and the most important,
altering this example video player and adding other functions was the most efficient way to go.
5.3.1 Controls
When a button is pushed in the GUI, or a slider is moved, a value is changed. On a slider, that
value could range from 0 to 100 or any other range, and on a button the value is true or false. The
changing of these values trigger functions within the GUI to update information. This
information is then appended together into the control packet. Once again, this packet is the same
on the surface as what the ROV expects to receive.
41
Rosario ROV
5.3.2 TCP Client
Recall that the GUI has the code for the TCP client on the surface side encoded in it. The client
works just like any other TCP client, requiring the IP address and port number of the server. The
GUI has built in libraries for TCP communication, making connecting to the server simple. It
must be admitted that this client is not as robust as the one on the ROV, and this could be
because of a lack of fail safes in the QT TCP library or just the lack of their implementation.
Regardless, at times the inability of the client code to correctly deal with the GUI client’s
inability to connect to the ROV server causes the program to crash. This must be rectified.
In the GUI, a separate function handles the client connecting to the ROV. This client function
runs every 250 milliseconds, meaning controls are sent every 250 milliseconds. The ROV server
cannot handle a connection coming in any more frequently because of the python script within
the C++ script, a problem already discussed.
5.3.3 Displaying Data
Data is displayed in the GUI in a simple way. Recall that the surface server saves the unedited
packet to a text file, with a string format containing delimiters. It only contains one line,
constantly being written over with the most recent data. The GUI simply opens this text file,
reads the line, parses it, and displays it in the appropriate manner.
The data is displayed in a few ways. First, there is the data displayed in text. The GUI simply
takes the value read and without changing it, displays it within the appropriate textbox. Other
data from the ROV, such as alerts which are either true or false, are displayed in different ways.
For the alerts, an error window is opened, forcing the user to address the problem, ensuring that
it doesn’t get overlooked. The user must address the problem before continuing.
Other data, such as the ROV’s heading, is more readily displayed by traditional analog
instruments, such as a compass. For displaying the heading, a compass widget was downloaded
and implemented into the program. This widget has a set range, and so data from the ROV must
be translated into the appropriate format to use the compass.
42
Rosario ROV
5.4 GUI Results
The resulting GUI is shown in the figure below.
Figure 13: GUI Main Window
There are a few noticeable changes from the first draft. First, the video feed is moved to the
upper left quarter. Because controls within the GUI have set sizes, having the feed in the upper
left optimizes both the size and overall available space of the GUI. Second, the controls for the
ROV’s movement, the camera’s movement, and the lights were allotted together beneath the
video feed. This arrangement is more practical, as like items are placed together. Similarly, the
data has been placed on the right side of the video feed, listed vertically, so the operator can look
to one place for any information gathered by the ROV. Below the data, an event log was added.
This event log may or may not prove to be a useful end feature, but for debugging and early
development, having the event log display the successes and failures in the program is a useful
feature. Real estate is still available, allowing for additions to be made both in controls and data
display.
43
Rosario ROV
5.5 GUI Problems and Continued Work
The GUI is far from finished, and is far from perfect. The most serious problem is the lack of a
robust TCP client. Using the QT TCP library makes connecting easy, but only if it works. If a
TCP connection from the GUI to the ROV is not successful, the program will crash. In fact, if
the server is not available from the start of the program, the GUI may not start. To rectify this
problem, there are a few options. The next project team could expand on the current TCP code
with the QT libraries. These libraries are most likely comprehensive enough to contain code to
deal with errors such as failed connections and interruptions. Or, since QT is written in C++,
scrap the QT library and use the already existing C++ server code in the GUI. This option was
not attempted, but may prove to be the better of the two.
A second problem deals with the video feed. When displaying the video feed, there needs to be
little to no latency in the feed. The feed is to be used for real-time control of the ROV, and even
latency as small as a second is noticeable enough to make control of the ROV extremely
difficult. With the current GUI, the latency is nearly two seconds. The cause of this delay is
known though. Thankfully, this is not a hardware issue but a software issue, one inside of VLC.
When streaming video, VLC buffers, two seconds by default, the stream in order to have a
smooth playback of the video. This of course introduces latency, but VLC is used primarily for
playing movies and videos, and so this smooth playback is desired. Here low latency is desired,
and thankfully the buffering can be set to closer to none in VLC. Unfortunately, the VLC port
into QT does not seem to have this option, or at least it hasn’t been found. Either the caching in
the VLC QT library needs to be turned off, or the video feed needs to be displayed through
another method.
Aside from these problems, work on the GUI is expanding upon what is already present.
Currently, the GUI only expresses two pieces of data and sends down two controls. The
remaining controls and data simply need to be coded in, and the code already present can simply
be copied and pasted.
44
Rosario ROV
6 Other Considerations
6.1 Social
The goal of this project was to design a ROV and interface that will one day be used by college
students for biological research. These college students will not necessarily be well versed in the
inner workings of a computer, computer language, video encoding, or even controlling a ROV.
For this reason, throughout the project, a constant consideration was the robustness of the system
and the ease of use of the system. The communications needed to be robust enough to deal with
failed connections itself, as the operator may not be able to fix the connection. The video feed on
the ROV needed to be constantly playing and recording, and in the case of failure, needed to be
easily restarted. It would simply be impractical to have the ROV hauled up from its depth of
1000 feet to the surface, simply to turn the camera back on. For ease of use of the system, the
GUI needed to be simple and intuitive. A powerful program can be utterly ruined if the user
interface is difficult to understand, so when designing the GUI, it was important to keep in mind
where things would be expected. Where would a person look first to find a tool or function they
would normally use in another program?
6.2 Sustainability
Not only did the project need to be easy to use for the operator, but future teams would be
working on this ROV. It was important to create a platform that would leave them plenty of
room for expansion. The GUI tools needed to be simple enough to learn again, TCP needed to be
set up to allow for good communication, and the video feed needed to work to allow other
projects to save their time and effort for continuing the project rather than fixing what was left to
them. The project needed to be a building platform for future generations. For this reason, the
work done needed to not lead into dead ends but set up for continued additions. The GUI tools
chosen are very powerful and contain enough resources to tackle almost any foreseeable
problem. The entire GUI could be coded now, with the only requirement being time. The
communications for the ROV were made to not only work, but to be easily fixable and
expandable. Rather than choose a communication packet that would work for a limited time,
proper designing allowed for a packet that would be expanded upon without the need for
scrapping and starting over. The video feed, once done, can theoretically be left alone once it is
working. The camera system is isolated from the rest of the ROV, and so although changes to the
ROV will be made, the concern of affecting the video need not be addressed.
45
Rosario ROV
7 Conclusions Over the course of the past two years, the following tasks were completed: initial ROV design
research, communication protocol selection, hardware selection, camera selection, capturing of
video, streaming of video feed, writing of communication scripts, implementation of scripts and
subsequent successful communications, development of GUI, and a successfully communicating
ROV computer and host computer. The main aspects of this project, the video feed,
communication, and GUI, have been thoroughly researched. This initial research and design has
paved the way for future engineering students to take this project on. This joint venture between
the engineering and biology departments has been a great means for Walla Walla University
departments to pool their knowledge and resources together to develop a project to be used by
the university and its students for years to come.
46
Rosario ROV
References
"Sockets Tutorial." Linux Howtos: C/C++ -. N.p., n.d. Web. 08 May. 2015.
<http://www.linuxhowtos.org/C_C++/socket.htm>.
Molloy, Derek. "Beaglebone Black Streaming Video Tutorial and Custom LibVLC Player |
Derekmolloy.ie | Derekmolloy.ie." Derekmolloyie. N.p. Web. 04 Mar. 2015.
Molloy, Derek. "The Beaglebone." YouTube. YouTube, n.d. Web. 22 May. 2015.
<https://www.youtube.com/playlist?list=PLF4A1A7E09E5E260A>.
Molloy, Derek. Exploring Beaglebone: Tools and Techniques for Building with Embedded
Linux. Hoboken, NJ: Wiley, 2015. Print.
Richardson, Matt. Getting Started with BeagleBone: Linux-powered Electronic Projects with
Python and JavaScript. Sebastopol, CA: Maker Media, 2013. Print.
VoidRealms. "C++ Qt Programming." YouTube. YouTube, n.d. Web. 04 Feb. 2015.
<https://www.youtube.com/playlist?list=PL2D1942A4688E9D63>.
47
Rosario ROV
Appendix A – BBB Pin Layouts
48
Rosario ROV
49
Rosario ROV
50
Rosario ROV
Appendix B – StartUSBNetworkBBB & StartUSBNetworkLX
StartUSBNetworkBBB
1. #!/bin/sh 2. 3. /sbin/route add default gw 192.168.7.1 4. echo "nameserver 8.8.8.8" >> /etc/resolv.conf
StartUSBNetworkLX
51
Rosario ROV
Appendix C – client.py
1. #!/usr/bin/python 2. import Adafruit_BBIO.ADC as ADC 3. import time 4. import socket 5. import sys 6. import datetime 7. import atexit 8. import smbus 9. #Adafruit's Python library is required for this script 10. from Adafruit_I2C import Adafruit_I2C 11. 12. # Select which of the two I2C buses from the BBB to use 13. bus = smbus.SMBus(1) 14. 15. # Host IP address and port for socket connection 16. HOST, PORT = "192.168.7.1", 50007 17. 18. # List of slave device addresses; 19. # NOTE that these addresses are arbitrarily assigned on the master and slave 20. LEAK_ADDRESS = 0x08 21. MPU9250 = Adafruit_I2C(0x69) 22. MAGNOMETER = Adafruit_I2C(0x0c) 23. 24. 25. ADC.setup() 26. 27. # In case of application closing errors, the socket must be closed before reassigned 28. def exit_handler(): 29. print "Application Ending" 30. client.close() 31. 32. # Closes socket from earlier program executions 33. atexit.register(exit_handler) 34. 35. while True: 36. # Tell socket location of server IP and port. 37. client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 38. 39. # Connects client once per packet 40. client.connect((HOST, PORT)) 41. 42. # Reads signed byte from the register on slave 43. accelxH = MPU9250.readS8(0x3b) 44. # Reads unsigned byte from register on slave 45. accelxL = MPU9250.readU8(0x3c) 46. 47. # Converts data to string type 48. if accelxH >= 0: 49. accelx = str((accelxH << 8) + accelxL) 50. else: 51. accelx = str((accelxH << 8) - accelxL) 52. 53. # Reads signed byte from the register on slave 54. tempH = MPU9250.readS8(0x41) 55. # Reads unsigned byte from register on slave 56. tempL = MPU9250.readU8(0x42) 57. 58. # Converts data to string type 59. if tempH >= 0:
52
Rosario ROV
60. temp = str((tempH << 8) + tempL) 61. else: 62. temp = str((tempH << 8) - tempL) 63. 64. # magxH = MAGNOMETER.readS8(0x03) 65. # magxL = MAGNOMETER.readU8(0x04) 66. # if magxH >= 0: 67. # magx = (magxH << 8) + magxL 68. # else: 69. # magx = (magxH << 8) - magxL 70. 71. # Declare variable 72. data = 0; 73. 74. # Reads different data type from Leak Detector 75. for i in range(0,4): 76. data += int(bus.read_byte(LEAK_ADDRESS)); 77. 78. timestamp = str(datetime.datetime.now()) 79. # Convert data to string 80. leak = str(data) 81. 82. # Construct packet by appending strings together with ";" 83. packet = ";".join([timestamp,accelx,temp,leak]) 84. 85. # Send the packet to the surface 86. client.send(packet) 87. #print(str(client.recv(17))) 88. 89. # Client receives confirmation of sent packet from surface for debugging purposes 90. client.recv(17) 91. 92. # Close socket after packet transferral 93. client.close() 94. 95. time.sleep(0.1)
53
Rosario ROV
Appendix D – SurfaceServer.cpp
1. //============================================================================ 2. // Name : SurfaceServer.cpp 3. // Author : Alexander Beckner 4. // Version : 5. // Copyright : Your copyright notice 6. // Description : Hello World in C++, Ansi-style 7. //============================================================================ 8. 9. # A simple server in the internet domain using TCP 10. 11. #include <stdio.h> 12. #include <stdlib.h> 13. #include <string.h> 14. #include <unistd.h> 15. #include <sys/types.h> 16. #include <sys/socket.h> 17. #include <netinet/in.h> 18. #include <iostream> 19. #include <fstream> 20. 21. 22. void error(const char *msg) 23. { 24. perror(msg); 25. exit(1); 26. } 27. 28. void fnExit (void) 29. { 30. printf("Exit function"); 31. } 32. 33. int main(int argc, char *argv[]) 34. { 35. atexit (fnExit); 36. std::ofstream datafile, controlfile; 37. 38. # Declaration of Variables 39. int sockfd, newsockfd, portno; 40. socklen_t clilen; 41. 42. # Declaration of buffer to receive data 43. char buffer[1024]; 44. struct sockaddr_in serv_addr, cli_addr; 45. int n; 46. /* 47. if (argc < 2) { 48. fprintf(stderr,"ERROR, no port provided\n"); 49. exit(1); 50. } 51. */ 52. # Assignment of port number to receive data on 53. portno = 50007; 54. 55. # Lots of complicated magic 56. sockfd = socket(AF_INET, SOCK_STREAM, 0); 57. if (sockfd < 0) 58. error("ERROR opening socket"); 59. bzero((char *) &serv_addr, sizeof(serv_addr));
54
Rosario ROV
60. //portno = atoi(argv[1]); 61. serv_addr.sin_family = AF_INET; 62. serv_addr.sin_addr.s_addr = INADDR_ANY; 63. serv_addr.sin_port = htons(portno); 64. 65. if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) 66. { 67. //close(sockfd); 68. error("ERROR on binding"); 69. } 70. while(1) 71. { 72. # Listens for incoming connection attempts 73. listen(sockfd,5); 74. clilen = sizeof(cli_addr); 75. 76. # Creation of socket 77. newsockfd = accept(sockfd, 78. (struct sockaddr *) &cli_addr, 79. &clilen); 80. if (newsockfd < 0) 81. error("ERROR on accept"); 82. 83. # Emptying of buffer 84. bzero(buffer,1024); 85. 86. # Reads transmission of from connection and places in buffer 87. n = read(newsockfd,buffer,1023); 88. if (n < 0) 89. error("ERROR reading from socket"); 90. // printf("Here is the message: %s\n",buffer); 91. printf("%s\n",buffer); 92. 93. # Opens text file to store data 94. datafile.open ("/home/alex/Desktop/data.txt", std::ios::app); 95. //myfile << "Writing this to a file.\n"; 96. 97. # Writes to datafile and closes file 98. datafile << buffer << "\n"; 99. datafile.close(); 100. 101. # Opens and writes to control file 102. controlfile.open ("/home/alex/Desktop/control.txt"); 103. //myfile << "Writing this to a file.\n"; 104. controlfile << buffer; 105. controlfile.close(); 106. 107. # For debugging, writes to client that data was received 108. n = write(newsockfd,"Message Received",16); 109. if (n < 0) error("ERROR writing to socket"); 110. close(newsockfd); 111. } 112. close(sockfd); 113. return 0; 114. }
55
Rosario ROV
Appendix E – BeagleServer.cpp
1. //============================================================================ 2. // Name : BeagleServer.cpp 3. // Author : Alex Beckner 4. // Version : 5. // Copyright : Your copyright notice 6. // Description : Hello World in C++, Ansi-style 7. //============================================================================ 8. 9. /* A simple server in the internet domain using TCP 10. The port number is passed as an argument*/ 11. #include <stdio.h> 12. #include <stdlib.h> 13. #include <string.h> 14. #include <unistd.h> 15. #include <sys/types.h> 16. #include <sys/socket.h> 17. #include <netinet/in.h> 18. #include <iostream> 19. #include <fstream> 20. #include <cstring> 21. #include <string> 22. #include <cstdlib> 23. 24. #define LED0_PATH "/sys/class/leds/beaglebone:green:usr0" 25. #define PWM_PATH "/sys/devices/ocp.2/pwm_test_P8_19.15/" 26. #define PWM_SETUP "/sys/devices/bone_capemgr.8/" 27. 28. void error(const char *msg) 29. { 30. perror(msg); 31. exit(1); 32. } 33. 34. # This function is an attempt at using PWM in C++ on the BBB 35. # It has nothing to do with server operation, and should be disused 36. # and only used for reference. 37. /* 38. void setPWM() 39. { 40. std::fstream fs; 41. fs.open( PWM_SETUP "slots", std::fstream::out); 42. fs << "am33xx_pwm"; 43. fs.close(); 44. fs.open( PWM_SETUP "slots", std::fstream::out); 45. fs << "bone_pwm_P8_19"; 46. fs.close(); 47. 48. fs.open( PWM_PATH "period", std::fstream::out); 49. fs << "20000000"; 50. fs.close(); 51. fs.open( PWM_PATH "polarity", std::fstream::out); 52. fs << "0"; 53. fs.close(); 54. } 55. */ 56. void removeTrigger() 57. { 58. // remove the trigger from the LED 59. std::fstream fs;
56
Rosario ROV
60. fs.open( LED0_PATH "/trigger", std::fstream::out); 61. fs << "none"; 62. fs.close(); 63. } 64. 65. int main(int argc, char *argv[]) 66. { 67. std::fstream fs; 68. setPWM(); 69. 70. int sockfd, newsockfd, portno; 71. socklen_t clilen; 72. char buffer[1024]; 73. char packet[1024]; 74. struct sockaddr_in serv_addr, cli_addr; 75. int n; 76. /* 77. if (argc < 2) { 78. fprintf(stderr,"ERROR, no port provided\n"); 79. exit(1); 80. } 81. */ 82. sockfd = socket(AF_INET, SOCK_STREAM, 0); 83. if (sockfd < 0) 84. error("ERROR opening socket"); 85. bzero((char *) &serv_addr, sizeof(serv_addr)); 86. //portno = atoi(argv[1]); 87. portno = 51717; 88. serv_addr.sin_family = AF_INET; 89. serv_addr.sin_addr.s_addr = INADDR_ANY; 90. serv_addr.sin_port = htons(portno); 91. if (bind(sockfd, (struct sockaddr *) &serv_addr, 92. sizeof(serv_addr)) < 0) 93. error("ERROR on binding"); 94. setPWM(); 95. 96. while(1) 97. { 98. listen(sockfd,5); 99. clilen = sizeof(cli_addr); 100. newsockfd = accept(sockfd, 101. (struct sockaddr *) &cli_addr, 102. &clilen); 103. if (newsockfd < 0) 104. error("ERROR on accept"); 105. bzero(buffer,1024); 106. n = read(newsockfd,buffer,1023); 107. if (n < 0) error("ERROR reading from socket"); 108. 109. strcpy (packet, buffer); 110. char* packet_read = strtok(packet, ";"); 111. 112. /* 113. removeTrigger(); 114. 115. fs.open (LED0_PATH "/brightness", std::fstream::out); 116. fs << packet_read; 117. fs.close(); 118. */ 119. fs.open (PWM_PATH "/duty", std::fstream::out); 120. fs << packet_read; 121. fs.close();
57
Rosario ROV
122. 123. 124. 125. // printf("%s \n",packet_read); 126. packet_read = strtok(NULL, ";"); 127. 128. 129. std::string command_0 = "python controls.py "; 130. std::string command = command_0+packet_read; 131. 132. system(command.c_str()); 133. /* 134. fs.open (PWM_PATH "/duty", std::fstream::out); 135. fs << packet_read; 136. fs.close(); 137. */ 138. 139. 140. //printf("%s \n",packet_read); 141. //packet_read = strtok(NULL, ";"); 142. 143. //printf("%s \n",packet_read); 144. // } 145. 146. 147. 148. 149. 150. //printf("Here is the message: %s and nothing but it\n",buffer); 151. 152. 153. 154. close(newsockfd); 155. } 156. close(sockfd); 157. return 0; 158. } 159. 160. /* 161. #include<iostream> 162. #include<fstream> 163. #include<string> 164. using namespace std; 165. 166. #define LED0_PATH "/sys/class/leds/beaglebone:green:usr0" 167. 168. void removeTrigger(){ 169. // remove the trigger from the LED 170. std::fstream fs; 171. fs.open( LED0_PATH "/trigger", std::fstream::out); 172. fs << "none"; 173. fs.close(); 174. } 175. 176. int main(int argc, char* argv[]){ 177. if(argc!=2){ 178. cout << "Usage is makeLED and one of: on, off, flash or status" 179. << endl; 180. cout << "e.g. makeLED flash" << endl; 181. } 182. 183. string cmd(argv[1]);
58
Rosario ROV
184. std::fstream fs; 185. cout << "Starting the LED flash program" << endl; 186. cout << "The LED Path is: " << LED0_PATH << endl; 187. 188. // select whether it is on, off or flash 189. if(cmd=="on"){ 190. removeTrigger(); 191. fs.open (LED0_PATH "/brightness", std::fstream::out); 192. fs << "1"; 193. fs.close(); 194. } 195. else if (cmd=="off"){ 196. removeTrigger(); 197. fs.open (LED0_PATH "/brightness", std::fstream::out); 198. fs << "0"; 199. fs.close(); 200. } 201. else if (cmd=="flash"){ 202. fs.open (LED0_PATH "/trigger", std::fstream::out); 203. fs << "timer"; 204. fs.close(); 205. fs.open (LED0_PATH "/delay_on", std::fstream::out); 206. fs << "50"; 207. fs.close(); 208. fs.open (LED0_PATH "/delay_off", std::fstream::out); 209. fs << "50"; 210. fs.close(); 211. } 212. else if (cmd=="status"){ 213. // display the current trigger details 214. fs.open( LED0_PATH "/trigger", std::fstream::in); 215. string line; 216. while(getline(fs,line)) cout << line; 217. fs.close(); 218. } 219. else{ 220. cout << "Invalid command" << endl; 221. } 222. cout << "Finished the LED flash program" << endl; 223. return 0; 224. } 225. */
59
Rosario ROV
Appendix F – controls.py
1. #!/usr/bin/python 2. 3. import sys 4. import smbus 5. import time 6. 7. #print 'Number of arguments:', len(sys.argv), 'arguments.' 8. intensity = int(sys.argv[1]) 9. 10. # for RPI version 1, use "bus = smbus.SMBus(0)" 11. bus = smbus.SMBus(1) 12. 13. # This is the address we setup in the Arduino Program 14. 15. address = 0x04 16. bus.write_byte(address, intensity)
60
Rosario ROV
Appendix G – mainwindow.cpp
1. #include "mainwindow.h" 2. #include "qcgaugewidget.h" 3. #include <QThread> 4. #include <QTextStream> 5. #include <QDesktopWidget> 6. #include <QScrollBar> 7. #include <QRegExp> 8. #include <QString> 9. #include <QFile> 10. #include <QTimer> 11. #include <QProcess> 12. 13. #include <QFileDialog> 14. #include <QInputDialog> 15. #include <QMessageBox> 16. 17. #include <vlc-qt/Common.h> 18. #include <vlc-qt/Instance.h> 19. #include <vlc-qt/Media.h> 20. #include <vlc-qt/MediaPlayer.h> 21. 22. 23. 24. MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), 25. _media(0) // Let's us promote widgets to Vlc Widgets 26. 27. { 28. // Initialize Variables 29. lights = 0; 30. QProcess *process = new QProcess(this); 31. 32. // Launch Surface server from within QT 33. QString file = QDir::homePath() + "ServerTest"; 34. process->start(file); 35. 36. 37. 38. 39. //connect(horizontalSlider, SIGNAL(valueChanged(int)), this, SLOT(setServoValue(hor
izontalSlider->value(int)))); 40. 41. // Read data file every 250 milliseconds and update GUI 42. QTimer *updateTimer = new QTimer(this); 43. connect(updateTimer, SIGNAL(timeout()), this, SLOT(update())); 44. updateTimer->start(250); // 250 milliseconds 45. 46. // Send control packet every 1 milliseconds 47. QTimer *controlTimer = new QTimer(this); 48. connect(controlTimer,SIGNAL(timeout()), this, SLOT(sendControls())); 49. controlTimer->start(1); 50. 51. // Create Compass Widget 52. { 53. mCompassGauge = new QcGaugeWidget; 54. 55. mCompassGauge->addBackground(99); 56. QcBackgroundItem *bkg1 = mCompassGauge->addBackground(92); 57. bkg1->clearrColors(); 58. bkg1->addColor(0.1,Qt::black);
61
Rosario ROV
59. bkg1->addColor(1.0,Qt::white); 60. 61. QcBackgroundItem *bkg2 = mCompassGauge->addBackground(88); 62. bkg2->clearrColors(); 63. bkg2->addColor(0.1,Qt::white); 64. bkg2->addColor(1.0,Qt::black); 65. 66. QcLabelItem *w = mCompassGauge->addLabel(80); 67. w->setText("W"); 68. w->setAngle(0); 69. w->setColor(Qt::white); 70. 71. QcLabelItem *n = mCompassGauge->addLabel(80); 72. n->setText("N"); 73. n->setAngle(90); 74. n->setColor(Qt::white); 75. 76. QcLabelItem *e = mCompassGauge->addLabel(80); 77. e->setText("E"); 78. e->setAngle(180); 79. e->setColor(Qt::white); 80. 81. QcLabelItem *s = mCompassGauge->addLabel(80); 82. s->setText("S"); 83. s->setAngle(270); 84. s->setColor(Qt::white); 85. 86. QcDegreesItem *deg = mCompassGauge->addDegrees(70); 87. deg->setStep(5); 88. deg->setMaxDegree(270); 89. deg->setMinDegree(-75); 90. deg->setColor(Qt::white); 91. mCompassNeedle = mCompassGauge->addNeedle(60); 92. mCompassNeedle->setNeedle(QcNeedleItem::CompassNeedle); 93. mCompassNeedle->setValueRange(0,360); 94. mCompassNeedle->setMaxDegree(360); 95. mCompassNeedle->setMinDegree(0); 96. mCompassGauge->addBackground(7); 97. mCompassGauge->addGlass(88); 98. } 99. // Place Compass Widget 100. setupUi(this); 101. gridLayout->addWidget(mCompassGauge); 102. 103. // Initialize GUI TCP Client 104. socket = new QTcpSocket(this); 105. connect(socket, SIGNAL(readyRead()), this, SLOT(readyRead())); 106. connect(socket, SIGNAL(connected()), this, SLOT(connected())); 107. 108. // Set Lights to maximum and set slider to maximum 109. textEdit->setText("Lights: Off"); 110. textEdit->verticalScrollBar()->setSliderPosition(textEdit-
>verticalScrollBar()->maximum()); 111. textEdit->append(QDir::currentPath()); 112. 113. // Create Video Player 114. _instance = new VlcInstance(VlcCommon::args(), this); 115. _player = new VlcMediaPlayer(_instance); 116. _player->setVideoWidget(video); 117. video->setMediaPlayer(_player); 118. connect(open, &QPushButton::clicked, this, &MainWindow::openUrl); 119. /*
62
Rosario ROV
120. ui->video->setMediaPlayer(_player); 121. ui->volume->setMediaPlayer(_player); 122. ui->volume->setVolume(50); 123. ui->seek->setMediaPlayer(_player); 124. 125. connect(ui-
>actionOpenLocal, &QAction::triggered, this, &DemoPlayer::openLocal); 126. connect(ui-
>actionOpenUrl, &QAction::triggered, this, &DemoPlayer::openUrl); 127. connect(ui-
>actionPause, &QAction::triggered, _player, &VlcMediaPlayer::paused); 128. connect(ui-
>actionStop, &QAction::triggered, _player, &VlcMediaPlayer::stop); 129. connect(ui-
>openLocal, &QPushButton::clicked, this, &DemoPlayer::openLocal); 130. connect(ui-
>pause, &QPushButton::clicked, _player, &VlcMediaPlayer::pause); 131. connect(ui-
>stop, &QPushButton::clicked, _player, &VlcMediaPlayer::stop);*/ 132. } 133. 134. // This function gives the option of viewing a local media file on the computer
135. void MainWindow::openLocal() 136. { 137. QString file = QFileDialog::getOpenFileName(this, tr("Open file"), QDir::hom
ePath(), tr("Multimedia files(*)")); 138. 139. if (file.isEmpty()) 140. return; 141. 142. _media = new VlcMedia(file, true, _instance); 143. _player->open(_media); 144. } 145. // This function opens and displays the video stream 146. void MainWindow::openUrl() 147. { 148. QString url = 149. QInputDialog::getText(this, tr("Open Url"), tr("Enter the URL you wa
nt to play"),QLineEdit::Normal, 150. "http://192.168.7.2:8
080/?action=stream"); 151. 152. if (url.isEmpty()) 153. return; 154. 155. _media = new VlcMedia(url, _instance); 156. _player->open(_media); 157. } 158. 159. 160. // This function triggers every time the connect button is pushed; 161. // The GUI client attempts to connect to the ROV Server 162. void MainWindow::on_actionConnect_triggered() 163. { 164. socket->connectToHost("192.168.7.2", 51717); 165. } 166. 167. // This function triggers every time the connect button is pushed; 168. // The GUI client attempts to connect to the ROV Server 169. void MainWindow::on_connectbutton_clicked() 170. {
63
Rosario ROV
171. socket->connectToHost("192.168.7.2", 51717); 172. textEdit->append(("Connecting: 192.168.7.2, 51717")); 173. textEdit->append(("Connection: Success")); 174. textEdit->verticalScrollBar()->setSliderPosition( 175. textEdit->verticalScrollBar()->maximum()); 176. } 177. 178. // This function triggers every time the light button is pressed 179. void MainWindow::on_lightsbutton_clicked() 180. { 181. // The lights are switched 182. lights = !lights; 183. // The light status is reflected in the bar 184. progressBar->setValue(lights*100); 185. // The action is recorded in the event log 186. textEdit->verticalScrollBar()->setSliderPosition(textEdit-
>verticalScrollBar()->maximum()); 187. 188. } 189. 190. 191. // This function converts all data to string format, 192. // and appends the data into the control packet 193. // The packet is then sent to the ROV 194. void MainWindow::sendControls() 195. { 196. // Initialize packet array 197. QByteArray packet; 198. // Append horizontalslider value 199. packet.append(QString::number(horizontalSlider->value())); 200. // Append Delimiter 201. packet.append(";"); 202. // Append light value 203. packet.append(QString::number(light_slider->value()*lights)); 204. socket->connectToHost("192.168.7.2", 51717); 205. // Write to ROV 206. socket->write(packet,1024); 207. } 208. // This function updates the GUI with the newest received data 209. void MainWindow::update() 210. { 211. // Read the newest data from the text file 212. QString filename1 = "/home/alex/Desktop/control.txt"; 213. QFile file1(filename1); 214. file1.open(QIODevice::ReadOnly|QIODevice::Text); 215. QTextStream in(&file1); 216. QString str = in.readLine(); 217. 218. //QString name = str.split(";")[1]; 219. // Parse the x acceleration data data 220. double data = str.split(";")[1].toDouble(); 221. // Display x acceleration data 222. xaccel_lcd->display(data); 223. 224. //QString name = str.split(";")[2]; 225. // Parse temperature data 226. data = str.split(";")[2].toDouble(); 227. // Display temperature data 228. temperature_lcd->display(data); 229. 230. // Parse the leak alert data 231. leak = str.split(";")[3].toDouble();
64
Rosario ROV
232. // Display leak alert data 233. leak_lcd->display(leak); 234. // Display a warning if the leak detected 235. if (leak > 100 && leak_known == false) 236. { 237. QMessageBox::critical(this, "Critical: Leak Detected","Leak Detected in
main housing."); 238. leak_known = true; 239. } 240. 241. 242. file1.close(); 243. } 244. 245. void MainWindow::on_leak_mode_clicked() 246. { 247. leak_known = false; 248. }
65
Rosario ROV
Appendix H – mainwindow.h
1. #ifndef MAINWINDOW_H 2. #define MAINWINDOW_H 3. 4. #include <QMainWindow> 5. #include <QTcpSocket> 6. #include <QAbstractSocket> 7. #include "ui_mainwindow.h" 8. #include "qcgaugewidget.h" 9. 10. class VlcInstance; 11. class VlcMedia; 12. class VlcMediaPlayer; 13. 14. class MainWindow : public QMainWindow, public Ui::MainWindow 15. { 16. Q_OBJECT 17. 18. 19. public: 20. MainWindow(QWidget *parent=0); 21. int lights; 22. int servo; 23. int leak; 24. bool leak_known; 25. 26. private slots: 27. 28. void on_actionConnect_triggered(); 29. void on_connectbutton_clicked(); 30. void on_lightsbutton_clicked(); 31. void update(); 32. void sendControls(); 33. 34. 35. void openLocal(); 36. void openUrl(); 37. 38. void on_leak_mode_clicked(); 39. 40. private: 41. QTcpSocket *socket; 42. QcGaugeWidget *mCompassGauge; 43. QcNeedleItem *mCompassNeedle; 44. 45. VlcInstance *_instance; 46. VlcMedia *_media; 47. VlcMediaPlayer *_player; 48. }; 49. 50. #endif // MAINWINDOW_H