Mixed Initiative planning and control ofUAV teams for Persistent Surveillance

Final Preparation Report

Joao Fortuna - ee07245@fe.up.ptJoao Tasso Sousa - Mentor

http://paginas.fe.up.pt/∼ee07245/

February 17, 2012

Contents

List of Figures ii

Acronyms iii

1 Introduction 1

2 Literature and State of the Art 22.1 UAV teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Mixed Initiative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.3 Control Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.4 Decision Support Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Background 53.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2 Arduino . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.3 Platforms available at LSTS . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.4 Operation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 The Problem 84.1 Related Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.2 Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

5 Planning and Tools 125.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Bibliography 15

i

List of Figures

2.1 Hierarchical Control Loops for a Single UAV (Adapted from [1]) . . . . . . 32.2 Agent Organization to complete a set of Tasks (Adapted from [2]) . . . . . 4

3.1 Neptus Tele-Operation and a Supervisory control consoles (from [3]) . . . . 53.2 UAV and AUV joint Mission . . . . . . . . . . . . . . . . . . . . . . . . . . 63.3 ANTEX-M model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.4 Medium class UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.5 Cularis model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.6 Alpha and ANTEX models line-up . . . . . . . . . . . . . . . . . . . . . . . 7

4.1 System Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.3 Scenario 1 - Area division with 4 Unmanned Aerial Vehicles (UAVs), sim-

plest situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.4 Scenario 2 - Area division with 4 UAVs, with a grey high interest area . . . 114.5 Scenario 3 - Area division with 4 UAVs, approximation to a rectangle . . . 114.6 Scenario 4 - Area division with 4 UAVs, transformation into a rectangle . . 11

5.1 Gantt Chart - Work Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125.2 System Breakdown Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 13

ii

Acronyms

UAV Unmanned Aerial Vehicle

UAS Unmanned Aerial System

DSS Decision Support System

PITVANT Projeto de Investigacao e Tecnologia em Veıculos Aereos Nao-Tripulados(Research and Technology in UAVs Project)

LSTS Laboratorio de Sistemas e Tecnologia Subaquatica (Underwater Systems and Tech-nology Laboratory)

FEUP Faculdade de Engenharia da Universidade do Porto

AFA Academia da Forca Aerea (Portuguese Air Force Academy)

SiL Software in the Loop

HiL Hardware in the Loop

DSV Decision Support Visualisation

GCS Ground Control Station

SA Situation Awareness

AI Artificial Intelligence

IMC Inter-Module Communication

SBS System Breakdown Structure

AUV Autonomous Underwater Vehicle

iii

Chapter 1

Introduction

UAVs are increasingly found in military, search and rescue operations. By removingthe human pilot from dangerous flight conditions and having smaller sizes than regularmanned aircrafts, hence making them harder to detect, UAVs are becoming essential forboth civilians and military forces all over the world. Even though some of these UAVs areable to perform fully autonomous flights, human operators are needed to assign tasks tothe vehicles as well as pilots for emergency situations.The Projeto de Investigacao e Tecnologia em Veıculos Aereos Nao-Tripulados (Researchand Technology in UAVs Project) (PITVANT) is a joint operation of Laboratorio deSistemas e Tecnologia Subaquatica (Underwater Systems and Technology Laboratory)(LSTS), at FEUP, and Academia da Forca Aerea (Portuguese Air Force Academy) (AFA)that aims at further developing UAV know-how and technology.This work will focus on developing/implementing algorithms for cooperative control ofUAVs to aid in (persistent) surveillance operations. To guarantee that UAVs can do itefficiently, several optimization algorithms (scheduling, path planning, etc.) need to beconsidered when creating/editing flight plans. However, even the best and more complexoptimization algorithms do not produce the best solution. It is so because it is not possi-ble to include all the information which influences the efficiency of the solution. Can theperformance of such algorithms be increased by introducing more sensors? Can an auto-mated system make decisions comparable with a restriction from an experienced humanoperator? Probably not. Nevertheless, the system can still provide important informationfor the operator and it can help him make decisions. A human operator included in theplanning and control loops introduces the concept of mixed initiative control.Systems known as Decision Support System (DSS) do what was just mentioned, theyintegrate sensor and user inputs and feed them to optimization algorithms to create aninitial flight plan, this plan is then presented to the human operator who may choose toaccept or refuse it.If the operator accepts the proposed plan, it is simply sent to the controllers responsiblefor flying the UAV. If not, another plan needs to be created. The optimization algorithmcannot be run with the same inputs, or it would produce the same plan, which has justbeen refused. To handle this problem more data is needed, a new restriction. The operatoris to be asked for this new restriction, and the system has to know what to ask for.What to ask, what to present and how to integrate optimization algorithms and restrictionsby human operators will be the focus of this work. These algorithms will be implementedusing technologies developed at LSTS.This report will start by giving an overview of the state of the art, then the problem willbe laid out. Following that, related and future work before a few final remarks.

1

Chapter 2

Literature and State of the Art

Persistent Surveillance - A collection strategy that emphasizes the abil-ity of some collection systems to linger on demand in an area to de-tect, locate, characterize, identify, track, target, and possibly provide bat-tle damage assessment and re-targeting in near or real-time. Persistentsurveillance facilitates the prediction of an adversary’s behaviour and theformulation and execution of pre-emptive activities to deter or forestallanticipated adversary courses of action. in Joint Publication 2-0, 2007 [4]

2.1 UAV teams

Teams of multiple UAVs, can accomplish different results than those of a single UAV.The possibilities offered by cooperative teams of UAVs are very appealing for search,localization (see [5]) and persistent surveillance applications.These teams can be composed of homogeneous UAVs, which means all of them have thesame capacities and tools. This situation is seen in [6]. However, a more challengingsituation occurs when the teams have different types of UAVs (heterogeneous) [7], theycan have different sizes, operational limits and sensors.

2.2 Mixed Initiative

UAVs are complex systems, and it is difficult for an autonomous system to fully under-stand all the restrictions and environmental variables that affect UAVs performance. Theconcept of mixed initiative is to allow experienced human Operators/Pilots to interactwith planning algorithms, by introducing restrictions which are not visible or obvious forthe algorithm. These restrictions which may seem simple for a human, sometimes wouldrequire vast sensor networks and time costly processing for an automated system to deducethem.In [8], some limitations of purely Artificial Intelligence (AI) based systems in comparisonwith Mixed Initiative ones are discussed.

2.3 Control Architecture

Currently, UAV operations require a ratio of humans per UAV greater than 1. However,there has been an effort to invert that ratio so that one operator can control severalUAVs. In [1] we learn that there are many human factors that cannot be quantified likeoverconfidence, tiredness, attention allocation, which are difficult to take into considerationwhen building a DSS. Results analysed in the same article show that for an operator tocontrol multiple UAVs, the Motion Control loop (see Fig. 2.1) has to be fully automatedand reliable. In that situation, with a Navigation Control loop which requires human

2

Navigation AutopilotMotion Control

Mission & Payload Management

System Health & Status Monitoring

Pitch, yaw, airspeed & altitude control

Planning & execution for obstacle avoidance &

route headings

Sensor, communications &

payload management

Figure 2.1: Hierarchical Control Loops for a Single UAV (Adapted from [1])

validation the optimal number of UAVs per operator is 2-4. If the Navigation Controlloop becomes fully automated, we can see a jump to a plateau of 8-12 UAVs per operator,who interacts only with the outer control loop, the Mission Management. These optimalvalues were obtained by extensive testing and observation by experts, cost of missed targetsand cost in terms of mission delays introduced by inefficient human interactions.In [9] it is possible to see that once again, the Motion Control has to be fully automated.The Navigation Control is defined as three controller processes: Waypoint, Orbit andVision. The Mission Management loop assigns tasks to UAVs, composed of chains ofNavigation processes, and tries to fulfil the objectives defined by the human operator,asking for interaction when the resources are not enough. [9] focuses in search and patrol,while [1] often mentions attacking the target, systems like these have a much higher needfor operator confirmation before the UAVs engage in offensive manoeuvres.We can estimate that a higher number of observer UAVs can be under a single operator’sresponsibility when compared to offensive ones.

2.4 Decision Support Systems

A Decision Support Visualisation (DSV) is an important part of a DSS because it canmake the interaction much easier for the human operator. In [10] we can read aboutdifferent types of DSVs and how well they perform. Some results were unexpected, andare explained by the human behaviour factor which was hard to predict. Although theDSVs described in this article present the data to the operator in a way that it is easier tounderstand and that increases the operator’s Situation Awareness (SA), they do not pro-vide any recommendations. That functionality would probably increase the performancegreatly.The systems described so far were implemented in a static operator architecture, butin [2] a different design is introduced. The UAV’s operator is a pilot flying on his ownaircraft. This requires the pilot to split his attention between managing the UAVs andflying himself a fast jet. The UAVs are controlled by a set of collaborating agents, andthe different AI planning techniques are wrapped inside specialist planning agents. EachUAV’s movement is controlled by a UAV Agent that is coordinated by a Group Agent whoin turn gets his tasks from the User Agent, the pilot (see Fig. 2.2). Another importantcontribution of [2] is the conclusion that sometimes a sub-optimal, but simpler system isa better choice for implementation because it allows for a easier interface with the pilotand faster algorithms.

3

User Agent

Group Agent Group Agent

Task CMission:

Tasks A & B

UAV Agent UAV Agent UAV Agent UAV Agent

Task C Task C Task A Task B

AI Wrapper Agent

Figure 2.2: Agent Organization to complete a set of Tasks (Adapted from [2])

4

Chapter 3

Background

3.1 Introduction

The Projeto de Investigacao e Tecnologia em Veıculos Aereos Nao-Tripulados (Researchand Technology in UAVs Project) is a joint operation of Faculdade de Engenharia daUniversidade do Porto and Academia da Forca Aerea (Portuguese Air Force Academy),it started in 2009 and is scheduled to finish at the end of 2015. So far, approximately 30UAVs have been/are part of PITVANT’s fleet and there is a constant expansion.At PITVANT there is an effort to develop cooperative control for multiple UAVs withmixed initiative, data fusion and navigation systems. There is also an objective to trainpersonnel who can define requirements, operate and maintain UAVs. Several tools andplatforms have been developed at Laboratorio de Sistemas e Tecnologia Subaquatica (Un-derwater Systems and Technology Laboratory):

• Neptus (Fig. 3.1)

• Dune

• IMC

• Platforms (see section 3.3)

they will be discussed later in this report.The Laboratorio de Sistemas e Tecnologia Subaquatica (Underwater Systems and Tech-nology Laboratory), as the name suggests was created for Autonomous Underwater Vehi-cle (AUV) research, so one of the interesting Mission Scenarios will be interaction betweenUAVs and AUVs, see Fig. 3.2.Persistent surveillance and mixed initiative control are of great interest for the Air Force,and an important reason for their investment in the PITVANT.

3.2 Arduino

Arduino is a cheap microcontroller with a programming language similar to C++. Thereis a community dedicated to developing an autopilot for Arduino, Ardupilot. Because ofthe low price, ease of programming and a base program to work on, Arduino is a good

Figure 3.1: Neptus Tele-Operation and a Supervisory control consoles (from [3])

5

Figure 3.2: UAV and AUV joint Mission

choice as a prototyping and testing tool.At the moment there are Ardupilot-mega available to start testing, while the new modelArdupilot-mega 2.0 will be available soon.

3.3 Platforms available at LSTS

There are three classes of UAVs defined by their size, as follows:

• Large - Ex.: ANTEX-X03 series (see Fig. 3.3), not at LSTS because of its size (seeFig. 3.6 for comparison) but a part of the PITVANT;

• Medium - Ex.: Pilatus (see Fig. 3.4a) and ANTEX-X02 (a.k.a. Alpha) (see Fig.3.4b) series;

• Small - Ex.: Cularis (see Fig. 3.5) series;

3.4 Operation Setup

UAV missions require a set of systems, some are autonomous but others have to managehuman decisions.A typical basic setup includes:

• Ground Control Station (GCS) - and its certified Operator

• Certified Pilot

• UAV - and autopilot

• Communication system

• Possible on-board data collection system (payload)

Medium and Small classes will be the main targets for this work, at least in a firstapproach.

6

Figure 3.3: ANTEX-M model

(a) Pilatus model (b) Alpha model

Figure 3.4: Medium class UAVs

Figure 3.5: Cularis model

Figure 3.6: Alpha and ANTEX models line-up

7

Chapter 4

The Problem

UAVs are suited for persistent surveillance applications, which require several UAVs andpossibly other systems to coordinate. There is a need for a system/tool that allows a singleOperator to efficiently plan a mission for multiple UAVs. Such system should considerinputs from AI algorithms and human Operators. LSTS has developed a set of tools whichcan integrate such Decision Support System.

4.1 Related Problems

Examples of operation scenarios for multiple UAVs systems are:

• Fire detection and tracking

• Coastline surveillance, of particular interest to Portugal considering the geography

• Target search

4.2 Statement

A DSS is to be developed that will enable a single Operator to coordinate multiple UAVs.Having Fig. 2.1 in mind, we need to abstract from the inner control loops, which havealready been fully automated, either by Dune (see 2) (from LSTS) or Ardupilot (see 3.2).Mission Management, the outer loop, can be used in a first stage of planning before theUAVs are deployed.With the current setup an operator is required to manually input all the waypoints foreach UAV, which is not practical at all and does not guarantee any kind of optimality inresource allocation.In order to solve this issue, we need a system that automatically generates plans formultiple UAVs simultaneously. This system will have as inputs:

• Mission Type (Persistent Surveillance),

• Mission Parameters (Area, Altitude, Priorities),

• Available UAVs,

• UAVs Status,

• UAVs Type and Tools,

and generate a plan for each UAV.The plan will be created in order to optimize the update rate of information captured(photographs for example) at each location, dividing the total area into n smaller areas,using n− 1 divisions, where n in the number of available UAVs [11]. See Fig. 4.3.

8

Path PlanningAgent

Plans

UAVs Status

.txt, .csv, .xmlType of Mission

Mission Parameters

Available UAVs

Decision Support System

User Inputs

Figure 4.1: System Interfaces

When defining areas to be patrolled, some sections of those areas may be of more interest,so they require a higher update frequency. See Fig. 4.4. Another area might be of highrisk so it is preferable to send a smaller and harder to detect UAV. The system should beable to understand these restrictions given by the operator to propose a better plan.Initially the system should divide the area in as many smaller areas as the number ofavailable UAVs. This division would take into consideration the previously mentionedrestrictions.Then, the path each UAV will follow inside its assigned area, will be determined by a pathplanning algorithm. This path planning algorithm can be a simple spiralling or zigzagingmanoeuvre during the development of this system, but later it may be replaced by asystem currently being worked on at LSTS.This system can be represented as in Fig. 4.1, and it can fit with the rest of the setup likein Fig. 4.2The previous cases require the defined area to be a rectangle, some additions to the simplecases could handle irregular four-side polygons. There are two possible ways to apply therectangle algorithm:

1. Approximate the area to a rectangle. See Fig. 4.5

2. Transform the area into a rectangle by stretching. See Fig. 4.6

After a first plan is presented to the Operator, some changes might be required by him/her:

• Change the proposed areas.

• Change the proposed altitude.

• Change the UAV proposed for a specific area.

These changes can be made before the UAVs are deployed, or during the mission. Theseinteractions of the Operator are classified as mixed initiative.When making changes during the mission execution, limits should be imposed to thosechanges. If an operator were to change plans every few seconds it would lead to systeminstability.

9

Waypoint

Waypoint

Decision Support System

Consult

Path Planning Agent

Status Ground Station

(Neptus)

Comm.(IMC)

Waypoints

Status

Autopilot(Dune) Waypoints

Status

Missions,Restrictions, Confirmation

Sensors/Actuators

Status

Create

Plans

Read

Suggestions,Input Requests

Waypoints

Operator(s)

Figure 4.2: System Architecture

D1

D2

D3

D4

Figure 4.3: Scenario 1 - Area division with 4 UAVs, simplest situation

10

D1 D3

D4 D2

Figure 4.4: Scenario 2 - Area division with 4 UAVs, with a grey high interest area

D1

D2

D3

D4

Figure 4.5: Scenario 3 - Area division with 4 UAVs, approximation to a rectangle

D1

D2

D3

D4

Figure 4.6: Scenario 4 - Area division with 4 UAVs, transformation into a rectangle

11

Chapter 5

Planning and Tools

Fig. 5.1 shows a simplified overview of the work plan, which may be subject to minorchanges during the semester.

1. Introduction to the PITVANTSince August 2011 an effort has been made to take part in the PITVANT missions.Fig. 5.2 shows a simple System Breakdown Structure (SBS) of the Unmanned AerialSystem (UAS), developed according to the current knowledge of the complete system.Only the basic system elements are represented (top levels of a complete SBS).

2. PITVANT Command and Control Architecture ReviewThe UAS needs several Software tools to accomplish the given tasks. The main onesare:

• A Decision Support System running on the base that will coordinate the indi-vidual UAV tasks in order to complete the common goal. This will be the focusof this work.

• Neptus - a GCS software, it can, among other things, show autonomous vehicleson a map, send them waypoints and paths and receive status updates (sensordata for example). See [12].

• Inter-Module Communication (IMC) - a message-oriented protocol, see [13].

• Dune - a task manager running on-board that executes the commands given bythe Ground Station.

Fig. 4.2 shows a basic System Architecture with the tools mentioned above.

3. State of the Art Survey

4. Neptus CourseNeptus [12] [14] is a GCS software developed at LSTS. As it will be an importantpart of the project do be developed it is necessary to understand how it works andhow it can include the DSS.

Figure 5.1: Gantt Chart - Work Plan

12

UAS

UAV GCS

Autopilot

Payload

Comms

Actuators

Comms Computer

Operators

Pilots

Development and Test

SiL

HiL

Platform Manufacturing

AFA

INEGI

OtS

Training/Certification

Pilots

Operators

Platforms

Platform

Ground Force

DecisionSupportSystem

Figure 5.2: System Breakdown Structure

5. Background ResearchNeptus, Dune and IMC have specific architectures. In order to communicate withthese systems it is important to understand their structures and interfaces.Platforms described in 3.3 have limitations, it is crucial to know them.

6. Requirements SurveyThese are some examples of requirements:

• Send commands to the GCS, to be executed by the UAVs

• Receive data from UAVs, static sensors, operators and other potentially relevantdata sources and use them to optimally complete de given objective

• Maintain persistent surveillance - update frequency of data from each areashould be lower than a maximum limit

7. Manoeuvres IdentificationFor example:

• Search/patrol area

• Area mapping

• Create network

8. Problem FormulationAfter the problem has been identified and the requirements have been surveyed itis important to define what will be the limitations of the solution so that it will befeasible. Compromises will have to be made with the information available at thetime.

9. ApproachDuring the development of this project, a gradually more complex system will becreated. This system could, at each big step, be able to:

(a) propose plans based on given data, before take-off

(b) propose new plans after human intervention, before take-off

(c) change a plan based on new data gathered from UAVs, after take-off

(d) change a plan after human intervention, after take-off

13

10. ImplementationAs a prototyping tool, Arduino is a good choice.Ardupilot uses the MAVLink communication protocol. To allow it to interface withNeptus, a MAVLink - IMC translator has been developed.

11. Simulation TestsUASs are expensive and their operation involves some risks so it is crucial to first doextensive Software in the Loop (SiL) and Hardware in the Loop (HiL) tests. Thereare several tools, either Open Source or developed at LSTS, that allow such tests.

12. Real-World TestsAfter the system has been tested in simulation it is important to validate it in thereal world, these tests will use PITVANT’s platforms and software.

5.1 Future Work

The Association for Unmanned Vehicle Systems International (AUVSI) holds an annualcontest for students, the tasks involved include target location and area search. If thiswork is successful it maybe possible to use some of the results to participate in the nextAUVSI Student UAS Competition.Another important development to build on the resulting system is an online system whichcan re-allocate UAVs as their autonomy decreases, other UAVs become available, and thesearch area or conditions change.

14

Bibliography

[1] M. L. Cummings, S. Bruni, S. Mercier, and P. J. Mitchell. Automation architecturefor single operator, multiple uav command and control. The International C2 Journal,1(2), 2007.

[2] J. W. Baxter, G. S. Horn, and D. P. Leivers. Fly-by-agent: Controlling a pool of uavsvia a multi-agent system. Knowledge-Based Systems, 21(3), 2008.

[3] Authority Sharing in Mixed Initiative Control of Multiple Uninhabited Aerial Vehicles,Orlando, Florida, USA, July 2011.

[4] Joint Intelligence. Joint publication 2-0, June 2007.

[5] J. Tisdale, A. Ryan, Z. Kim, D. Tornqvist, and K. Hedrick. A multiple uav system forvision-based search and localization. In IEEE American Controls Conference, June2008.

[6] J. B. Sousa, T. Simsek, and P. Varaiya. Task planning and execution for uav teams.In 43rd IEEE Conference on Decision and Control, December 2004.

[7] A. S. Ferreira. Interface de operacao para veıculos nao tripulados. Master’s thesis,FEUP, February 2011.

[8] S. F. Smith, O. Lassila, and M. Becker. Configurable, mixed-initiative systems forplanning and scheduling. In ARPI. Carnegie Mellon University, 1996.

[9] A. Ryan, X. Xiao, S. Rathinam, J. Tisdale, M. Zennaro, D. Caveney, R. Sengupta, andK. Hedrick. Collaborative flight demonstration with three unmanned aerial vehicles.Proceedings of the AIAA, Guidance, Navigation and Control Conference, Keystone,CO, 2006.

[10] M. L. Cummings and A. S. Brzezinski. Global vs. local decision support for multipleindependent uav schedule management. International Journal of Applied DecisionSciences, 3(3):188–205, 2010.

[11] N. Nigam and I. Kroo. Persistent surveillance using multiple unmanned air vehicles.In Aerospace Conference, 2008 IEEE, pages 1 –14, March 2008.

[12] P. S. Dias, R. M. F. Gomes, J. Pinto, G. M. Goncalves, J. B. Sousa, and F. L. Pereira.Mission planning and specification in the neptus framework. In IEEE InternationalConference on Robotics and Automation, 2006.

[13] R. Martins, P.S. Dias, E.R.B. Marques, J. Pinto, J.B. Sousa, and F.L. Pereira. Imc:A communication protocol for networked vehicles and sensors. In Proc. IEEE OceansEurope (OCEANS’09). IEEE, 2009.

[14] G. M. Goncalves, E. Pereira, J. B. Sousa, J. Morgado, R. Bencatel, J. Correia, andL. Felix. Unmanned air vehicles for coastal and environmental research. In 10thInternational Coastal Symposium - ICS 2009, 2009.

15