1

Incremental Evolution of Autonomous Controllers for

Unmanned Aerial Vehicles using Multi-objective Genetic

Programming

Gregory J. Barlow, Choong K. Oh, and Edward GrantNorth Carolina State University

U.S. Naval Research Laboratory

2

Overview

• Problem• Unmanned Aerial Vehicle Simulation• Multi-objective Genetic Programming• Fitness Functions• Experiments and Results• Conclusions• Future Work

3

Background

We have previously evolved unmanned aerial vehicle (UAV) navigation controllers able to:• Fly to a target radar based only on

sensor measurements• Circle closely around the radar• Maintain a stable and efficient flight

path throughout flight

4

Problem

• We are most interested in the more difficult radar types, particularly intermittently emitting, mobile radars

• Evolving controllers directly on the most difficult radars yields very low rates of success

• We would like to create controllers able to handle all of the radar types rather than having one controller for each type

5

Simulation

• To test the fitness of a controller, the UAV is simulated for 4 hours of flight time in a 100 by 100 square nmi area

• The initial starting positions of the UAV and the radar are randomly set for each simulation trial

6

Sensors

• UAVs can sense the angle of arrival (AoA) and amplitude of incoming radar signals

7

UAV Control

EvolvedController

AutopilotUAVFlight

Sensors

Roll angle

8

Transference

These controllers should be transferable to real UAVs. To encourage this:

• Only the sidelobes of the radar were modeled

• Noise is added to the modeled radar emissions

• The angle of arrival value from the sensor is only accurate within ±10°

9

Multi-objective GP

• We had four desired behaviors which often conflicted, so we used NSGA-II (Deb et al., 2002) with genetic programming to evolve controllers

• Each fitness evaluation ran 30 trials• Each run had a population size of 500• Computations were done on a Beowulf

cluster with 92 processors (2.4 GHz)

10

Functions and Terminals

Turns• Hard Left, Hard Right, Shallow Left,

Shallow Right, Wings Level, No ChangeSensors• Amplitude > 0, Amplitude Slope < 0,

Amplitude Slope > 0, AoA <, AoA >Functions• IfThen, IfThenElse, And, Or, Not, <, =<,

>, >=, > 0, < 0, =, +, -, *, /

11

Fitness Functions

Normalized distance• UAV’s flight to vicinity of the radar

Circling distance• Distance from UAV to radar when in-range

Level time• Time with a roll angle of zero

Turn cost• Changes in roll angle greater than 10°

12

Normalized Distance

1

i

0

T

i 0

i1

fitness

idistance

distance

T

distance

distance

Tfitness

minimize tois goal The

at time distance theis

radar the to UAV thefrom

distance initial theis

steps timeofnumber total theis

1

1

2

13

Circling Distance

2

i

T

iii2

fitness

in_range

N

distancein_rangeN

fitness

minimize tois goal The

otherwise 0 and range,-in is UAV when the1 is

target theof range-in steps timeofnumber theis

)(*1

1

2

14

Level Time

3

i

T

iii3

fitness

level

levelin_rangefitness

maximize tois goal The

otherwise 0 and level, are wings the when1 is

*)1(1

15

Turn Cost

4

i

i

T

i1iii4

fitness

iroll_angle

h_turn

roll_angleroll_angleh_turnT

fitness

minimize tois goal The

at time UAV theof angle roll theis

10 than moreby changed has angle roll theif 1 is

*1

1

16

Performance of Evolution

• Multi-objective genetic programming produces a Pareto front of solutions, not a single best solution.

• To gauge the performance of evolution, fitness values for each fitness measure were selected for a minimally successful controller.

17

Baseline Values

Normalized Distance ≤ 0.15• Determined empirically

Circling Distance ≤ 4• Average distance less than 2 nmi

Level Time ≥ 1000• ~50% of time (not in-range) with roll angle = 0

Turn Cost ≤ 0.05• Turn sharply less than 0.5% of the time

18

Experiments

Continuously emitting, stationary radar• Simplest radar case

Intermittently emitting, stationary radar• Period of 10 minutes, duration of 5 minutes

Continuously emitting, mobile radar• States: move, setup, deployed, tear down• In deployed over an hour before moving again

Intermittently emitting, mobile radar• Most difficult radar type for evolution

19

Direct Evolution

Radar TypeRuns Controllers

Total Succ. Rate Total Avg. Max.

Continuously emitting, stationary radar 50 45 90% 3,149 63 170

Continuously emitting, mobile radar 50 36 72% 2,266 45.3 206

Intermittently emitting, stationary radar 50 25 50% 1,891 37.8 156

Intermittently emitting, mobile radar 50 16 32% 569 11.38 93

20

Incremental Evolution

• Environmental incremental evolution was used to improve the success rate for evolving controllers

• A population is evolved on progressively more difficult radar types

21

Incremental Evolution

Radar TypeRuns Controllers

Total Succ. Rate Total Avg. Max.

Continuously emitting, stationary radar 50 45 90% 2,815 56.30 166

Continuously emitting, mobile radar 50 45 90% 2,774 55.48 179

Intermittently emitting, stationary radar 50 42 84% 2,083 41.66 143

Intermittently emitting, mobile radar 50 37 74% 1,602 32.04 143

22

Comparison

23

Intermittently emitting, mobile radar

24

Conclusions

• Autonomous navigation controllers were evolved to fly to a radar and then circle around it while maintaining stable and efficient flight dynamics

• Using incremental evolution dramatically increased the chances of producing successful controllers

• Incremental evolution produced controllers able to handle all radar types

25

Future Work

• We have successfully tested evolved controllers on a wheeled mobile robot equipped with an acoustic array tracking a speaker

• Controllers will be tested on physical UAVs for several radar types in field tests next year

• Distributed multi-agent controllers will be evolved to deploy multiple UAVs to multiple radars