1

Robustness Analysis of Genetic Programming Controllers for

Unmanned Aerial Vehicles

Gregory J. Barlow1,2 and Choong K. Oh2

1 The Robotics Institute, Carnegie Mellon University2 The U.S. Naval Research Laboratory

2

Motivation

• Evolutionary robotics (ER) controllers may evolve in simulation or on real robots, but the true test of performance must happen in real-world conditions

• Testing unfit controllers may be dangerous or expensive for some robots

3

Transference

• For controllers evolved in simulation, evaluation in a noisy environment does not ensure good transference if simulated noise is not consistent with true noise

• If a controller performs well over a wide range of state and sensor noise conditions in simulation, prior work suggests that the controller should transfer well

4

Evolving controllers for unmanned aerial vehicles

• Unmanned aerial vehicles (UAVs) require assurance of off-design performance

• Even under noise not considered during evolution, controllers must still be able to efficiently accomplish the task

• Poorly performing controllers could cause crashes, possibly destroying the UAV

5

Overview

• Controller evolution (Barlow et al., 2004)• Goals• Robustness testing• Results• Conclusions

6

Controller evolution

Evolve unmanned aerial vehicle (UAV) navigation controllers able to:• Fly to a target radar based only on

sensor measurements• Circle closely around the radar• Maintain a stable and efficient flight

path throughout flight

7

Controller Requirements

• Autonomous flight controllers for UAV navigation

• Reactive control with no internal world model

• Able to handle multiple radar types including mobile radars and intermittently emitting radars

• Robust enough to transfer to real UAVs

8

Simulation

• To test the fitness of a controller, the UAV is simulated for 4 hours of flight time in a 100 by 100 square nmi area

• The initial starting positions of the UAV and the radar are randomly set for each simulation trial

9

Sensors

• UAVs can sense the angle of arrival (AoA) and amplitude of incoming radar signals

10

UAV Control

EvolvedController

AutopilotUAVFlight

Sensors

Roll angle

11

Radars

• Stationary, continuously emitting• Mobile, continuously emitting• Stationary, intermittently emitting with

regular period• Stationary, intermittently emitting with

irregular period• Mobile, intermittently emitting with

regular period

12

Transference

• To encourage good transference to real UAVs, during evolution:

• Modeled only the sidelobes of radars• Added noise to the modeled radar emissions• Set accuracy of the angle of arrival values to be

within ±10°• Evolved controllers were successfully

tested on wheeled mobile robots (Barlow et al., 2005)

13

Multi-objective GP

• We had four desired behaviors which often conflicted, so we used NSGA-II (Deb et al., 2002) with genetic programming to evolve controllers

• Each evaluation ran 30 simulations• Each of 50 evolutionary runs had a

population size of 500• We used environmental incremental

evolution to produce controllers evolved for a total of 1800 generations

15

Functions and Terminals

Functions• Prog2, Prog3, IfThen, IfThenElse, And, Or, Not,

<, <=, >, >=, < 0, > 0, =, +, -, *, /, X < 0, Y < 0, X > max, Y > max, Amplitude > 0, AmplitudeSlope > 0, AmplitudeSlope < 0, AoA > Arg, AoA < Arg

Terminals• HardLeft, HardRight, ShallowLeft, ShallowRight,

WingsLevel, NoChange, rand, 0, 1

16

Considerations

• We have many acceptable controllers on the Pareto front, but we need to choose one “best” controller

• Controllers may be optimized to the simulation parameters, may not be robust to other noise levels or sources

• Fitness values are only measured over 30 trials

17

Goals

• Choose a single “best” evolved controller for future flight tests

• Evaluate the robustness of the best evolved controllers to sensor and state noise to assure off-design performance

• Compare evolved controllers to human designed controllers

18

Test functions

1. Flying to the radar• Percent error in time to radar

2. Circling the radar• Average circling distance

3. Efficient flight• Percent error in flying with a roll angle of zero

degrees

4. Stable flight• Cost of sharp, sudden turns

19

Test functions

T

iii

T

ii

in

T

iout

outintotal

rollanglerollangleT

testT

test

T

T

testT

TTtest

DD

T

TTT

11}10 angle {roll4

1}rangein if{2

expect

expect1

level} wings{if}rangein if{

3expect

expect1

expect

11

distance11

111

45

seconds 3600hour 1

knots 80

seconds 14400

20

Baseline Values

Flying to the radar ≤ 0.2• Error in flight time to radar less than 20%

Circling the radar ≤ 2• Average distance less than 2 nmi

Efficient flight ≤ 0.5• ~50% of time (not in-range) with roll angle = 0

Turn Cost ≤ 0.05• Turn sharply less than 0.5% of the time

21

Performance metrics

• Failures• Percent of trials that don’t meet the baseline values

• Normalized maximum• Magnitude of failure normalized by the baseline value

• Normalized mean• Means for each test function normalized by the baseline

value and then averaged

• Average rank• Combination of first three performance metrics

22

Performance metrics

3

14

1

1 13

,2

1

),(3

1),(

)),((min)),((max

)),((min),(),(

),(11

1),(

),(maxmax),(

),(

kk

kGkG

kGkk

M

m m

R

r

N

nmm

m

mmNRM

tfnormtfmetric

tgmetrictgmetric

tgmetrictfmetrictfnorm

baseline

nrtestNR

baseline

Mtfmetric

baseline

baselinenrtesttfmetric

N

Ftfmetric

23

Selecting controllers for testing

1. GP produced 25,000 controllers2. Based on prior work, 1,602 had

acceptable mean fitness values3. We ran 100 simulations on each of the

five radar types for each of these 1,602 and chose ~300.

4. We cut these down to 10 using the normalized maximum performance metric

24

Designed controllers

• Hand-written• Based on functions and terminals available

to GP and knowledge of good GP strategies

• Proportional-derivative (PD)• Takes AoA as input (approximates

derivative)• PID performed poorly with mobile radars,

so integral term was not used

25

Robustness tests

• Robustness tests fell into five categories: AoA error, amplitude error, varied airspeed, heading error, and wind effects (position error)

• For every combination of radar type and controller, we performed 10,000 simulations, for a total of 50,000 simulations per controller per test

26

Robustness tests

• Angle of arrival error ±{10, 15, 20, 30}°• Amplitude error {6, 12} dB• UAV airspeed {50, 80, 100} knots• Heading error {0, 0.5, 1, 1.5, 2}°• Wind (position error) {0, 5, 10, 20, 30}

knots

27

Results

• For each test, we ranked the 12 controllers based on the four performance metrics

• We combined these results into an overall ranking to determine the best controller

• The best evolved controller fails gracefully and compares well to the PD controller

28

Rankings

Overall ranking best 2 3 4 5 6 7 8 9 10 11 12

failures G D E F J H A C B pd I hd

norm maximum pd D I F G hd J E B A H C

norm mean D E hd G F J H pd A C B I

average rank D G E pd F J H hd A B C I

 

Control case ranking best 2 3 4 5 6 7 8 9 10 11 12

failures pd A B C D E F G H I J hd

norm maximum pd G E J F I B A D C H hd

norm mean pd I J A B C G E D F H hd

average rank pd J G I E B A F C D H hd

29

Comparison

 Stationary,

intermittently emitting

Mobile, intermittently

emitting

Average for all five radar types

Test type D pd D pd D pd

control case 1.18 0.07 13.87 0.03 7.13 0.04

AoA=15 6.55 100.0 16.74 100.0 9.88 100.0

AoA=20 20.10 100.0 24.81 100.0 16.15 100.0

AoA=25 35.39 100.0 34.90 100.0 23.49 100.0

AoA=30 77.15 100.0 85.25 100.0 90.24 100.0

Amp=12 1.41 0.04 13.21 0.07 7.24 0.04

Speed=50 0.56 100.0 16.53 100.0 6.63 100.0

Speed=100 2.63 92.23 14.83 93.00 8.96 92.69

 Stationary,

intermittently emitting

Mobile, intermittently

emitting

Average forall five radar

types

Test type D pd D pd D pd

Head=0.5 1.70 0.19 14.38 0.18 7.51 0.17

Head=1.0 4.48 2.50 16.08 2.60 9.66 2.56

Head=1.5 9.29 54.30 20.31 55.03 13.32 55.06

Head=2.0 19.85 98.55 29.38 98.64 19.58 98.60

Wind=5 2.40 0.01 15.70 0.14 8.41 0.12

Wind=10 21.13 1.39 30.82 1.11 24.98 0.51

Wind=20 49.96 95.12 60.77 94.83 66.77 96.45

Wind=30 63.43 96.64 86.10 99.67 75.44 98.39

30

Conclusions

• Selected a single best controller for future flight tests

• Established the off-design performance of evolved UAV controllers; evolved controllers failed gracefully

• Performance of the best controller compares favorably with PD control

• Established the limits of performance for these evolved controllers

31

Acknowledgements

• Financial support was provided by Swampworks project office of the Office of Naval Research

• The U.S. Naval Research Laboratory (Code 5730) provided computation time on their Beowulf cluster

• Gregory J. Barlow is supported by a National Defense Science and Engineering Graduate Fellowship