autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming

Integrated Computer-Aided Engineering 14 (2007) 141–159 141IOS Press

Autonomous and cooperative roboticbehavior based on fuzzy logic and geneticprogramming

James F. Smith III∗ and ThanhVu H. NguyenCode 5741, Naval Research Laboratory, Washington, DC, 20375-5320, USA

Abstract. Advances in a fuzzy decision theory that allow automatic cooperation between unmanned aerial vehicles (UAVs)are discussed. The algorithms determine points the UAVs are to sample, flight paths, and the optimal UAVs for the task andrelated changes during the mission. Human intervention is not required after the mission begins. The algorithms take intoaccount what is known before and during the mission about UAV reliability, fuel, and kinematics as well as the measurementspace’s meteorological states, terrain, air traffic, threats and related uncertainties. The fuzzy decision tree for path assignmentis a significant advance over an older fuzzy decision rule that was previously introduced. Simulations show the ability of thecontrol algorithm to allow UAVs to effectively cooperate to increase the UAV team’s likelihood of successfully measuring theatmospheric index of refraction over a large volume. A genetic program (GP) based data mining procedure is discussed forautomatically evolving fuzzy decision trees. The GP is used to automatically create the fuzzy decision tree for real-time UAVpath assignments. The GP based procedure offers several significant advances over previously introduced GP based data miningprocedures. These advances help produce mathematically concise fuzzy decision trees that are consistent with human intuition.

1. Introduction

Autonomous cooperative teams of robots will beused for many applications in the near future. Funda-mental to this process will be mission planning priorto the mission and algorithms for automatic controland cooperation of the robots in real-time during themission.

In the following, algorithms based on fuzzy logic aredescribed that can be used to plan missions for a coor-dinated team of flying robots. The robotic unmannedaerial vehicles (UAVs) will be rendered autonomous.Human intervention is not required after the missionbegins when the algorithms described below are used.

The UAVs’ goal is to cooperate to measure the atmo-spheric index of refraction. The fuzzy mission planningalgorithm uses human expertise to determine the pointsto be sampled, the points to avoid, the best flight paths

∗Corresponding author. Tel.: +1 202 767 5358; Fax: +1 202 4047690; E-mail: [email protected].

for sampling and a small number of the best UAVs toconduct the mission.

The planning algorithm takes into account the inputproperties of each UAV under consideration including:the UAVs risk tolerance, i.e., how much risk the UAVsowner will allow it to experience, expert estimates ofthe UAV’s sensor and non-sensor system reliabilities;and the amount of fuel the UAV carries. The planningalgorithm also takes into account what is known aboutthe atmospheric volume where measurements are to bemade, i.e., the measurement space prior to the mission.This information includes weather, e.g., rain, turbu-lence, and icing conditions; physical obstructions suchas mountains, and high tension wires; enemy behavior;air traffic such as civilian or military aircraft or animallife. This information is used to form a measure of risk,using a fuzzy risk tree. The risk tree allows a simplemathematical formulation of the mission risk.

Also, the planning algorithm takes into account elec-tromagnetic propagation [4,9] as well as human exper-tise to determine points in the atmosphere to sample,

ISSN 1069-2509/07/$17.00 2007 – IOS Press and the author(s). All rights reserved

142 J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming

RISK-TOL VALUE FAST LOW-RISK

VMR

RMP

SR NSR MP

RMP

SR NSR MP

AUP

RISK-TOL VALUE FAST LOW-RISK

VMR

RMP

SR NSR MP

RMP

SR NSR MP

AUP

Fig. 1. The AUP subtree.

their priority and a simple mathematical formulation ofthe mission priority.

A real-time algorithm expressed as a fuzzy decisiontree is formulated for assignment of the best UAV toeach path. The algorithm that “assigns UAVs to paths”(AUP) is referred to as the AUP fuzzy decision tree andit is depicted in Fig. 1.

The AUP fuzzy decision tree is a simple elegantmathematical formula determining the degree to whicheach UAV belongs to the path in terms of risk-tolerance,sensor and non-sensor reliability, UAV fuel limitations,mission priority and mission risk.

The predecessor to the AUP fuzzy decision tree wasthe AUP fuzzy decision rule [20,22,24]. Both the AUPdecision rule and AUP decision tree were initially con-structed based on human expertise. The AUP fuzzydecision rule had some of the properties of the tree,but was more limited in its decision making ability andsubject to certain types of errors that the tree was de-signed to avoid. The AUP decision tree is a significantadvance over the AUP decision rule [22,24].

The AUP fuzzy decision tree is used by both theplanning and real-time control algorithm. It can makeextremely fast assignments of UAVs to paths, whileallowing the various input concepts to remain explicitin the formulation.

Although the AUP decision tree was originally con-structed using human expertise, another method forevolving it using a genetic program (GP) [7] as a datamining (DM) function has been created. The GP iscapable of recreating the original tree, but has subse-quently produced trees that are different but arguablysuperior in their performance properties. The GP isguided by built in fuzzy logic based on partial exper-tise and related uncertainty. This procedure results in

fuzzy decision trees that are mathematically more con-cise. The trees also assume a form that is more con-sistent with human intuition making them easier to un-derstand. By making the tree more concise and easierto understand it is easier to introduce new rules on thetrees for the purpose of innovation. Also, concise andintuitive results frequently facilitate validation.

Classical if-then rules have been used in the pastto guide GP evolution, notably for the purpose of re-verse engineering hardware designs [21,25]. The useof fuzzy rules to guide the GP is a significant advanceover using classical if-then rules. Fuzzy logic offers abetter way of dealing with the uncertainties associatedwith partial knowledge.

A GP is a computer program based on the theoryof evolution that automatically evolves other computerprograms or mathematical expressions [7]. The math-ematical expressions evolved here are fuzzy decisiontrees.

The GP based procedure is a DM technique, a kindof pattern recognition. The GP mines a database ofscenarios to produce an optimal tree. An optimal so-lution is one that maximizes the fitness function. Thefitness function is constructed using the database ofscenarios. The GP is guided by fuzzy logic to improvethe GP’s convergence time, to reduce size of the treeand produce elegant mathematical formulations under-standable by human beings. Elegance and concisenessof form, and understandability are essential to furtherinnovation. The fuzzy decision trees that are created bythe GP based DM procedure are unlike black box algo-rithms like neural nets where understanding parameterrelations is generally out of the question.

Being able to automatically generate decision algo-rithms using GP based data mining is a significant ad-

J.F. Smith III and T.H. Nguyen / Autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming143

vance. It is frequently difficult to acquire enough if-then rules from experts to produce a full tree, whereasexperts can often provide opinions about the status ofa scenario.

Other approaches to creating optimal flight pathswith UAV assignments might use a genetic algorithm(GA) or dynamic programming. Both approachesmight be suitable for a pre-mission planning algorithmwhere there is plenty of time for computationally in-tensive algorithms to run. They are unlikely to be suit-able for real-time application, when dealing with slowlegacy processors. The AUP fuzzy decision tree re-quires little CPU time and can produce effective deci-sions even on slow legacy processors. Also, the one-time decisions made by the GA [8,27] or dynamic pro-gramming approaches [1,30] would be represented asnumbers, not mathematical expressions. It would bedifficult or impossible to extract explicit relationshipsbetween input quantities like reliability, risk-tolerance,etc., and output quantities like the ultimate assignmentof a UAV to a path.

The planning algorithm makes determinations priorto the mission’s execution. Inevitably, during the mis-sion, events will occur that demand changes to the pre-viously determined flight path. The real-time controlalgorithm allows the UAVs’ task to change during themission. These changes can include alteration in flightpath, changing sample points and the need for auto-matic cooperative behavior between UAVs. As pathschange in real-time the AUP fuzzy decision tree is usedfor reassignment.

A fuzzy decision rule referred to as the priority ofhelping (PH) decision rule is also providedas part of thecontrol algorithm that allows the UAVs to cooperate au-tomatically in three ways. This automatic cooperationis based on communication; there is no fixed or cen-tral command platform: the UAVs automatically self-organize. The information transferred between UAVsis a small number of fuzzy grades of membership, socommunication bandwidth requirements are very low.

The PH fuzzy decision rule allows UAVs to collabo-rate to make atmospheric measurements. It also allowsthem assist each other when malfunctions are suspectedor take over when a malfunction has occurred.

The PH fuzzy decision rule is a simple elegant mathe-matical relationship between various fuzzy concepts es-sential to collaboration. It makes explicit relationshipsthat would be impossible to discern using a black boxlike a neural net [6] or an optimization procedure [1,5,8,11,12,27,28,30] that delivers only a numerical outputlike a GA or dynamic programming.

The planning and control algorithms described be-low could be used for many different cooperative atmo-spheric measurement processes such as tracking chemi-cal plumes [26], determining properties of atmosphericducts [4], rain systems, etc. The application that moti-vated this work was a need to measure the atmosphericindex of refraction for the purpose of geo-location [20,24].

Section 2 discusses the electromagnetic measure-ment space, UAV risk, and the planning algorithm. Sec-tion 2 also discusses the UAV path construction algo-rithm that determines the minimum number of UAVsrequired to complete the task, a fuzzy logic based ap-proach for assigning paths to UAVs and which UAVsshould be assigned to the overall mission. Finally Sec-tion 2 discusses a genetic program based data miningprocedure for evolving the fuzzy decision tree for as-signing UAVs to paths. Section 3 describes the con-trol algorithm that renders the UAVs autonomous. Sec-tion 3 also describes the priority for helping algorithm,a part of the control algorithm based on fuzzy logic thatdetermines which UAV should support another UAV re-questing help. The three subclasses of help requests arealso discussed in this section. Section 4 discusses ex-perimental results including UAV path determination,UAV path assignment, determination of which UAVsshould fly the mission and the result of a request for helpduring the mission. Section 5 provides conclusions.Finally, Section 6 describes future research directions.

2. Planning, AUP tree and the evolution of logic

The measurement space consists of the electromag-netic propagation environment where UAVs and theIP make their measurements. This environment in-cludes sample points and the desirable neighborhoodsthat surround them. The sample points or the desirableneighborhoodsare where the UAVs will make measure-ments. The method of determining the sample pointsand desirable neighborhoods is described below.

The measurement space also includes taboo pointsand the undesirable neighborhoods that surround them.The taboo points are points of turbulence and other phe-nomena that could threaten the UAVs. The undesirableneighborhoods surrounding them also represent variousdegrees of risk. The method of specifying taboo pointsand quantifying the degree of risk associated with theirundesirable neighborhoods employs fuzzy logic and isdiscussed in this section.


The planning algorithm allows the determination ofthe minimum number of UAVs needed for the missionsubject to fuel constraints, risk, UAV cost, and impor-tance of various points for sampling. Risk refers toturbulent regions or regions undesirable for other rea-sons, e.g., the presence of enemy observers or physi-cal obstructions. The planning algorithm automaticallyestablishes the order in which to send the UAVs tak-ing into account the UAV’s value; onboard sensor pay-load; onboard resources such as fuel, computer CPUand memory; etc. The priority of sample points andtheir desirable neighborhoods are taken into account.The planning algorithm also calculates the optimal patharound undesirable regions routing the UAVs to or atleast near the points to be sampled.

In the planning phase, the location of the electro-magnetic source (EMS) is unknown. Some positionsare more likely than others for the EMS’s location.When establishing likely positions for the EMS, humanexperts are consulted. The experts provide subjectiveprobabilities of the EMS being located at a numberof positions. These likely EMS locations are referredashypothesis positions. Ray-theoretic electromagneticpropagation [4,20] is conducted from each hypothesisposition to each interferometer element on the interfer-ometer platform (IP). The points on the sampling gridnearest the points of each ray’s passage are the samplepoints. The priority of a sample point is related to thesubjective probability of the hypothesis position fromwhich the associated ray emerges. Sample points aris-ing from the highest probability hypothesis positionshave priority one; sample points associated with lowerprobability hypothesis positions, priority two; etc.

Each sample point is surroundedby what are referredto asdesirable neighborhoods. Depending on localweather, topography, etc., the desirable neighborhoodsare generally concentric closed balls with a degree ofdesirability assigned to each ball. The degree of de-sirability characterizes the anticipated variation in theindex of refraction. If for that region of the measure-ment space, the spatial variation of the index of refrac-tion is slow, the degree of desirability may assume itsmaximum value of unity for a ball of radius measuredin miles. For regions of space where the index of re-fraction’s spatial variation is greater, the degree of de-sirability may fall off much more rapidly, approachingthe minimum value of zero after just a mile or two.

The desirable region need not have spherical geom-etry. Rotational symmetry may be broken by a varietyof processes, e.g., an elevated duct, a radio hole, etc.

The notion of a desirable neighborhood is motivatedby the fact that a sample point may also be a taboo

point or reside within an undesirable neighborhood.In the case the sample point coincides with or is neara taboo point and at least part of the sample point’sdesirable neighborhood falls within the taboo point’sundesirable neighborhood, the UAV may only samplewithin a desirable neighborhood that is consistent withits risk tolerance.

A point may be labeled taboo for a variety of rea-sons. A taboo point and the undesirable neighborhoodscontaining the point generally represent a threat to theUAV. The threat may take the form of high winds,turbulence, icing conditions, mountains, etc. The un-desirable neighborhoods around the taboo point relateto how spatially extensive the threat is. A methodof quantifying risk and incorporating it into the pathassignment algorithm is presented that offers concep-tual improvements over an approach previously de-veloped [20]. This section uses fuzzy logic to quan-tify how much risk a given neighborhood poses for aUAV. This quantitative risk is then incorporated into theUAV’s cost for traveling through the neighborhood asdescribed in this section. Once the cost is establishedan optimization algorithm is used to determine the bestpath for the UAV to reach its goal.

When determining the optimal path for the UAVsto follow both the planning algorithm and the controlalgorithm running on each UAV take into account taboopoints and the undesirable neighborhood around eachtaboo point. The path planning algorithm and controlalgorithm will not allow a UAV to pass through a taboopoint. Depending on the UAV’s risk tolerance a UAVmay pass through various neighborhoods of the taboopoint, subsequently experiencing various degrees ofrisk. Both the concepts of risk and risk tolerance arebased on human expertise and employ rules each ofwhich carry a degree of uncertainty. This uncertaintyis born of linguistic imprecision [24,29], the inabilityof human experts to specify a crisp assignment for risk.Owing to this uncertainty it is very effective to specifyrisk and risk tolerance in terms of fuzzy logic.

2.1. Risk fuzzy decision tree

Risk is represented as a fuzzy decision tree [3,15–19]. The risk subtree defined below and displayed inFig. 2 is a subtree of the larger risk tree that was actuallyused. The risk tree is used to define taboo points andthe undesirable neighborhoods surrounding the taboopoints.

The root concepts on the risk tree use the membershipfunction defined in Eqs (1)–(3),


Weather

Temperature

Cold Heat

Rain

Precipitation

Snow

Sleet

Icing

Fog

Hail

Hostile Action/

Observation

Turbulence

Wind-Shear

Wind

Air Pocket

Traffic

Other UAVs Civilian

Own Military

Allied Military

Neutral Military

Birds/Insects

Air-Pollution

Physical Obstructions

Mountains Trees

Buildings

High Tension Wires

Smoke Plumes

Suspended

Sand

Suspended Systems &

Contaminants

Risk

Weather

Temperature

Cold Heat

Rain

Precipitation

Snow

Sleet

Icing

Fog

Hail

Hostile Action/

Observation

Turbulence

Wind-Shear

Wind

Air Pocket

Weather

Temperature

Cold Heat

Rain

Precipitation

Snow

Sleet

Icing

Fog

Hail

Hostile Action/

Observation

Turbulence

Wind-Shear

Wind

Air Pocket

Hostile Action/

Observation

Turbulence

Wind-Shear

Wind

Air Pocket

Traffic

Other UAVs Civilian

Own Military

Allied Military

Neutral Military

Birds/Insects

Traffic

Other UAVs Civilian

Own Military

Allied Military

Neutral Military

Birds/Insects

Traffic

Other UAVs Civilian

Own Military

Allied Military

Neutral Military

Birds/Insects

Air-Pollution


Mountains Trees

Buildings

High Tension Wires

Smoke Plumes

Suspended

Sand

Suspended Systems &

Contaminants

Air-Pollution


Mountains Trees

Buildings


Mountains Trees

Buildings

High Tension Wires

Smoke Plumes

Suspended

Sand

Suspended Systems &

Contaminants

Risk

Fig. 2. The fuzzy risk tree and its 25 fuzzy root concepts.

µγ (�qtaboo, �x) (1)

=

1, if r = 03/4, if 0 < r � 1 · ∆l1/2, if 1 · ∆l < r �

√2·∆l

1/4, if√

2·∆l < r �√

3·∆l

0, if r >√

3·∆l

r = ‖�x− �qtaboo‖ , (2)

�qtaboo = position of taboo point. (3)

where the “taboo point,”�qtaboo is the point at which therisk phenomenonhas been observed. The root conceptsused on the risk subtree are given in Eq. (4), and thesubscriptγ is an element of the root concept set,RC,i.e.,

γ ∈ RC = {Mountains, High Tension Wires,

Buildings, Trees, Smoke Plumes,

Suspended Sand, Birds/Insects,

Other UAVs, Air Polution, Civilian,

Own Military, Allied Military,

Neutral Military, Cold, Heat,

Icing, Rain, Fog, Sleet, Snow, Hail,

Air Pocket, Wind, Wind Shear,

Hostile Action/Observation}. (4)

The norm in Eq. (2) is typically taken as an Euclideandistance. The quantity∆l for most applications isgenerally assigned a value of one mile or more. Inextreme cases it can be much larger than a mile.

The fuzzy membership function for the compositeconcept “risk” is defined as

µrisk (�qtaboo, �x) = maxα∈RC

µα (�qtaboo, �x) . (5)


2.2. Optimal paths and AUP fuzzy decision tree

The best path algorithm is actually an optimizationalgorithm that attempts to minimize a cost functionto determine the optimal trajectory for each UAV tofollow, given a priori knowledge. The cost function forthe optimization algorithm takes into account variousfactors associated with the UAV’s properties, missionand measurement space. Two significant quantities thatcontribute to the cost are the effective distance betweenthe initial and final proposed positions of the UAV andthe risk associated with travel.

For purposes of determining the optimal path, theUAV is assumed to follow a rectilinear path consistingof connected lines segments, where the beginning andending points of each line segment reside on the UAV’ssampling lattice. Let A and B be two grid points onthe UAV’s sampling grid with corresponding positionvectors,�rA and�rB , respectively. Denote the Euclideandistance between A and B asd (�rA, �rB). Letv (�rA, �rB)be the speed at which the UAV travels in going from�rA to�rB. If both�rA and�rB are sample points then theUAV travels at sampling velocity, otherwise it travelsat non-sampling velocity. The path cost is given by

path cos t (�rA, �rB) =

d(�rA,�rB)+β·ntaboo∑

i=1

µrisk(�ti,�rB)

v(�rA,�rB)

. (6)

where ntaboo is the number of taboo points, i.e.,columns in the taboo point matrix

Taboo≡ [�t1,�t2, . . . ,�tntaboo

](7)

and�ti, i = 1, 2, . . . , ntaboo are the taboo points de-termined to exists in the measurement space whenpath cos t (�rA, �rB) is calculated. The number of pointsin the measurement space is finite sontaboo is finite.The quantity,β, is an expert assigned parameter. Notethat pathcos t (�rA, �rB) is an effective time. When risk

is not present, i.e.,β ·ntaboo∑i=1

µrisk

(�ti, �rB

)is zero, then

path cos t (�rA, �rB) is the actual travel time. When riskis present then the travel time is increased. The param-eterβ helps to penalize risky paths, thus decreasing theprobability they are selected.

If the candidate path for the mission consists of thefollowing points on the UAV lattice given by the pathmatrix in Eq. (8),

Pathi =[�r1, �r2, . . . , �rn

], (8)

then the total path cost is defined to be

total cost(Pathi) ≡n−1∑j=1

pathcost(�rj , �rj+1). (9)

Determining the optimal path for theith UAV con-sists of minimizing the total path cost given by Eq. (9)such that there is enough fuel left to complete the path.

An A-star algorithm [13] is used to determine thepoint that will ultimately minimize the cost function.A-star is a heuristic algorithm. It was selected becauseit is relatively fast and can be polynomial in time [13].Empirically it is significantly faster than the Dijkstraalgorithm [13]. It is very effective for cases where asignificant percentage of the atmospheric volume doesnot change over the course of a mission.

The A-star algorithm as implemented in the planningand control algorithms is an easily replaceable mod-ule. It may well be replaced by a more sophisticatedalgorithm in the future [5,11,12,28].

The planning algorithm determines the path eachUAV will pursue, which points will be sampled, theminimum number of UAVs required for sampling thepoints and makes assignments of UAVs for measure-ments at particular points. UAVs are assigned as afunction of their abilities to sample high priority pointsfirst. The planning algorithm determines flight paths byassigning as many high priority points to a path as pos-sible taking into account relative distances includingsampling and non-sampling velocity, risk from taboopoints, and UAV fuel limitations. Once flight pathsare determined, the planning algorithm assigns the bestUAV to each path using the fuzzy logic decision treefor path assignment described in this section.

The planning algorithm must assign UAVs to theflight paths determined by the optimization proceduredescribed below in this section. This is referred to asthe UAV path assignment problem (UPAP). The plan-ning algorithm makes this assignment using the follow-ing fuzzy logic based procedure. To describe the de-cision tree it is necessary to develop some preliminaryconcepts and notation.

Each UAV will fly from lattice point to lattice point,i.e., grid point to grid point, let one such route be givenby the matrix of points,

Path=[�P1, �P2, . . . , �Pnpath ,

�P1

](10)

where the ordering of points gives the direction ofthe route, i.e., starting at�P1 and ending at�P1. Letthe taboo points be those given in Eq. (7). Let thedegree of undesirability of the neighborhood associ-ated with taboo points,�ti, i = 1, 2, . . . , ntaboo be


denotedµrisk

(�ti, �Pj

)for the route points�Pj , j =

1, 2, . . . , npath. The definition of the mission risk (MR)is

missionrisk(Taboo,Pathk) ≡ntaboo∑i=1

npath∑j=1

µrisk

(�ti, �Pj

)(11)

The degree to which thekth path belongs to therelated fuzzy conceptMR is given by

µMR (Taboo,Pathk) ≡missionrisk(Taboo,Pathk)

maxj

{missionrisk

(Taboo,Pathj

)} (12)

The “max” operation in Eq. (12) is taken over theset of all possible UAVs that can be assigned to themission.

A fuzzy concept related to “mission risk” is “lowrisk.” The fuzzy membership function for “low risk”denoted asµLR is defined as

µLR (Taboo,Pathk) ≡min (1, α + 1 − µMR) (13)

whereα ∈ (0, 1) is an expert defined parameter. Thefunction ofα is to make sure that “low risk” does notdominate calculations developed below.

Within the path specified by Eq. (10), let there bethe following sample points to be measured,�Sj , j =1, 2, . . . , nsp. Let the functionprio assign prioritiesto the sample points, i.e, prio( �Sj) is the priority of thejth sample point. The values that prio( �Sj) can takeare positive integers with one representing the highestpriority, two the next highest priority, etc. The missionpriority (MP) for thekthPathk is defined to be

missionprio(Pathk) ≡nsp∑i=1

1

prio(�Si

) . (14)

The degree to which thekth path belongs to therelated fuzzy conceptMP is given by

µMP (Pathk) ≡ missionprio(Pathk)maxj

{missionprio(Pathj)} .(15)

The fuzzy degree of reliability experts assign to thesensors of UAV(i) is denoted asµsr (UAV (i)). This isa real number between zero and one with one implyingthe sensors are very reliable and zero that they aretotally unreliable. Likewise,µnsr (UAV (i)) is thefuzzy degree of reliability of other non-sensor systems

onboard the UAV(i). This fuzzy concept relates to anynon-sensor system, e.g., propulsion, computers, harddisk, deicing systems, etc. The value of UAV(i) inunits of $1000.00 is denoted asV (UAV (i)). Theamount of fuel that UAV(i) has at timet is denotedfuel (UAV (i) , t). All the UAVs participating in amission are assumed to leave base at time,t = to.

Let UAV(i)’s fuzzy grade of membership in thefuzzy concept “risk tolerance” be denoted asµ risk−tol

(UAV (i)). The quantity,µrisk−tol (UAV (i)), is anumber between zero and one and will be simply re-ferred to as UAV(i)’s risk tolerance. If the risk toler-ance is near zero then the UAV should not be sent onvery risky missions. If the UAV’s risk tolerance is nearone then it can be sent on very risky missions. It seemsnatural to compare “risk tolerance” to “Value.” So thecomparison can be carried out on the same footing, afuzzy concept of value should be defined.

The fuzzy grade of membership of each UAV thatcan be assigned to the mission in the fuzzy concept“Value” is defined as

µV (UAV (i)) ≡ εV · Value(UAV (i))maxj

{Value(UAV (j))} (16)

The quantityεV is an expert assigned value. It isa number between zero and one that insures that themost valuable UAV can still be assigned to a path. Themotivation for defining this parameter will be clearerafter the AUP subtree is defined below, since it can beobserved that ifεV is zero, then for the most valuableUAV in the team,µAUP will take the value zero. Thiswould prevent it from being assigned a path which isundesirable.

The advantage of the concept of “risk tolerance” isthat it gives the user an extra concept to exploit. If theUAV is not of great relative value, but it still might beneeded for a crucial mission after the current one, itmight be useful to give it a low risk tolerance so that itis not lost on the current mission. This may allow it tobe used on a subsequent mission.

Another fuzzy concept and related fuzzy member-ship function that will be defined is “fast.” A UAVis said to be fast if it permitted to pursue a particularpath and the time it would take to complete the pathis small. It is not allowed to travel the path unless theUAV’s fuel level, reliability, risk-tolerance and missionpriority exceed certain tolerances.

Let theT (UAV (i) ,Path) be the amount of time itwill take UAV(i) to fly and make measurements alongPath. The fuzzy membership function for the concept“fast” is defined as follows:


µfast (UAV (i) , Path) ≡ Λrrtmp· (17)

min

1, α + 1 − T (UAV (i) ,Path)

maxj

{T (UAV (j) ,Path)}

and

Λrrtmp ≡ χ [fuel(UAV (i) , to) + εfuel

−T (UAV (i) ,Path)] ·χ [min (µsr, µnsr) − ε1,rel· (18)

min(1 − µrisk−tol,

max(1 − µMP , ε2,MP )) − ε3,rel]

whereε1,rel, ε2,MP , ε3,rel, εfuel ∈ (0, 1] are expert as-signed parameters. The Heaviside step functions de-noted asχ in Eq. (18) takes the value one when itsargument is greater than or equal to zero and is zerootherwise.

The factor in Eq. (18) determines whether the UAVwill be permitted to travel the path at all. If its amount offuel, sensor and non-sensor reliabilities, risk-toleranceand mission priorities do not exceed certain expert de-fined tolerances it will not be permitted to travel and assuch it will certainly not be “fast.”

The termε1,rel ·min(1 − µrisk−tol,max(1 − µMP ,ε2,MP )) in the Heaviside step function’s argument inEq. (18) can result inΛrrtmpgoing to zero ifµrisk−tol

or µMP are small enough.The parameterα is selected so thatµfast in Eq. (17)

does not go to zero for the UAV that takes the longesttime to navigate Path. By preventingµfast from goingto zero in this case, the slowest UAV can be selected ifit can complete the path and its grades of membershipin the other fuzzy concepts found in Eq. (17) are highenough.

If “Risk tolerance” and “mission priority” take lowvalues then depending on the value ofε1,rel, the mem-bership function for the fuzzy concept “fast” may takethe value zero. The parameterε2,MP limits the effectof “mission priority.” Even if the mission priority isvery high, risk tolerance plays an important role. If theUAV has high risk tolerance and the path, high mis-sion priority the UAV must have a minimum reliabilitygiven byε3,rel. Finally, the motivation for the concept“fast” is that a fast UAV experiences a lower relativerisk since it is in the field less time and may be exposedto risk for a shorter duration.

A fuzzy concept that combines “Value” and “missionrisk” is “VMR” and its membership function denotedasµVMR is defined as

µVMR ≡ min [min (µrisk−tol, 1 − µV ) ,

AND2 (µfast, µLR)] (19)

The use ofAND2 in Eq. (19) allows distinctionsto be made between various values ofµ fast andµLR.If AND2 were replaced by amin in Eq. (19) then ifµfast is low enough thenmin (µfast, µLR) would takethe valueµfast independent of the value ofµLR thiswould not allow fine distinctions to be made.

The logical connectiveAND2 is defined as

AND2 (µA, µB) ≡ µA · µB (20)

The fuzzy concept “RMP” combines the fuzzy con-cepts sensor reliability (sr) non-sensor-reliability (nsr)and “MP.” The fuzzy membership function for “RMP,”denoted asµRMP is defined as

µRMP ≡ min (µsr, µnsr, µMP ) . (21)

Table 1 provides an example of the application ofEq. (21) for a three UAV scenario.

Both the membership functions for “VMR” and“RMP” can be represented as fuzzy decision trees.

Finally, the fuzzy membership function for the fuzzyconcept “assignment of UAV(i) to the path” (AUP) isdefined as

µAUP ≡ AND2[µRMP ,

AND2 (µRMP , µVMR)]

= µ2RMP · µVMR (22)

The fuzzy membership function for AUP is a deci-sion tree that combines both “VMR” and “RMP” assubtrees. The use ofAND2 in Eq. (22) in two placesrendersµAUP more sensitive to the values ofµRMP

andµVMR than it would be if the membership func-tion for AUP took the valuemin (µRMP , µVMR). IfµAUP were to take the valuemin (µRMP , µVMR) thena small value ofµRMP such thatµRMP < µVMR

would causeµAUP to take the value ofµRMP inde-pendent of the value ofµVMR. The use ofAND2 in-stead ofmin allows finer distinctions to be made. Thesecond degree dependence ofµRMP in Eq. (22) resultsin a small value ofµAUP if µRMP is small, butµAUPis still dependent onµVMR. This is consistent withexpertise. If the sensor or non-sensor reliabilities ormission priority are small,µAUP should be small. Lowreliability or priority results in a faster decline inµAUPthan high mission risk, high UAV value, low UAV risktolerance or the fact that a reliable and risk-tolerantUAV is slow.

It should be observed that the decisions made by AUPare taken over a pool of candidates. So it is reasonable


Table 1Three UAV example for the composite concept RMP

Three UAV Numerical Example for Use of RMP

Fuzzy Membership Function UAV 1 UAV 2 UAV 3

Sensor reliability,µsr 0.8 0.7 0.8Non-sensor reliability,µnsr 0.9 0.4 0.8Risk Tolerance,µrisk−tol 0.3 0.4 0.5Mission Priority,µMP 0.6 1 0.4µRMP ≡ min (µsr , µnsr , µMP ) 0.6 0.4 0.4

those UAVs in the pool that have greater sensor andnon-sensor system reliabilities and greater relative mis-sion priority, should have a higher fuzzy grade of mem-bership for assignment to the path under consideration.AUP’s second degree dependence on RMP may allowRMP to be too dominant in the calculations. In thegenetic program based approach described below ver-sions of RMP were evolved where the power of RMPwas between 1.5 and 1.7. These different versions ofRMP are still under study.

The fuzzy concept AUP is depicted as a tree in Fig. 1.Leaves of the tree, i.e., those vertices of degree one arelabeled by the names of the fuzzy concepts describedabove. Vertices are labeled by the specific logical con-nective used, i.e.,min or AND2. A circle on an edgeindicates the fuzzy logic modifiernot. The fuzzy mod-ifier not is defined as the complement of the fuzzy set,i.e., letµA be the fuzzy membership function for thefuzzy conceptA then membership function fornot Ais given by1 − µA. A three UAV example is providedin Table 1 to illustrate the use of the RMP subtree ofFig. 1.

Given the fuzzy grade of membership it is necessaryto defuzzify, i.e., make definite UAV-path assignments.Simply assigning the UAV with the highest fuzzy gradeof membership for a particular path to that path can giveless than desirable results. The approach to defuzzifi-cation taken is as follows: if the number of UAVs isdenoted asnUAV and likewise, the number of paths isdenoted bynpath, wherenUAV � npath then considerthe set of all possible permutations of thenpath UAVsselected from a total ofnUAV UAVs. For each assign-ment ofnpath UAVs to the paths, add up the values ofµAUP for that assignment over the paths. This sum isreferred to as the assignment benefit (AB). The assign-ment with the highest AB is the one selected. Finally,a similar procedure is followed ifnUAV < npath.

The decision tree for AUP given in Eq. (22) was con-structed using expertise provided by human experts.It is a significant improvement over a previously de-veloped fuzzy decision rule for path assignment alsoconstructed from expertise [24]. An alternate method

of obtaining Eq. (22) is to evolve it using a geneticprogram (GP). A GP is a computer program based onthe theory of evolution that evolves mathematical ex-pressions or computer programs that can be consideredoptimal in a sense. The GP has been used as a datamining function to create the decision tree in Eq. (22).The GP data mined a scenario database where eachscenario had been labeled by an expert. Expert ruleswere also incorporated to guide the evolutionary pro-cess and improve convergence time. The decision treein Eq. (22) has been evolved many times. The GP findsthe same AUP decision tree, over and over again inde-pendent of the seed of the random number generatorused to simulate a random evolutionary process. TheGP based procedure is described in greater detail in thenext subsection and an earlier version in [19].

2.3. GP creation of the AUP tree

The AUP tree given in the previous subsection arosefrom rules provides by human experts. In this subsec-tion a method of evolving the AUP tree using data min-ing is discussed. This procedure has been successfulin evolving the exactly same tree found in Fig. 1. Thefact that two significantly different methods of obtain-ing the AUP algorithm give the same results provides asignificant level of confidence in the AUP tree. Finally,the fact that the data mining approach is an optimiza-tion procedure, i.e., it gives results that are optimal in asense should further strengthen confidence in the AUPtree.

Data mining is the efficient extraction of valuablenon-obvious information embedded in a large quantityof data [2]. Data mining consists of three steps: theconstructionof a database that represents truth; the call-ing of the data mining function to extract the valuableinformation, e.g., a clustering algorithm, neural net,genetic algorithm, genetic program, etc; and finally de-termining the value of the information extracted in thesecond step, this generally involves visualization.

In a previous paper, a genetic algorithm (GA) wasused as a data mining function to determine parameters


for fuzzy membership functions [14]. Here, a differentdata mining function, a genetic program [7] is used. Agenetic program is a problem independent method forautomatically evolving computer programs or mathe-matical expressions.

The GP data mines fuzzy decision tree structure, i.e.,how vertices and edges are connected and labeled ina fuzzy decision tree. The GP mines the informationfrom a database consisting of scenarios.

The GP evolves a population of decision trees. Eachtree is a candidate solution for the AUP tree. To createthese candidate solutions the GP requires two inputsets. These are the terminal set and function set.

At the end of each generation the GP ultimately ratesor determines the fitness of each candidate solution pro-duced during that generation. Its method of rating thecandidate solutions is to calculate their fitness using afitness function. When using a GP as a data miningfunction the GP requires a third type of input. Thisthird input set is a scenario database where each sce-nario has been labeled by an expert with a real numberbetween zero and one. This label corresponds to thedecision that the optimal decision tree should make.So it is natural when calculating the fitness for a givencandidate solution for the AUP tree to compare its fi-nal composite concept fuzzy membership value to thevalue labeling that scenario. So the scenario databaseis used to construct the final fitness function for ulti-mately determining the optimal AUP decision tree asdetermined by the GP.

The terminal set, function set, and fitness functionsnecessary for the GP to be used as a data mining func-tion to automatically create the AUP tree are describedbelow. The terminal set used to evolve the AUP treeconsisted of the root concepts from the AUP tree andtheir complements. The terminal set, T, is given by

T = {risk-tol, value, fast, low-risk, sr, nsr,

MP, not-risk-tol, not-valuable, not-fast,

not-low-risk, not-sr, not-nsr, not-MP}. (23)

Let the corresponding fuzzy membership functionsbe denoted as

{µrisk−tol, µvalue, µfast, µlow−risk, µsr,µnsr, µMP , µnot−risk−tol,µnot−valuable, µnot−fast,µnot−low−risk, µnot−sr, µnot−nsr,µnot−MP } .

(24)

When mathematical expressions are constructed bya GP that reproduce the entries in a database withinsome tolerance, the process is referred to as symbolic

regression [10]. It is found in symbolic regressionthat candidate solutions are frequently not in algebraicsimplest form and this is the major source of their excesslength. When candidate solutions are too long this isreferred to as bloat [10].

By including in the terminal set a terminal and itscomplement, e.g., “risk-tol,” and “not-risk-tol”; “val-ue” and “not-valuable”; etc., it is found that bloat isless and convergence of the GP is accelerated. This isa recent innovation which was not used when anotherresource manager, i.e., the electronic attack resourcemanager (EARM) was evolved using GP based datamining (DM) [17]. Additional bloat control proceduresare described below and in [23] which provides muchless detail than found here.

The mathematical form of the complement whetherit appears in the terminal set or is prefixed with a “NOT”logical modifier from the function set is one minus themembership function. To make this more explicit

µNOT (A) = µnot−A = 1 − µA, (25)

whereNOT (A) refers to the application of the logi-cal modifierNOT from the function set to the fuzzyconceptA from the terminal set. The notation,not-Arefers to the terminal which is the complement of theterminalA.

The function set, denoted as F, consists of

F = {AND1, OR1, AND2, OR2, NOT } (26)

where the elements of Eq. (26) are defined in Eqs (20),(27)–(30). Let A and B represent fuzzy membershipfunctions then elements of the function set are definedas

AND1 (A,B) = min (A,B) ; (27)

OR1 (A,B) = max (A,B) ; (28)

OR2 (A,B) = A + B −A ·B; (29)

and

NOT (A) = 1 −A. (30)

The database to be data mined is a scenario databasekindred to the scenario database used for evolving theEARM [17]. In this instance scenarios are character-ized by values of the fuzzy membership functions forthe elements of the terminal set plus a number fromzero to one indicating the experts’ opinion about thevalue of the fuzzy membership function for AUP forthat scenario.

GPs require a fitness function [7]. As its name im-plies the fitness function measures the merit or fitness


of each candidate solution represented as a chromo-some. The fitness used for data mining is referred to asthe input-output fitness.

The input-output fitness for mining the scenariodatabase takes the form

fIO(i, ndb) ≡1

1 + 2 ·ndb∑j=1

|µgp (i, ej) − µexpert (ej)|. (31)

whereej is the jth element of the database;ndb isthe number of elements in the database;µgp(ej) is theoutput of the fuzzy decision tree created by the GP fortheith element of the population for database elementej ; andµexpert(ej) is an expert’s estimate as to what thefuzzy decision tree should yield as output for databaseelementej .

The AUP tree is evolved in three steps. The first stepinvolves evolving the VMR subtree; the second step,the RMP subtree and the final step, the full AUP tree.In the second and third steps, i.e., evolving the RMPsubtree and full AUP tree from the RMP and VMRsubtrees, only the input-output (IO) fitness in Eq. (31)is calculated, i.e., the rule-fitness described below isnot used.

When evolving the VMR subtree a rule-fitness iscalculated for each candidate solution. Only when thecandidate’s rule fitness is sufficiently high is its input-output fitness calculated. The use of the rule-fitnesshelps guide the GP toward a solution that will be con-sistent with expert rules. Also the use of the rule fitnessreduces the number of times the IO fitness is calculatedreducing the run time of the GP. After some prelimi-nary definitions of crisp and fuzzy relations, a set ofcrisp and fuzzy rules that were used to help acceleratethe GP’s creation of the VMR subtree are given. Therules are combined to formulate the rule fitness.

Let T be a fuzzy decision tree that represents a ver-sion of the VMR subtree, that is to be evolved by a ge-netic program. LetA andB be fuzzy concepts. Thenlet γshare (T,A,B) = 1 if A andB share a logicalconnective denoted asC and γshare (T,A,B) = 0,otherwise.

Furthermore, define the fuzzy relation

µcom (T,A,B,C) = (32)

0.4 if C = AND1 orAND20.1 if C = OR1 orOR20, otherwise

.

The following is a subset of the rules used to accel-erate the GP’s convergence and to help produce a resultconsistent with human expertise.

R1. “not-valuable” and “risk-tol” must share a log-ical connective, denoted asC1, i.e., it is desired thatγshare (T,not-valuable,risk-tol) = 1.

R2. “not-valuable” and “risk-tol” strongly influenceeach other, so they should be connected by AND1 orAND2. So it is desired that

µcom(T,not-valuable,risk-tol,C1) = 0.4.

R3. “fast” and “low-risk” have an affinity for eachother. They should share a logical connective, denotedasC2, i.e., it is desired thatγshare (T,fast,low-risk) = 1.

R4. The fuzzy root concepts “fast” and “low-risk”strongly influence each other, so they should be con-nected by AND1 or AND2. So it is desired thatµcom (T,fast,low-risk,C2) = 0.4.

R5. There is an affinity between the fuzzy root con-ceptsC1 (not-valuable,risk-tol) andC2 (fast,low-risk),they are connected by a logical connective denoted asC3, i.e., it is desired that,

γshare (T,C1 (not-valuable,risk-tol) ,C2 (fast,low-risk)) = 1 . (33)

When the EARM was evolved by GP based data min-ing [17] bloat was controlled using adhoc proceduresbased on tree depth and parsimony pressure. Most ofthe bloat in evolving mathematical expressions with aGP arises from the expressions not being in algebraicsimplest form [10]. With that observation in mind,computer algebra routines have been introduced thatallow the GP to simplify expressions. The followingis a partial list of algebraic simplification techniquesused during the evolution of the EARM and the AUPtree. The simplification routines used when evolvingAUP are more sophisticated than those applied to thecreation of EARM [17].

One routine simplifies expressions of the formNOT(NOT(A)) = A. This can be more complicated thanit initially appears, since the NOT logical modifiers canbe separated on the fuzzy decision tree.

Another simplification procedure consists of elimi-nating redundant terminals connected by an AND1 log-ical connective. An example of this isAND1(A,A) = A.Like the case with the logical modifier NOT there canbe a separation between the AND1s and the terminalsthat add complexity to the simplification operation.

The third algebraic simplification example is like thesecond. It involves simplifying terminals connected byOR1s. Like AND1, separation between terminals andOR1 can increase the complexity of the operation.


Other types of algebraic simplification use DeMor-gan’s theorems in combination with the above proce-dures. This can significantly reduce the length of anexpression.

Another algebraic procedure that reduces the lengthof expressions includes replacement of forms likeAND2(A,A) by the square of “A,” i.e.,A2. Still an-other length reducing simplification includes replacingNOT(A) with not-A, its complement from the terminalset listed in Eq. (23).

There is always a question of how much algebraicsimplification should be conducted from generation togeneration as such the simplification algorithm allowslevels of simplification. If a low level of simplificationis selected then some parts of an expression remain thatmight be eliminated during full simplification. This hastwo advantages: it leaves chromosome subcomponentsthat may prove useful during mutation or crossover andit takes less CPU time.

Algebraic simplification produces candidate solu-tions in simpler form making it easier for human ob-servers to understand what is being evolved. Havingcandidate solutions that are easier to understand canbe an important feature for improving the evolution ofGPs.

3. Control algorithm

Each UAV has a real-time algorithm onboard itthat allows recalculation of paths during flight due tochanges in environmental conditions or mission prior-ities. These changes typically become apparent afterthe planning algorithm has run during the pre-flightstage. As in the case of the planning algorithm the con-trol algorithm uses an A-star algorithm [13] to do thebest path calculation, employs fuzzy logic and solves aconstrained optimization problem. A-star’s and fuzzylogic’s CPU time requirements have been quite satis-factory for this application.

The control algorithms’ recalculation of flight pathscan be triggered by a number of events such as weatherbroadcasts that indicate new taboo regions or changesof priority of sample points. For those changes thatdo not require UAVs supporting each other, the controlalgorithm does not differ from the planning algorithm.The control algorithm is faster by virtue that it only needprocess those parts of the measurement space wherethere have been changes relative to sample or taboopoints. The A-star algorithm is particularly effective

in an environment where change is confined to smallregions.

A UAV may requests help if it discovers a potentialelevated system like a radio hole, malfunctions or sus-pected malfunctions. All of these conditions can resultin help messages being transmitted between the UAVs.These help messages can result in interactions betweenthe UAVs based on transmission of the results of prior-ity calculations for rendering support to the requestingUAVs.

Currently in the control stage, when a UAV discov-ers an interesting physical phenomenon, is malfunc-tioning, or suspects due to internal readings that it ismalfunctioning, it sends out a request for help. EachUAV receiving this message calculates its priorities forproviding assistance to the UAV in need. This prioritycalculation gives rise to a number between zero andone, inclusive, which is subsequently transmitted to theoriginal UAV desiring support. The requesting UAVsends out an omni-directional message with the ID ofthe UAV with highest priority for contributing support.The high priority UAV then flies into the necessaryneighborhood of the requesting UAV to provide help.

There are three classes of help request. The firstoccurs when a UAV, the requester, determines it mayhave discovered an interesting physical phenomenon.This phenomenon may be an elevated duct, radio hole,rain system or some other type of system with phys-ical extent. The requester desires to determine if thephenomenon has significant extent. It will request thata helping UAV or UAVs sample likely distant pointswithin this phenomenon.

The second class of help request relates to a UAV thataccording to internal diagnostics may be experiencinga sensor malfunction. This UAV will requests thatanother UAV or UAVs measure some of the points thatthe requesting UAV measured. This will help determineif the UAV is actually malfunctioning. If the requestingUAV is determined to be malfunctioning, then it willfly back to base, if it is capable. The determinationof whether it is actually malfunctioning requires someconsideration. Since the second UAV will probablybe measuring a distant point at a time different thanthe original requesting UAV made its measurements,potential variation in the index of refraction over timemust be taken into account.

When a UAV sends out an omni-directional requestfor help, those UAVs receiving the message will calcu-late their fuzzy priority for helping, denoted as “PH.”The UAV that will ultimately help the requester is theone with the highest fuzzy priority for helping. The


fuzzy priority for helping takes into account a varietyof properties of the potential helper. The set of UAVsthat receive the request for help from UAV(i) at timet is denoted as help(i, t). If UAV(i) request help attime t and UAV(j) receives the message then UAV(j)will take into account the amount of time, denoted,help time(UAV (j)), it will take it to fly from thepoint where it received the request to the point whereit would provide support. It also takes into account theamount of fuel UAV(j) has left at the time of the re-quest, denoted fuel(UAV (j) , t); UAV(j)’s fuzzy con-cept of price denoted as“price” , and UAV(j)’s fuzzyconcept of “mission priority” at time, t. Let the set ofrelevant UAV properties be denoted asUAV prop andbe defined as

UAV prop =

{help time,fuel,missionprio,price} (34)

The fuzzy priority for helping denoted asµPH takesthe form

µPH (UAV (i) , UAV (j)) =∑

δ∈UAV prop

wδ · µδ (UAV (j)) (35)

The quantitieswδ andµδ for δ ∈ UAV prop areexpert defined weights and fuzzy membership func-tions, respectively. The fuzzy membership functionsare defined in Eqs (36)–(39) and given below,

µhelp time (UAV (i) , UAV (j)) = (36) help time(UAV (j))

maxk∈help(i,t)

{help time(UAV (k))} + 1

−1

µfuel (UAV (i) , UAV (j)) = (37)

fuel(UAV (j))max

k∈help(i,t){fuel(UAV (k))}

µmission prio (UAV (i) , UAV (j)) = (38) missionprio(UAV (j))

maxk∈help(i,t)

{missionprio(UAV (k))} + 1

−1

µprice (UAV (i) , UAV (j)) = (39) Value(UAV (j))

maxk∈help(i,t)

{Value(UAV (k))} + 1

−1

It is assumed that all evaluations are processed attime,t, so time dependence is suppressed in Eqs (35)–(39) for notational convenience. A more sophisticatedversion of the control logic that takes path risk, changesin risk, UAV reliability, UAV risk-tolerance and missedsample points into account will be the subject of a futurepublication.

4. Computational experiments

The planning and control algorithms described in theprevious sections have been the subject of a large num-ber of experiments. This section provides a descriptionof a small subset of these experiments. They serve toillustrate how the algorithms were tested.

UAV experiments using only one UAV demonstratehow the planning and control algorithm will determinethe route the UAV flies so that it is successful in makingmeasurements at sample points in space, while the UAVavoids taboo points, that is points in space that coulddamage or destroy the UAV. Experiments using twoUAVs illustrate how the control algorithm allows theUAVs to automatically support each other to increasethe probability their joint mission is successful.

Figures 2–6 use the same labeling conventions. Sam-ple points are labeled by concentric circular regionscolored in different shades of gray. The lighter theshade of gray used to color a point, the lower the point’sgrade of membership in the fuzzy concept “desirableneighborhood.” The legend provides numerical valuesfor the fuzzy grade of membership in the fuzzy concept“desirable neighborhoods”. If the fuzzy degree of de-sirability is high then the index of refraction is consid-ered to be close to the index of refraction of the sam-ple point at the center of the desirable neighborhood.This allows the UAV to make significant measurementswhile avoiding undesirable neighborhoods.

Each sample point is labeled with an ordered pair.The first member of the ordered pair provides the in-dex of the sample point. The second member of theordered pair provides the point’s priority. For example,if there arensp sample points and theqth sample pointis of priority p, then that point will be labeled with theordered pair (q, p).

Points surrounded by star-shaped neighborhoodsvarying from dark grey to white in color are taboopoints. As with the sample points, neighborhoods withdarker shades of gray have a higher grade of member-ship in the fuzzy concept “undesirable neighborhood.”The legend provides numerical values for the fuzzy


PLANNING PHASEPLANNING PHASE

Fig. 3. One UAV trajectory as determined by planning algorithm.

CONTROL PHASECONTROL PHASECONTROL PHASECONTROL PHASE

Fig. 4. One UAV trajectory as determined by real-time control algorithm.

grade of membership in the fuzzy concept “undesirableneighborhood.” UAVs with high risk tolerance may flythrough darker grey regions than those with low risktolerance. When comparing planning and associatedcontrol pictures, if a point ceases to be taboo, the neigh-borhood where it resides is marked by a very dim gray

star as well as being labeled by a dialog box as being an“old taboo point.” New taboo points and their associ-ated undesirable neighborhoodsare labeled with dialogboxes indicating that they are “new.”

Each UAV has three states of motion, it can be at rest,traveling at sampling speed or non-sampling speed.


PLANNING PHASEPLANNING PHASE

Fig. 5. Trajectory of two UAVs as determined by planning algorithm.

CONTROL PHASECONTROL PHASE

Fig. 6. During flight, updates about changes cause the real-time control algorithms on the two UAVs to change their trajectories.

Within the simulation the UAV properties other thanspeed are its risk-tolerance, cost, fuel, sensor reliabilityand non-sensor reliability. The values associated witheach property can vary over the pool of UAVs in use.

UAVs start their mission at the UAV base which is

labeled with a diamond-shaped marker. They fly in thedirection of the arrows labeling the various curves inFigs 2–5.

In Figs 2–6,flight paths that can be represented in twodimensions have been selected for easy comprehension.


In the general case the flight paths are three dimensionalspatial curves, which can be difficult to visualize.

Figure 3 provides the sample points, taboo points andsample path for one UAV as determined by the planningalgorithm. It is important to notice that the UAV’s pathpasses directly through each sample point, i.e., throughthe center of the concentric circular regions represent-ing the fuzzy degree of desirability of neighborhoods.Fortuitously, the taboo points and their neighborhoodsare so positioned that they do not interfere with theUAV’s measurement process or its return to base.

Figure 4 depicts the actual path the UAV flies asdetermined by the UAV’s real-time control algorithm.The path determined by the control algorithm differsfrom the one created by the planning algorithm due toreal-time changes in taboo points. After leaving theUAV base new weather data was acquired informing theUAVs that the exact position of the third sample point,i.e., the one labeled (3,1) actually resides within anundesirable neighborhood. Due to the high priority ofthe sample point and the UAV’s risk-tolerance, the UAVflies into the taboo points’ undesirable neighborhoodas indicated in Fig. 4.

In both the planning and control algorithms the UAVmeasures sample points of two different priorities, withthe direction of the flight path selected so that the higherpriority points are measured first. By measuring highpriority points first, the likelihood of an important mea-surement not being made is diminished, if the UAV cannot complete its mission due to a malfunction, changein weather, etc.

Also, due to movement of old taboo points or theemergence of new taboo points which are marked“New,” the path determined for the UAV using the con-trol algorithm is significantly different than the one cre-ated by the planning algorithm. The path change rep-resents the control algorithm’s ability to reduce UAVrisk.

Figure 5 depicts the sampling path determined bythe planning algorithm for an experiment involving twoUAVs. The first, UAV(1) follows the dashed curve;the second, UAV(2), the solid curve. The UAVs wereassigned to the different paths by the fuzzy path assign-ment decision tree described in Section 2. UAV(1) isassigned to sample all the highest priority points, i.e.,the priority one points. UAV(2) samples the lower pri-ority points, i.e.; those with priority two. Due to thegreedy nature of the point-path assignment algorithm,the highest priority points are assigned for samplingfirst.

Figure 6 depicts the actual flight path the UAVs takeduring real-time. Initially, UAV(1) is successful in

measuring sample points one and two as assigned it bythe planning algorithm. Just beyond sample point two,UAV(1) experiences a malfunction. UAV(1)’s real-timecontrol algorithm subsequently sends out a help requestinforming the only other UAV in the field, UAV(2) ofthe malfunction. UAV(2)’s control algorithm deter-mines a new path for UAV(2) to fly so that the priorityone points, labeled (3,1) and (4,1), that UAV(1) wasnot able to sample are subsequently measured. Af-ter UAV(2) measures sample point five, its new flightpath allows it to measure sample points three and four.UAV(2)’s control algorithm determined it was very im-portant that these priority one points be measured. Un-fortunately, due to the extra fuel expended in reassign-ing sample points three and four to UAV(2), UAV(2)did not have enough fuel to measure sample pointsseven and eight which were of priority two. UAV(2)’sreal-time control algorithm determined the best possi-ble solution in the face of changing circumstances andlimited resources.

It is important to note that the control algorithmsrunning on UAV(1) and UAV(2) direct both UAVs toalter their return paths to the base due to the emer-gence of new taboo points making the planning algo-rithm determined flight paths too dangerous. The con-trol algorithm uses each UAV’s fuzzy risk-tolerance todetermine how near each UAV may approach a taboopoint.

Figure 7 provides an example of the AUP decisiontree’s assignment of three UAVs to three paths. Thehighest priority locations are assigned to UAV(1) as ithas the greatest fuel capacity, i.e., 90 minutes. UAV(1)however does not have enough fuel to handle the highpriority points located at positions six and seven andtherefore UAV(2) is assigned these points along withthe second degree high priority locations.

Table 2 provides numerical details of the tasks de-picted in Fig. 7. The column labels have the followinginterpretation: “Location,” the UAV coordinates on themap; “Fly mode,” whether the UAV sampled from itsprevious location to its current position. If the UAVsampled then a “S” was entered. “NS” was enteredif sampling did not occur. “Fuel Time” refers to howmuch fuel remained by the time the UAV reached theassociated location.

5. Conclusions and summary

Fuzzy logic algorithms for planning and control of acoordinated team of robots making atmospheric mea-


Table 2Details of three UAV mission depicted in Fig. 7

Three UAV Mission

UAV 1 MISSION UAV 2 MISSION UAV 3 MISSIONLocations Fly Fuel Time Locations Fly Fuel Time Locations Fly Fuel Time

Mode Remain Mode Remain Mode Remain(minutes) (minutes) (minutes)

Base 90.0 Base 85.0 Base 85.0(1,1) NS 76.5088 (6,1) NS 67.9691 (11,3) NS 64.2839(2,1) S 61.5088 (7,2) S 55.2412 (12,3) S 51.0412(3,1) S 54.2662 (8,2) S 47.9986 (13,3) S 39.5559(4,1) S 42.7809 (9,2) S 39.5133 (14,3) S 31.0706(5,1) S 28.2956 (10,2) S 22.028 Base NS 6.2574Base NS 6.7113 Base NS 11.7854

Y Plane

PLAN PHASE

X P

lane

Y Plane

PLAN PHASE

X P

lane

Fig. 7. Three UAV mission described in Table 2, an example of the AUP decision tree’s assignment.

surement have been developed. The algorithms exhib-ited excellent performance under extensive testing indigital simulation.

The fuzzy logic algorithms provide explicit, con-cise, elegant mathematical relationships between inputroot concepts like risk-tolerance,sensor reliability, non-sensor system reliability, mission-risk, mission-priorityand output composite fuzzy concepts like “assign UAVto path.” Such mathematical relationships are valuable

for subsequent innovation. Only having an implicitknowledge of the relationship between concepts as fre-quently occurs if only an optimization approach is usedis less desirable. Such implicit relationships can makeinnovation considerably more difficult.

The fuzzy logic based algorithms require little CPUtime and can function in real-time even on many slowlegacy processors. They also require very little memorystorage.


The fuzzy logic algorithms allow a group of flyingrobots to collaborate without a central or fixed com-mander. Decisions are made based on communica-tion. The team of robots automatically self-organize.This is valuable since the loss of anyone robot will notdestabilize the team. Late arriving robots may join thegroup and contribute to the team’s well-being withoutdifficulty.

Self-organization arises by the UAV’s transmittingonly small amounts of information. The UAVs onlytransmit processed information, fuzzy grades of mem-bership, resulting in very low bandwidth requirements.Since the bandwidth requirements are low, no expen-sive data compression is required. This means physicalpower and time are saved. This is a valuable feature forfuture inexpensive, disposable, low powered systems.

A method of creating fuzzy decision logic using analgorithm related to the theory of evolution, a geneticprogram (GP) is discussed. This algorithm’s evolutionis guided by expert provided scenarios in a data baseand expert rules in the form of fuzzy logic. Ultimately,the GP evolves a fuzzy decision tree that is optimal,with respect to the expertise provided for the desiredtask. The GP has been successful in reproducing knownresults as well as creating new distinct algorithms.

The GP’s output, the fuzzy decision logic is concise,elegant and understandable by humans. This is largelydue to the use of partial expertise embedded in the GPin the form of fuzzy rules and innovative bloat con-trol mechanisms like computer algebra. All the bloatcontrol mechanisms contribute to shorter solutions, butthe use of fuzzy rules within the GP to guide evolutionproduces results closer to human intuition.

In the case where the decision logic is the same asthat found by interviewing experts, it is easy to arguethe GP’s results are understandable by humans,after all,the GP has reproduced results handed down by humanexperts. In the case where the results are different fromthose obtained from expertise, they can be related tothe expert results and the differences understood.

6. Future directions

The various techniques for controlling GP basedbloat, i.e., generating more concise results must be ex-amined to determine their assets and liabilities. Also,various techniques for accelerating the convergence ofthe GP will be explored.

Additional fuzzy logic algorithms will be added togive the UAVs greater flexibility. This will require

consulting with experts to obtain rules. In the absenceof a good set of rules it should be possible to construct ascenario data base for mining by the GP. A partial set ofrules about the anticipated shape of fuzzy decision treescan be used to help accelerate the GP’s convergenceandproduce simpler, more concise results understandableby human beings as in the AUP decision tree case.

From the practical perspective of modeling physicalsystems, the fidelity of the underlying UAV models, theelectromagnetic propagation model,and environmentalmodels will be increased.

Increases in fidelity of the UAV model will includespecifics of the UAV engine model, e.g., how doesits efficiency change with altitude. The ray-theoreticelectromagnetic propagation model will be partially re-placed with a ray-wave-theoretic hybrid [9]. This willallow greater fidelity in modeling rough surface scat-tering from terrain and also various types of ductingphenomena. By using a hybrid model increases in CPUtime and memory requirements can be kept in checkas the fidelity of the modeling of underlying physicsincreases.

Changes in modeling fidelity will also result in al-terations of the cost function related to flight path de-termination. It will probably also prove useful, eventu-ally to explore more sophisticated algorithms for pathdetermination [5,11,12,28] that will replace A-star. Analgorithm of greater sophistication may be useful forrapidly changing environments or when there are moreobstructions or threats within the environment.

Acknowledgements

This work was sponsored by the Office of NavalResearch. The authors would also like to acknowledgeMr. Alan Schultz, Dr. Lawrence Schuette, Dr. JeffreyHeyer, Dr. Francis Klemm, and Dr. Gregory Cowart.

References

[1] M. Azizoglu, S. Subrarnaniam and A.K. Somani, Converterplacement onwavelength-routed network paths, in:Con-verter placement on wavelength-routed network paths III, JohnSenior, ed., San Diego, SPIE Proceedings, October 1997,pp. 265–275.

[2] J.P. Bigus, Data Mining with Neural Nets,New York,McGraw-Hill, 1996.

[3] S. Blackman and R. Popoli,Design and Analysis of ModernTracking Systems, Chapter 11, Boston, Artech House, 1999.

[4] L.V. Blake, Radar Range-Performance Analysis, ArtechHouse, Boston, 1986.


[5] K.R. Buck, R. Gassner, A.B. Poore and X. Yan, Visibility,Constrained Routing of Unmanned Aerial Vehicles, in:Sig-nal Processing, Sensor Fusion, and Target Recognition VIII,(Vol. 3720), I. Kadar, ed., SPIE Proceedings, Orlando, Florida,April 1999, pp. 256–266.

[6] C. Jeffries, Tracking, code recognition, and memory manage-ment with high order neural networks, in:Applications andScience of Artificial Neural Networks, Vol. 2492, S. Rogers,ed., SPIE Proceedings, Orlando, Florida, April 1995, pp. 364–373.

[7] J.R. Koza, F.H. Bennett III, D. Andre and M.A. Keane,GeneticProgramming III: Darwinian Invention and Problem Solving,Morgan Kaufmann Publishers, San Francisco, 1999.

[8] J. Latourell, B. Wallet and B. Copeland, Genetic Algorithmto Solve Constrained Routing Problem with Applications forCruise Missile Routing, in:Applications and Science of Com-putational Intelligence, Vol. 3390, S. Rogers, ed., SPIE Pro-ceedings, Orlando, Florida, April 1998, pp. 490–500.

[9] M. Levy, Parabolic equation methods for electromagneticwave propagation, Institution of Electrical Engineers, Herts,United Kingdom, 2000.

[10] S. Luke and L. Panait, Fighting Bloat with Nonparametric Par-simony Pressure, in:Parallel Problem Solving from Nature –PPSN VII, 7th International Conference Proceedings, LNCSVol. 2439, J.J.M. Guervos, ed., Berlin, Springer-Verlag, 2002,pp. 411–421.

[11] R. Mittu and J.K. Uhlmann, Routing and Advanced DisplayTechnologies within STOMPM, in:Robotic and Semi-RoboticGround Vehicle Technology Vol. 3366, G. Gerhart, ed., Or-lando, Florida, SPIE Proceedings, April 1998, pp. 177–188.

[12] H.G. Nguyen, H.R. Everett, N. Manouk and A. Verma,Autonomous mobile communication relays, in:UnmannedGround Vehicle Technology IV Vol. 4715,G.R. Gerhart, C.M.Shoemaker and D.W. Gage, eds, Orlando, Florida, SPIE Pro-ceedings, April 2002, pp. 50–57.

[13] S.J. Russel and P. Norvig,Artificial Intelligence: A ModernApproach, (2nd ed.), Englewood Cliffs, Prentice-Hall, 2002.

[14] J.F. Smith, III and R. Rhyne, II, A Resource Manager forDistributed Resources: Fuzzy Decision Trees and GeneticOptimization, in:Proceeding of the International Conferenceon Artificial Intelligence, IC-AI’99, Vol. II, H. Arabnia, ed.,Las Vegas CSREA Press, 1999, pp. 669–675.

[15] J.F. Smith, Co-evolutionary Data Mining to Discover Rules forFuzzy Resource Management, in:Proceedings of the Inter-national Conference for Intelligent Data Engineering and Au-tomated Learning, H. Yin, ed., Manchester, Springer-Verlag,August, 2002, pp. 19–24.

[16] J.F. Smith, Data Mining for Fuzzy Decision Tree Structurewith a Genetic Program, in:Proceedings of the InternationalConference for Intelligent Data Engineering and AutomatedLearning, H. Yin, ed., Manchester, Springer-Verlag, August2002, pp. 13–18.

[17] J.F. Smith, Fuzzy logic resource manager: decision tree topol-ogy, combined admissible regions and the self-morphing prop-erty, in: Signal Processing, Sensor Fusion, and Target Recog-nition XII, Vol. 5096, I. Kadar, ed., Orlando, SPIE Proceed-ings, April 2003, pp. 104–114.

[18] J.F. Smith, Fuzzy logic resource manager: real-time adapta-tion and self organization, in:Signal Processing, Sensor Fu-sion, and Target Recognition XIII, Vol. 5429, I. Kadar, ed.,Orlando, SPIE Proceedings, April 2004, pp. 77–88.

[19] J.F. Smith, Genetic Program Based Data Mining for Fuzzy De-cision Trees, in:Proceedings of the International Conferencefor Intelligent Data Engineering and Automated Learning,H.Yin, ed., Exeter, Springer-Verlag, August 2004, pp. 464–470.

[20] J.F. Smith and T.H. Nguyen, Distributed autonomous systems:resource management, planning, and control algorithms, in:Signal Processing, Sensor Fusion, and Target RecognitionXIV, Vol. 5809, I. Kadar, ed., Orlando, SPIE Proceedings,April 2005, pp. 65–76.

[21] J.F. Smith, III and T.H. Nguyen, Data mining based automatedreverse engineering and defect discovery, in:Data Mining, In-trusion Detection, Information Assurance, and Data NetworksSecurity 2005, B. Dasarathy, Vol. 5812, April 2005, Orlando,SPIE Proceedings, 2005, pp. 232–242.

[22] J.F. Smith and T.H. Nguyen, Fuzzy Logic Based UAV Allo-cation and Coordination, in:Proceedings of the Third Inter-national Conference on Informatics in Control, Automationand Robotics: Intelligent Control Systems and Optimization,J.A. Cetto, ed., Setubal, Portugal, ICINCO 2006 Proceedings,August 2006, pp. 9–18.

[23] J.F. Smith III and T.H. Nguyen, Guiding Genetic ProgramBased Data Mining Using Fuzzy Rules, in:Proceedings of theInternational Conference for Intelligent Data Engineering andAutomated Learning,H. Yin, ed., Burgos, Spain, Springer-Verlag, September 20–23, 2006.

[24] J.F. Smith and T.H. Nguyen: Resource manager for an au-tonomous coordinated team of UAVs, in:Signal Processing,Sensor Fusion, and Target Recognition XV: 62350C,I. Kadar,ed., Orlando, SPIE Proceedings, April 2006, 104–114.

[25] J.F. Smith, III and T.H. Nguyen, Genetic program based datamining to reverse engineer digital logic, in:Data Mining,Intrusion Detection, Information Assurance, and Data Net-works Security 2006, Vol. 6241, B. Dasarathy, Orlando, SPIEProceedings, April 2006.

[26] D. Spears and D. Zarzhitsky, Multi-Robot Chemical PlumeTracing, in:Multi-Robot Systems: From Swarms to IntelligentAutomata,Vol. III, A. Schultz, ed., New York, Springer, May2005, pp. 211–221.

[27] S.R. Thangiah and K.E. Nygard, School bus routing usinggenetic algorithms, in:Applications of Artificial IntelligenceX: Knowledge-Based Systems,Vol. 1707, G. Biswas, Orlando,Florida, SPIE Proceedings, April 1992, pp. 387–398.

[28] K. Thyagarajan, R. Batta and M.H. Karwan, Planning Dissim-ilar Paths for Military Units, Progress Report Department ofIndustrial Engineering, University of Buffalo SUNY, Buffalo,NY 14260, USA, 2004.

[29] L.H. Tsoukalas and R.E. Uhrig,Fuzzy and Neural Approachesin Engineering, Chapter 5, New York, John Wiley and Sons,1997.

[30] S. Verma and A.K. Saxena, Polynomial Time Algorithm forHop Constrained QoS Routing, in:Internet Quality of Service,Vol. 5245, M. Atiquzzaman and M. Hassan, eds Orlando,Florida, SPIE Proceedings, April 2003, pp. 131–135.

autonomous and cooperative robotic behavior based on fuzzy logic and genetic programming

Education