Target Tracking with Unmanned Aerial Vehicles: From Singleto Swarm Vehicle Autonomy and Intelligence

Ben Ludington, Johan Reimann,and George Vachtsevanos Irtaza Barlas

School of Electrical and Computer Engineering Impact Technologies, LLCGeorgia Institute of Technology Atlanta Division

Atlanta, GA 30345 USA 75 Fifth Street NW, Suite 312ben.ludington@gatech.edu Atlanta, GA 30308, USAjohan.reimann@gatech.edu irtaza.barlas@impact-tek.com

gjv@ece.gatech.edu

Abstract have broken the problem down into three steps.

Recent military and civil actions worldwide havehighlighted the potential utilityfor unmanned aerial ve- In the first step, the incoming video is processedhicles (UAVs). Bothfixed wind and rotary aircraft have using a particle filter to determine the location of thecontributed significantly to the success ofseveral mili- target. The particle filter is a sample-based approachtary and surveillance operations. Future combat oper- to Bayesian state estimation that is able to approximateations will continue to place unmanned aircraft in chal- non-Gaussian distribution that evolve according to non-lenging conditions such as the urban warfare environ- linear dynamics. The following section describes thement, where surveillance is particularly challenging. particle filter framework and discusses methods of deal-These challenges as well as the reduced autonomy, and ing with the algorithms large computational load.operator workload requirements of current unmannedvehicles present a roadblock to their success. It is antic- After the article filter estimates the osition of theipated thatfuture operations will require multiple UAVs p pprfomn in a coprtv moe shrin reore target, a reasoning layer provides situational awareness

and complemen t cother ati mo ground a esouaccu to deal with occlusions or other target tracking short-andl complementing oth?er air or groundl assets to accu- coig.Ti.ae se rt aneac syte torately calculate a target's position. This paper reviews comings.Thitlaerssesa truth maintenance system tothe current status of UAV target tracking technologies determine a set of scenarios that may have resulted in

with mphassonrecen devlopmets amed a UAV a degradation of the target tracking algorithm. The sys-withemproved a onomycand discussesnfuturedirecsandet Atem will then use these scenarios to develop an effective

tpehnol onogicalychallengedischussutueaddressednsindt strategy to remedy the situation. The truth maintenanceimmedhnologicatfuture. ges that must be addressed inthe

system is introduced following the particle filter track-immediatefuture. igagrtming algorithm.

1. INTRODUCTION Finally, an adversarial reasoning module is placedin the highest layer to produce strategies that deal with

Because of their ability to reach unique vantage evading targets using a swarm of UAVs. By using apoints without endangering a human operator, camera differential game framework, which is discussed afterequipped unmanned aerial vehicles (UAVs) are effec- the truth maintenance system, a team of air vehicles istive platforms for military and civilian surveillance mis- able to contain a target that is attempting to escape. Thesions. Small UAVs are particularly useful in urban mil- framework decomposes a complete game into a set ofitary operations when it is necessary to see around the two player games, which are more easily solved. Thecorner. However, urban surveillance is challenging be- adversarial reasoning module allows for group coordi-cause of the inherent clutter and occlusions in the en- nation as depicted on the Department of Defense's UAVvironment. To effectively deal with this challenge, we autonomy capability trend [1] in figure 1.

F-1YAAonoffos L S i i,frame to the next. Therefore, the particles are moved ac-1 LSws cording to a Gaussian random walk model. The secondg . WSNN component accounts for the times when the smoothness

lUCAR assumption is violated, which can occur if the target be-

I8 Dttibmcdwcomes occluded or briefly moves outside of the FOV.

7 ONW t This portion of the model is another Gaussian distrib-Gffll} ution that is centered around a randomly selected pixel

T AF UCV / oa where motion has been detected. Motion is detected in aQpi-!' natll'l pre-processing step. The two components of the model

ottow nwteare combined using a convex combination, and the size

4F Vlq! of each contribution is changed throughout the trackingi6atkiFArit & process depending on the performance of the system.

F4C res

R;<Tige*aXDS / Q GiobalHwk After the particles are moved, measurements are2 C Predw4at taken. Here, color and motion measurements are used.

I Guhdk;,idn However, other cues can also be used depending on thesensor capabilities. Color measurements are taken by

19taI96 15 19s5 N 2; 2Q15 2025 comparing the histogram of each particle to a referencehistogram as in [5]. The reference histogram is either

Figure 1. Autonomous control level trend. part of the a priori knowledge or is generated by manu-ally selecting the target in the first frame. The particlesthat have histograms that are close to the reference his-

2. PARTICLE FILTERS FOR VISUAL togram are assigned higher weights. Motion measure-TRACKING ments are taken by subtracting the value (in the hue-

saturation-value sense) of each pixel within the particleThe particle filter is a state estimation tool that from the corresponding pixel in the previous frame and

is able to approximate a non-Gaussian distribution taking the absolute value. The differences are summedthat evolves according to nonlinear dynamics using a over the particle and normalized by the particle area.randomly-selected set of weighted samples [2]. It is an The particles that have the higher motion measurementseffective tool for visual tracking because non-Gaussian are assigned higher weights. The total weight is gen-distributions are prevalent due clutter in each image erated by assuming the measurements are independentframe. As with the Kalman filter or other Bayesian tech- and taking the product ofthe weights generated by eachniques, the estimated distribution is recursively com- measurement. The contribution of each measurementputed using the two step process of prediction and up- source is changed throughout the tracking process de-date. In the prediction step, the particle set is propa- pending on the performance of the system.gated forward one step in time using the system update The performance of the filter is estimated by ex-model. In the update step, measurements are used to amining properties of the particle distribution. A neuralupdate the particle weights. The particles are resampled network is used to map the properties of the particleat each step by their weights. This results in the lower distribution, such as maximum weight, maximum colorweighted particles being replaced by higher weighted weight, and spatial spread ofthe particles to an estimateparticles. As the number of particles increases, the of the shape of the distribution. When a distinct peaksampled-based distribution converges to the true distri- is detected in the distribution, the particle filter is as-bution [3]. When using a particle filter to track targets in sumed to be performing well, and the parameters can beimages, the state space is typically four dimensional and adjusted accordingly. The system update and measure-describes a rectangle. Therefore, the higher weighted ment models can both be updated to reflect the filter'sparticles correspond to the peak of the distribution and performance. Also, the number of particles can be de-are rectangles that are near the target, while the lower creased when the filter is performing well. By reducingweighted particles correspond to the tails of the distrib- the number of particles, the inherently large computa-ution and are rectangles that are further away from the tional burden of the particle filter can be decreased.target. Figure 2 shows a set of output frames as the particle

A particle filter has been implemented that uses a filter tracked a truck moving across a field. The videosystem update model that is similar to [4], where the was taken from the GTMax research unmanned heli-model is made up of two components. The first com- copter [6] and the frames were processed offline afterponent assumes the target moves smoothly from one converting the video to a series of frames at 10 frames

per second. In this case, the helicopter was hovering and 3. TRUTH MAINTENANCE SYSTEMSthe orientation of the camera was manually controlled. FOR SITUATIONAL AWARENESSThe output of the particle filter is shown as the red andblue rectangles. The ten highest weighted particles are Tracking the target in real-time is a challengingrepresented by blue rectangles, while the remained of task. It forms the core of any architecture and providesthe particle set is represented by red rectangles. The information on which related reasoning and inferenc-weighted average of the particles is shown as the white ing mechanisms work. Tracking algorithms are limitedcrosshairs. As can be seen, the particle filter correctly in their reasoning abilities. There is usually no persis-estimated the position of the target. The tracking error tence associated with them. Also we do not get situa-is shown in Figure fig:terror tion awareness out ofthe algorithms themselves. In this

section we present a framework for automated reason-..... ~~~~~~ing that works on top of the tracking and image analy-

sis algorithms to generate an overall picture for the rea-soner and can provide insight into the state of the track-ing to a human observer. We use a popular problem-solving class of systems called Truth-Maintenance Sys-tems (TMS). The TMS work with inference enginesand maintain the truth in the system by revising setsof beliefs as new information becomes available thatmay contradict existing information. TMS can solveproblems where algorithmic solutions do not exist andare very suitable for large-scale spaces [7]. Like otherknowledge-based problem solvers, the TMS work ondomain knowledge. There are several shortcomings inconventional problem solvers that are addressed by aTMS. For example, a TMS by design provides an ex-planation for its reasoning process. It also recovers frominconsistencies such as incorrect inputs. This input canbe an erroneous sensor or it can make some incorrectassumptions. When the input makes the situation clear,the assumption may prove to be incorrect. At this point

Figure 2. Using a particle filter to track a truck the bad assumption and its related inferencing are re-from a UAV. tracted. A TMS also maintain its previous inferences

and therefore avoids performing the same processingagain and again [8].

Tracking problems require maintaining an internalTruckTraking Error awareness of the situation. This awareness becomes

45 especially useful in situations where the tracking algo-40 rithms begin to loose confidence in their tracking abil-5| 10 2 ities. Such situations may arise due to clutter in the

oe0 01 2 images, partial or full occlusions, change in the imageD 25 quality due to light or whether conditions, etc. A typi-W 20 F ,lj 1 cal tracker can potentially loose track of its target(s) intt'20 01 01 XI these situations and improving the tracking algorithm15 alone may not be enough to address this problem. As10 shown in Figure 4, we add a reasoning layer that main-

b000 j!I0i9fW >0$ § ltlli tains data structures across video frames. This layer

50 100 150200 250 300 receives tracking information about a target from theFrameInde 3 350 tracking layer that is also responsible for image analy-

sis. The reasoning layer maintains a database of currentsituation. The data is maintained in terms of the coordi-Figure 3. Particle filter tracking error. nates of the target, itS features being tracked, and otherrelevant information such as presence of buildings, trees

etc. As new situations arise, such as occlusion due to the themselves such that the target can be handed off effec-presence of a building, the reasoning layer (guided by tively. Naturally, to determine how the vehicles shouldtracking database) generates scenarios that may be rel- be deployed, it is necessary for the swarm determine theevant here. These scenarios are treated as assumptions target's escape strategies by considering the constraintsby the inferencing mechanism. Some assumptions may imposed by the environment, the targets maneuveringbe related to expected time of occlusion etc. As time capabilities, the current location of all the UAVs in thepasses by if the target does not emerge from occlusion, swarm and the maneuvering capabilities of the UAVs.new assumptions will be added, possibly taking into ac- To accomplish this, a differential pursuit-evasioncount other possibilities such as target is aware of track- game framework is used to determine the escape strate-ing, target's final destination, dismounted targets. The gies of the evading target and the actions needed to betracker is made aware of the situation and its control as taken by the swarm to ensure that the target does nota result may decide to hover around during this time. escape.

The main result from the work done on differentialgames is the so-called Hamilton-Jacobi-Bellman-Issacs

R n atbsng (HJBI) equation given byCoordinates, dV(1

F6t&es, - <-+H(x,t,DV) 0, (1)Faue, Possible action dtInmage componets|Tracking & Image | where the Hamiltonian H isAnalysis

H(x,t,DV) =minmax[fi (x,t,up,u,) +Up Ue dx1

dV+fN(X,t,Up,Uee )XN +L(X,t,up,ue)] (2)

given the cost functional

*< L J= 0~~~~~~~~~~~~~~~~((0(t) +J L (x,t,up,ue) dt (3)

The control inputs for the UAVs and the evading tar-get are up and ue, respectively, () is the terminationcost and Lo is the integrated cost functional. V is thevalue of the game, that is, given a particular state of the

Figure 4. Truth maintenance system for target game, V describes the cost of intercepting the evadingtracking targets. The system is governed by the following system

dynamics:

4. DIFFERENTIAL GAMES FOR AD- X fl (X,t,Up,Ue)VERSARIAL REASONING

XN fN(Xt,Up,Ue), (4)While a UAV is tracking a target, the target may

become aware of the UAV and attempt to avoid the air- where x C XN.craft. Due to limited maneuverability of the UAV and Since initial conditions are expressed over the en-possible obstacles such as buildings and trees, the target tire state space, the interception cost will have to bemay be able to leave the field of view of the UAV. How- determined for all the states. Hence, since the size ofever, when dealing with swarms of UAVs, it is possible the state space is N = m n, where m is the number ofto overcome the limitations of the individual UAV by players and n is the size of the individual players' stateallowing the vehicles to work in concert, thereby guar- space, the computational complexity of the problem in-anteeing that the target will remain within the field of creases exponentially with an exponent of m n. Theview of at least one UAV. Hence, when a target attempts computational complexity suggests that means must beto avoid the UAV, which is currently maintaining a vi- sought that will reduce the computational burden andsual lock on the target, the other UAVs must be able to make it feasible for real-time implementation of thepredict the future actions of the target and positioning scheme.

A hierarchical decomposition technique is used toreduce the computational burden ofthe multiplayer sto- -chastic game problem. The decomposition is done inthree stages. In the lowest level, the interception strat- P 1egy for each of the pursuers is determined by solv-ing several two-player stochastic differential pursuit-evasion games. In the middle level, one or more pur- Ev ersuers are assigned to each evading target based eitheron a target importance measure or on the time to inter-cept. In the highest level, a dynamic performance-basedregion of responsibility (DPRR) is computed for each ofthe pursuers assigned to a particular target. ' _ _ _ _ _ _ _

The decomposition approach essentially reducesthe multiplayer differential game problem into severalmuch simpler two-player problems as shown in Figure5 By performing the decomposition, the cooperation be- Figure 6. Three pursuers attempt to capture

the evading target located at the center. Oneof the pursuers attempts to intercept the target

-U directly, while the other pursuers perform con-tainment maneuvers.

be noted that the DPRR for each of the pursuers is up-dated regularly; consequently, the tasks assigned to thepursuers also changes frequently depending on the ac-tions taken by the evading target.

Figure 5. The decomposition of the multiplayer The decomposition approach is advantageous,game into several two-player problems since it is computationally much simpler to solve the

two-player games than general multiplayer games. Thetween the pursuing players is not considered. Hence, approach is able to adapt rapidly to changes in the sce-the mid- and high-level steps in the decomposition ap- nario, that is, if new pop-up targets are encountered theproach are designed to reintroduce cooperation in an in- roles of each of the pursuers is reassigned to effectivelytelligent fashion. Based on the estimated interception handle the unexpected change to the scenario. Addi-time derived in at the low-level decomposition stage, tionally, the algorithm can easily be implemented in aeach of the pursuers is assigned a DPRR. If an evader is distributed fashion, that is, the computational resourcesin a pursuer's DPRR, it is that pursuer's responsibility onboard all of the UAVs can be utilized effectively.to intercept the evading target. However, if there are noevaders in a particular pursuer's DPRR, the pursuer willmove toward a virtual target. The virtual target is the 5. CONCLUSIONSpoint on the boundary of the pursuer's DPRR with thelargest difference between the estimated time to captureand the time it takes the evader to reach the point. This As the autonomy and reliability ofUAVs improve,point is where it is most likely for the evader to cross UAVs will find themselves being relied upon in moreinto the pursuer's DPRR. Figure 6 illustrates the target complex situations. This paper presented a target track-assignment. ing system that can be used in the complex situation of

The evading target in the center of the figure is at- tracking targets in urban warfare environments. The hi-tempting to escape the three pursuers. Only the pur- erarchical system was presented from the bottom up. Itsuer at the top of the picture is attempting to intercept includes a particle filter visual tracker, a truth mainte-the evading target directly. The other two pursuers are nance system for situational awareness, and an adver-heading toward the virtual targets in an attempt to block sarial reasoner to allow effective UAV teaming. Suchthe evader's possible escape routes, that is, they are es- a system will provide robust and reliable informationsentially performing containment maneuvers. It should while keeping a human operator out of harms way.

6. ACKNOWLEDGEMENTS

This work was partially supported by a US ArnySBIR Phase II Project under contract W91 INF-06-C-OO 18 via Impact Technologies LLC and by theDARPA/IXO HURT program and the Air Force Re-search Laboratory under contract FA865004-C-7142via Northrop Grumman. The authors thank these or-ganizations for their support.

References

[1] "Unmanned Aerial Vehicles Roadmap 2002-2027," Of-fice of the Secretary of Defense, Washington, DC 20301,Tech. Rep., December 2002.

[2] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp,"A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking," IEEE Transactions on Sig-nal Processing, vol. 50, no. 2, pp. 174-188, February2002.

[3] D. Crisan and A. Douocet, "A survey of convergence re-sults on particle filtering," IEEE Transactions on SignalProcessing, vol. 50, no. 3, pp. 736-746, January 2002.

[4] P. Perez, J. Vermaak, and A. Blake, "Data fusion for vi-sual tracking with particles," Proceedings of the IEEE,vol. 92, no. 3, pp. 495-513, March 2004.

[5] P. Perez, C. Hue, and J. Vermaak, "Color-based proba-bilistic tracking," in Proceedings of the European Con-ference on Computer Vision, May 2002, pp. 661-675.

[6] E. N. Johnson and D. P. Schrage, "System integration andoperation of a research unmanned aerial vehicle," JournalofAerospace Computing, Information, and Communica-tion, vol. 1, no. 2, pp. 5-18, January 2004.

[7] M. Stanojoevic and S. Vroanes, "Using truth maintenancesystems: A tutorial," IEEE Expert, pp. 46-56, 1994.

[8] K. D. Forbus and J. deKleer, Building Problem Solvers.MIT Press, 1993.