AFOSR MURI. Salem, MA. June 4, 2002. 1/10

Coordinated UAV Operations:Perspectives and New Results

Vishwesh Kulkarni

Joint Work with

Jan De Mot, Sommer Gentry, Tom Schouwenaars, Vladislav Gavrilets, and Prof. Eric Feron

at the Laboratory for Information and Decision Systems, MIT.

AFOSR MURI ONR YIA

AFOSR MURI. Salem, MA. June 4, 2002. 2/10

• Efficient multi-agent operations require robust, optimal coordination policies.

• UAV specifications constrain deployable coordination policies.• How may we improve our understanding of these constraints?• How may we use it to synthesize more efficient coordination policies?

Overview

We view spatial distribution of the UAVs as a key factor and present original

results concerning the UAV separations and the UAV placements.

O bstacles

D anger Zones

• Coordinated Path Planning• Surveillance

Number of UAVs

Efficiency =

??

1

cost per UAV

AFOSR MURI. Salem, MA. June 4, 2002. 3/10

Coordinated Path Planning (CPP)

Questions

• What is the spatial distribution under an optimal policy? We have characterized the separation bounds.

• How many UAVs are needed? We do not know the full answer yet!

CPP Problem Setting

• UAVs need to go from a point s to a point t.• Environment is dynamic and uncertain.• UAVs cooperate by sharing the acquired local information.• UAVs have limited resources. GOAL: Optimize the traversal efficiency.

AFOSR MURI. Salem, MA. June 4, 2002. 4/10

Related Past Works

We present new results in a coordinated target acquisition setting using DP.

Multi-Agent Exploration of Unknown Environments

Probabilistic map building of Burgard et al [2002] uses deterministic value iteration to determine the next optimal observation point.

The market architecture of Zlot et al [2002] auctions off the next optimal observation points obtained by solving a TSP.

• The end goal is spanning rather than CPP.

CPP as Multi-Agent MDPs

• Boutilier et al [2000]. We consider partially observable MDPs.

• Greedy policy pursuit-evasion games of Hespanha et al [2002].

known region

new region

agent

unknown region

AFOSR MURI. Salem, MA. June 4, 2002. 5/10

Our CPP Problem

Terrain is mapped into regions having payoffs.

Terrain traversal becomes graph traversal.

s t...

G 2 S 1 S i

...

S N

• UAVs share local information.• Partially known, uncertain environment• On-board sensors reduce uncertainty in a direction dependent manner.

• Lookahead link costs are deterministic, others i.i.d.

Goal: Find a path for each agent that minimizes the expected aggregate cost.

... ...

AFOSR MURI. Salem, MA. June 4, 2002. 6/10

The CPP Separation Results

G7, infinite horizon, discount factor = 0.8

2.5 2

1 2

2.5 2

2 3

1 2 1

2.5 2 0.5 1.5

2 3

2.5 2.5

1 2 1

2.5 2 0.5 1.5 3

2 3 1.5 0.5

2.5 2.5

1 2 1

2.5 2 0.5 1.5 3

2 3 1.5 0.5 1

2.5 2.5

1 2 1

2.5 2 0.5 1.5 3

2 3 1.5 0.5 1 2

2.5 2.5

1 2 1

2.5 2 0.5 1.5 3 1 3

2 3 1.5 0.5 1 2 1.5

2.5 2.5

1 2 1

2.5 2 0.5 1.5 3 1 3 2.5

2 3 1.5 0.5 1 2 1.5

2.5 2.5 2.5 2.5

1 2 1

2.5 2 0.5 1.5 3 1 3 2.5

1.5 1

2 3 1.5 0.5 1 2 1.5 1 1.5

2.5 2.5 2.5 2.5

1 2 1

2.5 2 0.5 1.5 3 1 3 2.5 3

1.5 1

2 3 1.5 0.5 1 2 1.5 1 1.5

2.5 2.5 2.5 2.5

1 2 1

Conjecture 1: The UAV separation is bounded in

Extra nodes should not affect the separation adversely.

Conjecture 2: The UAV separation is bounded in in a pair-wise sense.

Conjecture 1 should hold pair-wise in the n-agent setting.

.mG

mG

Cluster Separation Lemma

Using optimal paths for two agents in , configurations ,

, and do not evolve into configurations with l > 2.

The UAV separation is bounded in

0C

lC1C 2C7G

7.G

Communication power, hierarchy tier sizes

AFOSR MURI. Salem, MA. June 4, 2002. 7/10

Surveillance as CPP

Surveillance Problem Setting

• Terrain as regions with dynamic, uncertain payoffs. • UAVs face dynamic, uncertain threats. • Limited communication capacity and efficiency.• Efficiency decreases with distance. • UAVs cooperate by repositioning and handoffs. Goal: Maximize the net minimal spare UAV capacity.

Questions

• What is the spatial distribution under an optimal policy? Characterized by the separation results.

• How many UAVs are needed? We do not know the full answer yet!

1log(1 SNR)

2

efficiency

SNR

AFOSR MURI. Salem, MA. June 4, 2002. 8/10

Related Efficiency Results

• capacity … Gupta-Kumar [2000]

• capacity … Grossglauser-Tse [2000]

• Dumb Antennas … Viswanath et al [2002]

• Space-Time Codes … Tarokh et al [2000]

(1/ )O n

(1)O

Commonalities with Cellular Network Concepts

• i.i.d. uniformly distributed payoffs

• path loss decrease in efficiency

• How many Network Capacity

r

Cellular network understanding has promise in the UAV setting.

• Techniques to exploit the UAV mobility

AFOSR MURI. Salem, MA. June 4, 2002. 9/10

Future Directions

probability

link cost

1

• Extensions for larger and heterogeneous clusters

Dynamic program modifications

• More incremental on-board information gathering

Gradual link cost change from i.i.d. to deterministic

Sets of possible link cost distributions

• Separation and efficiency properties for large scale systems

Curse of dimensionality

Neuro-Dynamic programming for approximate solutions

• To add or not to add (a UAV) … Brute force iterative DP-based solution Binary search for an optimum number

efficiency per UAV

number of UAVs

??

AFOSR MURI. Salem, MA. June 4, 2002. 10/10

Questions ??

http://www.mit.edu/people/vishwesh/vishwesh@mit.edu

Joint work being done at MIT with Prof. Eric Feron’s group, supported by his AFOSR MURI and ONR Young Investigator Award grants.

Thank You !