comotion computational methods for collaborative motion pursuit evasion games for networks of uuvs...
Post on 25-Dec-2015
217 Views
Preview:
TRANSCRIPT
CoMotionComputational Methods for
Collaborative Motion
Pursuit Evasion Games for Networks of UUVs
November 2004
Mike Eklund, Jonathan Sprinkle, Shankar Sastry
Pursuit Evasion Games for Multiple UUVs
• UUVs have the potential to provide an effective defense against submersible threats to military and civilian assets
• The strategies and protocols for their operation are at least as big a challenge as the design and construction of the vehicles themselves
• This project address the strategies and coordination protocols necessary to enable this technology
• Defense against enemy subs is the number one FNC that is called for in every briefing since 2000.
Bear UUV Being DevelopedLt Tulio Celano III, USN
UUV Resistance Curve
0
10
20
30
40
50
60
0 2 4 6 8 10 12 14 16
Water Speed (knots)
To
tal
Dra
g (
lbf)
UUV Range-Speed Curve
0
10
20
30
40
50
60
70
80
90
100
0 5 10 15 20
Water Speed (knots)
Ran
ge
(NM
) (B
lue)
0
5
10
15
20
25
30
35
40
45
50
Ho
urs
on
Sta
tio
n (
hr)
(P
ink)
8 ft X 10 in., 400 lbs. displacement, upto 100 ft. depth, Top speed 12 knots, effective cruising speed 5 knots, endurance at 5 knots is 45 hours, Modular design
Prior PEG Experience and Lessons Learned
• Pursuit evasion games (PEGs) have been demonstrated within our group using unmanned aerial and ground vehicle (UAVs and UGVs) in both two and three dimensions.
• Additionally, they have also been performed in both symmetrical and asymmetrical games
Previous UC Berkeley PEG Experiments
• Berkeley Aerobot Project (BEAR)– Goal: to build a coordinated, intelligent network with multiple
heterogeneous agents• 11 Rotorcraft-based unmanned aerial vehicles (UAVs) • 5 Unmanned ground vehicles (UGVs)• Shipdeck simulator (landing platform)
• Stochastic Pursuit-Evasion Games (PEG)– Self-localization– Target detection– Map building– Pursuit policy– Trajectory generation– Control / Action
Previously: PEGs with 4 UGVs and 1 UAV
• Sub-problems for Pursuit Evasion Games– Sensing
• Navigation sensors -> Self-localization• Detection of objects of interest
– Framework for communication and data flow
– Map building of environments and evaders• How to incorporate sensed data into agents’ belief states • probability distribution over the state space of the world• (I.e. possible configuration of locations of agents
and obstacles)• How to update belief states
– Strategy planning • Computation of pursuit policy• mapping from the belief state to the action space
– Control / Action
PEG Experiment with UAV/UGVsPEG with four UGVs• Global-Max pursuit policy• Simulated camera view
(radius 7.5m with 50degree conic view)• Pursuer=0.3m/s Evader=0.5m/s MAX
Evaluation of Policies for different visibility
• Global max policy performs better than greedy, since the greedy policy selects movements based only on local considerations.
• Both policies perform better with the trapezoidal view, since the camera rotates fast enough to compensate the narrow field of view.
Capture time of greedy and glo-max for the different region of visibility
of pursuers
3 Pursuers with trapezoidal or omni-directional view
Randomly moving evader
Evader’s Speed vs. Intelligence
• Having a more intelligent evader increases the capture time
• Harder to capture an intelligent evader at a higher speed
• The capture time of a fast random evader is shorter than that of a slower random evader, when the speed of evader is only slightly higher than that of pursuers.
Capture time for different speeds and levels of intelligence of the evader
3 Pursuers with trapezoidal view & global maximum policy
Max speed of pursuers: 0.3 m/s
SEC Capstone Demo: Fixed-Wing PEGs
• Capstone Demonstrations were proposed to highlight and test the technologies developed in the SEC program
• One was a fixed-wing UAV flight test– 6 participant technology developers (TDs)
• Honeywell, Northrop Grumman, U Minnesota, MIT, Stanford, and UCB/U Colorado/CalTech
– System Integrator was Boeing– OCP would be software framework– A T-33 trainer as UAV surrogate– An F-15 as wingman/opponent
• 13 month schedule May 03 – June 04• UCB Contribution: Fixed-Wing PEGs
Nonlinear Model Predictive Trajectory Control
• Explicitly addresses nonlinear systems with constraints on operation and performance
• A cost minimization problem in the presence of state and input constraints– Control resulting in the minimum cost is determined over a
model predicted horizon
• Previously demonstrated in rotary wing UAVs[1]
[1] H.J. Kim, D.H. Shim and S. Sastry, ACC, 2002
Demo: UCB PEG Development
• 20 – 60 min. games confirm NMPTC feasibility at real-time
– Evader goal: get to final waypoint or avoid evader
– Pursuer goal: ‘target’ evader
• Pursuer and evader restricted to same performance limits
• Planes on the same logical plane, but separated by 6000ft altitude at all times
• Evader and pursuer have a few scenarios
– UAV as evader – UAV can become pursuer
OCP Experiment Controller SnapshotT-33: Evader (yellow)F-15: Pursuer (blue)
Demo: Controller Coverage
• Final waypoint is the major component
• Travel to the waypoint can be interrupted by the actions of the pursuer
– Relative positions– Angle-off-tail– Distance from each other– Proximity to boundaries of the
testing area– Physical limitations of the
aircraft• Velocity, climb/turn rate
• Use a simpler model for predictive behavior under certain conditions
– Still use the Boeing-supplied (DemoSim) model for behavioral simulations
Approx. same timePursuer goes fortarget cone
Evader turns away(regardless of endpoint)
Endpoint
Target cone definition (θ=10˚,d=3 nm)Left: F15 not behind UAV, middle: F15 not pointed at
UAV, right: F15 behind AND pointed at UAV
F15
UAV
F15
UAV
F15
UAV
(b)(c)(a)
F15
UAV
F15
UAV
F15
UAV
(b)(c)(a)
UCB PEA Experiment #1 Test Plan:UAV as Evader
• UAV attempts to cross Scenario Area (SA) from East to West without being targeted by the F15
• UAV “wins” by:– Reaching the RVPT– Not being targeted for 20 minutes
• F15 “wins” by targeting the UAV
• Note: F15 performance is restricted
UCB PEA Experiment #2 Test Plan:UAV as Evader and Pursuer
• UAV attempts to cross Scenario Area (SA) from East to West without being targeted by the F15, however, UAV will attempt to target F15 if suitable conditions arise
• UAV “wins” by:– Reaching the END ZONE– Not being targeted for 20 minutes– Targeting the F15
• F15 “wins” by targeting the UAV
Note: F15 performance is restricted
Flight Test 1 (UAV as evader)
Flight Test 2 (UAV as evader/pursuer)
UUV PEGs: Multiplayer Games
• In littoral waters the pursuit evasion game consists of an enemy submarine attempting to cross a line of UUVs which are protecting an asset
• The enemy submarine has a speed advantage over the blue force UUVs
• UUVs play a role in between a sensor web and a group of pursuers
• Research aimed at determining new approaches to teaming for multi-player games. Current literature focuses exclusively on either Nash or Stackleberg solutions
Multiplayer PEG Challenges
• The research challenge includes extending the strategies to: – Large multi-player teams– Asymmetric platform characteristics– Limited communications– High level of uncertainly
UUV PEG: Approach
• The UUV PEG involves two distinct phases:– Detection phase
• Maximize chances of detection constrained by:– Area to cover
– Number of UUVs available
– Possible evader strategies
– Capabilities of UUVs and evader (sensors and noise signatures)
– Response phase• Maximize chances of catching the pursuer constrained by:
– Capabilities of UUVs and evader (speed, manueverability and communications)
– Number of UUVs available
• These two phases also depend on each other as both must succeed– How to share resources to maximize overall chance of success
– How to overlap the strategies: detectors are responders as well
Multiplayer PEGs: Proposed Solution
• A close analogy is football or other team games:– Multi player
– Initial (global) strategies well defined
– Limited (local) coordination after the snap
• What can we learn?• How can we apply this?• How far does the analogy
go?
Multiplayer PEGs
• Preseason (Off-line precomputed strategy)– Play book:
• Evaluate strategies and configurations that will maximize chance of success based on best estimate of other team’s tactics
– Practice and preseason games:• Test playbook and find problems
• Game time (On-line adaptive strategy)– Choose play based on best knowledge and experience
• Line up (in best detection configuration, not necessarily static)
– Execute the play• Active and reactive actions (respond to detected evader)• Local communication• Adapt to evolving behavior
• Learn from experience, repeat as necessary (Learning by Doing)
Pre-game: Strategies for Detection
• Maximize the chance of detecting the evader• Tradeoffs
– Movement of the pursuer:• Moving quickly covers more area, but
• This makes it easier for evader to see the pursuer and avoid the pursuer
– Sensors:• Using passive sonar reduces the range of detection
• Using active sonar reveals the pursuers location
– Number of pursuers in detector role:• Increases chance of detection
Basic principle: Defense in depth
• May not be the optimal with limited resources,– for instance if there are not
enough UUVs to ensure detection of the evader by the front line.
Options: Zone Defense
• Would this leave seams for an evader to exploit if they have superior sensors, for instance?
• Is communication necessary to make such a zone defense effective?
• Is there an alternative?
Options: Channeling the evader
• A coordinated, heterogenous detection strategy.
• For instance, some pursuers could use a very active strategy that exposed them to intentional detection by the evader, with the intension of “channeling” the evader towards other more passive pursuers.
Pursuer Strategies: Capture
• Speed disadvantage means that simply optimizing the detection probability is not sufficient
• Reachability of UUVs must be known• Communication and coordination will be necessary to
overcome speed disadvantage
Defining the Problem:Basic UUV PEG Scenarios
• Scenario 1– Single evader infiltration– Objectives
• Red: pass through game area undetected• Blue: detect red team only
• Scenario 2– Single evader attack– Objectives
• Red: get within weapons range of some objective• Blue: prevent red attack on objective
– Capabilities• Red: limited number of torpedoes available to attack target or blue team
UUVs• Red: suicide attacks only, must get within effective range
• Scenario 3– Multiple evader attack– Objectives & Capabilities
• Same as Scenario 2
UUV Characteristics in General
• Performance– Speed and acceleration– Maximum rate of turn– Maximum rate of ascent/descent (not symmetric in general)– Maximum depth
• Sensors– Effective range in passive detection mode– Effective range in active detection mode– Deployable sonar buoys
• Communications– Effective communication range (variable in general)
UUV Characteristics, cont.
• Method of attack, including:– Self detonation, effective range and perhaps effectiveness as a
function of range– Missile (i.e. torpedo) capabilities
• Counter measures, including:– Sonar buoys– Noise canisters
• Noise signature as a function of:– Speed– Acceleration– Rate of turn– Rate of ascent/descent– Sensor mode– Communication mode
• And others
The First Problem Definition
• Based on “Scenario 2”:– Single evader attack, many defenders– Objectives
• Red: get within weapons range of some objective• Blue: prevent red attack on objective
– Capabilities• Red team:
– Limited number of torpedoes available to attack target or blue team UUVs
– 3 times speed advantage over the Blue (pursuer) team
• Blue team:– Suicide attacks only, must get within effective range– 3 times manuever (turn rate ) advantage over the Red team– Limited communication range– Passive and active sensors available
The First Problem Definition, cont
• Characteristics– Speed
• 3X advantage to Red Team
– Maneuver advantage• 3X to Blue Team
– Detection a function only of • speed,• communication use • Distance
– Sonar• Active & passive available
– Communications available for each team:
• range a function of power• detectability also a function
of power
0 5 10 15 20 25 300
0.5
1
1.5Noise profile of a UUV
Det
ectio
n ra
nge
(nm
)
Evader speed (knots)
Detection: Strategy Comparison
-25 -20 -15 -10 -5 0 5 10 15 20 25-20
-15
-10
-5
0
5
10
15
20
-20 -10 0 10 20-20
-15
-10
-5
0
5
10
15
20
Detection: Monte Carlo results
• Goals: – Statistical model as a
function of configuration, spacing, etc.
– Test strategies
0 10 20 30 40 50 60 7030
40
50
60
70
80
90
100Detection probability for line abreast
Evader speed (knots)
Pro
babi
lity
of d
etec
tion
(%)
0 10 20 30 40 50 60 7030
40
50
60
70
80
90
100Detection probability for staggered configurationt
Evader speed (knots)
Pro
babi
lity
of d
etec
tion
(%)
0 5 10 15 20 25 300
0.5
1
1.5Noise profile of a UUV
Det
ectio
n ra
nge
(nm
)
Evader speed (knots)
Line abreast Staggered
(Recall: detection function)
Capture: Strategies based on reachability (Mitchell’s Level Set Toolbox)
From Airplane example Still even, turn rate reduced to 1/3
Pursuer speed reduced to 1/3 Pursuer speed reduced to 1/3, turn rate increased by 3
Combined the Strategies: Advances in Game Theory
• These PEGs fit Game Theory descriptions as: – mixed strategy– simultaneous move– multiplayer– coordinated games – games with incomplete information
• Specific tactics can be evaluated to find the equilibria and optimal strategies in certain situations, e.g.:– the best search patterns within a single zone for one UUV,
or for two adjacent zones/UUVs
• Statistical methods or Monte Carlo methods could be used to determine the changes of success for each player
Communication Between UUVs
• The issue of the short range of underwater communications with multiple players is very similar to the problem of ad hoc wireless networks of motes devices
• This experience is directly adaptation to the multiple, coordinated UUV scenarios and used to disseminate information within the team regarding: – Detection of an evader– Likely detection by the other team– Coordination instructions– Strategic commands – etc.
• This is integral into the simulation environment
Future Research
• We are building a group of UUVs at the USNA in Anapolis. These are also being shared with NUWC, Newport
• We will develop a theory of multi-player pursuit evasion games with off-line strategies (using robust optimization, the level set toolbox), on-line modifications of the strategy using Model Predictive Control, and an outerloop of learning
• Expect to have simulation results by June 05, experiments for single UUV by June 05, multiple UUV experiments by June 06
top related