multi-agent coordination for multi-robot task allocation and area coverage
DESCRIPTION
Multi-Agent Coordination for Multi-robot Task Allocation and Area Coverage. Raj Dasgupta C-MANTIC Group Computer Science Department University of Nebraska at Omaha. Presentation at INAOE August 13, 2012. Outline. Introduction and Preliminaries Multi-robot Coverage Robotic Team Formation - PowerPoint PPT PresentationTRANSCRIPT
MULTI-AGENT COORDINATION FOR MULTI-ROBOT TASK ALLOCATION AND AREA COVERAGE
Raj Dasgupta
C-MANTIC Group
Computer Science Department
University of Nebraska at Omaha
Presentation at INAOE
August 13, 2012
04
/19
/20
23
2
INA
OE 2
01
2 - R
aj D
asg
upta
OUTLINE
Introduction and Preliminaries Multi-robot Coverage
Robotic Team Formation Flocking Coalition Game
Multi-robot Task Allocation Swarm-based Auction-based
Ongoing and Future Work
04
/19
/20
23
3
INA
OE 2
01
2 - R
aj D
asg
upta
RESEARCH PROBLEM (1)
How to coordinate a set of agents (each agent is situated on a robot) to perform a set of complex tasks in a collaborative manner Complex task: single robot does not have
resources to complete the task individually Coordination can be synchronous or
asynchronous Robots might or might not have to perform the task at
the same time Distributed Autonomous
04
/19
/20
23
4
INA
OE 2
01
2 - R
aj D
asg
upta
RESEARCH PROBLEM (2)
Performance metric(s) need to be optimized while performing tasks Time to complete tasks, distance traveled,
energy expended Robots are able to communicate with each
other Bluetooth, Wi-fi IR
04
/19
/20
23
5
INA
OE 2
01
2 - R
aj D
asg
upta
APPLICATIONS
Humanitarian de-mining (COMRADES) Autonomous exploration for planetary
surfaces (ModRED) Automatic Target Recognition (ATR) for
search and recon (COMSTAR) Unmanned Search and Rescue Civlian and domestic applications like
agriculture, vaccum cleaning, etc.
04
/19
/20
23
6
INA
OE 2
01
2 - R
aj D
asg
upta
ROBOT PLATFORMS
E-puck mini robot - suitable for table-top experiments for proof-of-concept
Coroware Corobot (indoor robot)- suitable for experiments in indoor arena within lab; hardware and software compatible with Coroware Explorer robot Coroware Explorer
(outdoor robot) – all terrain robot for outdoor experiments
All techniques are first verified on Webots simulator using simulated models of e-puck and
Corobot robots
04
/19
/20
23
INA
OE 2
01
2 - R
aj D
asg
upta
7
SOLUTIONS Swarming or emergent
computing for low-level coordination Fast, easy to implement No guarantee of achieving
desired outcome always When swarming-based
coordination fails... Low-level
coordinationSwarming Layer
04
/19
/20
23
INA
OE 2
01
2 - R
aj D
asg
upta
8
SOLUTIONS Swarming or emergent
computing for low-level coordination Fast, easy to implement No guarantee of achieving
desired outcome always When swarming-based
coordination fails...use a higher-level coordination mechanism
Game theory Branch of micro-economics
that gives rules of encounter between humans
Low-level coordination
Swarming Layer
Rules of Encounter Game Theoretic
Layer
04
/19
/20
23
9
INA
OE 2
01
2 - R
aj D
asg
upta
OUTLINE
Introduction and Preliminaries Multi-robot Coverage
Robotic Team Formation Flocking Coalition Game
04
/19
/20
23
10
INA
OE 2
01
2 - R
aj D
asg
upta
DISTRIBUTED MULTI-ROBOT COVERAGE
Efficiency is measured in time and spaceTime: reduce the time required to cover the
environmentSpace: avoid repeated coverage of regions that
have already been covered
• Use a set of robots to perform complete coverage of an initially unknown environment in an efficient mannerThe region of the environment that passes under the swathe of the robot’s coverage tool is considered as covered
Tradeoff in achieving both simultaneously
Source: Manuel Mazo Jr. and Karl Henrik Johansson, “Robust area coverage using hybrid control,”, TELEC'04, Santiago de Cuba, Cuba, 2004
FLOCKING-BASED CONTROLLER FOR MULTI-ROBOT TEAMS
04/19/2023 11INAOE 2012 - Raj Dasgupta
Works with physical characteristics such
as wheel speed, sensor reading,
pose, etc.
ControllerLayer (uses
flocking)
04
/19
/20
23
12
INA
OE 2
01
2 - R
aj D
asg
upta
MULTI-ROBOT TEAMS FOR AREA COVERAGE
Flocking based formation using Reynolds’ flocking model
Theoretical analysis: Forming teams gives a significant speed-up in terms of coverage efficiency
Simulation Results: The speed-up decreases from the theoretical case but still there is some speed-up as compared to not forming teams
04
/19
/20
23
13
INA
OE 2
01
2 - R
aj D
asg
upta
COVERAGE WITH MULTI-ROBOT TEAMS
Square
Corridor
Office
20 robots in different sized teams, in different environments over 2 hours
P. Dasgupta, T. Whipple, and K. Cheng, "Effects of Multi-Robot Team Formation on Distributed Area Coverage," International Journal on Swarm Intelligence Research (IJSIR), vol. 2, no. 1, 2011, pp. 44-69.
04
/19
/20
23
14
INA
OE 2
01
2 - R
aj D
asg
upta
DYNAMIC RECONFIGURATIONS OF MULTI-ROBOT TEAMS
Having teams of robots is efficient for coverage Having large teams of robots doing frequent
reformations is inefficient for coverage Can we make the robot teams change their
configurations dynamically Split and merge teams based on their recent
performance
LAYERED CONTROLLER FOR DYNAMICALLY REFORMING MULTI-ROBOT TEAMS
04/19/2023 15INAOE 2012 - Raj Dasgupta
Works with agent utility, agent strategies,
equilibrium points, etc.
Works with physical characteristics such
as wheel speed, sensor reading,
pose, etc.
Coalition GameLayer (uses
WVG)
ControllerLayer (uses
flocking)
Mediator
Map from agent strategy to robot
action, sensor reading to agent
utility, maintain data structure for
mappingLow-level
coordinationSwarming Layer
Rules of Encounter
Game Theoretic Layer
04
/19
/20
23
16
INA
OE 2
01
2 - R
aj D
asg
upta
COALITION GAME-BASED TEAM RECONFIGURATION
Coalition games provide a theory to divide a set of players into smaller subsets or teams
We used a form of coalition games called weighted voting games (WVG) R: set of players (robots) Each player i is assigned a weight wi
q: threshold value called quota Solution concept: What is the minimum set of players
whose weights taken together can reach q
subject to S wi >=q for all S subset of R
minimize |S|
i e S
04
/19
/20
23
17
INA
OE 2
01
2 - R
aj D
asg
upta
COALITION GAME-BASED TEAM RECONFIGURATION
Coalition games provide a theory to divide a set of players into smaller subsets or teams
We used a form of coalition games called weighted voting games (WVG) R: set of players (robots) Each player i is assigned a weight wi
q: threshold value called quota Solution concept: What is the minimum set of players
whose weights taken together can reach q
subject to S wi >=q for all S subset of R
minimize |S|
i e S
Gives each robot’s coverage
performance
Minimum coverage quality reqd. by a
team
04
/19
/20
23
18
INA
OE 2
01
2 - R
aj D
asg
upta
COALITION GAME-BASED TEAM RECONFIGURATION
Coalition games provide a theory to divide a set of players into smaller subsets or teams
We used a form of coalition games called weighted voting games (WVG) R: set of players (robots) Each player i is assigned a weight wi
q: threshold value called quota Solution concept: What is the minimum set of players
whose weights taken together can reach q
subject to S wi >=q for all S subset of R
minimize |S|
i e S
Minimum Winning
Coalition (MWC)
Gives each robot’s coverage
performance
Minimum coverage quality reqd. by a
team
04
/19
/20
23
19
INA
OE 2
01
2 - R
aj D
asg
upta
quota=3.5
OUTLINE OF WVG ALGORITHM FOR ROBOT TEAM FORMATION
Each follower robot in a team reports its coverage ratio in last T time steps as its ‘weight’ in the WVG to leader robot Represented as a ratio; low values correspond to bad
(repeated area), higher values to good (new area) coverage
Two heuristics proposed to calculate MWC BMWC: Enumerates all MWCs and finds the one with
most robots that have closest pose and shortest distance to form a team with leader – O (R2)
Greedy: Stops as soon as it finds first MWC – O (R log R)
P. Dasgupta and K. Cheng, “Distributed Multi-robot Team Reconfiguration usingWeighted Voting Game,” DARS 2010 + forthcoming journal publication
04
/19
/20
23
21
INA
OE 2
01
2 - R
aj D
asg
upta
WVG TEAM FORMATION RESULTS
Time spent by 5-robot team in different reconfiguration
operations
Percentage of a 4 m2 environment covered by 5-robot team in 30 mins for
different percentage of free space in env.
04
/19
/20
23
22
INA
OE 2
01
2 - R
aj D
asg
upta
CONCLUSIONS AND LESSONS LEARNED
Proposed a new concept of using coalition game for dynamic team reconfiguration of robots
Stablity and convergence verified analytically and experimentally
Generates fewer coalitions (lesser running time) than existing multi-robot coalition generation algorithm by Vig et al [IEEE TRO 2006]
Current technique: If the team is getting obstacles, retain q% (e.g., 70%)
of the team How to adapt this value of q?
Transfer learning: store patterns of obstacles encountered in the past and learn a mapping from obstacle pattern to best possible action of team (preliminary work in ARMS 2011)
04
/19
/20
23
23
INA
OE 2
01
2 - R
aj D
asg
upta
OUTLINE
Introduction and Preliminaries Multi-robot Coverage
Robotic Team Formation Flocking Coalition Game
Multi-robot Task Allocation Auction-based Swarm-based heuristics
04
/19
/20
23
INA
OE 2
01
2 - R
aj D
asg
upta
24
MULTI-ROBOT TASK ALLOCATION
Our solutions: Swarming heuristics-based: Simple
to implement, high comm. overhead.
Market (auction)-based: Guarantees per task completion
time, time-out when a task cannot be completed, trade-off (efficiency loss) in waiting for better solutions
Almost 90% lower comm. overhead than heuristics approaches
T tasks, R robots, T >> R Constraints:
Each robot can communicate with a subset r of other robots (can change over time)
Robots do not know T, has to be discovered online by robots
Each task has to be done by rcap robots
Each task has time constraint Problem: How to find an
allocation from 2R -> T while minimizing the time (distance traveled) and communication overhead to all robots NP-complete problem
TaskRobot
Direction of movement
Legend
04
/19
/20
23
25
INA
OE 2
01
2 - R
aj D
asg
upta
AUCTION-BASED MRTA An agent discovers a task and begins an
auction Other agents in communication range hear
about the auction If an agent does not have a full task list, it
bids in the auction with its distance to the task, via any tasks on its task list
R
4R’s task list
4
04
/19
/20
23
26
INA
OE 2
01
2 - R
aj D
asg
upta
AUCTION-BASED MRTA: FIXEDBIDS An agent discovers a task and begins an
auction Other agents in communication range hear
about the auction If an agent does not have a full task list, it
bids in the auction with its distance to the task, via any tasks on its task list – because it can’t decommit from a previously committed/won auction
R
4 3
5
R’s task list
4
= 7 = 4+3R’s bid on
04
/19
/20
23
27
INA
OE 2
01
2 - R
aj D
asg
upta
FIXEDBIDS BIDDING STRATEGY PROBLEM
Previous strategy: Bid with distance from current location to the task’s location, via any tasks on its task list (No de-commitment!) Constraint: Newly arriving tasks added at end of
task list Problem: Creates very inefficient routes
R
4 3R’s task list
4
7 = 4+3
R’s bid on = 7+6=13
6
04
/19
/20
23
28
INA
OE 2
01
2 - R
aj D
asg
upta
DYNAMICBIDS AUCTION
Soft bids: Bid with a lower bound and an upper bound value (instead of a single bid)
Agent can then replan its path while keeping its edges (to different tasks already on its task list) between bidl and bidu
R
4 3
R’s task list
(4, 8)
(7, 12)
04
/19
/20
23
29
INA
OE 2
01
2 - R
aj D
asg
upta
DYNAMICBIDS AUCTION
Soft bids: Bid with a lower bound and an upper bound value (instead of a single bid)
Agent can then replan its path while keeping its edges (to different tasks already on its task list) between bidl and bidu
R
4 3
R’s task list
(4, 8)
(7, 12)
04
/19
/20
23
30
INA
OE 2
01
2 - R
aj D
asg
upta
DYNAMICBIDS AUCTION
Soft bids: Bid with a lower bound and an upper bound value (instead of a single bid)
Agent can then replan its path while keeping its edges (to different tasks already on its task list) between bidl and bidu
R
33
R’s task list
(4, 8)
(7, 12)
R’s actual cost
5
8
(2, 5) 2
2
04
/19
/20
23
31
INA
OE 2
01
2 - R
aj D
asg
upta
BIDDER ALGORITHM WITH DYNAMICBIDS
On receiving information about a new task (request for bid)Solve a TSP (approximation algorithm) with
nodes corresponding to the existing tasks in task list and new task
If bidu is not exceeded for any task already committed to (existing in task list) as per TSP solution Insert new task into task list at position given by TSP Send the cumulative path cost of new task as bidl
Calculate bidu = bidl + ~N(rc/2,1.0) Send (bidl, bidu) to auctioneer as soft bid for new task
Else don’t bid on new task
04
/19
/20
23
32
INA
OE 2
01
2 - R
aj D
asg
upta
DYNAMICBIDS AUCTIONEER SIDE
Two criteria for selecting winners from the set of biddersWho has the least cost (distance) to get here
(bidl)What could be the delay in executing the task
(and consequent decay in pheromone) if the bidder revises its bid to bidu later on Called “Loss in efficiency” Given by (1-e(bidl-bidu)/v)/(nbid e bidl/v+1)
Selects bidder based on a weighted product of these two criteria
04
/19
/20
23
33
INA
OE 2
01
2 - R
aj D
asg
upta
ANALYTICAL RESULTS: AUCTIONEER
DynamicBids algorithm is robust Handles livelocks gracefully
Analytically proven upper bounds on How many bids an auctioneer should wait for How much time it should wait for getting those
bids How much can auctioneer “lose” if a bidder
revises its bids Is DynamicBids always better? Theorem:
Marginal cost for every task using the DynamicBids algorithm < Marginal cost of the task using the FixedBids algorithm.
04
/19
/20
23
INA
OE 2
01
2 - R
aj D
asg
upta
34
EXPERIMENTAL SETUP Webots robotic
simulation platform 4 X 4 m2
environment 9-27 e-pucks, 5-60
tasks placed randomly, averaged over 10 runs
E-puck sensors IR Bluetooth Camera Overhead camera
based localization
04
/19
/20
23
37
INA
OE 2
01
2 - R
aj D
asg
upta
SWARM-BASED HEURISTICS FOR MRTA
Distance-based heuristic Select a task that is “closest to me and has highest amount
of pheromone” Robot density-based heuristic
Each robot selects a task that has least number of robots in its vicinity, lowest pheromone (starved tasks first)
Preference-based heuristic Density-based heuristic + amount of task outstanding
(starved tasks nearing completion first) Proximity-based heuristic
Density-based heuristic + effect of other robots - how many other robots are likely to be headed (ahead of me) to the task?• D. Miller, P. Dasgupta, T. Judkins, "Distributed Task Selection in Multi-agent based Swarms using Heuristic
Strategies,“ LNCS vol. 4433 (Proc. 2nd Swarm Robotics Workshop, Rome, Italy), 2006, pp. 158-172.• P. Dasgupta, "Multi-Robot Task Allocation for Performing Cooperative Foraging Tasks in an Initially Unknown Environment,“ Innovations in Defense Support Systems - 2 , Springer, Studies in Computational Intelligence, vol. 338, 2011, pp. 5-20.
Swarm-based heuristics• Require flooding of task updates to all robots in communication
range• Robots make decisions asynchronously
04
/19
/20
23
39
INA
OE 2
01
2 - R
aj D
asg
upta
AUCTION VS. SWARM-BASED HEURISTICS FOR MRTA
0
2000
4000
6000
8000
10000
12000
14000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Robot Number
To
tal
no
. o
f b
yte
s s
en
t o
r re
ceiv
ed
Auction Based
Market Based DF
Comparison of total number of bytes between auction protocoland swarming-based heuristic protocols
04
/19
/20
23
41
INA
OE 2
01
2 - R
aj D
asg
upta
CONCLUSIONS AND LESSONS LEARNED
Market-based MRTA offers an inherently distributed and robust technique for MRTA Communication overhead is significantly lower
than swarm-heuristic based techniques Soft guarantees can be made about time (no. of
rounds) required to complete tasks Market-based MRTA algorithms are not the
silver bullet Open Problem: Is there a relationship
between spatial and temporal distribution of tasks and type of MRTA algorithm used?
MRTA results (time) are very susceptible to underlying robot path planning algorithm
04
/19
/20
23
44
INA
OE 2
01
2 - R
aj D
asg
upta
ONGOING WORK
Coverage Dynamic coverage information compression
(ICINCO 2011) Using Voronoi Partitions for Coverage (ACODS
2012, SPIE 2012, ICRA 2012, ARMS 2012) MRTA
Spatial Queueing Theory for MRTA (ICINCO 2012) Reconfiguration Planning for modular robots
(ModRED project) Distributed Information Aggregation (Foretell
project)
04
/19
/20
23
46
INA
OE 2
01
2 - R
aj D
asg
upta
CONCLUSION
Multi-agent and multi-robot systems appear to be made for each other Collaboration through coordination is cornerstone
of both MAS and MRS Bridging the gap
Abstractions, assumptions vs. crisp definitions Robustness in operation, data (sensor error) Computational overhead vs. fast operation Simulations vs. physical experiments Scientists vs. engineers
04
/19
/20
23
47
INA
OE 2
01
2 - R
aj D
asg
upta
ACKNOWLEDGEMENTS
We thank our sponsors DoD Navair US Office of Naval Research NASA (ESPCoR program)
C-MANTIC Research Group Members Coverage: Taylor Whipple, Dr. Ke Cheng, Task Allocation: Taylor Whipple, Matthew Hoeing,
David Miller, Timothy Judkins http://cmantic.unomaha.edu