distributed multiagent resource allocation in diminishing marginal return domains yoram...

Distributed Multiagent Resource Allocation In Diminishing Marginal Return Domains

Yoram Bachrach(Hebew University)

Jeffrey S. Rosenschein (Hebrew University)

Outline

Multiagent Resource Allocation (MARA) General problem Applications

Centralized and decentralized mechanisms Selfish behavior challenge

Specific restricted domain VCG solution in restricted domain

Allocation by interaction Market motivation behind method

Allocation protocol and suggested strategies Convergence to optimal allocation Strategic and selfish behaviour Expected time to convergence

Conclusions and future research

Multiagent Resource Allocation

Allocating resources to users Scarce resources Selfish agents with private information

Both users and resource owners

An allocation maps resources to users

MARA Applications

Industrial procurement Satellite resources Tasks in manufacturing systems Grid computing RF spectrum and coverage …

MARA Domain Properties

Divisible / Indivisible Can parts of a single resource be allocated to several agents?

Sharable / Non-Sharable Can a resource be allocated to several agents simultaneously?

Single-Unit / Multi-Unit Are there bundles of identical resources?

Transferable / Non Transferable Utility Can agents compensate by transferring utility among them?

MARA Approaches

Attempt to maximize social welfare Other possible goals – Maximin, fairness, … There may be more than one optimal allocation

Centralized mechanisms A central mechanism gets the agents’ preferences and chooses

an outcome

Decentralized approaches Agents actively participate in choosing the outcome

Problem – agents are selfish and attempt to maximize their own utility

Centralized Mechanisms

The mechanism must elicit the agents’ private information about allocations But agents may manipulate to increase their own utility

We are interested in incentive compatible mechanisms Agents reply truthfully, under a certain rational behavior Rational behavior captured in a game theoretic solution concept

Vickery-Clarke-Groves (VCG) approach Tax agents to make truth telling is a dominant strategy Strategyproof, allocatively efficient but only weakly budget balanced

Distributed Mechanisms

Central mechanisms may not be appropriate in distributed environmentsHard to establish a trusted central authorityScalability concerns – the central mechanism may be a

performance bottleneck Have agents interact among themselves to

choose the allocationNeed to define the protocol for interactionSelfish agents may still manipulate

Specific Domain

Set of identical agents Each agent only requires a single resource, and does not benefit

from being allocated more than one resource

Set of resources Cannot be divided among agents Can be shared among agents

Diminishing marginal production The total utility of the agents who are allocated a certain

resource drops as more agents use that resource

Diminishing Marginal Return

10

10

7

14

7

5

5

15

5


10

10

7

14

7

5

5

15

5

Total production is 10


10

10

7

14

7

5

5

15

5

Total production increases to 14


10

10

7

14

7

5

5

15

5

Total production increased by 4 when adding a single agent Marginal production of 4


10

10

7

14

7

5

5

15

5

Total production increased by 1 when adding a single agentMarginal production of 1

What needs to be decided?

A mechanism must decide: An allocation – which agent gets which resource

We want to maximize the social welfare – total production

Utility transfers Agents gain utility due to the allocation

Resource owners receive nothing

Resource owners hold the private information Eliciting this information requires incentivizing the resource owners to report

their production function Requires giving resource owners some of the utility

We assume the total production across all the resources can be redistributed in any way

VCG in Restricted Domain

Easy to compute an optimal allocation Resources report total production functions Find maximal social welfare by a greedy algorithm

Assign to the resource with maximal marginal production

Induce truthfullness by VCG tax Requires establishing a trusted central authority

Trust and security issues, central bottleneck, … Weakly budget balanced – some of the total production is

kept in the mechanism and not distributed

Allocation by Interaction

Define a protocol for interaction between agents and resource owners Simulate a market for services

Interaction proceeds in discrete time rounds Each round determines both an allocation and transfers

Design protocol and suggest interaction strategies so that the optimal allocation is always reached

Challenges Achieve the optimal allocation despite selfishness Make sure the optimal allocation is reached quickly

Interaction Protocol

R1

R2

R3

Round Payment (5)

Currently on R1, getting utility 5


R1

R2

R3

Resource Request

Currently on R1, getting utility 5


R1

R2

R3

Payment Bid (10)


R1

R2

R3

Accept

Switch to R2 with utility 10


R1

R2

R3

Decline

Stay on R1, with utility 5


R1

R2

R3

Round Payment 10

Currently on R2 with utility 10


R1

R2Payment Change (5)

Currently on R2 with utility 5

The Resource Owner’s Perspective

4

4

13

4

5

5

12

Production – 12Payments – 10Utility – 2

Production – 13Payments – 12Utility – 1

Chosen Allocation

The interaction decides both the allocation and redistribution of the utility Agents are allocated the last resource whose bid they accepted Agents get the utility as in the last payment bid they accepted Resource owners keep the reminder of the production on the

resource not redistributed to the agents

The allocation may change at the end of every round An allocation is stable if once reached it never changes

Depends on the strategies of the participants Agents and resource owners

Suggested Strategy - Agents

Each round, randomly choose a resource and request using the resource If the bid in that resource is better than the

current bid, switch to that resource (accept) If the bid is lower than the current resource

offers, stay with current resource

Suggested Strategy – Resource Owners Keep the agents’ share of the utility in the level of

the marginal production on the resource On round start, offer all the agents allocated to the

resource the current last marginal production Answer resource requests with bid of the next

marginal production on the resource If accepted, set the bid for all the agents to the new

marginal production by a Payment Change message If declined – do nothing

Resource Owners - Example

10

10

4

14

4

1

1

15

1

MP = 4 MP = 1

Protocol Stable Allocation

Given a set of strategies for the agents and resource owners, a protocol stable allocation is one that, once reached, never changes Under these strategies, no interaction results in an agent switching to a

different resource Protocol stable under the suggested strategies

No agent is ever given a bid higher than what he is currently getting on his current resource

Resource owners bid the next marginal production There is no resource where the next marginal production is greater than

the current marginal production on other resources Similar to greedily allocating agents to resources according to marginal production

Convergence to Optimum

Under the suggested strategies, the chosen allocation always converges to the optimal allocation Monotonic improvement

If an agent switches resources, the social welfare increases Stability in optimum

The optimal allocation is protocol stable No “local” optima – protocol stable is optimal

If a non optimal allocation is chosen, there is a possible round where an agent switches resources

What about strategic behavior?

Strategic Behavior

Agents and resource owners have to follow the protocol, but not the suggested strategies Might obtain higher utility by choosing a different strategy

Agents may accept a bid lower than what they currently have Resource owners may suggest a bid different than the current

marginal production Higher, to attract more agents Lower, to give a lower share of utility to the agents

Is such strategic behavior rational for self interested agents?

Strategic Agents (Our domain)

If an agent gained from strategic behavior, we still reach an optimal allocation If a single agent has deviated from the

suggested strategy and gained utility Gained utility: a protocol stable allocation has been

reached, in which the agent gets a higher utility

Then the reached protocol stable allocation is also optimal

Strategic Resource Owners

Resource owners who set too high a bid Attract more agents but pay more and lose utility

Resource owners who set too low a bid Pay less, but lose agents to competing resources

who offer higher bids

When the domain is competitive for resource owners, such a manipulation is irrational

Highly competitive settings Condition that occurs mostly in environments where there are

many resources with similar marginal production values Similar resources or slight changes in marginal production

Strategic Resource Owners

In our specific domain Diminishing marginal return Highly competitive for resource owners

If a resource owner gained from strategic behavior, we still reach an optimal allocation If a single resource owner has deviated from the

suggested strategy and gained utility Gained utility: a protocol stable allocation has been reached, in

which the resource owner gets a higher utility

Then the reached protocol stable allocation is optimal

Convergence Time

When agents and resource owners behave rationally, we converge to an optimal allocation But how quickly is the optimal allocation reached?

Under the suggested strategiesExpected time to convergence: Bound on convergence time:

Quick polynomial convergence

Related Work

TFG-MARA survey Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J. A.

Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation.

Distributed mechanism design approaches J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: Recent results and

future directions.

Scheduling domains B. Heydenreich, R. Muller, and M. Uetz. Decentralization and mechanism design for online

machine scheduling. Negotiations over resources

U. Endriss, N. Maudet, F. Sadri, and F. Toni. Negotiating socially optimal allocations of resources. T. W. Sandholm. Contract types for satisficing task allocation.

Conclusions

A distributed approach to resource allocation in a specific domain Achieves optimal allocation (maximal social welfare) No central authority required All utility divided among agents and resource owners

“Strongly budget balanced”

Quick convergence

Can a similar approach be applied to other domains (or more general domains)?

distributed multiagent resource allocation in diminishing marginal return domains yoram...

Documents