Multi-Agent Resource allocation

Notes prepared by Ulle EndrissUsed with permission

What is MARA?

A tentative definition would be the following:Multiagent Resource Allocation (MARA) is the process of distributing a number of items amongst a number of agents.

What kind of items (resources) are being distributed?How are they being distributed? And finally, why are they being distributed?

OutlineConcerning the specification of MARA problems:– Overview of different types of resources– Representation of the preferences of individual agents

(done)– Notions of social welfare to specify the quality of anallocation (partly done already)• Concerning methods for solving MARA problems:– Discussion of allocation procedures– Some complexity results concerning allocation

proceduresY. Chevaleyre, P.E. Dunne, U. Endriss, J. Lang, M. Lemaˆıtre, N. Maudet,J. Padget, S. Phelps, J.A. Rodr´ıguez-Aguilar and P. Sousa. Issues in MultiagentResource Allocation. Informatica, 30:3–31, 2006.

Types of Resources

• A central parameter in any resource allocation problem is the nature of the resources themselves.

• There is a whole range of different types of resources, and each of them may require different techniques . . .

• Distinguish properties of the resources themselves and characteristics of the chosen allocation mechanism. Examples:

– Resource-inherent property: Is the resource perishable?– Characteristic of the allocation mechanism: Can theresource be shared amongst several agents?

Continuous vs. Discrete ResourcesResources may be continuous (e.g. energy) or discrete(e.g. fruit).• Discrete resources are indivisible; continuous resources may betreated either as (infinitely) divisible or as indivisible (e.g. onlysell orange juice in units of 50 liters ; discretization).• Representation of a single bundle:

– Several continuous resources: vector over non-negative reals– Several discrete resources: vector over non-negative integers– Several distinguishable discrete resources: vector over {0, 1}

• Classical literature in economics mostly concentrates on asingle continuous resource; recent work in AI and ComputerScience focuses on discrete resources.

Divisible or not

Resources may be treated as being divisible or indivisible.

• Continuous/discrete: physical property of resources• Divisible/indivisible: feature of the allocation

mechanism

Sharable or not

• A sharable resource can be allocated to a number of different agents at the same time. Examples:– – a photo taken by an earth observation satellite– – path in a network (network routing)

• More often though, resources are assumed to be non-sharable and can only have a single owner at a time. Examples:– – energy to power a specific device– – fruit to be eaten by the agent obtaining it

Static or not

Resources that do not change their properties during a negotiation process are called static resources. There are at least two types of resources that are not static:– consumable goods such as fuel– perishable goods such as food

In general, resources cannot be assumed to be static. However, in many cases it is reasonable to assume that they are as far as the negotiation process at hand is concerned.

Single-unit vs. Multi-unit• In single-unit settings there is exactly one copy of each type ofgood; all items are distinguishable (e.g. several houses).• In multi-unit settings there may be several copies of the sametype of good (e.g. 10 bottles of milk).• Note that this distinction is only a matter or representation:

– Every multi-unit problem can be translated into a single-unit problem by introducing new names for the items

– (inefficient, but possible).– Every single-unit problem is in fact also a (degenerate) multi-

unit problem.• Multi-unit problems allow for a more compact representation of

allocations and preferences, but also require a richer language (variables ranging over integers, not just binary values).

Resources vs. Tasks

• Tasks may be considered resources with negative utility.• Hence, task allocation may be regarded a MARA

problem.• However, tasks are often coupled with constraints

regarding their timing.

Remark: From now on, we are going to deal with the allocation of static indivisible resources that are available in single units and that cannot be shared . . .

Setting

Set of agents A = {1..n} and finite set of indivisible resources R.

An allocation A is a partitioning of R amongst the agents in A.

Each agent i A has got a utility function ui : 2R R.We usually write ui(A) as a shorthand for ui(A(i)).We shall sometimes refer to the preference relation i

induced by the utility function ui: R ≤i R’ iff ui(R) ≤ ui(R’).Remark: MARA with indivisible resources is a prime

example for a combinatorial domain.

Social WelfareAn important parameter in the specification of a MARA problemconcerns our goals: what kind of allocation do we want to achieve?• Success may depend on a single factor (e.g. revenue of anauctioneer), but more often on an aggregation of preferencesof the individual agents in the system.• Concepts from social choice theory and welfare economics canbe useful here (“multiagent systems as societies of agents”).• Here we use the term social welfare in a broad sense, todescribe the quality of an allocation in view of a suitableaggregation of the individual agent preferences.

Pareto optimality is probably the most basic criterion for socialoptimality, but there are many others . . .

Social Welfare Orderings

A collective utility function is a function W : RnR mapping utility vectors to the reals. Here we define them over allocations A (inducing utility vectors):

• The utilitarian social welfare is defined as the sum of utilities:

• The egalitarian social welfare is given by the utility of the agent that is currently worst off:

• The Nash product is the product of the individual utilities:

Social Welfare Orderings (cont.)

• The elitist social welfare is given by the utility of the agent that is currently best off:

• Let uA be the ordered utility vector induced by allocation A (from lowest to highest). Then the k-rank dictator CUF (collective utility function) swk is defined as the kth element of the vector. When k=1, it is egalitarian.

• The leximin-ordering is a social welfare ordering that may be regarded as a refinement of the egalitarian collective utility function: a is less than b if a lexically precedes b in the uA order

Normalized UtilityIt can be useful to normalize utility functions before aggregation:• If A0 is the initial allocation, then we may restrict attention toallocations A that Pareto-dominate A0 and use utility gainsui(A) − ui(A0) rather than ui(A) as problem input.

• We could evaluate an agent’s utility gains relative to the gainsit could expect in the best possible case. Define an agent’smaximum utility with respect to a set Adm of admissible allocations: bui = best of possible allocationsThen define the normalized individual utility of agent i:u’i(A) = ui(A)/bui

The optimum of the leximin-ordering wrt. normalized utilitiesis known as the Kalai-Smorodinsky solution.

Ordered Weighted Averaging

We can build families of parameterized collective utility functions that induce several social welfare orders. An example are the ordered weighted averaging operators.

Let w = <w1,w2, . . . ,wn> be a vector of real numbers. Define:

This generalizes several other social welfare orders:• If w is such that wi = 0 for all i ≠k and wk = 1, then we haveexactly the k-rank dictator collective utility function.• If wi = 1 for all i, then we obtain the utilitarian collective utility function.• If wi =α i−1, with α > 0, then the leximin-ordering is the limit of the social welfare order induced by sww as goes to 0.

Envy-Freeness

An allocation is called envy-free iff no agent would rather have one of the bundles allocated to any of the other agents: A(i) i A(j)

Recall that A(i) is the bundle allocated to agent i in allocation A. Note that envy-free allocations do not always exist (at least not if we require either complete or Pareto optimal allocations).

ExampleConsider the following example with two agents and three goods:A = {1, 2} and R = {a, b, c}. Suppose utility functions are additive:u1({a}) = 18 u1({b}) = 12 u1({c}) = 8u2({a}) = 15 u2({b}) = 8 u2({c}) = 12Let T be the allocation giving {a} to agent 1 and {b,c} to agent 2.• T has maximal egalitarian social welfare (18); utilitarian socialwelfare is not maximal (38 rather than 42); and neither iselitist social welfare (20 rather than 38).• T is Pareto optimal and leximin-optimal, but not envy-free.• There is no allocation that would be both Pareto optimal andenvy-free. But if we change u1({a}) = 20 (from 18), then Tbecomes Pareto optimal and envy free.

Degrees of EnvyAs we cannot always ensure envy-free allocations, another

approach would be to try to reduce envy as much as possible.But what does that actually mean?A possible approach to systematically defining different ways ofmeasuring the degree of envy of an allocation:• Envy between two agents:max{ui(A(j)) − ui(A(i)), 0} [or even without max]• Degree of envy of a single agent:0-1, max, sum• Degree of envy of a society:max, sum [or indeed any social welfare order/collective utility

function]

Allocation Procedures and Complexity• We have now seen the various components that are

needed tospecify a MARA problem (type of resource, agent

preferences,optimality criterion).• In the future, we are going to see how agents can negotiate

optimal allocations in a distributed manner and could consider a centralized allocation procedure (combinatorial auctions).

• Now we are going to look into the computational complexity of the problem of finding an optimal allocation, independently from any specific allocation procedure.

Resource Allocation Problems

For the purpose of formally stating the resource allocationproblems for which we want to analyze the complexity, let

aresource allocation setting <A,R, U> be given by:• A = {1, 2 . . . , n} is a set of n agents;• R = {r1, r2, . . . , rm} is a collection of m resources; and• U = {u1, u2, . . . , un} describes the utility function ui :

2RQ for the agent i in A.The set of allocations A is the set of partitionings of R

amongst A (or equivalently, the set of total functions from R to A).

Welfare Optimization

How hard is it to find an allocation with maximal social welfare?

Rephrase this optimization problem as a decision problem:

Welfare Optimization (WO)Instance: <A,R, U>; K RealsQuestion: Is there an allocation A such that swu(A) > K?Unfortunately, the problem is intractable:

Envy-Freeness

Checking whether a given setting admits an envy-free allocation (if all goods need to be allocated) is again NP-complete:

Allocation Procedures

To solve a MARA problem, we firstly need to decide on anallocation procedure. This is a complex issue, involving at

least:• Protocols: What types of deals are possible? What

messages do agents have to exchange to agree on one such deal?

• Strategies: What strategies can agents use for a given protocol? How can we incentivize agents to behave in a certain way?

• Algorithms: How do we solve the computational problemsfaced by agents when engaged in negotiation?

Centralized vs. Distributed NegotiationAn allocation procedure to determine a suitable allocation ofresources may be either centralized or distributed:• In the centralized case, a single entity decides on the finalallocation, possibly after having elicited the preferences of theother agents. Example: combinatorial auctions

• In the distributed case, allocations emerge as the result of asequence of local negotiation steps. Such local steps may ormay not be subject to structural restrictions (for instance, theprotocol may only allow for bilateral deals – mutual obligations).Which approach is appropriate under what circumstances?

Advantages of the Centralized ApproachMuch recent work on negotiation and resource allocation(particularly in the MAS community) has concentrated oncentralized approaches, in particular on combinatorial auctions.There are several reasons for this:• The communication protocols required are relatively simple.• Many results from economics and game theory, in particular

on mechanism design, can be exploited.• Recent advances in the design of powerful algorithms forwinner determination in combinatorial auctions.

Disadvantages of the Centralized ApproachBut there are also some disadvantages to the centralized

approach:• Can we trust the centre (the auctioneer)?• Does the centre have the computational resources required?(but beware: distributing it doesn’t dissolve NP-hardness)• Less natural to take an initial allocation into account (in anauction, usually the auctioneer owns everything to begin with).• Less natural to model step-wise improvements.• Arguably, only the distributed approach is a seriousimplementation of the MAS paradigm (that is, whileadmittedly being difficult, we would really like to understandhow to make distributed decision making work . . . ).

Auction Protocols

Auctions are centralized mechanisms for the allocation of goods amongst several agents. Agents report their preferences (bidding) and the auctioneer decides on the final allocation (and on prices).

The Contract Net ProtocolOriginally developed for task decomposition and allocation, butalso applicable to distributed negotiation over resources. Each agentmay assume the roles of manager and bidder. The Contract Netprotocol is a one-to-many protocol matching an offer by a managerto one of potentially many bidders. There are four phases:• Announcement phase: The manager advertises a deal to anumber of partner agents (the bidders).• Bidding phase: The bidders send proposals to the manager.• Assignment phase: The manager elects the best bid andassigns the resource(s) accordingly.• Confirmation phase: The elected bidder sends a confirmation.

R.G. Smith. The Contract Net Protocol: High-level Communication and Controlin a Distributed Problem Solver. IEEE Trans. Comp., 29:1104–1113, 1980.

ExtensionsThe immediate adaptation of the original Contract Net protocolonly allows managers to advertise a single resource at a time, and a

bidder can only offer money in return for that resource (not other items).

Possible extensions:• Allow for negotiation over the exchanges of bundles of items.• Allow for deals without explicit utility transfers (monetarypayments). The announcement phase remains the same, butbids are now about offering resources in exchange, not money.• Allow agents to negotiate several deals concurrently and tode-commit from deals within a certain period.• In leveled-commitment contracts, agents are also allowed tode-commit, but have to pay a pre-defined penalty.

Properties of Allocation ProceduresWe may study different properties of allocation procedures:• Termination: Is the procedure guaranteed to terminateeventually?• Convergence: Will the final allocation be optimal according toour chosen social welfare measure?• Incentive-compatibility: Do agents have an incentive to reporttheir valuations truthfully? (mechanism design)• Complexity results: What is the computational complexity offinding a socially optimal allocation of resources?

Next, we are going to see an example for a convergence property . . .

Negotiating Socially Optimal Allocations

We are now going to analyze a specific model of distributed

negotiation (defined on the next slide).We are not going to talk about designing a concrete

negotiation protocol, but rather study the framework from an abstract point of view. The main question concerns the relationship between

• the local view: what deals will agents make in response to their individual preferences?; and

• the global view: how will the overall allocation of resources evolve in terms of social welfare?

An Abstract Negotiation Framework• Finite set of agents A and finite set of indivisible resources R.• An allocation A is a partitioning of R amongst the agents in A.Example: A(i) = {r5, r7} — agent i owns resources r5 and r7• Every agent i A has got a utility function ui : 2R R.Example: ui(A) = ui(A(i)) = 577.8 — agent i is pretty happy• Agents may engage in negotiation to exchange resources inorder to benefit either themselves or society as a whole.• A deal = (A,A’) is a pair of allocations (before/after).• A deal may come with a number of side payments tocompensate some of the agents for a loss in utility.A payment function is a function p : A R withExample: p(i) = 5 and p(j) = −5 means that agent i pays $5,while agent j receives $5.• What are the benefits of side-payments?

The Global/Social PerspectiveAs emphasized in earlier lectures, there are many different (fairnessor efficiency) criteria that we could use to define our goals.For now, suppose that as system designers we are interested inmaximizing utilitarian social welfare:swu(A) = ui(A)Observe that there is no need to include the agents’ side-payments

into this definition, because they’d always add up to 0.While the local perspective is driving the negotiation process, weuse the global perspective to assess how well we are doing. Note that trying to improve social welfare is at odds with being self-

interested.

ExampleLet A = {ann, bob} and R = {chair, table} and suppose our agentsuse the following utility functions:uann({ }) = 0 ubob({ }) = 0uann({chair}) = 2 ubob({chair}) = 3uann({table}) = 3 ubob({table}) = 3uann({chair, table}) = 7 ubob({chair, table}) = 8Furthermore, suppose the initial allocation of resources is A0 withA0(ann) = {chair, table} and A0(bob) = { }.Social welfare for allocation A0 is 7, but it could be 8. By movingonly a single resource from agent ann to agent bob, the formerwould lose more than the latter would gain (not IR). The onlypossible deal would be to move the whole set {chair, table}.What would the appropriate side payment be?

Linking the Local and the Global PerspectivesIt turns out that individually rational deals are exactly those dealsthat increase social welfare:Lemma 1 (Rationality and social welfare) A deal = (A,A1’with side payments is individually rational iff swu(A) < swu(A’).Proof: “ →”: Rationality means that overall utility gains outweighoverall payments (which are = 0).“←”: The social surplus can be divided amongst all dealparticipants by using, say, equal division.

Discussion: The lemma confirms that individually rationalbehavior is “appropriate” in utilitarian societies.

ConvergenceIt is now easy to prove the following convergence result (originally

stated by Sandholm in the context of distributed task allocation):Theorem 1 (Sandholm, 1998) Any sequence of IR deals willeventually result in an allocation with maximal social welfare.Proof: Termination follows from our lemma and the fact that thenumber of allocations is finite. So let A be the terminal allocation.Assume A is not optimal, i.e. there exists an allocation A’ withswu(A) < swu(A’). Then, by our lemma, deal = (A,A’) is individuallyrational → contradiction. Discussion: Agents can act locally and need not be aware of theglobal picture (convergence is guaranteed by the theorem).

Multilateral NegotiationOn the downside, outcomes that maximize social welfare can onlybe guaranteed if the negotiation protocol allows for deals involvingany number of agents and resources:

Theorem 2 (Necessity of complex deals) Any deal δ= (A,A’)may be necessary, i.e. there are utility functions and an initialallocation such that any sequence of individually rational dealsleading to an allocation with maximal social welfare would have toinclude (unless is δ “independently decomposable”).The proof involves the systematic definition of utility functionssuch that A’ is optimal and A is the second best allocation.Independently decomposable deals (to which the result does notapply) are deals that can be split into two sub-deals concerningdistinct sets of agents.

Negotiation in Restricted DomainsMost work on negotiation in MAS is concerned with bilateralnegotiation or auctions. THUS Multilateral negotiation is difficult!Maybe we can guarantee convergence to a socially optimalallocation for structurally simpler types of deals if we restrict therange of utility functions that agents can use?First, two negative results:• Theorem 2 continues to hold even when all agents have to usemonotonic utility functions. [R1 R2 ) ui(R1) ≤ ui(R2)]• Theorem 2 continues to hold even when all agents have to usedichotomous utility functions. [ui(R) = 0 or ui(R) = 1]NOTE: that often progress is made by limiting the cases.

Modular DomainsA utility function ui is called modular iff it satisfies the followingcondition for all bundles R1,R2 R:

ui(R1R2) = ui(R1) + ui(R2) − ui(R1R2)

That is, in a modular domain there are no synergies between items; youcan get the utility of a bundle by adding up the utilities of its elements.

→Negotiation in modular domains is feasible:Theorem 3 (Modular domains) If all utility functions are modular,then individually rational 1-deals (each involving just one resource)suffice to guarantee outcomes with maximal social welfare.We also know that the class of modular utility functions is maximal : nostrictly larger class could still guarantee the same convergence property.

Simulation and ExperimentsWhile we know from Theorem 3 that 1-deals (blue) guarantee an

optimal result, an experiment (20 agents, 200 resources, modular utilities) suggests that general bilateral deals (red) achieve the same goal faster. The graph shows how utilitarian social welfare (y-axis) develops as agents attempt to contract more an more deals (x-axis) amongst themselves.

Communication Complexity

We should also consider the communication complexity of negotiating socially optimal allocations: focus on the length of negotiation processes and the amount of information exchanged, rather than just on computational aspects.

U. Endriss and N. Maudet. On the Communication Complexity of Multilateral Trading. Journal of Autonomous Agents and MAS, 11(1):91–107, 2005.

Aspects of Complexity(1) How many deals are required to reach an optimal allocation?– communication complexity as number of individual deals(2) How many dialogue moves are required to make one such deal?– affects communication complexity as number of moves(3) How expressive a communication language do we require?– Minimum requirements: propose, accept, reject + content language to specify multilateral deals– affects communication complexity as number of bitsexchanged(4) How complex is the reasoning task faced by an agent whendeciding on its next dialogue move?– computational complexity (local rather than global view)

Number of DealsThere are two results on upper bounds pertaining to the first variant

of our negotiation framework (with side payments, general utility functions, and aiming at maximizing utilitarian social welfare):

Theorem 4 (Shortest path) A single rational deal is sufficientto reach an allocation with maximal social welfare.Proof: Use Lemma 1δ= (A,A0) is IR iff swu(A) < swu(A’)]. Theorem 5 (Longest path) A sequence of rational deals canconsist of up to |A||R| − 1 deals, but not more.Proof: No allocation can be visited twice (same lemma) and thereare |A||R| distinct allocations ) upper bound follows.To show that the upper bound is tight, we need to show that it ispossible that all allocations have distinct social welfare . . .

Path Length in Modular DomainsIf all agents are using modular utility functions and only negotiate1-deals, then we obtain the following bounds:• Shortest path: ≤ |R|• Longest path: ≤|R| · (|A| − 1)There are similar results for a framework without monetary sidepayments (where the goal is to reach a Pareto optimal allocation).Dunne (2005) has also worked on the topic of communication

complexity in distributed negotiation, but generally this is still very much an under-explored area . . .

P.E. Dunne. Extremal Behaviour in Multiagent Contract Negotiation. Journalof Artificial Intelligence Research, 23:41–78, 2005.

More on Convergence

Generally, it is interesting to see for what kind of combination of deals and optimality criteria we can get convergence results:

• Deals: structural constraints and rationality criteria• Optimality criteria: various social welfare orders, degrees

of envy, . . .For example, the result (Theorem 1) shows that by using the

rationality criterion given by our definition of individual rationality and by not imposing any structural constraints, we can guarantee convergence with respect to the optimality criterion given by the notion of utilitarian social welfare.