Reaching Agreements II

2

• What utility does a deal give an agent?• Given encounter T1,T 2 in task domain T,{1,2},c • We define the utility of a deal for agent k as

Uk ( ) = c (Tk) c ( )The difference between

• The cost of agent k doing the tasks Tk it was originally assigned• The cost of the tasks it is assigned in

• Pure deal = T1,T 2 is called the conflict deal– Because no-one agrees to execute tasks other than its own

Utility of Deals

3

Utility of Deals in the Delivery Domain• The stand-alone cost

– For agent 1 is 1– For agent 2 is 3

• Deal {a},{b} gives – Agent 1 no utility – Agent 2 utility of 1

• Deal {a,b},{} gives – Agent 1 utility of -2 – Agent 2 utility of 3

Distribution Point

21

Warehouse a Warehouse b

4

Dominating DealsFor two pure deals 1 and 2, 1 dominates 2, denoted as 1 > 2, if, and only if,

– For all i {1,2}, Ui(1) Ui(2) • deal 1 is at least as good as 2 for all agents…

– There is an i {1,2}, Ui(1) > Ui(2)• deal 1 is better for some agent(s)…

1 weakly dominates 2, denoted as 1 2, iff – For all i {1,2}, Ui(1) Ui(2)

1 is equivalent to 2, denoted as 1 2, iff – For all i {1,2}, Ui(1) = Ui(2)

• In our delivery domain, we have the following relationships between deals:

{},{a,b} {a},{a,b} {a,b},{a,b}{a},{b} {a},{a,b} {a,b},{a,b}

5

Dominating Deals in Delivery Domain

6

The Negotiation Set• A deal is called individual rational if • A deal is called Pareto optimal if there does not exist

another deal ’ such that ’ • The set of all deals that are individual rational and Pareto

optimal is called the negotiation set.• The negotiation set for the delivery domain is:

{{},{a,b},{a},{b}}– N.B.: We don’t have {a,b},{} because it is not individual rational

for agent 1

7

Monotonic Concession Protocol (1)• Negotiation proceeds in a series of rounds• On the first round, both agents simultaneously propose a

deal from the negotiation set• An agreement is reached

– If the agents propose deals 1 and 2 such that eitherU1(2) U2(1) or U2(1) U1(2)

That is, an agent finds that the deal proposed by the other is as good as or better than the proposal it made

• If an agreement is reached, the agreement deal:– If both agent’s offers match or exceed those of the other agent,

then one of the proposals is selected at random– If only one proposal exceeds or matches the other’s proposal,

then this is the agreement deal

8

Monotonic Concession Protocol (2)• If no agreement is reached, then negotiation proceeds to

another round of simultaneous proposals– In round u + 1, no agent is allowed to make a proposal less

preferred by the other agent than deal proposed at round u• If neither agent makes a concession in round u > 0, then

negotiation terminates, with the conflict deal

9

Monotonic Concession Protocol (3)• For example,

– If agent 1 offers a deal that gives agent 2 a utility of 14,– While 2 proposes a deal that gives himself utility of 11,– Then the agreement will be 1’s offer.

• If both offer such a deal…– An arbitrary choice is made between them.

• If neither agent offers such a deal…– The negotiation continues to another round.

• In each round, – An agent may propose the same utility to the other or– Offer more than it did in the previous round.

10

Monotonic Concession Protocol (4)• Finally, if neither agent concedes in a round, the agreement

reached is the conflict deal.

The monotonic concession protocol over pure deals in any encounter in any task environment will terminate after a finite number of rounds.

A2’s best dealA1’s best deal

11

Negotiation Strategy• Given a negotiation protocol,

A negotiation strategy is a function from the history of the negotiation (run of the negotiation process until now) to the current offer consistent with protocol.

• An example of a strategy:– If the other agent conceded the last n rounds, I will concede with

a probability of 1/n+1.• We assume that an agent’s strategy doesn’t change during

negotiation.

12

Choosing a Negotiation Strategy• What negotiation strategy should be adopted?• Agents using the monotonic concession protocol can run

into conflict if neither concedes during a round; – The deal is then the conflict deal…

• This can happen even if there are better deals for both agents– Such a result is not efficient!

• However, to avoid an inefficient outcome, an agent may concede at every step– This is not stable!– The best response to this strategy is to never concede!

13

An Initial Strategy• Consider this strategy:

– Start by making an offer that is best for me.– Make a minimal concession in the next round.– Then, use the rule: if my opponent has conceded x% of the time, I

will concede with probability (100-x)%.• This, however, is not stable! Consider 2 agents:

– They start by conceding, but as negotiation proceeds, the chance of further concessions diminishes.

– Reaching conflict when they can do better is inefficient, and if an agent knows the other is using this strategy the best it can do is never concede.

14

Risk Evaluation• One way of thinking about which agent should concede is

to consider how much each has to lose by running into conflict.

• An agent that has made many concessions has less to lose if conflict is reached.

• This is the inverse of the initial strategy considered: – The more concessions you’ve already made, the less likely you will

make the next concession.

15

• For each round t, and each agent i , let it be the deal offer

made by i in round t.• The degree of willingness to risk conflict is

Riskit =

• More formally:

Zeuthen Risk Evaluation (1)

utility agent i loses by conceding and accepting agent j ’s offer

utility agent i loses by not conceding and causing a conflict

H G Zeuthen 1, if Ui (it ) = 0

Ui (it ) Ui (j

t ) otherwise

Ui (it )

Riskit =

16

• If t is not the last round in the negotiation protocol, then the risk is always between 0 and 1, for both agents.

• As Risk1t increases, agent 1 has less to lose from a conflict

deal, and will be more willing to concede.• Let’s consider the strategy where the agent with a smaller

risk will make the next concession – Call this the Zeuthen Strategy.

Zeuthen Risk Evaluation (2)

17

The Zeuthen Strategy• In round 1:

– Offer the deal that is best for you among all those in the negotiation set.

• In round t >1: – Calculate your risk Riski

t and the risk of your opponent Riskjt

– If your risk is smaller than or equal to that of your opponent, then make an offer that involves the minimal sufficient concession from your point of view.

– Otherwise, offer the same deal that you offered previously

18

Efficiency of the Zeuthen Strategy• Two agents using the Zeuthen strategy will not run into

conflict.• Further, they will agree on a deal that maximises the

product of their utilities; i.e. it is Pareto Optimal.

19

Stability of the Zeuthen Strategy• A negotiation strategy s is in symmetric Nash equilibrium

– If assuming that agent 1 uses s, – Agent 2 can do no better by using a strategy other than s.

• The Zeuthen Strategy is not in equilibrium.• If two agents are one step from agreement:

– It is sufficient for one of them to concede.– If one agent knows the other is using the Zeuthen Strategy, it

could diverge from the strategy and gain benefit from the other conceding.

20

Last Round in Zeuthen StrategySuppose 2 agents, each must deliver to warehouse a.• Negotiation set = {{},{a}, {a},{}}• Agent 1 starts by offering {},{a}• Agent 2 starts by offering {a},{}• If they follow the strategy they must concede so that 1

offers {a},{}) and 2 offers {},{a}.– Tossing a coin would be the only solution.

• However,– If an agent doesn’t concede (breaks from the strategy) and the

other does conform to the strategy, – Then it will receive a utility of 1.

21

Zeuthen Last Round: A Game of Chicken

50% chance ofwinning the faircoin toss.

Other agentdelivers bothcontainers.

Agent 1

Don’t Concede

Concede

Don’t Concede

00

01

Concede

10

0.50.5

Agent 2

22

Summary (1)• Monotonic concession protocol is

– Symmetric– Distributed – Guaranteed to terminate,

It also fits well with our intuition about negotiations• However

– Termination depends on common knowledge about the negotiation set.

• The Zeuthen strategy – Ensures that agents move toward agreement and, – Because it uses the concept of risk, can be seen as fair. – Agents maximise the product of their utilities.

23

Summary (2)• The Zeuthen strategy has problems in the last step.

– If an agreement will be reached when one agent concedes, it becomes a “game of chicken”

– This means that agents must have additional strategies for the last step.

• Therefore, there is a finite and positive probability that conflict will occur, leading to inefficient solutions.

24

Summary (3)• The Zeuthen strategy is not particularly simple.

– Agents need to compute the entire negotiation set – This could be very large…

• It also involves many steps to reach agreement– A potentially high communication overhead

• Finally, and most importantly,– An agent needs to know the other agent’s utility function

25

• An Introduction to Multi-Agent Systems, M. Wooldridge, John Wiley & Sons, 2002. Chapter 7.

Recommended Reading