Use of Markov Chains to Design an Agent Bidding Strategy for

Continuous Double Auctions

Sunju ParkManagement Science and Information Systems Department

Rutgers Business School, Rutgers University

Edmund H. DurfeeArtificial Intelligence Laboratory, University of Michigan

William P. BirminghamMath & Computer Science Department, Grove City College

Presenter: TinTin Yu {tiyu@mtu.edu}

Introduction

Not like tradition auctions Single seller and multiple buyers (e.g. eBay)

Continuous Double Auctions (CDA) Buyers place bids, and sellers place offers to the same items. We have a match whenever a buyer’s bid is higher than a

seller’s offer. (e.g. Name your price (hotel.com?)

Goal To determine the optimal price/offer for a seller in order to gain

the maximum profit.

Definitions

Notation: bbssp

b: buyer’s bid; s: seller’s offer sp: seller’s offer that was just submitted bbssp: a queue in ascending order (of price)

Clearing Price (CP) bspbs: When an offer is less than a bid sp<=CP<=b (the right most b) We use sp in this paper.

Definitions

Markov Chains (Markov state machine) Probabilistic finite state machine Input is ignored We uses first-order Markov chain only

First-order means the probability of the present state is a function only to its direct predecessor states.

p-strategy Algorithm (1/2)

p-strategy Algorithm (2/2)

Information used by p-strategy

Step1: Building Markov Chains (1/3)

Given a current state (bbs). When the p-seller (a seller use p-strategy) submit its offer sp,

there are four possible next auction states.

We make these states the initial states of the Markov Chain.

Step1: Building Markov Chains (2/3)

From the initial states, we keep populate the (bbss) queue by either submitting a new buyer bid or a seller offer.

If we have a match, it goes to the SUCCESS state. If it goes out of the bound (maximum number of standing offers), it goes to

the FAIL state.

Step1: Building Markov Chains (3/3)

The MC model of the CDA with starting state (bbs) and the number of bids and offers are limited to 5 each.

Step2: Compute Utilities (1/5)

Step2.1: The utilities function

Ps(p): probability of success at price p U(Payoffs(p)): utilities of payoff if the offer receives a match

CP: clearing price C: cost TD(s/f): delay overhead

Step2: Compute Utilities (2/5)

Things we need to compute for each p

Step2: Compute Utilities (3/5)

Step2.2.1: Transition Probabilities Going from state (bbs) to (bbssp) at time step n That is P(bbssp | bbs); Applying Baye’s rule; Evaluating using probability density function

(PDF), f(s); bababa…

Step2: Compute Utilities (4/5)

Step2.2.2: TD(s/f): delay overhead Too complex to cover in details It involves building a transition probability matrix P from the states of the

Markov Chain we built in step1. Here is listed equations:

: reward = c (a constant) except for the initial states and the absorbing states

: the number of visits to state (…) until it goes to S.

Step2: Compute Utilities (5/5)

Plug in the numbers and we will get a expected utility value associated with price p.

The algorithm find the optimal price p by looping through all p in a possible range.

Time complexity of the algorithm is O(n3), where is the number of possible prices, n is the number of MC states.

Benchmark (1/6)

Agents used for comparison FM: Fixed-Markup

bids its cost plus some predefined markup

RM: Random-Markup bids its cost plus some random markup

CP: Clearing-Price obtains a clearing-price quote (similar to FM agent)

OPT: Post-facto Optimal our benchmark strategy. Given it “knows” exactly everything about

the future (no uncertainty at all), it returns the maximum profit an agent may have achieved.

Benchmark (2/6)

Benchmark (3/6): p-strategy vs other

Results: Arrival rate:

0.4=high 0.1=low

negotiation zone narrow: =5

Benchmark (4/6): p-strategy vs other

Results: Arrival rate:

0.4=high 0.1=low

negotiation zone narrow: =25

Benchmark (5/6): p-strategy vs itself

Results Profit of individual

p-agent decreaseas the numberof p-agents increase.

However, when thereis more buyers,p-agents are able togain similar profitat the expense of buyers.

Benchmark (6/6): CP vs multiple p and CP

Results CP-strategy agents are

able to raises profit as the number mixed p-agents andCP-agents increase.

Conclusion

Summary: p-strategy is based on stochastic modeling of the auction process. It works while it does not need to consider much about the other

individual agents. Time complexity only depends on the number of MC states, not the number of agents.

It out performs other agents (FM/ RM/ CP)

Future Work Similar strategy can be apply to buyers. Analysis shows an average of 20% gap between p-strategy and the

optimal one. Ongoing work: hybrid strategy. This adaptive approach allow the agent

to figure out when to use stochastic model and when to use some simpler strategies.

Question to think about

Human can think very differently:e.g. Selling a 50” plasma HDTV

Place a very low selling price like $1.00 without a hidden limit.

Shipping cost = $3000.00 ?!

Can artificial intelligent agents think outside the box?

Your Questions

Bibliography

Park, S., Durfee, E.H. and Birmingham, W.P. (2004) "Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions", Volume 22, pages 175-214.