conversational game theory thomas k harris graduate seminar on dialog processing november 25, 2003

Conversational Game Theory

Thomas K Harris

Graduate Seminar on Dialog Processing

November 25, 2003

Conversational Game Theory 2

• Why look to Game Theory?– “…studying the nature of the rules of games must be

useful for the study of grammatical rules, since it is beyond doubt that there is some sort of similarity between them” –L. Wittgenstein (1958)

• Game Theory Intro– von Neumann, Morgenstein, Nash…

• A Conversational Game Theory– Power, Houghton, Kowtko and Isard…

• Conversational Game Theory SDS in Practice– Lewin @ SRI Cambridge & later with the EU’s TRINDI

• Some Evaluation


Why Look at Game Theory?

• Wittgenstein– “The use of a word in the language is its meaning.

The grammar describes the use of the words in the language. So it has somewhat the same relation to the language as the description of a game, the rules of the game, have to the game.”

• Dialogue grammars and user preferences can be coded as game rules and payoffs. Game Theory provides a mechanism and a justification for choosing/predicting among possible utterances (dialogue moves) in a game.


Game Theory Intro


Game Theory Origins

• “Studies the behavior of rational agents in competitive and collaborative situations.” Christos Papadimitriou

• Conceptualized and clearly defined by John von Neumann c. 1928 and 1937.

• Little interest until the publication of von Neumann and Morgenstern’s Theory of Games and Economic Behavior [1944].


Would you like to play Thermonuclear War?

• c. 1950’s Military think tanks esp. the Rand Institute become very interested in game theory for logistics, submarine search, air defense…

• The MAD concept is formalized in game theory. Equilibrium -> Truce

• A beautiful mind expands the theory from competitive to collaborative games.


Are You a Rational Agent?

• “Studies the behavior of rational agents in competitive and collaborative situations.”

• The following 6 slides describe an axiomatic treatment of utility for a rational agent.

• BTW, There’s a related course on this here (Philosophy Dept) : 80-305 “Rational Choice”


Can you consistently order your alternatives?

• A preference ordering exists between any two outcomes, and it is transitive.


Are you indifferent to compound lotteries?

• Compound lotteries can be reduced to simple lotteries.

• ½ + ½ ( ½ + ½ )• ≡ ½ + ¼ + ¼


Are Your Preferences Continuous?

• Each outcome Ai is indifferent to some lottery ticket involving just A1 and Ar, where for each Ai, A1 Ai and Ai Ar.

• i.e. There exist a probability p such that • p + (1-p) ≡ • Note that this says nothing about the value of p other

than p [0,1].• In particular, note that [.5 $10 + .5 $0 ≡ $2] may be

possible (risk aversion, or non-linear value of money).• Think for a sec, however, about these three outcomes:

$1, ¢1, burning at the stake. What’s your p?


Are you indifferent to prize substitutions?

• If you’ve already claimed an indifference between say, the car and the cash prize, then you should also be indifferent the substitution of one for the other inside a lottery.


Are you consistent with lotteries as well as your prizes?

• Transitivity among lottery tickets applies, that is,

• If (p1 ,p2 ) (q1 ,q2 )• And (q1 ,q2 ) (r1 , r2 )• Then (p1 ,p2 ) (r1 , r2 )


Is more of a good thing always better?

• Lotteries are monotonic.• Assuming

• (p1 , (1-p1) ) (p2 , (1-p2) )

• if and only if p1 > p2


So What?

• If you answered yes to the last 6 questions, you are a rational agent.

• This is a minimum set of assumptions for mathematically tractable theories of behavior.


Irrational Agents

• What about fewer assumptions?

• Mathematical intractability; unsolvable solutions; ambiguous results.

• Still can be good science, more apt to be called psychology.


Super-Rational Agents

• What about more assumptions?• Probably incorrect description of human

behavior; overgeneralization of human preferences; sub optimal decisions made on behalf of humans.

• May still work for games with highly proscribed objectives, e.g. parlor games, or potentially super-rational agents, e.g. virus’s or other simple automata.


Ontology of games

• “Studies the behavior of rational agents in competitive and collaborative situations.”

• # of players: 2-person, n-person

• utility relationship: zero-sum, non-cooperative, cooperative

• information: perfect information, risk, uncertainty


Games

• Chess: 2-player, perfect information, zero-sum

• Bridge: 2-player!, risk, zero-sum

• Rock-Paper-Scissors: 2-player, perfect-information, zero-sum

• Prisoners dilemma: 2-player, perfect-information, non-zero-sum

http://www.worldrps.com/advanced.html


A Common Game Tree


Conversational Games


Game Theory and Conversation

• Dialog management is decision making based on utility under uncertainty.

• This is exactly the domain of Game Theory.

• Presupposes linguistic “rules” that define how to achieve non-linguistic goals in the context of other players.


Conversational Game Types

Question Game

QW

QW-R

interrupt

pardon

RW confirmation



Pardon Game

Unrecognized Pardon



Confirmation Game

Explicit Confirmation

Yes

Implicit Confirmation

Mod

No



Interruption Game

Unimportant information



Information Game

Information confirmation



Hello Game

Hello Hello


An Illustrative Example

1 s: What time do you want to travel?

2 u: Pardon?

3 s: Please state a departure time.

4 u: Five o’clock in the evening.

5 s: Is the departure time at seventeen hundred hours?

6 u: Yes.


Parsing the Game Tree

1 s: What time do you want to travel?

QW



2 u: Pardon?

QW

Pardon

Unintelligible



3 s: Please state a departure time.

Pardon QW

Unintelligible



4 u: Five o’clock in the evening.

Pardon QW Reply

Unintelligible



5 s: Is the departure time at seventeen hundred hours?

Pardon QW Reply Conf

Unintelligible



6 u: Yes.

YesPardon QW Reply Conf

NoUnintelligible


Game Types and Move Types

• Game Types are sets of States and Move Types, and are operators on commitments.

• Move Types edges between states, can be either Game Types or Atomic Types, and are operators on propositions.


Game Types

Game Type Operation

Question add λ(p,q).[pq]

Pardon copy λ(p,q).[p]

Information add λ(p,q).[pq]


Atomic Move Types

Move Type Operation

Hello copy λ(p,q).p

Reply-Yes promote λ(p,q).[promote(p,q)]

Reply-No delete λ(p,q).[p - q]


Realizing Games

• A game is realized with a preposition under discussion q.

• For the question game in the example, the question type was realized as the preposition travel_time(x) or “a query game about travel time”.

• A confirmation game might be realized with the preposition travel_time(17:00).


Plans and Preferences

• Since games are routes toward committed propositions, plans can be made that are simply partially ordered stacks of games.

• Plans can be formed by Horn clause solvers, or other means.

• Preferences about how to choose and parse moves can be adjusted with probabilistic game tree parsing and high-level features.


Conclusions - Pros

• Conversational Game Theory appears to be loosely based on Game Theory, with many added complications.

• It’s an interesting way to define the intentional structure of dialogue into a declarative compositional data structure.

• This intentional data structure can be computed over to generate and interpret dialogue, with high-level parameters that correlate with a theoretically sound notion of utility.


Conclusions - Cons

• It seems very untested. There’s not much literature and even fewer working systems. The working systems are toys.

• So is it easy to develop more complex systems?

• Is it generic enough for a wide range of domains?

• Not everyone likes formal systems.


Another Con

• A problem with logical omniscience.– K(t, p) == “t knows that p”– r => (p => q) == “r implies that p implies q”– [K(t, p) and K(t, r)] => K(t, q) ??

• Always assumed in game theory, but even Sherlock Holmes fails this sometimes (and Watson fails often).

• Probably not a big deal; when introduced to p, r, and q, t will immediately accept q.

• There is research in psychology that may qualify the logic of the K proposition a little better.