graphical models for strategic and economic reasoning michael kearns computer and information...
Post on 19-Dec-2015
213 views
TRANSCRIPT
![Page 1: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/1.jpg)
Graphical Models forStrategic and Economic
Reasoning
Michael KearnsComputer and Information Science
University of Pennsylvania
BNAIC 2003
Joint work with: Sham Kakade, John Langford,Michael Littman, Luis Ortiz, Satinder Singh
![Page 2: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/2.jpg)
Probabilistic Reasoning
• Need to model a complex, multivariate distribution• Dimensionality is high --- cannot write in “tabular” form• Examples: joint distributions of alarms and earthquakes,
diseases and symptoms, words and documents
• The world is not arbitrary:– Not all variables (directly) influence each other– True for both causal and stochastic influences– Many probabilistic independences hold– Interaction has (network) structure– Should ease modeling and inference
• The answer: graphical models for probabilistic reasoning
![Page 3: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/3.jpg)
Structure in Probabilistic Interaction
[Frey&MacKay 98]
[Horvitz 93]
Engineered “Natural”
![Page 4: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/4.jpg)
International Trade
[Krempel&Pleumper]
Embargoes, free trade, technology, geography…
![Page 5: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/5.jpg)
Corporate Partnerships
[Krebs]
![Page 6: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/6.jpg)
Internet Connectivity
[CAIDA]
![Page 7: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/7.jpg)
Structure in Social and Economic Analysis
• Trade agreements and restrictions• Social relationships between business people• Reporting and organizational structure in a firm• Regulatory restrictions on Wall Street• Shared influences within an industry or sector• Geographical dispersion of consumers• Structural universals (Social Network Theory)
Goal: Replicate the power of graphical modelsfor problems of strategic reasoning.
• Strategic Reasoning:– Variables are players in a game, organizations, firms, countries…– Interactions characterized by self-interest, not probability– Foundations: game theory and mathematical economics
![Page 8: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/8.jpg)
Outline
• Graphical Games and the NashProp Algorithm– [K., Littman & Singh UAI01]; [Ortiz & K. NIPS02]
• Correlated Equilibria, Graphical Games, and Markov Networks– [Kakade, K. & Langford EC03]
• Arrow-Debreu and Graphical Economics– [Kakade, K. & Ortiz 03]
![Page 9: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/9.jpg)
Graphical Games and NashProp
![Page 10: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/10.jpg)
Basics of Game Theory
• Have players 1,…n (think of n as large)• Each has actions 1,…,k (think of k as small)• Action chosen by player i is a_i • Vector a is population joint action• Player i receives payoff M_i(a)• (Note: M_i(a) has size exponential in n!)
• (Nash) equilibrium: – Choice of mixed strategies for each player– No player has a unilateral incentive to deviate– Mixed strategy: product distribution over a
• Exists for any game; may be many
![Page 11: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/11.jpg)
![Page 12: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/12.jpg)
Graphical Models for Game Theory
• Undirected graph G capturing local (strategic) interactions• Each player represented by a vertex• N_i(G) : neighbors of i in G (includes i)• Assume: Payoffs expressible as M_i(a’), where a’ over only N_i(G)• Graphical game: (G,{M’_i})• Compact representation of game; analogous to graph + CPTs• Exponential in max degree (<< # of players)• As with Bayes nets, look for special structure for efficient inference• Related models: [Koller & Milch 01] [La Mura 00]
2
4
3
5
8
7
6
1
![Page 13: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/13.jpg)
The NashProp Algorithm
• Message-passing, tables of “conditional” Nash equilibria• Approximate (all NE) and exact (one NE) versions, efficient for
trees • NashProp: generalization to arbitrary topology (belief prop)• Junction tree and cutset generalizations [Vickrey & Koller 02]
U1 U2 U3
W
V
T(w,v) = 1 <--> an “upstream” Nash where V = v given W = w <--> u: T(v,u_i) = 1 for all i, and v is a best response to u,w
![Page 14: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/14.jpg)
• Table dimensions are probability of playing 0• Black shows T(v,u) = 1• Ms want to match, Os to unmatch• Relative value modulated by parent values• =0.01, = 0.05
![Page 15: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/15.jpg)
![Page 16: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/16.jpg)
Experimental Performance
number of players
com
pu
tati
on
tim
e
![Page 17: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/17.jpg)
Correlated Equilibria, Graphical Games and Markov Networks
![Page 18: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/18.jpg)
The Problems with Nash
• Technical:– Difficult to compute (even in 2-player, multi-action
case)
• Conceptual:– Strictly competitive– No ability to cooperate, form coalitions, or bargain– Can lead to suboptimal collective behavior
• Fully cooperative game theory:– Somewhat of a mathematical mess
• Alternative: correlated equilibria
![Page 19: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/19.jpg)
Correlated Equilibria in Games[Aumann 74]
• Recall Nash equilibrium is a product distribution P(a)• Suffices to guarantee existence of equilibrium• Now let P(a) be an arbitrary distribution over joint
actions• Third party draws a from P and gives a_i to player i • P(a) is a correlated equilibrium:
– Conditioned on everyone else playing P(a|a_i), playing a_i is optimal– No unilateral incentive to deviate, but now actions are correlated– Reduces to Nash for product distributions
• Alternative interpretation: shared randomness• Everyday example: traffic signal
![Page 20: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/20.jpg)
Advantages of CE
• Technical:– Easier to compute: linear feasibility formulation– Efficient for 2-player, multi-action case
• Conceptual:– Correlated actions a fact of the real world– Allows “cooperation via correlation”– Modeling of shared exogenous influences– Enlarged solution space: all mixtures of NE, and more– New (non-Nash) outcomes emerge, often natural ones– Avoid quagmire of full cooperation and coalitions– Natural convergence notion for “greedy” learning
• But how do we represent an arbitrary CE?– First, only seek to find CE up to (expected) payoff
equivalence– Second, look to graphical models for probabilistic reasoning!
![Page 21: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/21.jpg)
Graphical Games and Markov Networks
• Let G be the graph of a graphical game (strategic structure)• Consider the Markov network MN(G):
– Form cliques of the local neighborhoods of G
– Introduce potential function c on each clique c
– Joint distribution P(a) = (1/Z) c c(a) • Theorem: For any game with graph G, and any CE of this game,
there is a CE with the same payoffs that can be represented in MN(G)
• Preservation of locality• Direct link between strategic and probabilistic reasoning in CE• Computation: In trees (e.g.), can compute a CE efficiently
– Parsimonious LP formulation
![Page 22: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/22.jpg)
From Micro to Macro:Arrow-Debreu and Graphical
Economics
![Page 23: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/23.jpg)
Arrow-Debreu Economics• Both a generalization (continuous vector actions) and
specialization (form of payoffs) of game theory• Have k goods available for consumption• Players are:
– Insatiable consumers with utilities for amounts of goods– “Price player” (invisible hand) setting market prices for goods
• Liquidity emerges from sale of initial endowments– Alternative model: labor and firms
• At equilibrium (consumption plans and prices):– Each consumer maximizing utility given budget constraint– Market clearing: supply equals demand for all goods– May also allow supply to exceed demand at 0 price (free disposal)
• ADE always exists• Very little known computationally
– [Devanur, Papadimitriou, Saberi, Vazirani 02]: linear utility case
![Page 24: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/24.jpg)
Graphical Economics
• Again wish to capture structure, now in multi-economy interaction
• Represent each economy by a vertex in a network– “Economies” could be represent individuals or sovereign nations– From international relations to social connections
• Same goods available in each economy, but permit local prices• Interpretation:
– Allowed to shop for best prices in neighborhood– Utility determined only by good amounts, not their sources
• Stronger than ADE: graphical equilibrium– Consumers still maximize utility under budget constraints– Local clearance in all goods (domestic supply = incoming demand)
2
4
3
5
8
7
6
1
![Page 25: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/25.jpg)
Graphical Economics: Results
• Graphical equilibria always exist (under ADE condition analogues)– does not follow from AD due to zero endowments of foreign goods– appeal to Debreu’s quasi-rationality: zero wealth may ignore zero
prices– Wealth Propagation Lemma: spread of capital on connected graph– relative gridding of prices and consumption plans
• ADProp algorithm:– computes controlled approximation to graphical equilibrium– message-passing on conditional prices and inbound/outbound demands– efficient for tree topologies and smooth utilities
2
4
3
5
8
7
6
1
![Page 26: Graphical Models for Strategic and Economic Reasoning Michael Kearns Computer and Information Science University of Pennsylvania BNAIC 2003 Joint work](https://reader035.vdocument.in/reader035/viewer/2022081519/56649d3e5503460f94a16a31/html5/thumbnails/26.jpg)
Conclusion• Use of game-theoretic and economic models rising
– Evolutionary biology– Behavioral game theory and economics NYT 6/17– Neuroeconomics– Computer Science– Electronic Commerce
• Many of these uses are raising– Computational issues– Representational issues
• Well-developed theory of graphical models for GT/econ– Structure of interaction between individuals and organizations
• What about structure in– Utilities, actions, repeated interaction, learning, states,…
[email protected]/~mkearns