arrow’s impossibility theorem and experimental tests of...

Arrow’s ImpossibilityTheorem and Experimental

Tests of PreferenceAggregation

Todd DaviesSymbolic Systems Program

Stanford University

joint work with Raja Shah, Renee Trochet, andKatarina Ling

Decision Behavior: Theory and Evidence (Symsys 170/270)

May 26, 2010

Questions1. Does intuitive preference aggregation robustlyviolate Arrow’s criteria of IIA and collectiverationality?

2. Can we characterize intuitive preferenceaggregation judgments in a descriptive framework?

3. What is the relationship between intuitivepreference aggregation and normative approaches tosocial choice?

DEFINITION: Preference relations R, P, and Irespectively exist between two outcomes wheneverthe outcomes can be matched with labels x and y suchthat:

• x R y, meaning x is preferred or indifferent to y(weak preference).

• x P y, meaning x is preferred to y, and holds iffnot y R x. (strict preference).

• x I y, meaning x is indifferent to y, and holds iff xR y and y R x (indifference).

DEFINITION: A preference relation R is weaklyordered iff for all outcomes x, y, and z: in a set ofoutcomes X:

• x R y or y R x (completeness).• x R y and y R z imply x R z (transitivity).

DEFINITION: A social welfare function is afunction f such that if

• X is the set of all conceivable social outcomes orstates (|X| ≥ 3),

• N is a set of individuals (2 ≤ N = n), and• D is a domain of preference profiles RN =

<R1,R2,…Rn>, such that• for all i in N, Ri is a weakly ordered preference

relation for individual i over X (individualrationality),

then f maps D into a range R of possible socialpreference relations RN on X for group N.

DEFINITION: A social welfare function f is anArrow social welfare function if

• the domain D of f contains all possiblepreference profiles RN (universal domain),

and• the range R of f is the set of weakly ordered

social preference relations RN (collectiverationality).

THEOREM: Arrow's Impossibility Theorem(Arrow, 1951/1963). There can be no Arrow socialwelfare function f satisfying all of the following:

• for all social outcomes x and y in X, if apreference profile RN obeys x Pi y for eachindividual i in N, then f(RN) yields x RN y (weakPareto efficiency);

• for all preference profiles RN and RN' in D, andall social outcomes x and y in X, if RN and RN'obey x Ri y iff x Ri' y for all individuals i in N,then f(RN) and f(RN') yield x RN y iff x RN' y(independence of irrelevant alternatives);

and• there is no individual d in N such that for all

preference profiles RN in D, and all socialoutcomes x and y in X, if x Pd y then f(RN) yieldsx PN y (non-dictatorship).

Proof. See Arrow (1963) and Mas-Colell, Whinstonand Green (1995).

Arrow on IIA

“The essential argument in favor of this principle isits direct appeal to intuition.” (1952)

“Stricter than desirable” but has practical benefits,necessary to keep the lid on the need to gatherlimitless information on unavailable options (1967)

Dead candidate example:If an election were held and one of the candidatesthen died: "Surely the social choice should be madeby taking each of the individual's preference lists,blotting out the dead candidate's name, andconsidering only the orderings of the remainingnames…" (1951/1963).

DEFINITION: A social choice rule C maps thedomain D of preference profiles, together withenvironments S ⊆ X, into subsets of S.

SIDE NOTE: The above definition does not assumethat C obeys an ordering. Arrow did assume this,however:

DEFINITION: A social choice rule C is an Arrowsocial choice function if it is determined by anArrow social welfare function f and for allenvironments S ⊆ X, C(S,f(RN)) = {x: x is in S and,for all y in S, f(RN) yields x RN y}.

DEFINITION: Given a set X of social outcomes, apreference profile RN, and a social choice rule C, xRN

* y (meaning there is a revealed social preferencefor x in relation to y) iff there is some environment S⊆ X, such that x and y are in S and x is in C(S,RN).

DEFINITION: Weak axiom of revealed socialpreference. Given a set X of social outcomes, apreference profile RN, and a social choice rule C, asocial preference relation RN satisfies inter-menuconsistency1 (IMC) iff it is a revealed socialpreference relation under C for all pairs of socialoutcomes in X, and the following condition holds:For all environments S ⊆X, if x and y are in S and x isin C(S,RN), then for all environments S' ⊆ X such thatx and y are in S', if y is in C(S',RN), then x is inC(S',RN).

THEOREM: If RN* is a revealed social preference

relation associated with a set X of social outcomes, apreference profile RN, and a social choice rule C, andRN

* violates inter-menu consistency, then it violatescollective rationality.

PROOF: Suppose not. Then there is a social welfarefunction f such that f(RN) = RN

* and f is collectively rational.By hypothesis, RN

* violates inter-menu consistency, so there1 This term is from Sen (1993).

exist x, y, S, and S' such that x is in C(S,RN), y is in C(S',RN),x is not in C(S',RN). Thus, by the definition of a revealedsocial preference relation: x RN

* y, y RN* x, and there is a z in

S' such that not x RN* z. So z PN

* x by the definition of strictpreference. But since y is in C(S',RN), y RN

* z. By thedefinition of collective rationality, RN

* is weakly ordered.Applying transitivity and the definition of strict preference:y RN

* z, z PN* x, and x RN

* y imply y PN* y, implying not y

RN* y, which violates the completeness condition for x = y.

DEFINITION: A social choice rule C, defined on aset X of social outcomes and a domain D ofpreference profiles satisfies independence ofunavailable alternatives (IUA) iff for allenvironments S ⊆ X, and all preference profiles RN

and RN' in D, the following condition holds: If forall x, y in S, RN and RN' obey x Ri y iff x Ri' y for allindividuals i in N, then C(S,RN) = C(S,RN').

Figure 1 – Profile triplets (Rank-form presentation)

In each triplet, picking X in top profile and Y in middle profile violates IMC.Picking Y in middle profile and X in bottom profile violates IIA. Triplets differ inextent of “Borda reversal” between profiles (number of X-Y rank shifts)

5-2

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4 Voter 5X X X Y YY Y Y X X

5-3

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4 Voter 5Z X X Y YX Y Y Z ZY Z Z X X

5-3'

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4 Voter 5X X X Z YZ Z Y Y XY Y Z X Z

4-3

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4X X Y YY Z X XZ Y Z Z

4-6

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4U X Y YX Z W VY Y V UZ U U WV V X XW W Z Z

4-6'

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4U X Y YX Z X XZ U W VV V V UW W U WY Y Z Z

3-2

Ranked Alternatives

Voter 1 Voter 2 Voter 3X X YY Y X

3-4

Ranked Alternatives

Voter 1 Voter 2 Voter 3X X YY Y ZZ W WW Z X

3-4'

Ranked Alternatives

Voter 1 Voter 2 Voter 3X X ZZ W YW Y XY Z W

7-2

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4 Voter 5 Voter 6 Voter 7X X X X Y Y YY Y Y Y X X X

7-3

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4 Voter 5 Voter 6 Voter 7Z Z X X Y Y YX X Y Y Z Z ZY Y Z Z X X X

7-3'

RankedAlternatives

Voter 1 Voter 2 Voter 3 Voter 4 Voter 5 Voter 6 Voter 7X X X X Y Y YZ Z Z Z X X XY Y Y Y Z Z Z

Experimental VariationsBetween Subjects

Each survey taker aggregates only one profile ina given triplet, is asked “Which alternativeshould be chosen for the group?”

Within SubjectsEach survey taker’s aggregations are tested foradherence to IIA and/or collective rationality

Ordering TaskEach survey taker asked to rank order thealternatives for the group instead of choosingjust one

Pairwise FormatProfile is presented to survey taker as a matrix ofindividuals’ pairwise preferences instead of inthe rank format shown above

Restricted Availability Independence of unavailable alternatives(IUA). Survey taker is shown profile but toldthat only options S and T are available.

Indifference OptionSurvey taker is told they s/he can circle morethan one option for a group of voters if bothoptions are equally desirable given the profile.

Presentation Order VariationsExample: Some survey takers see one profile in atriplet before another, with other survey takersseeing them in the reverse order.

Profile Relationship VariationsDifferent profiles are presented to a survey takeras representing different voters and/or differentalternatives, or changes in the menu of options.

Graphs 1.1-1.3: IMC and IIA Between subjects (strict choice only) results for 5-voterprofile pair

IMC Between subjects Results: 5-2, 5-3

34

11

38

0510152025303540

X Y

Alternative selected

N Reduced (N=35)Expanded (N=39)

IIA Between subjects: 5-3

1

3841

30

10

20

30

40

50

X Y


N 5-3 (N=39)5-3' (N=46)



37

2

9

26

0510152025303540

X Y




9

26

42

20

10

20

30

40

50

X Y


N 3-4 (N=35)3-4' (N=46)



36

40

32

0510152025303540

X Y




0

32

40

30

10

20

30

40

50

X Y


N 4-6 (N=33)4-6' (N=46)

Ordering Task:

RankedAlternatives

51% 49%L PO LM OP NN QQ M

Out of 59 subjects, 55 ranked P ahead of M, whileonly 4 ranked M ahead of P. Assuming that theywould have ranked M ahead of P if only shown thevoting blocs’ preferences over these two alternatives,we can say that the subject group, as a whole, isviolating the IMC criterion.

Within-subjects: Tests for 5 voters x 3 alternativespairings, rank format presentation

Test Violators NonviolatorsIMC 84% 16%IIA 92% 8%

Question 1: Does intuitive preference aggregationrobustly violate Arrow’s criteria of IIA andcollective rationality?

Answer: Yes.

Rank Format results summary:

For each of the four profile triplets, when presentedin rank format, a majority of survey takers’aggregations violate both collective rationality (CR)and IIA, both between and within subjects.Examples: In the 5-voter triplet…– Between subjects, 34/35 pick S in 5-2, 38/39pick T in 5-3, and 41/44 pick S in 5-3’– Within-subjects test finds 84% violate CR and92% violate IIA.

Pairwise presentation:

5-3 (Pairwise)

X vs. Y Y vs. Z Z vs. X

Voter AX Z Z

Voter BX Y X

Voter CX Y X

Voter DY Y Z

Voter EY Y Z

5-3' (Pairwise)

X vs. Y Y vs. Z Z vs. X

Voter FX Z X

Voter GX Z X

Voter HX Y X

Voter IY Z Z

Voter JY Y X

Presentation format results summary:

For the 4-voter and 7-voter triplets, survey takersrobustly violate both CR and IIA when profiles arepresented in pairwise format as well, but pairwiseformat can induce adherence to CR and IIA when theBorda reversal is weak (e.g. the 3- and 5-votertriplets). Examples: Between subjects…– 90/90 pick T in 4-6, and 70/90 pick S in 4-6’– 100/100 pick S in 5-3’, and 55/100 pick S in 5-3

Other findings:

Availability of indifference option does notsignificantly increase adherence to IIA or CR.

Order of profile presentation does not affect resultswhen Borda swings are large enough, but can materwhen they are small.

Attendees of the Public Choice Society annualmeeting show the same pattern of results as naïvesurvey-takers.

Question 2: Can we characterize intuitive preferenceaggregation judgments in a descriptive framework?

Answer: Survey-takers appear to apply a variety ofprinciples to preference aggregation, none of which isfully determinative when they conflict:

• PRINCIPLE 1: Independence of irrelevantvoters (IIV). Social choices should be basedonly on the preferences of individuals affectedby the choice.

• PRINCIPLE 2: Interpersonal comparisons ofutility (ICU). Social choices should take intoaccount, as much as possible, information aboutthe relative strengths of preference thatindividuals in the choice group have for onealternative over another.

• PRINCIPLE 3: Inter-menu independence (IMI).Social choices should take into account only thealternatives that are available, and not utilizeinformation about how individuals rank theseavailable alternatives among other sets whichinclude unavailable outcomes.

• PRINCIPLE 4: Independence of unavailablealternatives (IUA). Social choices should be thesame whenever the set of available outcomesdoes not change, even if individuals change theirpreferences involving unavailable outcomes.(weakly supported)

Adherence to IUA is somewhat stronger thanadherence to either IIA or CR, and for weak Bordareversals shows majority adherence when IIA and CRare violated

There probably exist other empirically supportableprinciples as well.

Question 3: What is the relationship betweenintuitive preference aggregation and normativeapproaches to social choice?

Two suggested answers:

A. No principle of social choice should be regardedas indispensable if there are situations in which thefully-informed opinions of a large majority ofintelligent people think that the best social decision isone that entails violating the principle.

B. Surveys that systematically assess neutralsubjects’ aggregation across a large number ofabstract preference profiles could be the basis forinducing a weighted, multi-criterion hybrid socialchoice procedure that would optimally characterizepeople’s intuitions about fair social choice.

arrow’s impossibility theorem and experimental tests of...

Documents