asymmetrical adjustments in preference elicitation · multicriteria decision car 14 3.25 0 784...

Asymmetrical adjustments in

preference elicitation.

The effect of multicriteria conflict.

Stéphane Deparis, Ph.D.

IBM Research

Ireland Lab

OUTLINE

1. Multicriteria decision aid : normative, descriptive, prescriptive.

2. Multicriteria conflict

3. An experiment on matching judgments

4. Results

5. Recommendation for the practice of elicitation

2

MULTICRITERIA DECISION

Car 14 3.25 0 784

Subway 22 1.70 3 13

Bus 32 1.70 8 352

Taxi 11 18.60 2 637

Walking 50 0 0 30

3

Duration(min) Cost

(Euros)

Total waiting

time

(min)

CO2 emission

(g)

Montparnasse → Etienne Marcel

• Evaluations of the options

• Preferences of the decision-maker

a g1(a) g2(a) … gn(a)

b g1(b) g2(b) … gn(b)

… … … … …

e g1(e) g2(e) … gn(e)

4

g1 g2 … gn

a ↔ (g1(a), g2(a), … , gn(a))

Preference structure:

• total order b > e > d > a > c

• total preorder b > e ~ d > a > c

• partial preorder b > a, d > c ~ e and a ? c

EVALUATION MODEL

TWO TYPES OF PREFERENCE MODELS

SYNTHESIS CRITERION OUTRANKING

Aggregating performances

in a global performance.

ex : additive aggregation

g(a) = ∑ kivi( gi(a) )

Comparisons

Total preorder

g(a) > g(b) = g(c)…

Pairwise comparisons

« Majority vote » by criteria

Aggregating

Incomplete structure

5

m

i=1

aSb ¬(aSb)

bSa a~b a<b

¬(bSa) a>b a?b

AMPLITUDE OF MULTICRITERIA CONFLICT

6

Taxi 11 18.60

Car 14 5.25

Subway 21 1.70

Bus 32 1.70

Walk 50 0

Time

(min)

Cost

(Euros)

Dominance

AMPLITUDE OF MULTICRITERIA CONFLICT

7

Low conflit

Taxi 11 18.60

Car 14 5.25

Subway 21 1.70

Bus 32 1.70

Walk 50 0

Time

(min)

Cost

(Euros)

AMPLITUDE OF MULTICRITERIA CONFLICT

8

High conflit Time

(min)

Cost

(Euros)

Taxi 11 18.60

Car 14 5.25

Subway 21 1.70

Bus 32 1.70

Walk 50 0

THREE APPROACHES IN DECISION-AID

9

Prescriptive Recommendation

Descriptive Real behaviour

ex: Kahneman et Tversky

Normative Rationality

ex: Von Neumann and Morgenstern1944

THREE APPROACHES IN DECISION-AID

10

Prescriptive Recommandation

Descriptive Real behaviour

Normative Rationality

PRESCRIPTIVE APPROACH

11

Considering the preferences of a real decision-maker

to parameterize a formal model.

elicitation

Preferences

Preferential

parameters Set of alternatives

Multicriteria

aggregation

Preference Structure

THREE APPROACHES IN DECISION-AID

12

Prescriptive Recommandation

Descriptive Real behaviour

Normative Rationality

DESCRIPTIVE APPROACH

• Procedural invariance

« Normatively equivalent procedures for assessing

preferences should give rise to the same preference order. »

Kahneman et al. 1988

13

Most important criterion

favored (Kahneman, Sattath, Slovic 1988)

Choice

(a1, a2) ? (b1, b2)

Matching

(a1, a2) I (b1, ? )

EXPERIMENT

16

Objective :

investigate the effect of

multicriteria conflict

on the expression of

incomplete preferences

When conflict induces the expression of incomplete

preferences, European Journal of Operations Research, 221

(2012), pp. 593-602

ISOPREFERENCE CHAINS : A MATTER OF TASTE

17

0

200

400

600

800

1000

1200

14005 4 3 2 1

Ren

t

Distance to city center

attractive

neutral

unattractive

COMPARISONS WITH VARYING CONFLIT

18

ISOPREFERENCE CHAINS : A MATTER OF TASTE

19

0

200

400

600

800

1000

1200

14005 4 3 2 1

Ren

t

Distance to city center

attractive

neutral

unattractive

ISOPREFERENCE CHAINS : A MATTER OF TASTE

20

0

200

400

600

800

1000

1200

14005 4 3 2 1

Ren

t

Distance to city center

attractive

neutral

unattractive

ISOPREFERENCE CHAINS : A MATTER OF TASTE

21

0

200

400

600

800

1000

1200

14005 4 3 2 1

Ren

t

Distance to city center

attractive

neutral

unattractive

RESULTS

24

P 73%

I 9%

R 18% I 21%

P 79%

IMPLICATIONS FOR THE ELICITATION OF PREFERENCES

Pairwise comparisons → trade-off information

(Ratio, Smart, Swing weights)

When there is a large multicriteria conflict between two

options, an « indifference » answer is ambiguous:

equivalent or incomparables ?

Authorize the expression of incomparability

or restrain to low conflict comparisons.

25

OUTLINE

1. Multicriteria decision aid : normative, descriptive, prescriptive.

2. Multicriteria conflict

3. An experiment on matching judgments

4. Results

5. Recommendation for the practice of elicitation

26

EXPERIMENT

27

Objective :

investigate the effect of

multicriteria conflict

on the response asymmetry

during a double matching

DOUBLE MATCHING

28

A

B B

A’ A

Indifference Indifference

forward backward

Asymmetry δ = A’1 – A1

PARTICIPANTS : 29 students (ECP and Paris-Dauphine)

928 double matchings

OPTIONS : Apartments

CRITERIA : Rent (€)

Time to go to the city-center (min)

CONTEXT

29

VARIABLE : DIRECTION OF MATCHING

30

improving

degrading

improving

degrading

VARIABLE : MULTICRITERIA CONFLICT

31

ideal 5

attractive 15

rather attractive 25

neutral 35

rather unattractive 60

unattractive 90

worse 120

criterion: time (min)

Conflict level 1

lowest

VARIABLE : MULTICRITERIA CONFLICT

32

criterion: time (min)

Conflict level 2

ideal 5

attractive 15

rather attractive 25

neutral 35

rather unattractive 60

unattractive 90

worse 120

VARIABLE : MULTICRITERIA CONFLICT

33

criterion: time (min)

Conflict level 4

maximal

ideal 5

attractive 15

rather attractive 25

neutral 35

rather unattractive 60

unattractive 90

worse 120

OBSERVED ASYMMETRY (DEGRADING MATCHINGS)

34 δmoy = -16 min t(28)= -11.1, p<.001

Also observed by

Willemsen et Keren

2003

backward

forward

Commuting time

rent

OBSERVED ASYMETRY (IMPROVING MATCHINGS)

35 δmoy = +11 min t(28)= 13.7, p<.001

Non significant in

Willemsen et Keren

2003

forward

backward

rent

Commuting time

EFFECT OF MULTICRITERIA CONFLICT

36

Effect of conflict : F(3, 81)=142.7, p<.001

Effect of direction : F(1,27)=13.2, p=.001

Interaction : F(3, 81)=7.5, p<.001

unfavourable

favourable

Conflict

EFFECT OF MULTICRITERIA CONFLICT

37

Effect of conflict : F(3, 81)=142.7, p<.001

Effect of direction : F(1,27)=13.2, p=.001

Interaction : F(3, 81)=7.5, p<.001

unfavourable

favourable

Conflict

Asymmetry

towards stimulus,

In minutes

EFFECT SIZE

38

63 min

29 min

EXPERIMENT ON BIMATCHING : SUMMARY

• Greater knowledge of the multicriteria decision behaviour in

response to matching.

• The amplitude of multicriteria conflict magnifies the

asymmetry between forward and backward matchings.

• Effect of strong magnitude. Larger on unfavourable double

matchings.

39

IMPLICATIONS FOR THE ELICITATION OF PREFERENCES

Matching tasks are widely used to measure trade-off

information (ex : Keeney et Raiffa 1976).

• Use low conflict tasks.

• Use favouring matching tasks.

40

STIMULUS AS A REFERENCE POINT

41

|v(-∆l)| = v(+∆t) v(-∆’t) = v(+∆l)

or v(+∆l) < |v(-∆l)|

v(+ ∆’t ) < v(-∆’t) < v(+∆t)

INTRANSITIVITY OF INDIFFERENCE

42

a I b

b I a’ but a’ ∆ a

a I b

b I c

c I d

d I e

but a R e

• G. Fischer et al., Goal-based construction of preferences: Task goals

and the prominence effect, Management Science, 45(8), pp. 1057-1075,

1999

• R.L. Keeney and H. Raiffa. Decisions with multiple objectives :

Preferences and value trade-offs. J. Wiley, New York, 1976.

• M.C. Willemsen and G. Keren. The meaning of indifference in choice

behavior: asymmetries in adjustments embodied in matching.

Organizational Behavior and Human Decision Processes, 90(2), pp.342-

359, 2003.

• A. Tversky, S. Sattath, and P. Slovic. Contingent weighting in judgment

and choice. Psychological review, 95(3):371, 1988.

• J. Von Neumann, O. Morgenstern, A. Rubinstein, and H.W. Kuhn.

Theory of games and economic behavior. Princeton Univ Pr, 2007.

REFERENCES

46