decision making - the university of edinburgh · 2019. 3. 21. · 22/03/2019 3 heuris’cs and...

DecisionMakingin Robots and Autonomous Agents

BehaviouralAspectsofHumanChoice:

Heuris;cs&Biases

SubramanianRamamoorthySchoolofInforma;cs

22March,2019

TheRa'onalAnimalTheGreeks(Aristotle)thoughtthathumansarera'onalanimals.

–  Manyancientphilosophersagreed–  Clearly,evenAristotlewasawarethatpeople’s

judgments,decisions&behaviorarenotalwaysra'onal.–  Amoreplausiblerevisedclaim:allnormalhumanshave

thecompetencetobera'onal.ArgumentthusfarinDMRhasbeeninthisspirit:

–  “correct”principlesofreasoning&decisionmakingareinourminds,evenifwedon’talwaysusethem

–  “Excep'ons”werethingslikeBernoulli’sobserva'onthatwebecomeprogressivelymoreindifferenttolargergains

22/03/2019 2

22/03/2019 3

Heuris'csandBiases

•  TverskyandKahnemandrama'callyextendedBernoulli’saccount,andprovidedextensivepsychologicalevidencefortheirmodel.

•  Whattheyshowedisthathumansdepartinmul'plewaysfromaclassicalpictureofra'onalbehavior.

•  ManyhaveinterpretedtheKahneman&TverskyprogramasshowingthatAristotlewaswrong.–  mostpeopledonothavethecorrectprinciplesforreasoning&decisionmaking

–  wegetbywithmuchsimplerprincipleswhichonlysome/mesgettherightanswer

Amos Tversky1937 - 1996

Daniel Kahneman

AnIntui'onfromGigerenzer

Which US city has more inhabitants, San Diego or San Antonio?•  62%ofAmericansgotthiscorrect•  100%ofGermansgotthiscorrect

TheRecogni'onHeuris'c:Ifoneoftwoobjectsisrecognizedandtheotherisnot,theninferthattherecognizedobjecthasthehighervalue.

EcologicalRa'onality:Heuris'cworkswhenignoranceissystema'c(non–random),i.e.,whenlackofrecogni'oncorrelateswiththecriterion.

22/03/2019 4Goldstein & Gigerenzer, 2002, Psychological Review

ExperimentalStudiesofHumanReasoning

•  TheConjunc'onFallacy•  BaseRateNeglect•  Overconfidence•  Framing

22/03/2019 5

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Please rank the following statements by their probability, using 1 for the most probable and 8 for the least probable.

(a) Linda is a teacher in elementary school. (b) Linda works in a bookstore and takes Yoga classes. (c) Linda is active in the feminist movement. (d) Linda is a psychiatric social worker. (e) Linda is a member of the League of Women Voters. (f) Linda is a bank teller. (g) Linda is an insurance sales person. (h) Linda is a bank teller & is active in the feminist movement.

Naïve subjects: (h) > (f) 89%

Sophisticated subjects: 85% 22/03/2019 6

Johnis19,wearsglasses,isalidleshyunlessyoutalktohimaboutStarTrekorLordoftheRings,andstaysuplatemostnightsplayingvideogames.Whichismorelikely?1.  JohnisaCSmajorwholovescontemporarysculpture

andgolf,or2.  Johnlovescontemporarysculptureandgolf?

Onproblemslikethis,peopletendtopick(1)above(2),incontradic'ontoprobabilitytheory.

22/03/2019 7



22/03/2019 8

A panel of psychologists have interviewed and administered personality tests to 30 engineers and 70 lawyers [70 engineers and 30 lawyers], all successful in their respective fields. On the basis of this information, thumbnail descriptions of the 30 engineers and 70 lawyers [70 engineers and 30 lawyers], have been written. You will find on your forms five descriptions, chosen at random from the 100 available descriptions. For each description, please indicate your probability that the person described is an engineer, on a scale from 0 to 100.

22/03/2019 9

Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies which include home carpentry, sailing, and mathematical puzzles.

Dick is a 30-year-old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues.

22/03/2019 10

BaseRateandRepresenta'veness

Withnopersonalitysketch,simplyaskedfortheprobabilitythatanunknownindividualwasanengineer–  subjectscorrectlygavetheresponses0.7and0.3

Whenpresentedwithatotallyuninforma'vedescrip'on

–  thesubjectsgavetheprobabilitytobe0.5

Kahneman&Tverskyconcludedthatwhennospecificevidenceisgiven,priorprobabili'esareusedproperly;whenworthlessevidenceisgiven,priorprobabili'esareignored.

22/03/2019 11

If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming that you know nothing about the person’s symptoms or signs?

____%

Harvard Medical School: “2%” = 18%; “95%” = 45%

22/03/2019 12

22/03/2019 13

Whyisthecorrectanswer2%?

•  Thinkofapopula'onof10,000people.•  Wewouldexpectjust10peopleinthispopula'ontohavethe

disease(1/1000x10,000=10)•  Ifyoutesteverybodyinthepopula'onthenfalseposi'verate

meansthat,inaddi'ontothe10peoplewhodohavethedisease,another500(5%)willbewronglydiagnosedashavingit.

•  Inotherwordsonlyabout2%ofthepeoplediagnosedposi've(10/510)actuallyhavethedisease.

•  Whenpeoplegiveahighanswerlike95%theyareignoringtheverylowprobability(i.e.rarity)ofhavingthedisease.Incomparison,probabilityofafalseposi'vetestisrela'velyhigh.



22/03/2019 14

In each of the following pairs, which city has more inhabitants?

§  (a) Las Vegas (b) Miami How confident are you that your answer is correct? 50% 60% 70% 80% 90% 100%

§  (a) Sydney (b) Melbourne How confident are you that your answer is correct? 50% 60% 70% 80% 90% 100%

§  (a) Hyderabad (b) Islamabad How confident are you that your answer is correct? 50% 60% 70% 80% 90% 100%

22/03/2019 15

Overconfidenceeffect

•  Person'ssubjec'veconfidenceinhisorherjudgementsisreliablygreaterthantheobjec'veaccuracyofthosejudgements–  especiallywhenconfidenceishigh

•  Manifestedindifferentways:–  overes/ma/onofone'sactualperformance–  overplacementofone'sperformancerela'vetoothers–  overprecisioninexpressingunwarrantedcertaintyintheaccuracyofone'sbeliefs.

22/03/2019 16



22/03/2019 17

Imagine the U.S. is preparing for the outbreak of anunusual Asian disease which is expected to kill 600people. Two alterna've programs to combat thedisease have been proposed. Assume that the exactscien'fices'matesoftheconsequencesoftheprogramareasfollows:

22/03/2019 18

–  IfprogramAisadopted,200peoplewillbesaved.

–  IfprogramBisadopted,thereisa1/3probabilitythat600peoplewillbesavedanda2/3probabilitythatnopeoplewillbesaved.

22/03/2019 19



22/03/2019 20

–  IfprogramCisadopted,400peoplewilldie.

–  IfprogramDisadopted,thereisa1/3probabilitythatnobodywilldieanda2/3probabilitythat600peoplewilldie.

22/03/2019 21



22/03/2019 22





22/03/2019 23

ProgramAversusB

•  TverskyandKahneman(1981)foundthatthemajoritychoice(72%)forananalogueofproblem1usingananonymous“Asiandisease”(notSARS)wasanswerA.

•  Theprospectofsaving200liveswithcertaintywasmorepromisingthantheprobabilityofaone-in-threechanceofsaving600lives.

22/03/2019 24

AandBhavetheSameU'lity

•  Thestandardwayofcalcula'ngexpectedu'lity:•  ΣiP(oi)xU(oi)•  whereeachoiisapossibleoutcome,P(oi)isits

probabilityandU(oi)isitsu'lityorvalue.

•  Onthisbasis,op'onB(ariskyprospect)wouldbeofequalexpectedvaluetothefirstprospectA.

•  Sointhiscasesubjectsarenotsimplyusingexpectedu'lity:theyareriskaverse.

22/03/2019 25

ProgramCversusD

•  Themajorityofrespondentsinthesecondproblem(78%)chosetheriskierop'on.

•  Thecertaindeathof400peopleisapparentlylessacceptablethanthetwo-in-threechancethat600peoplewilldie.

•  Onceagain,thereisnodifferencebetweentheop'ons(CandD)instandardexpectedu'lity.

•  Wesayinthiscasethatpeopleareriskseeking.

22/03/2019 26

Framing

•  TverskyandKahneman’sinten'onwasthatProblem1andProblem2areunderlyinglyiden'cal,butpresentedindifferentways.

•  Totheextentthattheyareright,wesaythattheyaredifferentframingsofthesameproblem:thedifferentresponsesforthetwoproblemsarewhatwecallframingeffects.

•  Yetanotherwayinwhichpeople’sdecisionsdifferfromwhatwouldbepredictedonthestandardviewofexpectedu'lity.

22/03/2019 27

22/03/2019 28

ProspectTheory:S-curveofValue

Actual gain

Perceivedgain

Actual loss

PerceivedLoss

•  Small gain good,•  Big gain not much

better,•  Small loss terrible,•  Big loss not much

worse.

Thus we are generally:•  Loss averse,•  Risk averse for gains,

but•  Risk seeking for losses.

Reference point

22/03/2019 29

Es'ma'onofRisk

Estimates of Probabilities of Death From Various Causes:

Cause Stanford graduate estimates Statistical estimatesHeart Disease 0.22 0.34Cancer 0.18 0.23Other Natural Causes 0.33 0.35All Natural Causes 0.73 0.92

Accident 0.32 0.05Homicide 0.10 0.01Other Unnatural 0.11 0.02All Unnatural 0.53 0.08

(BasedonastudybyAmosTversky,ThayerWatkin’sreport)

BigandSmallProbabili'es

•  Smallprobabilityeventsareoverrated.

•  Moregenerally,behaviordepartsfromclassicalra'onalitysubstan'allyasregardsverysmallandverylargeprobabili'es.

•  Evenwhentoldthefrequencyofevents,subjectstendtoviewthedifferencebetween0and0.001(orthatbetween0.999and1)asmoresignificantthanthatbetween.5and.5001.

22/03/2019 30

The“BleakImplica'ons”Hypothesis

•  These results have“bleak implica'ons” for the ra'onalityofordinarypeople(Nisbedetal.)

•  “Itappearsthatpeoplelackthecorrectprogramsformanyimportant judgmental tasks…. We have not had theopportunity to evolve an intellect capable of dealingconceptually with uncertainty.” (Slovic, Fischhoff andLichtenstein,1976)

22/03/2019 31

The“BleakImplica'ons”Hypothesis

“I am par'cularly fond of [the Linda] example, because Iknow that the [conjunc'on] is least probable, yet a lidlehomunculus ismy head con'nues to jump up and down,shou'ngatme–“butshecan’t justbeabankteller;readthe descrip'on.” … Why do we consistently make thissimple logical error? Tversky and Kahneman argue,correctlyIthink,thatourmindsarenotbuilt(forwhateverreason) towork by the rules of probability.” (Stephen J.Gould,1992,p.469)

22/03/2019 32

ChallengefromEvolu'onaryPsychology

•  The“frequen'st”hypothesis–  Usefulinforma'onaboutprobabili'eswasavailableto

ourancestorsintheformoffrequencies•  e.g.,3ofthelast12huntsnearriversweresuccessful

–  Informa'onabouttheprobabili'esofsingleeventswasnotavailable

–  Soperhapsweevolvedamentalcapacitythatisgoodatdealingwithprobabilis'cinforma'on,butonlywhenthatinforma'onispresentedinafrequencyformat.

22/03/2019 33

MakingtheConjunc'onFallacy“Disappear”…

22/03/2019 34

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Please rank the following statements by their probability, using 1 for the most probable and 8 for the least probable.

(a) Linda is a teacher in elementary school. (b) Linda works in a bookstore and takes Yoga classes. (c) Linda is active in the feminist movement. (d) Linda is a psychiatric social worker. (e) Linda is a member of the League of Women Voters. (f) Linda is a bank teller. (g) Linda is an insurance sales person. (h) Linda is a bank teller and is active in the feminist movement.

22/03/2019 35

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

There are 100 people who fit the description above. How many of them are:

…

(f) bank tellers?

(h) bank tellers and active in the feminist movement?

...

(h) > (f) = 13%

22/03/2019 36

Level-kReasoning

Thecen'pedegame:

•  Intheonlyequilibriuminvolvingra'onalplayers,player1choosesdowninthefirst'memove

•  Whatexactlydorealpeopledo?–  Humansubjectsdon’tchoosethedownop'onrightaway–  Howdoweunderstandtheirbehaviour?

22/03/2019 37

HideandSeekGames

•  Youropponenthashiddenaprizeinoneoffourboxesarrangedinarow.

•  Theboxesaremarkedasshownbelow:A,B,A,A•  Yourgoalis,ofcourse,tofindtheprize.•  Hisgoalisthatyouwillnotfindit.•  Youareallowedtoopenonlyonebox.•  Whichboxareyougoingtoopen?

22/03/2019 38

FolkWisdom

•  “…inLakeWobegon,thecorrectanswerisusually‘c’.” –GarrisonKeillor(1997)onmul'ple-choicetests

•  CommentonpoisoningofUkrainian’spresiden'alcandidate:“Anygovernmentwan'ngtokillanopponent…wouldnottryitatamee'ngwithgovernmentofficials.” –ViktorYushchenko(2004)

22/03/2019 39

UniqueEquilibriumofHideandSeekGame

•  Ifbothplayersrandomizeuniformly:

Expectedpayoffs:Hider3/4,Seeker1/4

22/03/2019 40

BehaviouralExperimentinHideandSeek

•  CentralA(or3)ismostprevalentforbothHidersandSeekers•  CentralAisevenmoreprevalentforSeekers

–  Asaresult,Seekersdobederthaninequilibrium•  Shouldn’tHidersrealizethatSeekerswillbejustastemptedto

lookthere?•  “Thefindingthatbothchoosersandguessersselectedtheleast

salientalterna'vesuggestslidleornostrategicthinking.”

22/03/2019 41

p-BeautyContestGames(Keynes,1936)

“…professionalinvestmentmaybelikenedtothosenewspapercompe''onsinwhichthecompe'torshavetopickoutthesixprevestfacesfromahundredphotographs,

theprizebeingawardedtothecompe'torwhosechoicemostnearlycorrespondstotheaveragepreferencesofthecompe'torsasawhole….

Itisnotacaseofchoosingthose[faces]that,tothebestofone’sjudgment,arereallytheprevest,noreventhosethataverageopiniongenuinelythinkstheprevest.

22/03/2019 42

p-BeautyContestGames

•  Wehavereachedthethirddegreewherewedevoteourintelligencesto…an'cipa'ngwhataverageopinionexpectstheaverageopiniontobe.

•  Andtherearesome,Ibelieve,whoprac'cethefourth,fiIhandhigherdegrees.”

–Keynes,GeneralTheory,1936,pp.155-56

22/03/2019 43

Level-kReasoning

Onecouldplayatvaryinglevelsofdepthofreasoning:•  Level-0:Randomplay•  Level-1:BRtoRandomplay•  Level-2:BRtoLevel-1•  Nash:PlayNashEquilibrium(level-∞?)•  Worldly:BRtodistribu'onofLevel-0,Level-1andNashtypes

StahlandWilson(GEB1995)

22/03/2019 44

Level-kforHideandSeek

•  Level-k:EachroleisfilledbyLktypes:L0,L1,L2,L3,orL4(probabili'estobees'mated…) –Note:InHideandSeekthetypescycleaxerL4…

•  HightypesanchorbeliefsinanaïveL0typeandadjustswith

iteratedbestresponses: –L1bestrespondstoL0(withuniformerrors) –L2bestrespondstoL1(withuniformerrors) –LkbestrespondstoLk-1(withuniformerrors)

22/03/2019 45

Level-kforHideandSeek

•  L0HidersandSeekersaresymmetric–Favorsalientloca'onsequally1.Favor“B”:choosewithprobabilityq>1/42.Favor“endA”:choosewithprob.p/2>1/4 –Choiceprobabili'es:(p/2,q,1-p-q,p/2)

•  Note:Specifica'onofAnchoringTypeL0isthekeytomodel’sexplanatorypower –Can’tuseuniformL0(coincidewithequilibrium)…

22/03/2019 46

ModernPerspec've:DiversityinModesofThinking

•  Automa'cSystem(Type1)–  Fast,unconscious–  Parallel,associa've–  Lowenergy–  “Doer”

•  Reflec'veSystem(Type2)–  Slow,conscious–  Serial–  Expensive–  “Planner”

22/03/2019 47

TrainingtoGoBetweenModesExample:BoardMemoryinChess

hdps://www.youtube.com/watch?v=rWuJqCw|jc

22/03/2019 48

U'lisingTypeIThinking

•  frombehavioralscience•  arguesthatposi've

reinforcementandindirectsugges'onstotrytoachievenon-forcedcompliancecaninfluencemo'ves,incen'vesanddecisionmaking

•  atleastaseffec'vely–ifnotmoreeffec'vely-thandirectinstruc'on,legisla'on,orenforcement

22/03/2019 49

Applica'onExample:PersuasiveTechnology(Beeminder)

22/03/2019 50

Applica'onExample:ChangingDrivingBehaviour-LoderyIncen've

•  Enterpeopleinaloderyiftheycometocampusoff-peak•  Observethatthissmallprobabilityofearningsactuallycauses

non-essen'altravellerstoaltertheirbehaviour

22/03/2019 51

Summary

•  Muchofthecourseprogressedundertheassump'onofafairlystrictformofra'onality–  Nobodyactuallybelievesthatthisiswhatpeoplealwaysdo–  Instead,animplicitclaimisthatpeoplehavethecapacityforthiskind

ofreasoning

•  Inthislecture,welookedathowpeoplesystema'callydeviatefromsuchano'onofra'onality

•  Thishasmajorimplica'onsforcomputa'onalagents–  Intheend,youwantthemtoworkwithrealpeople–  Modelling“realpeople”ischallenging,butul'matelytheraisond’etre

22/03/2019 52

Acknowledgements

Firstsec'ononheuris'csandbiasesisadaptedfromJurafskyetal.’sSymbolicSystemscontentatStanford

22/03/2019 53

decision making - the university of edinburgh · 2019. 3. 21. · 22/03/2019 3 heuris’cs and...

Documents