trinity09 bossaerts paper

8/2/2019 Trinity09 Bossaerts Paper

1/28

WhatDecisionNeuroscienceTeachesUsAboutHuman

DecisionMakingUnderUncertainty

PeterBossaerts

CaliforniaInstituteofTechnology,Pasadena,CAand

EcolePolytechniqueFdraleLausanne,Switzerland

Abstract:

Humandecisionmakingunderuncertaintyistheoutcomeofcomplexneuro

physiologicalprocessesinvolving,amongothers,constantreevaluationofthestatisticsoftheproblemathand,balancingofthevariousemotionalaspects,andcomputationoftheveryvaluesignalsthatareatthecoreofmoderneconomicthinking.Theevidencesuggeststhatemotionsplayacrucialsupportingroleinthemathematicalcomputationsneededforreasonedchoice,ratherthaninterferingwithit,evenifemotions(andtheirmathematicalcounterparts)maynotalwaysbebalancedappropriately.Decisionneurosciencecanbeexpectedinthenearfuturetoprovideanumberofeffectivetoolsforimprovedfinancialdecisionmaking.

1.Introduction

Financestudiesdecisionmakingunderuncertainty.Foralongtime,thisstudyhasbeennormative,addressingthequestion:howshouldonechoosewhenfacedwithuncertainoutcomes?Thishasledtothedevelopmentofutilitytheory,andmorespecifically,expectedutilitytheoryandmeanvarianceanalysis.Thelatterhasproventobemosthelpfulwhenmultipleoptionscanbechosenatonetime(portfolioanalysis).

Normativedecisionanalysisstilldominatesassetpricingtheory,basedontheideathatthereisamarginalinvestor(oranaggregateinvestor)andthat

somehowshemustbedoingthingsright.Fromexperiments,however,weknowthatthisisfarfromaforegoneconclusion.Forinstance,insituationsofambiguity(someoftheoutcomeprobabilitiesareunknown),theirrationalinvestormaydeterminepricesevenifheisclearlynotthemarginalinvestor(Bossaertsetal2008a).

Attheindividuallevel,normativeanalysishaslongbeenknowntomakeinvalidpredictions.Eversince(Allais1953),choiceshavebeenconsistentlyfoundtoviolatethetwofoundationalaxiomsofexpectedutility,namely,theindependenceaxiomandthesurethingprinciple.Thishasledtotheemergenceofbehavioralfinance.


2/28

Behavioralfinancealsostartsfromchoice,butrejectstheideathatitreflectsmaximizationofarational(expected)utilityfunction.Nevertheless,ithasoneimportantfeatureincommonwithnormativefinance:choicesstillareinterpretedasiftheymaximizesomeutility.Eventhechoiceofutilityprofileissuggestiveofnostalgiatowardexpectedutility:thereareunderlyingstatesof

nature,andutilityisobtainedasthesummationoverthesestates,ofstateprobabilities(perhapsweighted)andutilitiesofstatespecificoutcomes.

Indeed,prospecttheory(Kahneman&Tversky1979)canbeviewedasanattempttocharacterizeactualhumanchoice[andrecentlyalsononhumanprimatechoice(Chenetal2006)]withinthestraightjacketofnormativetheory.Areferencepointneededtobeintroduced,probabilitiesneededtobebiasedinspecificways,andtheideaofdecreasingmarginalutilityhadtobeabandonedforthelossdomain.

Butisitreallytruethatchoiceistheoutcomeofautilitymaximizationprocess?

Takeriskseekingforlosses,forinstance.Whatifthisisabiasthatcomesaboutbecauseofthewaydecisionsaremade,andnotreallytheresultofincreasingmarginal(hedonic)utilityforoutcomesbelowthereferencepoint?

Ostensiblyinnocuousconsequencesfollowfrompracticalimplementationoftheideathatchoiceunderuncertaintyreflectsmaximizationof(prospecttheory)utility.E.g.,wealthmanagersshouldencouragetheirclientstobuystructuredproductsthatimplementshortpositionsinputoptions.

Buthowcouldweknowwherechoicereallycomesfrom?Itisherethatneurosciencecomesin.Neuroscienceallowsustostudytheentireprocessof

decisionmaking,frominitialperceptionofastimulus(whichconveysnewinformationand/ornewinvestmentoptions),tovaluationandmotivation,andtheveryactofchoosing.

Whenlookingforinspirationinneuroscience,itisimportantfromtheoutsettorealizethatneuroscienceisasubfieldofbiology,andhence,thatitusesanoverarchingprincipletounderstandneurobiologicalphenomena(analogoustoabsenceofarbitrageortheefficientmarketshypothesisofneoclassicalfinance),namely,evolutionaryfitness.Specifically,thebrainisviewedasanorganthatisoptimized,throughevolutionarypressure,todothingsrightinitsnaturalenvironment.

Thereisampleevidenceforthisinsensoryprocessinginthebrain.E.g.,auditorysignalsareoptimallymixedwithvisualsignalstogenerateaccurateguessesofthelocalizationofthesourceofthesignals(Shamsetal2005).Optimalmeans:inaccordancewithBayeslaw.

Whenchoiceunderuncertaintyisconcerned,however,thetypesofrisksthatthebrainwouldbeoptimizedformaybeverydifferentfromrisksroutinelyencounteredinthefinancialsphere.Indeed,whilemanyecologicallyrelevantrisks(e.g.,temperaturechanges)canreadilybedescribedasGaussianandpredictable,returnsgeneratedthroughorganizedfinancialmarketsare

distinctlyleptokurtoticandunpredictable.So,itmakessensetoaskwhetherthereisamismatchbetweenthebrainandfinancialmarkets.Butwecannottell


3/28

aslongaswedontknowhowdecisionscomeaboutinthebrain.Thatisoneofthetasksofanareaofneurosciencecalleddecisionneuroscience.

Whenexploringdecisionneuroscience,oneisimmediatelyconfrontedwithaninspiringdifferentiation.Indecisiontheory,allrisksarethesamethereisno

distinctionbysource.Yetthebrainevidentlydistinguishesbetween(atleast)twotypes:uncertaintygeneratedbyanunintentionalsource(nature;arandomnumbergenerator;afinancialmarketwithoutinsiders),andthatgeneratedbyanintentionalsource(astrategicopponent;afinancialmarketwithinsiders).Thedifferentiationmanifestsitselfintheengagementofdifferentfunctionalregionsofthebrain,thebehavioraleffectswhenlesionsselectivelyaffecttheseregions,anddifferencesacrosshumansinattitudestowardseithersource.Still,theseparationisnotextreme;commonregionsdogetinvolved,asweshallseelater.Wewillmostlydealherewithuncertaintyfromanunintentionalsource,leavingintentionaluncertaintyforalatersection.

Decisionneurosciencehasprogresseddramaticallyoverthelasttenyears,followingthediscoveryoftheroleofdopamineneuronsinpredictionunderuncertainty(Schultzetal1997).Thespeedofdiscoveryhasbeengrowingoverthelastfiveyears,sothatsurveyingtheaccomplishmentsbecomeslikedescribingamovingtarget.Bythetimethisreviewispublished,someclaimsmadeherewillbeoutdated,andquestionsraisedmayhavebeenanswered.

Thepickupinspeedisinlargepartduetotheintroductionofconcepts,theoriesandresultsfromeconomictheoryandexperimentaleconomics.Hencesomewouldrefertodecisionneuroscienceasneuroeconomics,or,whenitconcernsriskanduncertainty,neurofinance.Iwouldliketorefrainfromusingthese

termshere,primarilytoemphasizethatneurosciencehadmuchtosayaboutdecisionmakingevenbeforetheadventofneuroeconomics.

Onthematterofneuroeconomics,anumberofreviewshaverecentlyappeared.Alltoooftenthesereviewswerenotreallysurveysbutcriticalassessmentsfrombothproponentsandopponentsofthefieldbasedononlyafewarticleswithadistincteconomicsangle,effectivelybypassingthevastandsophisticatedliteratureindecisionneuroscience.Thegoalinthisreview,incontrast,isnotonlytolettheeconomisttalk,butalsothedecisionneuroscientist.

2.ExperimentalFinance

Virtuallyallempiricalworkinfinanceuseshistoricaldatafromthefield,i.e.,dataastheyaregeneratedbynaturallyoccurring(financial)markets.Thisisincontrastwithdecisionneuroscience,wherecontrolledexperimentationisthenorm.Whilethissometimesinvolvesfieldexperiments(Coates&Herbert2008,Lo&Repin2002),itmostlymeanslaboratoryexperiments.Iffinanceistobeenlightenedbydecisionneuroscience,itneedstoappreciatethemeritsofcontrolledexperimentation.Thisisfarfromaforegoneconclusion,soitispertinenttoelaborate.


4/28

Theuseofexperimentsisnotonlydictatedbypracticalconsiderations(itwouldbehardtohaveatraderperformhisorherdaytodaydutieswhileinanfMRIscanner),butforemostbecauseofthenecessitytocontrolparameters.Indeed,naturerarelycaresaboutourinterests.WemaywanttostudythebasicprinciplesunderlyingCAPM(Bossaerts&Plott2004),yetnaturemayhave

decidedtoprovideuswithanenvironment(multiperiodsecuritieswithskewedpayoffs,delegatedportfoliomanagement,)leastpropitiousforCAPMtoemerge.Togeneratefinancialuncertaintycausedbyanintentionalsource,onemaywanttocontrolthenumberofinsiders(Bruguieretal2008),yetthenumberofinsidersisgenerallyunknowninfieldmarkets,eveniftheirpresencemaybesuspectedattimes[andexploited:see(Bossaerts&Hillion1991)].

Lackofexternalvalidityisperhapsthemostimportantreservationonecanhaveaboutlaboratoryexperimentation.Inexperimentalfinance,thismostlyconcernsthesizeofpaymentsandrisks,whicharegenerallythoughttobeinadequatefortheresultstohaveanyrelevancefortherealworld(asifalaboratoryisnot

real,butthatisnotthepoint).

Interestingly,decisionneurosciencegivesanindicationofhowtothinkaboutthisissue.Indeed,oneofthekeybrainregionsthattracksrewardpredictionerrorshasrecentlybeendemonstratedtoencoderewardsadaptively:whenthehighrewardofabinarygambleisdelivered,firingofdopamineneuronsinthebrainstemisindependentofthesizeofthereward;evenwhenincreasingtherewardtenfold,activationremainsthesame.Converselyfordeliveryofthelowrewardofsuchgambles.SeeFig.1.Fromthepointofviewoftherewardpredictionsysteminthebrain,workingwithsmallgamblesorwithlargegamblesisthesame;everythingisjustscaled.

Thereisofcourseasimpleexplanationforadaptiveencoding(orscaling):thereisaphysicalconstraintontheamountoffiringagiventypeofneuroncanachieve.Sincetherangeofstimulithattheneuronsarepossiblyconfrontedwithisfarbeyondthatoftheirfiring,thebrainhasoptedforscaling.

Adaptiveencodingofrewardscanalsobeobservedelsewhere,suchasintheorbitofrontalcortex(theregionrightabovetheeyes),whichhasbeenshowntoencodeutility(moreonthatlater).Whilethiswasfirstdemonstratedforthemonkeybrain(Tremblay&Schultz1999),thishasrecentlybeenconfirmedforthehumanbrain(Elliottetal2008).

Adaptiveencodingmaynotbesufficienttoestablishexternalvalidityoflaboratoryexperiments.Indeed,whileadaptiveencodinghasbeenobservedfortherangesofoutcomesandrisksinthelaboratory,thereisnoguaranteethatitextrapolateswithoutlimit.Specifically,beyondacertainrange,brainregionsmaybecomeinvolvedthataretypicallynotactivatedinthelaboratory.

Butthebeliefthatdecisionmakingunderuncertaintyintherealworldisfundamentallydifferentrequiresproofthatthebrainregionsthatareactivatedinthelaboratorybecomedisengaged.Thisishighlyunlikely,becausetheregionsthatareactiveinsidethelaboratoryhappentobepowerfulandpervasive,aswe

shallsee,involvingcrucialneurotransmitters(suchasdopamine)andtriggering


5/28

strongemotionalreactions(arousal,fear,)thatwouldneedtobeoffsetorneutralizedsomehowforthemtobecomeirrelevant.

Infact,alltheevidencepointstothecontrary.Decisionmakingbyprofessionaltradershasbeenshowntobehighlyemotional(Lo&Repin2002),andit

actuateshormonessuchastestosteroneandcortisol(Coates&Herbert2008)thathaveadirectinfluenceontheverybrainregionsweseeatworkinthelaboratory.Likewise,theallegedwidespreaduseofpharmacologicalenhancerssuchascocainesuggestsattemptsbyprofessionaltraderstodirectlyimpactthedopamineneuronsreferredtoabove.Indeed,cocaineisknowntoaffectthesynapticstrengthofdopamineneurons,causingprojectionneuronstobecomehypersensitive,specificallytopositiverewardpredictionerrors(Saaletal2003).

Manyfinancialdecisionproblems,suchasretirementportfolioallocation,haveasignificanttimecomponent.Itisaseriouschallengetoemulatesuchproblemsinthelab.Thetimehorizoninrecentexperimentshasbeenextendedtoseveral

weeks(Kable&Glimcher2007,McClureetal2004),butnotyears.Nevertheless,cleverchoiceofexperimentaldesigndoesallowexperimenterstoexposesubjectstomultiyearhorizons,evenifonlyindirectly.Forinstance,whensubjectsarefacedwithnutritiousfoodandtastyfood,choiceeffectivelyimpliesasubstantialtimedimension,becauseofconsequencesforhealthfarintothefuture(Hareetal2008a).

Ofcourse,onecouldalwaysresorttothoughtexperiments,byaskingsubjectsforhypotheticalchoiceinimaginedsituations[asintheoriginalexperimentsin(Kahneman&Tversky1979)].Buttheproblemisthatthesethoughtexperimentsmaynotengagethesamebrainprocessesaswhenexperiencingthe

outcomes.Emotions,forinstance,areanintegralpartofreasoneddecisionmakingunderuncertainty,asweshallsee.Ifthehypotheticalquestionscannotinducethesameemotionalreactionaswhentherisksathandareexperiencedfirsthand,predictedeffectsmaynotemerge.Thiscouldexplainwhyexperimentaleconomistshavenoticedthattheyobtainbetterresultswhentheypayforperformance.Fielddataalsotellusthatoutcomesmayneedtobeexperiencedbeforetheysignificantlyinfluencechoice(Feng&Seasholes2005).

3.PerceptionOfRiskAndReward

Oneofthemostfascinatingfindingsofthelasttwoorthreeyearsisthatthebrainseparatelyencodestheriskandtherewardwhenconfrontedwitha

gamble.Thissuggeststhatthebrainengagesinmeanvarianceanalysis,wherebygamblesarerepresentedintermsoftheirstatisticalproperties(moments)ratherthanasarandomvariablelivingonanabstractstatespace.Thesuggestionisallthemorecrediblebecausethemetricofriskencodedinthebrainappearstobevariance.(Ongoingworkextendsthistohighermoments,suchasskewnessandkurtosis.)

Howdoweknowthis?Fromexperimentslikethefollowing.Subjectsarepresentedwithadeckoftenrandomlyshuffledcards,numbered1to10.Twocardswillbedrawnfromthedeck,withoutreplacement.Subjectsareaskedtobetwhetherthesecondcardisgoingtobelowerthanthefirst.Oncethesubjectpostsherbet(for$1),thetwocardsaredisplayed,oneaftertheother.Weareinterestedinbrainactivationwhenthecardsarerevealed,becausethatiswhen


6/28

thebrainshouldencodethecorrespondingchangesinexpectedrewardandrisk(Fig.2A).

Noticethatthisisaverysimplegame,involvingcards,whichmostsubjectsshouldbeintimatelyfamiliarwith.Probabilitiesarenotmentionedexplicitly,to

shortcutthemathematicalregionofthebrainthatwouldbesummonedtotranslatetheabstractnumericaldataintosomethingtherewardandriskprocessingregionscandealwith.Indeed,explicitrepresentationofprobabilitieswouldaddalayerofcomplexity,whichwouldbeinteresting,butwhichwemaynotbereadytoventureintoyet.

Also,noticethatsubjectsreallydonothavetomakeanydecisionatthepointsintimethatinterestus(revelationofthetwocards).Again,thisisdeliberatelydone.Thereisalongpathinthebrainfromperceptiontochoice,andweareatfirstinterestedinperception.Doesthatmeanthattheresultingactivationsareirrelevantforchoice?No:bynowitisknownthattheseactivationspredict

choice(Christopoulosetal2008,Kuhnen&Knutson2005).Asamatteroffact,activationsinapurelyperceptualtasklikeourshavebeenfoundtobetterpredictchoicethan(past)choiceitself,especiallywhenoneisclosetoindifference(Bernsetal2008).Thatis,perceptualneuralsignalsofvaluationparametersprovidebetterforecaststhanpreferencesrevealedthrough(past)choice.Thisisnottosaythattheperceptualactivationsdeterminechoice.Insituationsoffreechoice,additionalengagementisneeded,bothsubcortical(O'Dohertyetal2004)andcortical(Hamptonetal2006).

Inourcardgame,expectedreward(mathematicalexpectationofpayoff)andrewardrisk(measuredasvariance)changeeachintheirownwayasafunction

ofprobabilityofreward.Specifically,expectedrewardincreaseslinearlyandriskdisplaysasymmetric,invertedUshapedpattern,withpeakatprobability0.5(Fig.2B).Ifexpectedrewardandriskareencodedsomewhereinthebrain,activationoughttoexhibitthesamepatterns.

Fig.3AdisplaysacrosssectionofthehumanbrainhighlightingareasthatwereidentifiedbecauseactivationinafMRI(functionalmagneticresonanceimaging)scannerincreasedlinearlyinrewardprobability.Rewardprobabilitywascomputedfromthebetthatthesubjectchoseandthenumberonthefirstcard.Activationismeasuredinthefirstsecondafterthedisplayofcard1(modulothehemodynamicresponsedelay).Fig.3Bshowsmeanactivationsacross19

subjectsand95%confidenceintervals,stratifiedbylevelofrewardprobability.Thisverifiesthattheactivationpatternsareindeedincreasingandlinear,whichisrequirediftheactivationsaretobeassociatedwithexpectedreward.

Withtheexceptionofasmallactivationintheupperleftpartofthebrain,theregionshighlightedinFig.3Aaresubcorticalprojectionareasofthedopamineneuronsinthebrainstem,inparticular,theventralstriatum.Wealreadydiscussedtheimportanceofthedopamineneuronsintrackingrewards.Traditionally,however,theywerethoughttocausearousal.Thelinkbetweenthisemotionalreactionandpredictionerrorsisintriguing,andweshallelaboratelater.Fig.3Csituatestherewardtrackingdopamineneuronsintheventraltegmentalareaofthebrainstem,andtheyprojecttotheventralstriatum.Therearealsodopamineneuronsinthesubstantianigra,buttheseareprimarily


7/28

engagedinothertasks(anddamagetotheseneuronsleadstoParkinsonsdisease).

TheresultsofFig.3havebeenreplicatedmanytimes.Infact,wenowknowthatthisactivationreflectsrewardpredictionerrors,i.e.,thedifferencebetweenwhat

wasexpectedbeforethedisplayofthestimulus(card1)andtheupdatedexpectationoroutcome(McClureetal2003,O'Dohertyetal2003).Assuch,theactivationisbackwardlooking,and,consistentwiththis,itisphasic(meaning:ofshortduration)andtimelockedtothestimulusdisplay.Thattheactivationsrepresentrewardpredictionerrorshasrecentlybeenconfirmedinananalysisthatrequiredminimalassumptionsaboutthepriorexpectation(Caplinetal2008,Nivetal2008).

Fig.4Ashowsacrosssectionofthehumanbrainwithregionswhereactivationchangesquadraticallywithrewardprobability.Fig.4Bdisplaysthepatternofactivationinoneofthoseregions,namely,rightinsula.Asrequiredifthis

activationistoreflectrewardvariance,meanactivationislowestforzeroandunitrewardprobability,highestatabout0.5,andsymmetricaround0.5.TheregionsindicatedinFig.4A,leftandrightinsula,andthethalamus,areinvolvedinrelayinginformationaboutbodilystateemotionsandsensoryinput(e.g.,visualcues)tocorticalareas.Theresultshavebeenreplicatedmanytimes[e.g.,(Dreheretal2005)].Activationinatleasttwoadditionalcorticalregionscorrelateswithrisk(Huetteletal2005).Oneofthem,inferiorfrontalgyrus,willbebrieflymentionedlater:itsshallowlocationallowsfordirectelectricalormagneticmanipulation,withinterestingbehavioraleffects.

ThetimingandmodulationofthesignalsdisplayedinFig.4aresignificantly

differentfromthoseinFig3.Specifically,thesignalsemergelater,aresustained,andseemtobetimelockedtothedisplayofcard2.Thisisconsistentwiththeinterpretationthattheyreflectexpectationofriskthatistoberesolveduponrevelationofcard2.Assuch,itisforwardlooking,andthereforenotapredictionerror,butanticipation.Asweshallsee,thebraindoesalsoencodeacorrespondingpredictionerror,whichinthiscaseshouldbereferredtoastheriskpredictionerror.

Inourcardtask,probabilitiesareknown,inthatsubjectsshouldbefamiliarwithbasicfeaturesofcardgames.Thecontrastofactivationwhenprobabilitiesareknown(whicheconomistswouldrefertoaspurerisk)againstwhenthese

arenotknown(ambiguity)showsthatthedopamineprojectionareassuchasventralstriatumarelessactivatedunderambiguity(Hsuetal2005).Interestingly,anotheractivationemerges,namely,inamygdala(locatedapproximatelywherethenumber3isdisplayedinFig.3C,butmorelateral).

Amygdalaformspartofacircuit(whichalsoincludesthedopaminesystem)thathasbeenassociatedwithgoaldirectedlearning(Balleineetal2003).Amygdalaappearstosignalaneedtostartlearning.Econometricianswouldrefertothissignalasparameterestimationuncertainty(oritsinverse,precision),tobedistinguishedfromtheirreducibleriskthatremainsinanystochasticenvironmentevenwhenprobabilitiesareperfectlyknown.Itisinterestingtoobservethatthebrainlikewisedifferentiatesbetweenthesetwoaspectsofuncertainty.Itconfirmsbehavioralevidencethathumanstreatpureriskand


8/28

ambiguitydifferently,butthelinkwithfundamentalconceptsfromeconometricssuggeststhatthedifferentiationisnotnecessarilytobeinterpretednegatively,incontrastwithtradition(Ellsberg1961).Indeed,arecentexperimentdemonstratesthathumanmanagetoexploitthedifferentiationtooptimizelearningunderambiguity(Payzan&Bossaerts2008).

Amygdalahasbeenassociatedwithlearningonlyrecently.Amygdalahadalwaysbeenviewedasthefearcenterofthebrain(Adolphsetal1995,LeDouxetal1990,Morrisetal1996).Thetwoviewsarenotmutuallyexclusive.Fearmaybetheemotionalexpressionofestimationuncertainty,justlikearousalaccompaniespositiverewardpredictionerrors(andthelattertwoarebothrelatedtoactivationofthedopaminesystem).

Asmentionedbefore,thedopaminesystemencodesarewardpredictionerror,andtherefore,playsacrucialroleinlearningexpectedrewards.Signalscorrelatingwithriskpredictionerrorinourcardgamehavealsobeenlocated,in

insula(Preuschoffetal2008),overlappingwiththeareaswhereriskpredictionhadbeenfoundinpriorstudies.SeeFig.5.Mathematically,theriskpredictionerrorissimplythedifferencebetweenthesquaredrewardpredictionerroranditsanticipation,rewardvariance.EconometricianswouldrecognizethisasthedrivingtermofanARCHprocess(Engle2002).Riskpredictionerrorscanalsobeobservedintheothercorticalstructureswhererisksignalshavebeenfound(d'Acremontetal2008).

Oneofthestrikingaspectsaboutbrainactivationinourcardgameisthatsignalsofexpectedrewardandrisk(andcorrespondingerrors)appeareverytimethegameisplayed.Evidently,thebraininsistsonreevaluationofbasicstatistical

featuresratherthansimplyretrievingsomevaluation(utility)numberfrommemory.Ifthisisageneralproperty,thebrainisbestviewedasacomputerwithincredibleprocessingcapacitybutasmallharddisk.Thisshouldinspirenewmodelingofboundedrationality.

Butconstantreevaluationofthestatisticsofagambletakestime.Onethereforewonderswhetherthishasaneffectonchoicewhensubjectsareforcedtomakedecisionsbeforeallneuralcomputationscanbecompleted(whichwouldbewithin23secondsafterstimulusdisplay).Revealedpreferencesareindeedsignificantlydifferentwhensubjectshavetoindicatetheirchoiceinthe2ndsecond,comparedtochoiceinthe4thor6thsecond(Bollardetal2008).This

obviouslyhasimportantimplicationsfordesignofcontinuoustradingmechanisms,whereprofitopportunitiescansometimesonlybeachievedifactionistakeninasplitsecond.

Sofar,wehavediscussedencodingofstatisticalfeaturesofagambleinthehumanbrain.Infinancialdecisiontheory,thesearemerelytheinputstovaluation,i.e.,toexpectedutility.Where,then,arethevaluesignalsinthebrain?

PerhapstheactivationdisplayedinFig.3actuallydoesnotreflectexpectedrewardprediction(errors),butexpectedutility(errors).Wedontseeanynonlinearity,however(concavityforriskaverseandconvexityforriskseeking

subjects).Thenagain,oneshouldnotoverinterprettheevidence,asitisnotentirelyclearwhetherthefMRIsignalislinearinfiringofindividualneurons.As


9/28

amatteroffact,evenpatternsinneuronalfiringshouldbeinterpretedwithcare.Indeed,sincesignalprecisiondecreaseswithfiringrate,onecouldexpecttoobserveaconvexrelationshipbetweenrewardprobabilityandfiring,eveniftheneuronathandmerelyencodesexpectedreward.

Also,inourcardgame,onlyprobabilitieswerevaried.Significantly,wekept(reward)magnitudethesame:betswerealwaysfor$1.Onerecentstudydidmanipulatemagnitudes.Thisrevealedstrictconcavityintherewardpredictionerrorsignalsinventralstriatum.Moreover,thedegreeofconcavityincreasedwithriskaversion(Tometal2007).

Butifthismeansthatthesignalsinventralstriatumreflectexpectedutility,andnotexpectedreward,whataretherisksignalsininsulaorinferiorfrontalgyrusabout?Dothesereflectvarianceofutility?Atthismoment,wedontknow.Preliminaryevidence(Christopoulosetal2008)suggeststhattheriskinducedactivationsininsulaandinferiorfrontalgyruscorrelatewithriskaversion,and

hence,doreflectsubjectivevariance.Whiletheideaofseparatelyencodingutilityvarianceandexpectedutilitymayseemstrangeatfirst,itshouldberecalledthatitwasAllaiswhofirstsuggesteditasanexplanationfortheviolationsofexpectedutilityaxiomsthatheobservedinhumanchoice(Allais1953).

Tocomplicatematters,inventralstriatum,signalsencodingprobabilityofrewardandmagnitudeofrewardappeartolivealongsidesignalsthatcorrelatewithexpectedreward(Tobleretal2007,Yacubianetal2006).Probabilityandmagnitudeareofcoursethecrucialingredientsofexpectedutility.Theprobabilitysignalinventralstriatumevenshowsbiasesthatarereminiscentof

probabilityweightinginprospecttheory(Bernsetal2008,Hsuetal2008b).

Whyhasthebrainchosentoencodeallfourfeaturesofagamble:expectedreward,rewardvariance,rewardprobabilityandrewardmagnitude?Afterall,(expected)utilitycanbecomputedeitherbysummingmagnitudesaftermultiplicationwithprobabilities,orbycombiningexpectedrewardandrisk.Perhapsthesetwoapproachesarenotequivalent,incontrastwithreceivedwisdom(Krolletal1984)?Potentialfundamentaldifferenceshaverecentlybeenexploredboththeoreticallyandempirically(d'Acremont&Bossaerts2008).

Separateencodingofthekeydecisiontheoreticfeaturesofaproblemisnot

limitedtogambles.Thehumanbrainlikewiseseparatelytracksequityandefficiencyinproblemsofmoraljudgment(Hsuetal2008a).Interestingly,equitycorrelateswithactivationinregionsengagedinriskencoding,suchasinsula,whileefficiencyactivatesventralstriatum,involvedinexpectedrewardencoding.Thisfindingisunlikelytobecoincidental.Itcertainlylendssupporttomodelsofmoraldecisionmakingbasedonatradeoffofequityagainstefficiency,justliketheevidenceinFigs.3and4supportsmeanvarianceanalysisasamodelofhumanfinancialvaluation.

4.Emotions

Theregionsofthehumanbrainwhereweseeclearriskandrewardsignals

includethedopaminesystem,insula,andamygdala.Theseregionshaveincommononeimportantfeatureofthehumancondition,namely,emotions.We


10/28

alreadymentionedarousalinthecontextofdopamine(sometimesinflatedartificially,throughcocaineforinstance).Amygdalahasbeenassociatedwithfear,andinsula(oratleasttheanteriorpart,whereweobserverisksignals),isknowntobeacrucialrelayofemotionstothecortex.

Atthesametime,theevidencesurveyedintheprevioussectionsuggeststhatactivationinthesesamebrainregionsreflectsencodingofcoolmathematicalquantitiessuchasexpectedreward,risk(measuredasvariance),rewardpredictionerrors,ARCHlikeriskpredictionerrors,estimationuncertainty,etc.Hasthebraindecidedtorelatethesemathematicalquantities,usuallyassociatedwithcoolheadedness,toemotions?

Inthesamevein,recentstudieshaveshownthatstrongemotionssuchasdisappointment,elation,regret,relief,envy,gloatingareassociatedwithmathematicalerrorsignalssuchas(inthesameorder):(negativeandpositive)rewardpredictionerrors,fictiveplaypredictionerrors,andpredictionerrors

relativetootherpeoplesrewards(Baultetal2008,Coricellietal2005).Allthesecanbefoundinthebrainstructuresthatwealreadydiscussed.

Inthecontextofinsula,weshouldmentionastudywheretheabilitytosenseonesownheartbeatwasfoundtocorrelatewiththesizeoftheinsula(Critchleyetal2004).Togetherwiththefindingsthat(i)theheartbeatofprofessionaltraderscorrelateswithrisk(Lo&Repin2002)andthat(ii)insulaencodesrisk(Fig.4),onemayconjecturethatthehumanbodyhaschosentoemployasomaticmarker(heartbeat)toencoderisk.

Whenthatsomaticmarkerisabsent,theconsequencesmaybeharmful.

(Becharaetal1997)showthatsubjectswhofailtoexpressemotionalanticipationwhentakingriskydecisions(becauseofspecificbrainlesions)consistentlygoforinferiorfinancialchoices.Thisledtheauthorstopositthatemotionalprocessesguidereasoneddecisionmaking,whichtheyreferredtoastheSomaticMarkerHypothesis(Bechara&Damasio2005).

Becauseofthemathematicalaccuracyofthesignalsencodedintheemotionalregionsofthehumanbrain,anupdatedinterpretationofthefindingswouldbethatemotionsaresomehowsubsumedinthemathematicsofoptimaldecisionmaking.Thisofcoursewouldexplainwhy(emotional)experienceisimportantforrationalchoice(Feng&Seasholes2005).

Anotherwaytoputthisisthatreasonedchoiceistheresultofthebalancingofemotionalinputs.Humandecisionmakingunderuncertaintythustakesonfeaturesofthecelebratedmultiattributechoicetheory(Lancaster1966),albeitthatcharacteristicsneedtobereplacedbytheemotionalaspectsofagamble.Emotionsarenotbadperse,andirrationalregretavoidance,ambiguityaversion,etc.aretobeviewedastheresultofmaladaptedweighingofthedifferentemotionalaspectsofagamble.

Itisimportanttorealizethatemotionbaseddecisionmakingmaybeentirelysubconscious.Asamatteroffact,itsspeedmayprecludeconsciousbrain

processesfrominterfering.Attimes,suchdecisionmakingmaybeinconflictwithchoicebasedonexplicitreasoning,andthefactthathumansoftenshow


11/28

signsofbothfast,emotionbaseddecisionmakingaswellasslow,logicallyreasonedchoicehasledmanytoproposedual-systemtheory(Evans2003,Kahneman&Frederick2002).Accordingtothistheory,humansarecapableofopportunisticallyswitchingbackandforthbetweenthesetwowaysofgeneratingdecisions.Prefrontalandlateralorbitofrontalcorticalregions,aswell

astheadjacentanteriorcingulatecortexarethoughttobeinvolvedinexplicit,consciousdecisionmaking,butthepreciseneurobiologyremainspoorlyunderstood.

Arecentstudyonframing(DeMartinoetal2006)doesillustratehowthismaywork.Whenprospectswereframedintermsoflosses,subjectsappearedtotakeonmorerisk,andthisbehaviorwasassociatedwithincreasedactivationofamygdala(thefearcenterofthehumanbrain,asmentionedbefore).Nevertheless,subjectswhoshowedleast(orno)impactfromframingalsosimultaneouslydisplayedincreasedprefrontalactivation,suggestingattemptsoftheconscioussystemtocontrolquickemotionalreactions(Bossaertsetal

2008b,Kahneman&Frederick2007).

Howdoweknowthatthesecondaryactivationisreallyconscious?Wedont(yet).However,becauseitallowsforbettertimeresolution,EEGmayprovideinsight.Inarecentstudyontheimpactofdisappointmentonsubsequentdecisionmaking,asecondaryEEGsignalemergedinfrontalregionsafter375millisecondsbutonlyinindividualswhosuccessfullyovercamethedetrimentaleffectofreflexivereactiontodisappointment(Tzieropoulosetal2008).Whilethearrivalofthissecondarysignalissurprisinglyquick,itisnotoofastforconsciousexperience.

Itshouldberepeatedthatemotionbaseddecisionmakingneednotbebadorunsophisticated.ThemathematicalprecisionwithwhichactivationininsulacorrelateswithanARCHlikeriskpredictionerror(Fig.5)underscoresthis.Asdotherewardpredictionerrorsthatdopamineneuronsencode.Indeed,thelatterrevealalearningalgorithmthatisexquisitelyadaptedtoenvironmentswithmultiplestimuli,actionpossibilities,andrewardopportunities(Montagueetal1996,Schultzetal1997).Thisverysamealgorithm,calledtemporaldifference(TD)learning,isusedinmachinelearningtosolvecomplexdynamicprogrammingproblems.

5.ValueSignals

Standardeconomicthinking(includingbehavioralfinance)startsfromthepremisethatchoiceresultsfrommaximizationofvalue.Partlyundertheinfluenceofeconomists,oneofthemainissuesthatdecisionneuroscientistshavebeeninterestediniswhethertherearevaluesignalsinthebrain.Thefirsthandbookinneuroeconomicsismostlydevotedtothisissue(Glimcheretal2009).

Atonelevel,theexistenceofavaluesignalislikelytobeconfirmed.Indeed,ifactsfollowtheinstructionsofmotororpremotorneuronsthatfiremostintensely,thisfiringmustbeinaccordancewithmaximizationofsomeutilityfunctioniftheresultingactssatisfytheaxiomsofrevealedpreference.Consistentwiththisconjecture,(Platt&Glimcher1999)foundthatfiringofneuronsinthe


12/28

intraparietalsulcus(afoldinthelateralcortexbeyondthebrainmidline)oftheMacaquebraincorrelatedwithexpectedutility.

Butwhatabouttheexecutiveregionsinthefrontofthebrain,farremovedfromthemotorcortex?Bynow,theevidenceinfavorofsignalsthateconomistswould

recognizeasvalueiswellestablished.ValuationsignalshaveindeedbeendiscoveredintheorbitofrontalcortexoftheMacaquemonkey(PadoaSchioppa&Assad2006,Tremblay&Schultz1999).Thishasrecentlybeenconfirmedforthehumanbrain(Plassmannetal2007),andextended(Hareetal2008b):activationinmedialprefrontalcortexcorrelateswithwillingnesstopay,andmorelateralactivation(andcorrespondingtotheactivationidentifiedintheMacaquebrain)correlateswitheconomicsurplus(i.e.,utilitynetofcost),whileactivationinventralstriatumrepresentsvaluepredictionerrors(asalreadydiscussed).

Inthecontextofgambles,thesignalsinlateralorbitofrontalcortexexhibitthe

characteristicsonewouldexpectiftheyaretoconveyexpectedutility:theyincreasewithexpectedreward,yetdecreasewithrisk(variance),forriskaversesubjects,whiletheyincreasewithriskforriskseekingsubjects(Fig.6).

Thepresenceofvaluesignalsishandyinmanyrespects.Foronething,evolutionofthissignalovertimeteachesusmanythingsaboutlearning.E.g.,(Hamptonetal2006)havebeenabletoshowthatvaluesignalsinarandomlyrevertingtwoarmedbandittaskreveallearningthatisdistinctlyBayesian.ItistruethatthebehavioraldataalreadymarginallyfavoredBayesianupdating(oversimplereinforcementlearning),butthevaluesignalsshowtherightpropertiesattherighttimesothateventhemostskepticalmindwouldbeconvincedofthe

hypothesisofBayesianlearning.

Valuesignalshavealsoshedlightontheendowmenteffect.Specifically,thiscognitivebiasisnotdirectlyreflectedintheabovevaluesignals,butmaybetheresultofreferencebasedencodinginventralstriatumandinsula(DeMartinoetal2009).Inthecontextofgambles,activationininsulasuggeststhattheendowmenteffectistheconsequenceofthelevelofriskthatthesubjectperceives.Specifically,thewillingnesstopay(WTP)islowerbecausesheperceiveslessriskinherstartingposition(cashendowment).Thewillingnesstoask(WTA)ishigher,becausesheisnowendowedwiththegambleandisfacedwithexchangingthisforanotherriskyposition.Thelattermayseemstrange,

becausethegambleistobesoldforcash(i.e.,ariskfreeposition).ButthisignoresthefactthatWTPisgenerallyelicitedusingtheBeckerDeGrootMarshak(BDM)mechanism,whichinducesrisk.Thefindingsuggestsnovelhypotheses,suchasthedependenceofthedifferencebetweenWTPandWTAonriskaversion.ItalsodemonstratesthattheendowmenteffectinthiscaseresultsfromafundamentalmisconceptionoftheBDMmechanism(oraviolationoftheindependenceaxiom).Hence,itshouldbeavoidablethroughappropriatetraining.Whichispreciselywhathasbeenshownrecently(Plott&Zeiler2005).


13/28

6.UncertaintyCreatedByAnIntentionalSource

Sofar,wehaveconsidereduncertaintythatisobjective,inthesensethatitisgeneratedbyasourcewithoutgoaldirectnessorintentionbehindit.Opposedtothisaresituationswhereastrategicagentcausestheuncertainty(ananimal;ahuman;asophisticatedcomputer;orperhapsamarketwithinsiders?).Game

theoryhaslonghadtocopewiththistypeofuncertainty,buthassettledonanalysesthatdonotdistinguishitfromobjectiveuncertainty.Valuationisperformedusingstandardexpectedutility,andupdatingfollowsBayesianprinciples.TheresultingequilibriumconceptsarestraightforwardBayesianextensionsofNashequilibrium.

Butneurosciencehassuggestedthattherearefundamentaldifferencesinthewayhumansapproachuncertaintyfromanintentionalchoice.Forinstance,activationinasimplegameofimperfectinformationismorelikethatinsituationsofambiguity,notpurerisk(Hsuetal2005).Likewise,psychologyhaslongdifferentiateddealingwithanintentionalopponent.Assessingtheriskin

suchasituationhasbecomeknownasTheoryofMindormentalizingandspecificregionsofthebrainseemtobededicatedtoit,mostimportantlyparacingulatecortex(theregionofthefrontalcortexabovetheorbitofrontalcortex),cytoarchitecturallyarelativelyrecentbrainregion(Gallagher&Frith2003).

Paracingulatecortexisindeedpreferentiallyactivatedduringplayofstrategicgameswithotherhumans,moresothanwithcomputersprogrammedwithasimplerule(Gallagheretal2002,McCabeetal2001).(Hamptonetal2008)haveexploredthemathematicalcomputationsinthisregionduringgameplayanddiscoveredthatitencodedaninfluenceupdate,namely,theerrorfrompredictinghowonesopponentwouldshiftstrategyasaresultofonesownmoves.

Assuch,thissignaleffectivelypusheshumangameplaybeyondsimplereinforcementlearning,fictitiousplayorsomemixture[experienceweightedattractionlearning(Camerer&Ho1999)].Mostsignificantly,itdemonstratesthathumansrecognizetheintentionalityofthetarget(opponent)theyarelearningabout,apossibilitythathasbeenraisedonlyoccasionallyinthegametheorylearningliterature(Camereretal2002,Stahl2000).

Exploitingthesimpleformalstructureofthebeautycontest,thisfindinghas

recentlybeenconfirmed(Coricelli&Nagel2008).Significantly,inneitherstudy,regionstraditionallyassociatedwithformalmathematicalandlogicalthinkingwereengaged,andindeed,(Coricelli&Nagel2008)showhowskillinthebeautycontestandmathematicalproficiencywereunrelated,despitethefactthatthemathematicstestsrequiredtheverycalculationsthatareimplicitinsuccessfulgameplay.Thisdiscoverystillobtainswhenthegameisfarmorecomplex.Infinancialmarketswithinsiders,skillinpredictingfuturepricemovementsdoesnotcorrelatewithmathematicalproficiency(Bruguieretal2008).


14/28

7.Discussion

Whatshouldafinancescholarmakeofallthisneurobiology?Inthefirstplace,thatitprovidesplausibilityanddisciplinetomodelingofhumandecisionmaking.Forinstance,itsuggestswhereoneshouldplaceemotions.Andconsciouscontrol.Itteachesusthatvaluesarenotmerelyretrievedfrom

memory,butrecomputedeverytime,withtheriskofoccasionalmisestimation.Thatis,choiceistheoutcomeofaprocess,fromperceptiontoaction,notmaximizationofutility.Likewise,itsuggestswhyhumanscanbegoodat,say,playingstrategicgamesevenif,whenaskedexplicitly,theydonotseemtobegoodatthemathematicalcomputationsimplicitintheirplay.

Neurobiologydoesaskustoevaluateeverythingintermsofasinglecriterion,namely,evolutionaryplausibility.Allegedcognitivebiasesremainsuspectuntiltheymakesenseinanecologicallyrelevantsetting.Suchacriterionshouldbefamiliartofinancescholars,whohavebeenapplyingtheirown,toevaluatepricingpatternsinfinancialmarkets,namely,absenceofarbitrageopportunities,

or,althoughonlyforneoclassicists,theefficientmarketshypothesis.

Novelconjectureswillbethebyproductofanyinsistencethattheoriesofhumandecisionmakingbevalidatedneurobiologically.Forinstance,itiswellknownthathumansenvyandgloat,andthattheseemotionalreactionsappeartoaffectutility,andhence,futurechoice(Baultetal2008).Whywouldenvyandgloataffectutility?Itcanbeconjecturedthatthisleadstherewardpredictionerror(differencebetweenexperiencedutilityandexpectedutility)toreact,notonlytowhatanindividualdidherself(asintraditionalreinforcementlearning)orwhatshefailedtodo[fictivelearninglikein(Lohrenzetal2007)],butalsotowhatotherschosetodo.Assuch,therewardpredictionerrorautomaticallyincludesoptionschosenbyothers.Learningofoptimalstrategiesistherebyenhanceddramatically.Thisshouldmatterespeciallyinsituationswheretheconsequencesofunchosenactionsarehardtoassess,sothatfictivelearningisineffective.Thealternativeforthebrainwouldhavebeentosetupacostlyparallellearningsystemthatsimulatestheutilityofothers,withoutreferencetoonesownutility.Oritcouldhavechosenamirrorsystem(Rizzolatti&Craighero2004)thatstoresactionsofotherswithoutrecordingtheirvalue,buttheblindnessofsuchasystemrendersitscognitiveusequestionable.

Choicebiasescanbeexpectedtoarisewhenthebrainfacesanewadversaryor

anunknownenvironment.Financialmarketsmaybeaprimeexampleofthebrainchartingnewterritory.Nevertheless,eveninthiscontext,humansoftenappeartodoremarkablywell.Forinstance,thesuccessofthecontinuousdoubleauctioningeneratingefficientoutcomesdespiteitsstrategiccomplexity(Friedman&Rust1993)andthefacilitywithwhichhumanscandetectandexploitinsiderinformationinit(Bruguieretal2008)suggestthatcertaindesignsexploitthecomputationalcapabilitiesofthehumanbrainbetterthanothers.

Butthedevelopmentofthedoubleauctionwasaccidental,inthesensethatitdidnotemergefromathoroughstudyofthehumanbrainanditscapacity.Itis

hoped,however,thatfuturediscoveriesindecisionneurosciencewillfacilitatetargeteddevelopmentofnewtoolsforimprovedfinancialdecisionmakingand


15/28

exchange.Forinstance,furtherstudyofhowlongittakesthebraintoreevaluategambleseachtimetheyarepresentedshouldinspiretimingrestrictionsontradingplatformswiththeaimofavoidingchoicebiasesinducedbytimepressure.

Amongsuchtools,onecanofcoursealsoenvisagesimpleaffectcontrol(Kuhnen&Knutson2008).Pharmacologicalremedyisalsowithinreach[e.g.,glutocorticoidreceptoragonistsblockthesynapticeffectsofstressondopamineneuronsintheventraltegmentalarea(Saaletal2003)].Andinvestorsshouldbewarnedforthesideeffectsofmedication[suchasLDopa(Gschwandtneretal2001)]thatinterfereswiththebrainsystemsinvolvedindecisionmakingunderuncertainty.Directcontrolofriskattitudesisnowpossible,throughelectrical(Fecteauetal2007)ormagnetic(Knochetal2006)stimulationoftheinferiorfrontalgyrus,where,aswediscussedbefore,arisksignalresides.Butsuchcontrolraisesethicalissues.

Onealsowonderswhetheritispossibletoteachhumanstobecomebetterfinancialdecisionmakers,withoutresortingtotoolssuchasdoubleauctions,timingrestrictions,pharmacologicalremedies,etc.?Totheextentthatdecisionmakingunderuncertaintyengagespowerfulemotionalprocessesandpervasiveneurotransmitterrelease,thiswillbedifficult.Still,wecanteachtricks,suchaslookingforGARCHfeaturestodetectthepresenceofinsiders(Bruguieretal2008),justlikeautisticpatientscannowbeinstructedtolookintheeyesofotherstoovercometheirhandicapindiscerningtheintentionofothers(Spezioetal2007).Suchtricksareuseful,buttheirscopeislimited,becausewhatweareaskingforistheequivalentofcontrollingavisualbias.Avisualbiasisharmlessifoneknowswhenitoccurs,butitreallydoesnotgoaway,andthebrainwillbefooledanytimeitisinattentive.

Theforgoingalsoanswersacommonlyraisedquestion:shouldcomputersreplacehumansinfinancialmarkets?Theanswerisno.Humansaremuchbetteratcertaintasksthancomputers,buthumanshaveclearlimitationswhenaskedtosolveproblemsforwhichtheirbrainsaremaladapted.Thegoalofdecisionneuroscienceistoidentifythelimitationsandtointroducecomputersonlywheninneedtoovercomethem.Justasthehorizonwasinventedasatooltohelphumanpilots(whoincidentallyhavestillnotbeenreplacedwithcomputers)Thehorizonhelpsbecauseorientationfailswhenflyingthroughclouds.This

failureshouldnotbesurprising,becausethehumanbrainanditsevolutionaryantecedentshaveneverhadtofly.Nordidtheyhavetobeconcernedwithfinanceuntilrecently.


16/28

FigureCaptions

Figure1MedianfiringofdopamineneuronsinthebrainsoftwoMacaquemonkeysafterreceivingnoorlittlerewardvs.varioushigherrewards(juice).Rewardmagnitudeschangedfromonetrialtoanother,butwereperfectly

anticipated,andthemonkeyhadlearnedthatthehigherrewardwouldbedeliveredwith50%chanceinallcases.Firingshowsadaptationtosituation:reactiontodeliveryof0.05mlisnodifferentfromthatwhententimestheamountisdelivered(0.50ml;AnimalAandB).Likewise,deliveryofalowrewardof0.15mlcausesthesamefiringaswhenalowrewardof0mlisanticipated(AnimalB).Source:(Tobleretal2005b).

Figure2A.Timelineofcardgame.Ofinterestarebrainactivationsresultingfromdisplayofcard1(andcard2),whenbeliefs(expectedreward,rewardvariance)areupdatedinfunctionofthenumberonthecardandthebet.B.Expectedrewardandrewardvarianceasafunctionofprobabilityofreward.

Brainsignalsthatencodethesestatisticalfeaturesshouldshowsimilarrelationships.Source:(Preuschoffetal2006).

Figure3A.Locationof(someofthe)activationsthatincreaselinearlyinrewardprobability:LeftandRightventralstriatum(vst).B.MeanactivationinLeftventralstriatum(vst)across19subjects(diamonds)stratifiedbyrewardprobability;verticallinesindicate95%confidenceintervals.C.Crosssectionofthebrainfromfront(Right)toback,showingkeyprojectionareasofdopamineneuronsinthemidbrain.Dopamineneuronsencodingrewardpredictionerrorscanbefoundintheventraltegmentalarea.Theyproject,amongothers,totheventralstriatum(regionactivatedinA).Source:(Preuschoffetal2006)and

openlearn.ac.uk

Figure4A.Locationof(someofthe)activationsthatarequadraticinrewardprobability:LeftandRightanteriorinsula(ins)andmediodorsalnucleusofthethalamus(md)(frontofthebrainisontop).B.MeanactivationinRightanteriorinsula(ins)across19subjects(diamonds)asafunctionofrewardprobabilityshowstheinvertedUshapedpatternneededforittoencoderewardvariance;linesegmentsindicate95%confidenceintervals.Source:(Preuschoffetal2006).

Figure5A.Locationofactivationthatcorrelateswithriskpredictionerrorupondisplayofcard1:LeftandRightanteriorinsula.B.MeanactivationinRight

anteriorinsulaacross19subjects(diamonds)asafunctionofriskpredictionerrorupondisplayofcard1(red)andcard2(blue).Outlieratzeroriskpredictionerrorreferstotrialswhennoriskremainedaftercard1.Linesegmentsindicate95%confidenceintervals.Source:(Preuschoffetal2008).

Figure6A.Locationofactivationcorrelatingwithexpectedutilityofgambles:prefrontalcortex,slightlytotherightofthemedialline(eyeballisvisibleinbottomrightcorner).B.Activationinthislocationasafunctionofexpectedrewardandrewardvariance,persubject,stratifiedbyrevealedriskattitude.Source:(Tobleretal2007).


17/28

SummaryPoints

1. Choiceistheoutcomeofalongneurophysiologicalprocess,fromperceptiontoaction.Inthecontextofgambles,thisprocessincludesconstantreevaluationofstatisticalfeatures.

2. Fromthestatisticsofgambles,thebrainconstructstheveryvaluations(utilities)thatareatthecoreofmoderneconomicthinking.Thesevaluationsignalscanbefoundinexecutiveareasofthehumanbrainthatareremotefromthemotorcortex,andhence,thatarenotdirectlyinvolvedintheactsthatimplementachoice,butintheevaluationoftheoptionsathand.

3. Brainregionsthatencodethemathematicalfeaturesofadecisionproblemareoftenalsoinvolvedinthecorrespondingemotionalreactions,suggestingthatemotionsmaylargelybesubsumedinthemathematicsofdecisionmodels.

4. Emotionsareanintegralpartofreasoneddecisionmaking.Choicecanbeviewed,toalargeextent,astheresultofadelicatebalancingoftheemotionalfeaturesofadecisionproblem,suchasdisappointment,elation,regret,relief,envyandgloating.

5. Theclaimthatlaboratoryexperimentsinfinancedonothaveexternalvaliditycanonlybeupheldifitisdemonstratedthatthebraincircuitryengagedinthelaboratorysomehowbecomesneutralized.Thisishighlyunlikely,becausethatcircuitryisincontrolofpervasiveneurotransmittersandofstrongemotionalresponses.

6. Cognitivebiasesneedtobeevaluatedagainstevolutionaryfitness,justlikefinancialmarketphenomenaarejudgedagainstabsenceofrbitrage(ortheefficientmarketshypothesis).Cognitivebiasesneedtobeunderstoodasamismatchbetweenanovelenvironmentandaneurobiologicalsystemthatisoptimizedforadifferent,ecologicallymorerelevant,problem.

7. Whilethetheoryofgamesofimperfectinformationborrowslargelyfromfinancialdecisiontheory,thehumanbraindealsdifferentlywithuncertaintygeneratedbyanintentionalsourcethanwithobjective

uncertainty.

8. Animprovedunderstandingoftheneurobiologicalfoundationsofhumandecisionmakingunderuncertaintyshouldleadtothedevelopmentofeffectivetoolsofassistance,suchasappropriateemotionalcontrol,tradinginterfacesthatexploitnaturalskills,andevenpharmacologicaldecisionsupport.


18/28

Sidebar

TrackingNeuronalFiring

Firingofindividualneuronscanbetrackedbyinsertinganelectrodeinthebrain,aratherinvasiveproceduredoneinhumansonlyifthereisamedicalneedand

eventhenthelocationoftheelectrodeisdeterminedbymedicalconcernandnotscientificvalue.Recently,anumberofimagingtechniqueshavebecomeavailablethatallowonetononinvasivelybutindirectlymeasurefiringofclustersofneuronsor,moreaccurately,activationofprojectionareastriggeredbyfiringofupstreamneurons(Goense&Logothetis2008).Functionalmagneticresonanceimaging(fMRI)isoneofthem.Basically,fMRIlocalizesthepresenceofoxygenrichblood.Whileoxygenrichbloodwillconcentratewherethereisactivation,theeffectcomeswithsomedelay(about4s),referredtoashemodynamicresponsedelay.ThefunctionalinfMRIstandsforthefactthatalargepartofthebrainisscannedinitsentiretyaboutevery2seconds,sothatchangesinactivationcanbepickedupatdifferentstagesofatask.PicturesofbrainactivationlikeFig.3Aareeffectivelymapsoftstatisticsofthetreatmenteffects,wherebyonlyactivationsbeyondacutoffplevelareretained(generally0.001orless).


19/28

MiniGlossary

NeuronalFiring

Electricalimpulsesthataresentfromtheneuroncellbodyoutwardthroughitsaxon(anextensionofvaryinglength).

Neurotransmitter

Chemicalreleasedfromtheaxonfollowingfiringthatactivatesadendriteoftheprojectionneuronacrossthesynapticcleft.

Dopamine

Onetypeofneurotransmitter(othersincludeseretonin,acetylcholine,norepinephrine,glutamate).

PrimaryReinforcers

Physicalrewardssuchasjuice

SecondaryReinforcers

Indirectrewardssuchasmoneyorsocialreputation.Engagevaluationcircuitryasiftheywereprimary(Izumaetal2008).

EEG(Electroencephalogram)

Recordingonthescalpofelectricalactivitygeneratedbythebrain,withexcellenttimeresolution,butdifficultsourcelocalization.


20/28

Keywords

Neuroeconomics,ExperimentalFinance,Risk,Emotions

Acknowledgment

TheauthorthankstheSwissFinanceInstituteandtheU.S.NationalScienceFoundation(GrantSES0527491)forfinancialsupport.


21/28

References

AdolphsR,TranelD,DamasioH,DamasioAR.1995.Fearandthehumanamygdala.J.Neurosci.15:587991

AllaisM.1953.LeComportementdel'HommeRationneldevantleRisque:

CritiquedesPostulatsetAxiomesdel'EcoleAmericaine.Econometrica21:50346

BalleineBW,KillcrossAS,DickinsonA.2003.TheEffectofLesionsoftheBasolateralAmygdalaonInstrumentalConditioning.J.Neurosci.23:66675

BaultNg,CoricelliG,RustichiniA.2008.InterdependentUtilities:HowSocialRankingAffectsChoiceBehavior.PloSone3:e3477

BecharaA,DamasioAR.2005.Thesomaticmarkerhypothesis:Aneuraltheoryofeconomicdecision.GamesandEconomicBehavior52:33672

BecharaA,DamasioH,TranelD,DamasioAR.1997.DecidingAdvantageouslyBeforeKnowingtheAdvantageousStrategy.Science275:12935

BernsGS,CapraM,ChappelowJ,MooreS,C.N.2008.NonlinearNeurobiologicalProbabilityWeightingFunctionsForAversiveOutcomes.NeuroImage

BollardA,LiuR,NursimuluAD,RangelA,BossaertsP.2008.RiskandRewardPreferencesUnderTimePressure.UnderReview

BossaertsP,GhirardatoP,GuarnaschelliS,ZameWR.2008a.Ambiguityinassetmarkets:theoryandexperiment.UnderReview

BossaertsP,HillionP.1991.MarketMicrostructureEffectsofGovernmentInterventionintheForeignExchangeMarket.TheReviewofFinancialStudies4:51341

BossaertsP,PlottC.2004.BasicPrinciplesofAssetPricingTheory:Evidence

fromLargeScaleExperimentalFinancialMarkets.ReviewofFinance8:13569

BossaertsP,PreuschoffK,HsuM.2008b.Theneurobiologicalfoundationsofvaluationinhumandecisionmakingunderuncertainty.InNeuroeconomics:DecisionMakingandtheBrain,ed.PWGlimcher,CFCamerer,EFehr,RAPoldrack.London:AcademicPress

BruguierA,QuartzS,BossaertsP.2008.ExploringtheNatureofTradingIntuition".UnderReview

CamererC,HoTH.1999.ExperienceweightedAttractionLearninginNormalFormGames.Econometrica67:82774

CamererCF,HoTH,ChongJK.2002.SophisticatedExperienceWeightedAttractionLearningandStrategicTeachinginRepeatedGames.JournalofEconomicTheory104:13788

CaplinA,DeanM,GlimcherPW,RutledgeRB.2008.Measuringbeliefsandrewards:Aneuroeconomicapproach.UnderReview

ChenMK,LakshminarayananV,SantosLR.2006.HowBasicAreBehavioralBiases?EvidencefromCapuchinMonkeyTradingBehavior.JournalofPoliticalEconomy114:51737

ChristopoulosG,ToblerP,BossaertsP,DolanR,SchultzW.2008.Neuralcorrelatesofexpectedvalue,risk,andriskaversioncontributingtodecisionmakingunderrisk.UnderReview


22/28

CoatesJM,HerbertJ.2008.EndogenoussteroidsandfinancialrisktakingonaLondontradingfloor.ProceedingsoftheNationalAcademyofSciences105:616772

CoricelliG,CritchleyHD,JoffilyM,O'DohertyJP,SiriguA,DolanRJ.2005.Regretanditsavoidance:aneuroimagingstudyofchoicebehavior.NatNeurosci

8:125562CoricelliG,NagelR.2008.Beautycontestinthebrain;Aneuralbasisofstrategic

thinking.UnderReviewCritchleyHD,WiensS,RotshteinP,OhmanA,DolanRJ.2004.Neuralsystems

supportinginteroceptiveawareness.NatNeurosci7:18995d'AcremontM,BossaertsP.2008.Neurobiologicalstudiesofriskassessment:A

comparisonofexpectedutilityandmeanvarianceapproaches.Cognitive,Affective,&BehavioralNeuroscience8:36374

d'AcremontM,LuZL,LiX,VanderLindenM,BecharaA.2008.Neuralcorrelatesofriskpredictionerrorsduringreinforcementlearninginhumans.UnderReview

DeMartinoB,KumaranD,HoltB,DolanRJ.2009.Theneurobiologyofreferencedependentvaluecomputation.JournalofNeuroscienceInPrint

DeMartinoB,KumaranD,SeymourB,DolanRJ.2006.Frames,Biases,andRationalDecisionMakingintheHumanBrain.Science313:6847

DreherJC,KohnP,BermanKF.2005.NeuralCodingofDistinctStatisticalPropertiesofRewardInformationinHumans.CerebCortex

ElliottR,AgnewZ,DeakinJFW.2008.Medialorbitofrontalcortexcodesrelativeratherthanabsolutevalueoffinancialrewardsinhumans.EuropeanJournalofNeuroscience27:22138

EllsbergD.1961.Risk,Ambiguity,andtheSavageAxioms.TheQuarterlyJournal

ofEconomics75:64369EngleRF.2002.NewfrontiersforARCHmodels.JournalofAppliedEconometrics

17:42546EvansJSBT.2003.Intwominds:dualprocessaccountsofreasoning.Trendsin

CognitiveSciences7:4549FecteauS,KnochD,FregniF,SultaniN,BoggioP,PascualLeoneA.2007.

DiminishingRiskTakingBehaviorbyModulatingActivityinthePrefrontalCortex:ADirectCurrentStimulationStudy.J.Neurosci.27:125005

FengL,SeasholesMS.2005.DoInvestorSophisticationandTradingExperienceEliminateBehavioralBiasesinFinancialMarkets?ReviewofFinance9:30551

FriedmanD,RustJ,eds.1993.TheDoubleAuctionMarket:Institutions,Theories,andEvidence:WestviewPress.429pp.

GallagherHL,FrithCD.2003.Functionalimagingof'theoryofmind'.TrendsinCognitiveSciences7:7783

GallagherHL,JackA,RoepstorffA,FrithC.2002.Imagingtheintentionalstanceinacompetitivegame.NeuroImage16:81421

GlimcherPW,CamererCF,FehrE,PoldrackRA,eds.2009.Neuroeconomics.DecisionMakingandTheBrain.London,UK:AcademicPress

GschwandtnerU,AstonJ,RenaudS,FuhrP.2001.PathologicGamblingin

PatientswithParkinson'sDisease.Clinicalneuropharmacology24:170


23/28

HamptonAN,BossaertsP,O'DohertyJP.2006.TheRoleoftheVentromedialPrefrontalCortexinAbstractStateBasedInferenceduringDecisionMakinginHumans.J.Neurosci.26:83607

HamptonAN,BossaertsP,O'DohertyJP.2008.Neuralcorrelatesofmentalizingrelatedcomputationsduringstrategicinteractionsinhumans.

ProceedingsoftheNationalAcademyofSciences105:67416HareTA,CamererCF,RangelA.2008a.Selfcontrolindecisionmakinginvolves

modulationofthevmPFCvaluationsystem.UnderReviewHareTA,O'DohertyJ,CamererCF,SchultzW,RangelA.2008b.Dissociatingthe

RoleoftheOrbitofrontalCortexandtheStriatumintheComputationofGoalValuesandPredictionErrors.J.Neurosci.28:562330

HsuM,AnenC,QuartzSR.2008a.TheRightandtheGood:DistributiveJusticeandNeuralEncodingofEquityandEfficiency.Science320:10925

HsuM,BhattM,AdolphsR,TranelD,CamererCF.2005.Neuralsystemsrespondingtodegreesofuncertaintyinhumandecisionmaking.Science310:16803

HsuM,KrajbichI,ZhaoC,CamererCF.2008b.Neuralresponsetorewardanticipationunderriskisnonlinearinprobabilities.UnderReview

HuettelS,SongA,McCarthyG.2005.DecisionsunderUncertainty:ProbabilisticContextInfluencesActivationofPrefrontalandParietalCortices.J.Neurosci.25:330411

KableJW,GlimcherPW.2007.Theneuralcorrelatesofsubjectivevalueduringintertemporalchoice.NatNeurosci10:162533

KahnemanD,FrederickS.2002.RepresentativenessRevisited:AttributeSubstitutioninIntuitiveJudgment.InHeuristicsandBiases:ThePsychologyofIntuitiveJudgment,ed.TGilovich,DWGriffin,DKahneman,

pp.4981:CambridgeUniversityPressKahnemanD,FrederickS.2007.Framesandbrains:elicitationandcontrolof

responsetendencies.TrendsinCognitiveSciences11:456KahnemanD,TverskyA.1979.Prospecttheory:AnAnalysisofDecisionUnder

Risk.Econometrica47:26391KnochD,GianottiLRR,PascualLeoneA,TreyerV,RegardM,etal.2006.

DisruptionofRightPrefrontalCortexbyLowFrequencyRepetitiveTranscranialMagneticStimulationInducesRiskTakingBehavior.J.Neurosci.26:646972

KrollY,LevyH,MarkowitzHM.1984.MeanVarianceversusDirectUtilityMaximization.JournalofFinance39:4761

KuhnenCM,KnutsonB.2005.Theneuralbasisoffinancialrisktaking.Neuron47:76370

KuhnenCM,KnutsonB.2008.TheInfluenceofAffectonBeliefs,PreferencesandFinancialDecisions:SSRN

LancasterKJ.1966.ANewApproachtoConsumerTheory.TheJournalofPoliticalEconomy74:132

LeDouxJE,CicchettiP,XagorarisA,RomanskiLM.1990.Thelateralamygdaloidnucleus:sensoryinterfaceoftheamygdalainfearconditioning.J.Neurosci.10:10629

LoAW,RepinDV.2002.Thepsychophysiologyofrealtimefinancialrisk

processing.JCognNeurosci14:32339


24/28

LohrenzT,McCabeK,CamererCF,MontaguePR.2007.Neuralsignatureoffictivelearningsignalsinasequentialinvestmenttask.ProceedingsoftheNationalAcademyofSciences104:94938

McCabeK,HouserD,RyanL,SmithV,TrouardT.2001.Afunctionalimagingstudyofcooperationintwopersonreciprocalexchange.Proceedingsof

theNationalAcademyofSciences98:118325McClureSM,BernsGS,MontaguePR.2003.Temporalpredictionerrorsina

passivelearningtaskactivatehumanstriatum.Neuron38:33946McClureSM,LaibsonDI,LoewensteinG,CohenJD.2004.SeparateNeural

SystemsValueImmediateandDelayedMonetaryRewards.Science306:5037

MontaguePR,DayanP,SejnowskiTJ.1996.AframeworkformesencephalicdopaminesystemsbasedonpredictiveHebbianlearning.JNeurosci16:193647

MorrisJS,FrithCD,PerrettDI,RowlandD,YoungAW,etal.1996.Adifferentialneuralresponseinthehumanamygdalatofearfulandhappyfacialexpressions.Nature383:8125

NivY,EdlundJA,DayanP,O'DohertyJP.2008.Abirdinthehandversustwointhetree:Neuralpredictionerrorsrevealsubjectiverisksensitivity.UnderReview

O'DohertyJ,DayanP,SchultzJ,DeichmannR,FristonK,DolanRJ.2004.DissociableRolesofVentralandDorsalStriatuminInstrumentalConditioning.Science304:4524

O'DohertyJP,DayanP,FristonK,CritchleyH,DolanRJ.2003.TemporalDifferenceModelsandRewardRelatedLearningintheHumanBrain.Neuron38:329

PadoaSchioppaC,AssadJA.2006.Neuronsintheorbitofrontalcortexencodeeconomicvalue.Nature441:2236

PayzanE,BossaertsP.2008.LearningandChoiceunderNonstationaryandUnknownProbabilities.UnderReview

PlassmannH,O'DohertyJ,RangelA.2007.OrbitofrontalCortexEncodesWillingnesstoPayinEverydayEconomicTransactions.J.Neurosci.27:99848

PlattML,GlimcherPW.1999.Neuralcorrelatesofdecisionvariablesinparietalcortex.Nature400:2338

PlottCR,ZeilerK.2005.TheWillingnesstoPayWillingnesstoAcceptGap,the"EndowmentEffect,"SubjectMisconceptions,andExperimentalProceduresforElicitingValuations.TheAmericanEconomicReview95:53045

PreuschoffK,BossaertsP,QuartzS.2006.NeuralDifferentiationofExpectedRewardandRiskinHumanSubcorticalStructures.Neuron51:38190

PreuschoffK,QuartzSR,BossaertsP.2008.HumanInsulaActivationReflectsRiskPredictionErrorsAsWellAsRisk.J.Neurosci.28:274552

RizzolattiG,CraigheroL.2004.TheMirrorNeuronSystem.AnnualReviewofNeuroscience27:16992

SaalD,DongY,BonciA,MalenkaRC.2003.DrugsofAbuseandStressTriggeraCommonSynapticAdaptationinDopamineNeurons.37:57782

SchultzW,DayanP,MontaguePR.1997.ANeuralSubstrateofPredictionandReward.Science275:15939


25/28

ShamsL,MaWJ,BeierholmU.2005.Soundinducedflashillusionasanoptimalpercept.NeuroReport16:1923

SpezioM,AdolphsR,HurleyR,PivenJ.2007.AbnormalUseofFacialInformationinHighFunctioningAutism.JournalofAutismandDevelopmentalDisorders37:92939

StahlDO.2000.RuleLearninginSymmetricNormalFormGames:TheoryandEvidence.GamesandEconomicBehavior32:10538

ToblerPN,FiorilloCD,SchultzW.2005a.Adaptivecodingofrewardvaluebydopamineneurons.Science307:16425

ToblerPN,FiorilloCD,SchultzW.2005b.AdaptiveCodingofRewardValuebyDopamineNeurons.Science307:16425

ToblerPN,O'DohertyJP,DolanRJ,SchultzW.2007.Rewardvaluecodingdistinctfromriskattituderelateduncertaintycodinginhumanrewardsystems.JNeurophysiol97:162132

TomSM,FoxCR,TrepelC,PoldrackRA.2007.TheNeuralBasisofLossAversioninDecisionMakingUnderRisk.Science315:5158

TremblayL,SchultzW.1999.Relativerewardpreferenceinprimateorbitofrontalcortex.Nature398:7048

TzieropoulosH,GravedePeraltaR,BossaertsP,GonzalezS.2008.InMan,madnessistherule,rationalitytheexception.Underreview

YacubianJ,GlascherJ,SchroederK,SommerT,BrausDF,BuchelC.2006.DissociableSystemsforGainandLossRelatedValuePredictionsandErrorsofPredictionintheHumanBrain.J.Neurosci.26:95307


26/28

Figure1

Figure2


27/28

Figure3

C

Figure4

A B


28/28

Figure5

Figure6

trinity09 bossaerts paper

Documents