1

Theseductionsofclarity

C.ThiNguyenUniversityofUtah

c.thi.nguyen@utah.edu

(Thisisapre-print.Pleasecitethefinalversion,forthcomingintheRoyalInstituteofPhilosophySupplements)

Thefeelingofclaritycanbedangerouslyseductive.Itisthefeelingassociatedwithunderstandingthings.Andweusethatfeeling, intherough-and-tumbleofdaily life,asasignalthatwehaveinvestigatedamattersufficiently.Thesenseofclarityfunctionsasathought-terminatingheuris-tic.Inthatcase,ouruseofclaritycreatessignificantcognitivevulnerability,whichhostileforcescantrytoexploit. Ifanepistemicmanipulatorcanimbueabeliefsystemwithanexaggeratedsenseofclarity,thentheycaninduceustoterminateourinquiriestooearly—beforewespottheflawsinthesystem.Howmightthesenseofclaritybefaked?Let’sfirstconsidertheobjectofimi-tation:genuineunderstanding.Genuineunderstandinggrantscognitivefacility.Whenweunder-standsomething,wecategorizeitsaspectsmoreeasily;weseemoreconnectionsbetweenitsdis-parateelements;wecangeneratenewexplanations;andwecancommunicateourunderstand-ing.Inordertoencourageustoacceptasystemofthought,then,anepistemicmanipulatorwillwantthesystemtoprovideitsuserswithanexaggeratedsensationofcognitivefacility.Thesys-temshouldprovideitsuserswiththefeelingthattheycaneasilyandpowerfullycreatecategori-zations,generateexplanations,andcommunicatetheirunderstanding.Andmanipulatorshaveasignificantadvantageinimbuingtheirsystemswithapleasurablesenseofclarity,sincetheyarefreedfromtheburdensofaccuracyandreliability. Ioffertwocasestudiesofseductivelyclearsystems:conspiracytheories;andthestandardized,quantifiedvaluesystemsofbureaucracies.

Hereisaworryingpossibility:thereisasignificantgapbetweenourfeelingthatsome-

thingisclearandouractuallyunderstandingit.Thesenseofclaritycanbeamarkerofcog-

nitivesuccess,butitcanalsobeseductive.Oversimplificationsslipeasilyintoourmindsand

connivethemselvesintoourdeliberativeprocesses.

Inthatcase,thesenseofclaritymightbeintentionallyexaggeratedforexploitativeends.

2

Outsideforces,withaninterestinmanipulatingourbeliefsandactions,canmakeuseofclar-

ity’sappeal.Seduction,afterall,ofteninvolvesaseducer.Romanticseduction, initsmore

maliciousform,involvesmanipulatingtheappearancesofintimacyandromanceinorderto

subverttheaimsoftheseduced.Thereisananalogousformofcognitiveseduction,where

hostileforcesplaywiththesignalsandappearancesofclarityinordertoleadourthinking

astray.

Thesenseofclarityisapotentfocalpointformanipulationbecauseofitscrucialrolein

managing our cognitive resources. After all, we only have somuchmental energy to go

around;weneedtoprioritizeourinquiries.Inparticular,weneedsomewaytoestimatethat

we’veprobablythoughtenoughonsomematterforthemoment—thatit’sprobablysafeto

moveontomorepressingmatters,evenifwehaven’tgottentotheabsoluterockbottomof

thematter.Oursenseofclarity,anditsabsence,playsakeyroleinourcognitiveself-regula-

tion.Asenseofconfusionisasignalthatweneedtothinkmore.Butwhenthingsfeelclear

tous,wearesatisfied.Asenseofclarityisasignalthatwehave,forthemoment,thought

enough.Itisanimperfectsignal,butitisoneweoftenactuallyuseinthequick-and-dirtyof

everydaypracticaldeliberation.Thisshowswhy,say,manipulativeinterestsmightbepar-

ticularlyinterestedinapingclarity.Ifthesenseofclarityisathought-terminator,thensuc-

cessfulimitationsofclaritywillbequitepowerful.Ifsomebodyelsecanstimulateoursense

ofclarity,thentheycangaincontrolofaparticularcognitiveblindspot.Theycanhidetheir

machinationsbehindaveilofapparentclarity.

Here’sanotherwaytoputit:themomentwhenwecometounderstandoftenhasapar-

ticularfeelto it—whatsomephilosophershavecalledthe“a-ha!”moment.Themoment

whenwecometounderstand,saysAlisonGopnik,issomethinglikeanintellectualorgasm

3

(Gopnik,1998).And,asJohnKvanvigsuggests,itisourinternalsenseofunderstanding—

oursenseof “a-ha!”and“Eureka!”—thatprovidesasenseofclosure toan investigation

(Kvanvig,2011,88).The“a-ha”feelingisbothpleasurableandindicatesthatamatterhas

been investigatedenough. If, then,hostile forcescan learntosimulate that“a-ha” feeling,

thentheywillhaveaverypowerfulweaponforepistemicmanipulation.

Ioffertwosustainedcasestudiesofcognitivesubversionthroughtheseductionsofclar-

ity.First,Iwilllookatthesortsofbeliefsystemsoftenpromulgatedbymoralandpolitical

echochambers,whichoffersimplisticpicturesofaworldfullofhostileforcesandconspiracy

theories.Suchbeliefsystemscancreateanexaggeratedsenseofclarity,inwhicheveryevent

canbeeasilyexplainedandeveryactioneasilycategorized.Second,Iwilllookattheseduc-

tiveclarityofquantification.Iborrowmyuseof“seduction”fromSallyEngleMerry’sThe

SeductionsofQuantification(2016),astudyintohowglobalinstitutionsdeploymetricsand

indicatorsintheserviceofpoliticalinfluence.Merryfocusesonthegenerationofindicators

andmetricsontheglobalstage,suchastheHumanDevelopmentIndex,whichattemptsto

sumupthequalityoflifeacrosseachcountry’sentirecitizenshipinasingle,numericalscore.

TheHDIthencompilesthesescorestoofferasingleapparentlyauthoritativerankingofall

countriesby theirqualityof life. Suchsystemsofquantificationcanofferanexaggerated

senseofclaritywithoutanaccompanyingamountofunderstandingorknowledge.Theircog-

nitiveappealcanoutstriptheircognitivevalue.

It isstrikinghowquantifiedpresentationsofvalueseemtohaveaprofoundcognitive

stickiness. Themotivational drawof quantified values has beenwell-documented across

many terrains (Porter 1996;Merry2016;Espeland and Sauder2016). Thismotivational

4

poweriswhysomanycompaniesandgovernmentshavebecomeinterestedinthetechnol-

ogies of gamification. Gamification attempts to incorporate the mechanics of games —

points,experiencepoints,andlevelingup—intonon-gameactivities,inordertotransform

apparently“boring”activityasworkandeducationintosomethingmoreengaging,compel-

ling,andaddictive(McGonigal2011;Walzetal2015;Lupton2016).Iamworried,however,

thatgamificationmight increasemotivation,butonlyat thecostofchangingourgoals in

problematicways.Afterall,stepcountsarenotthesameashealth,andcitationratesarenot

thesameaswisdom(Nguyen2020,189-215;forthcoming).Theseductionsofclarityare,I

believe,oneimportantmechanismthroughwhichgamificationworks.

Letmebeclear:thepresentinquiryisnotastudyinidealrationality,norisitastudyof

epistemicviceandcarelessness. It isastudy in thevulnerabilitiesof limited,constrained

cognitiveagents,andhowenvironmentalfeaturesmightexploitthosevulnerabilities.Itisa

forayintowhatwemightcallhostileepistemology.Hostileepistemologyincludestheinten-

tionaleffortsofepistemicmanipulators,working toexploit thosevulnerabilities for their

ownends.Wemightcallthestudyoftheseintentionalepistemichostilitiescombatepiste-

mology.Hostileepistemologyalsoincludesthestudyofenvironmentalfeatureswhichpre-

sentadangertothosevulnerabilities,madewithouthostileepistemicintent.Hostileenvi-

ronments,afterall,don’talwaysarisefromhostileintent.Hostileenvironmentsincludein-

tentionallyplacedminefields,butalsocrumblingruins,thedeepsea,andMars.Anepistem-

icallyhostileenvironmentcontainsfeatureswhich,whetherbyaccident,evolution,orde-

sign,attackourvulnerabilities.

Iwillfocusfortheearlypartsofthispaperoncasesofcombatepistemology.Ithinkthis

istheeasiestplacetoseehowcertainsortsofsystemshaveahostileepistemicfunction.The

5

cases of intentionallymanufactured hostile environmentswill then help us to recognize

casesof theunintentional formationofhostileepistemicenvironments.Hostileepistemic

environmentscanarisefromentirelywell-intentioned,andevensuccessful,pursuitsofother

purposes.Aculinarilyextraordinarypastryshopalsopresentsanenvironmenthostiletomy

attemptsathealthyeating.Inmanybureaucraticcases,aswewillsee,systemsofquantifica-

tionoftenariseforverygoodreason:toefficientlymanagelargeandcomplexinstitutional

data-sets,or to increaseaccountability (Scott1998;Perrow2014).But theseverydesign

featuresalsomake them intoepistemicallyhostileenvironment.Becauseof themagnetic

motivationalpullofquantification,theveryfeatureswhichrenderthemgoodforefficient

administrationalsofunctionstoimbuethemwithseductiveclarity.1

Otherrecentinquiriesintohostileepistemologyincludediscussionsofepistemicinjus-

tice,propaganda,echochambers,fakenews,andmore(Fricker,2007;Medina2012;Dotson

2014;Stanley,2016;Rini2017;Nguyen2018b).Importantly,thestudyofhostileepistemol-

ogyisdistinctfromthestudyofepistemicvice.Thestudyoftheepistemicvices—suchas

closed-mindedness,gullibility,activeignorance,andcynicism—isastudyofepistemically

problematiccharacter traits. It is thestudyof failings in theepistemicagents themselves

(SullivanandTuana2007;ProctorandSchiebinger2008;Cassam2016;Battaly2018).Hos-

tileepistemology,ontheotherhand,isthestudyofhowexternalfeaturesmightsubvertthe

1IaminfluencedherebyA.W.Eaton’sdiscussionofartifactfunction,whichdrawsonanddevelopsRuthMilli-kan’snotionoffunction(Millikan1984,Eaton2020).Eatonarguesthattheintentofanartifact’sdesignerdoesnotdeterminethatartifact’sfunction.Shesuggestsamoreevolutionarymodel:Anartifactmaybeunintention-allyimbuedwithtrait,butinsofarasthattraitisselectivelyreproducedinfutureartifacts,thenitseffectispartofthoseartifacts’function.So,ifabureaucracygeneratesaquantifiedmetricforaccountingpurposes,butthatquantifiedmetricsurvivesandisreproducedinfurtherbureaucraticsystemsbecauseofitsseductiveeffect,thentheseductivenessispartofthosesystems’function.

6

effortsofepistemicagents.Ofcourse,viceandhostilityareoftenentangled.Hostileenviron-

mentspressonourvicesandmakeiteasierforustofallmoredeeplyintothem.Butviceand

hostilityrepresenttwodifferentpotentiallociofresponsibilityforepistemicfailure.

Thisallmightjustseemlikecommonsense.Ofcoursepeoplearedrawntooversimplifi-

cations;what’snewinthat?Butthereareimportantquestionshere,aboutwhywe’redrawn

tooversimplificationandhowculpableweareforgivingintoit.Importantly,manytheorists

treatourinterestinoversimplificationasstraightforwardlyirrational.Inthepsychological

andsocialsciences,theappealofoversimplificationisusuallyexplainedasamistakewhich

canbeunderstoodintermsofindividualpsychologicaltendencies,suchasmotivatedrea-

soningortheundueinfluenceoftheemotions.Weacceptoversimplifications,itisthought,

becausetheymakeusfeelsmug,theycomfortus,ortheyreinforceoursenseoftribalidentity

(KahanandBraman2006;Sunstein2017).Similarly,manyphilosophicalaccountstreatour

susceptibility tooversimplificationas aproblemarisingwholly froman individual’sown

personal failuresofcharacter– fromtheirepistemicvices.QuassimCassam, forexample,

tellsthestoryofOlivertheconspiracytheorist,whobelievesthat9/11wasaninsidejob.

SaysCassam,thereisn’tagoodrationalexplanationforOliver’sbeliefs.Thebestexplanation

isafailureofintellectualcharacter.Oliver,saysCassam,isgullibleandcynical;helacksdis-

cernment(162-3).

Iwillpresentapicturethatisfarmoresympathetictotheseduced.Itisapictureinwhich

exaggerated clarityplaysupon specific structuralweaknesses inour cognition.As cogni-

tivelylimitedbeings,weneedtorelyonvariousheuristics,signals,andshort-cutstomanage

thecognitivebarrage.Butthesestrategiesalsoleaveusvulnerabletoexploitation.Seductive

7

claritytakesadvantageofourcognitivevulnerabilities,whicharise,inturn,fromourper-

fectlyreasonableattemptstocopewiththeworldusingourseverelylimitedcognitivere-

sources.And,certainly, thepullofseductiveclaritywillbeworse ifwegive in tovarious

epistemicvices.And,certainly,oncewerealizeallthis,wewillwanttoactmorevigorously

tosecurethevulnerablebackdoorstoourcognition.Thegeneralpoint,however,isthatgiv-

ingintotheseductionsofclarityisn’tjustsomebruteerror,ortheresultofsheerlaziness

andepistemicnegligence.Rather,itisdriven,insignificantdegree,bysystemsandenviron-

mentswhichfunctiontoexploitthecognitivevulnerabilitiesgeneratedbythecopingstrate-

giesofcognitivelyfinitebeings.

Clarityasthought-terminator

Ihavebeenspeakinglooselysofar;letmenowstipulatesometerminology.Ontheone

hand,thereareepistemicallypositivestates:knowledge,understanding,andthelike.Onthe

otherhand,therearethephenomenalstatesthatareconnectedtothoseepistemicallyposi-

tivestate.Thesearetheexperiencesofbeing inanepistemicallypositivestate—likethe

senseofunderstanding,thefeelingofclarity.Loosely:understandingisoursuccessfulgrasp

ofpartsoftheworldandtheirrelationships,andthesenseofclarityisthephenomenalstate

associatedwithunderstanding.Forbrevity’ssake, letmeusetheterms“clarity”and“the

senseofclarity”interchangeably,torefertothephenomenalexperienceassociatedwithun-

derstanding.Idonotmeantobeusing“clarity”intheCartesiansense,whereitisaperfect

guaranteeofknowledge.Clarity,inmyusage,ismerelyanimpressionofacertainkindof

8

cognitivesuccess—whatJ.D.Trouthascalledthesenseofunderstanding(Trout2002).Clar-

itymayoftenaccompanygenuineunderstanding,butitisbynomeansaperfectindicator

thatwedo, in fact,genuinelyunderstand.Soexternal forcescanexploit thegapbetween

genuineunderstandingandthefeelingofunderstanding–thatsenseofclarity.

Therearetwogeneralstrategiesforepistemicmanipulation.Thereisepistemicintimida-

tion:thestrategyoftryingtogetanepistemicagenttoacceptsomethingbymakingthem

afraidoruncomfortabletothinkotherwise.Thereisalsoepistemicseduction:thestrategyof

manipulatingpositivecognitivesignalstogetanepistemicagenttoacceptsomething.The

manipulationofclarityisaformofepistemicseduction.Itistheattempttouseourowncog-

nitiveprocessesagainstus,whisperingpleasantlyallthewhile.

Howmightclarityseduce?Therearemanypotentialpathways.Foronething,clarityse-

ducesbecauseitispleasurable.Butfortheremainderofthisdiscussion,I’llfocusanother,

evenmoredangerousfeature:thatthesenseofclaritycanbringustoendourinquiriesinto

atopictooearly.Thispossibilityarisesbecauseoftheprofoundlyquick-and-dirtynatureof

dailydecision-making.Wearefinitebeingswithlimitedcognitiveresources.2Indailylife,we

needtofigureoutwhattodo:wheretospendourmoney,whotovotefor,whichcandidate

toback.Wefaceaconstantbarrageofpotentiallyrelevantinformation,evidence,andargu-

ment—farmorethanwecouldassessinanyconclusivemanner.Soweneedtofigureout

thebestwaytoallocateourcognitiveresourceswhileleavingmostofourinvestigationsun-

finished,insomecosmicsense.

2Twoparticularlyrelevantdiscussionsoncognitivelimitationandepistemologyare(Wimsatt2007;Dallman2017).

9

Whenpracticallyreasoningaboutthemessycomplexitiesoftherealworld,weareun-

likelytoarriveatanyconclusiveground-floor,wherewecanknowwithanycertaintythat

we’redone.3So,foreverydaypracticaldeliberation,weneedsomemethodfordetermining

thatwe’vethoughtenough.4Andthatbasisoftenneedstobefastandloose,tocopewiththe

fastandloosemannerofeverydaypracticaldeliberation.Weneedsomebasisforestimating

thatourunderstandingisprobablygoodenough,sothatwecanmakeadecisionandmove

on.Weneedsomethinglikeaheuristicforterminatingthought.

Here,then,istherulingsuppositionformyinquiry:thesenseofclarityisoneofthesig-

nalswetypicallyusetoallocateourcognitiveresources.(Idonotclaimthatitistheonly

signal, thoughIdoclaimit isasignificantone.)Weoftenuseoursenseofconfusionasa

3AsElijahMillgramputs it,practicalreasoningdoesn’tresult insettledargumentstofinalizedconclusions.Practicalreasoningproducesonlytentativeconclusions.Practicalconclusionsarealwaysopentodefeatfromunexpectedangles,andnewformsofdefeatmayalwayssurpriseus(Millgram1997).Theclosestwecangettoconclusivenessistothinkthatacertainpieceofpracticalreasoningseemsgoodenough,sofaraswecantell.AndevenifyourejectMillgram’sviewandbelievethattherewerefirmpracticalconclusionsthatwemighteventuallyreach—surely,findingsuchfirmconclusionsiswellbeyondthereachofmosthuman-scalepracticaldeliberationineverydaycircumstances.4Verylittlehasbeenwrittenonhowwedecidetoendourinquiriesinpracticaldeliberation.Andmuchofthatworkhasfocused,notonfast-and-loosedailyheuristicsforterminatinginquiry,butonwhenwecanconclu-sivelyterminateinquiry.See,forexample,AlanMillarandKvanvig’sdebateaboutwhetherweneedmerelyneedknowledgetoconclusivelyterminateinquiry,orwhetherweneedtoreflectivelyknowthatweknowinordertoterminateinquiry(Millar2011;Kvanvig,2011).Trouthimselfarguesthatthe“senseofunderstanding”—that“a-ha”feeling—isnotofparticularuseinthesciencesbecauseitisquitevulnerabletocognitivebiasesandothercorruptingpsychologicalinfluences.InTrout’sterms,themeresenseofunderstandingdoesn’tgrantuswhatwereallywantinscience,whichisgoodexplanations.Wehaveotherwaysofrecognizinggoodexpla-nations,farmoreaccuratethanmereinternalfeelings.Weknowwehaveagoodscientificexplanationwhenourscientificmodelmakesgoodpredictions.Weshould,saysTrout,thereforelargelyignorethevariousinter-nalsignalsofunderstanding,whichwillsimplyleadusastray.Weshould,instead,remainfirmlyfixedontheevidencethatourscientificmodelprovidesgoodexplanations,whicharemeasuredintheusualscientificmeth-ods:prediction,testing,andthelike(Trout2002,2017).Notice,however,thatthissortofapproachimaginestherelevantepistemicagentstobecognitivelyidealbeingswithessentiallyunlimitedresources.Itthenaskshowsuchcognitivebeingsshouldgoaboutgettingthingsrightonceandforall.Andthatmightbetherightidealizationforthinkingabouthowweshouldpursuelong-termepistemicprojectsaspartsof intergenera-tionalcommunities,aswedoinphilosophyandscience.Butthingslookverydifferentforcognitivelylimitedbeingsinthequick-and-dirtyofday-to-daydecision-making.Sometimeswemightbeabletoadoptsomemeth-odologywithapre-establishedthresholdforterminatingthought.Consider,forexample,thecognitivestrategyofsatisficing:takingthefirstsolutionwhichcrossessomepre-establishedminimalthreshold(Simon1956).Butwhatdowedowhenwearen’tsatisficing?Inmanycases,ourinvestigationsaremoreopen-ended,withoutanysortofpre-establishedminimalthreshold.Forthosesortsofinvestigations,weneedsomeheuristicbasisforattentionalmanagement.

10

signal thatweneed to keep investigating, andour senseof clarity as a signal thatwe’ve

thoughtenough.5Oursenseofclarityisasignalthatwecanterminateaninvestigation.When

asystemofthoughtseemscleartous,thenwehaveaheuristicreasontostopinquiringinto

it.6

I’mnotclaimingthatthisheuristicisanecessarypartofallpracticalreasoning–only

thattheheuristiciscurrentlyundercommonusage.Afterall,heuristicsareusuallycontin-

genttendenciesandnotnecessarypartsofourcognitivearchitecture.Infact,someresearch

suggeststhatwecanslowlychangetheheuristicsweuse(ReberandUnkelbach2010).

Here’smyplan.First,we’llstarttothinkabouthowpowerfulitwouldbeifthissupposi-

tionweretrue,andthereweresuchapleasurableandthought-terminatingheuristic.I’lllook

atsomeevidencefromtheempiricalliteratureoncognitiveheuristicsthatsupportssome-

thinginthevicinityofmysupposition.I’llshowhowthesupposition,whichconcernshow

weuseourfeelingofunderstanding,emergesfromarecentdiscussioninthephilosophyof

scienceaboutthenatureofgenuineunderstanding.Then,I’llusethesuppositiontothink

aboutwhatsortssystemsandenvironmentsmightsuccessfullyexploitthesenseofclarity.

I’lldigintosomehistoricalandsociologicalliteratureonechochambersandonthesocial

5MydiscussionhereheavilyborrowsstructuralfeaturesfromElijahMillgram’sdiscussionofthefunctionofboredomandinterestinpracticalreasonandagency.Millgramarguesthatasenseofinterestisoursignalthatourvaluesaregoodonesforustohave,andasenseofboredomisoursignalthatourvaluesarebadforustohave,soweshouldchangethem(Millgram2004).6AsfarasIknow,JustinDallmanofferstheonlycontemporaryaccountofhowourcognitivelimitationsforceustomanageoureffortsofinquiry.Thebestproceduretocopewithcognitivelimitation,hesays,istosetupapriorityqueue.Weassignpriority levels toourvariousoutstanding investigations,andthenweproceed inorderfromhighestprioritytolowest(Dallman2017).Butwhatbasisdowehaveforassigningprioritylevels?ToputmysuggestionintoDallman’sterms,weneedsomeheuristicforquicklyestimatingpriorities,andoursenseofclarityfunctionsasaheuristicbasisassigningalowprioritytoitsinvestigation.Asenseofclaritycanthusterminatealineofinquiry—notconclusively,butbyloweringitsprioritybelowthebarrageofother,morepressingmatters.

11

effectofsimplisticquantification.Thesuppositionwillturnouttoprovideaunifyingexpla-

nationformanyofthedocumentedeffectsofechochambersandquantification.Myargu-

mentinfavorofthesupposition,then,willbethatitprovidesaunifyingexplanationforvar-

ious observations from cognitive science, sociology, andhistory,while integratingneatly

withastandardaccountofthenatureofunderstanding.Butthismodeofargumentationcan

onlyrenderthesuppositionaplausiblehypothesis;moreempiricalinvestigationiscertainly

calledfor.

Clarityasvulnerability

Suppose,then,thatthesenseofclarityplaysacrucialroleintheregulationofourcogni-

tiveresources,functioningasasignalthatwecansafelyterminateaparticularlineofinquiry.

Obviously, thesenseofclaritycancomeapart fromactual fullunderstanding.7 Itmust, in

orderforittoplayaheuristicroleinquick-and-dirtydailydeliberation.8Inordertoknow

thatwefullyunderstoodsomething,wewouldneedtoconductanexhaustiveandthorough

investigation.Thesenseofclarityisfarmoreaccessibletous,sowecanuseittomakerough

estimatesaboutwhetherwe’veinquiredenough.

Ifahostileforcecouldapesuchclarity,thentheywouldhaveapotenttoolforgettingus

toaccepttheirpreferredsystemsofthought.Thisisbecausefalseclaritywouldprovidean

7Foranin-depthdiscussionofthispoint,seeTrout’sdiscussionofthegapbetweenthesenseofunderstandinginscience,andactuallypossessingagenuineunderstanding(Trout2002).Thereisausefulfurtherdiscussionin(Grimm2012,106-109),whichdefendsTrout’sclaimsagainstLindaZagzebksi’sclaimthatwealwaysknowwhenweunderstand(Zagzebski2001,247).Seealso(Strevens2013).8Iamdrawingherefromthecognitivescienceliteratureonheuristics.Keyrelevantmomentsinthatliteratureinclude(GigerenzerandGoldstein1996;Kahneman2013).

12

excellentcoverforintellectualmalfeasance.Asenseofclaritycouldbringustoterminateour

inquiryintosomethingbeforewecoulddiscoveritsflaws.Itwouldbesomethinglikeanin-

visibilitycloak—onethatworksbymanipulatingourattention.Ourattention,afterall,is

narrow.Webarelynoticewhat‘soutsidethefocusedspotlightofourattention.Wecanmake

somethingeffectivelydisappearsimplybydirectingtheirattentionelsewhere.9Onewayto

make something cognitively invisible, then, isbymaking it signalunimportance.The spy

novelistJohnLeCarre—whohadactuallyworkedinBritishintelligence—describes,inhis

novelTinkerTailorSoldierSpy,whatagenuinelyeffectivespylookslike.Theyaren’tdashing

andhandsome,likesomeJamesBondfigure.Aneffectivespypresentsasentirelynormal,

bland,anddull.Theycandisappearbecausetheyhavelearnedtomagnifythesignalsofbor-

ingness.Similarly,thetechniquesofstagemagicinvolveattentionalmisdirection.Stagema-

gicianslearntosignalboringnesswiththeactivehandwhiledirectingsignalsofinteresting-

nesselsewhere,inordertocontroltheiraudience’sattention.Thesenseofclaritycanwork

inananalogousstrategyofattentionalmisdirection.Anepistemicmanipulatorwhowants

ustoacceptsomesystemofthoughtshouldimbuethatsystemwithasenseofclarity,sothat

cognitiveresourceswillbe less likelytobedirectedtowards it.Thestrategywillbeeven

moreeffectiveiftheysimultaneouslyimbuesomeothertargetwithasenseofconfusion.The

confusingobjectseizesourattentionbysignalingthatweneedtoinvestigateit,whichmakes

iteasierfortheclear-seemingsystemtorecedeintotheshadows.Themanipulatorcanthus

9ThelocusofthemoderndiscussionofthissortofattentionalblindnessisinChristopherChabrisandDanielSimons’sinfluentialexperiments,including,famously,anexperimentwherehalfofthestudysubjectsfailedtonoticeapersoninagorillasuitwalkingacrossaroom,andpoundingtheirchest,whenthesubjectswerein-structedtoperformarelativelysimplecountingtask(ChabrisandSimons2011).

13

gaincontroloftheirtarget’sattentionbymanipulatingtheirtargets’priorityqueueforin-

vestigation.

Thus,hostileforcescanmanipulatethecognitivearchitectureofresource-management

inordertobypassthesafeguardsprovidedbythevariousprocessesofcognitiveinquiry.In

themovies,thecrooksarealwayshackingthesystemwhichcontrolsthesecuritycameras.

Epistemiccriminalswillwanttohackthecognitiveequivalent.

Easeandfluency

Theexperienceofclarityiscomplexanditsphenomenalmarkersmany.Let’sstartwith

acasestudyinonesmallandsimpleaspectofclarity—onewhichhasbeenrelativelywell-

studiedinthepsychologicalsciences.Considertheexperienceofcognitiveease—therela-

tivedegreetowhichitiseasytothinkaboutsomething.Intheliteratureoncognitiveheu-

ristics,cognitiveeaseispartofthestudyof“cognitivefluency”,whichisthe“subjectiveex-

perienceofeaseordifficultywithwhichweareabletoprocessinformation”(Oppenheimer

2008,237).Researchhasdemonstratedthatwedo,infact,oftenusefluencyasacognitive

heuristic. Ifwecomprehendan ideaeasily,wewillbemore likely toaccept it. Cognitive

difficulty,ontheotherhand,makesitmorelikelythatwewillrejectanidea.Thisheuristicis

notentirelyunreasonable:weoftenexperiencecognitiveeaseinadomainpreciselybecause

wehavealotofexperiencewithit.Cognitiveeaseoftencorrelateswithexperience,which

correlateswithskillandaccuracy.But,obviously,easeisseparablefromaccuracy.Studies

havedemonstratedthatone’smerefamiliaritywithanideamakesonemorelikelytoaccept

14

it.Familiaritycreatesasenseofcognitiveease,butwithouttheneedforanyrelevantskillor

expertise.Studieshavealsoshownthatwearemorelikelytobelievesomethingwrittenina

morelegiblefont.Legibilityleadstoeasierprocessing,whichleadstoreadieracceptance.In

otherwords:weareusingourcognitiveeasewithsomepropositionordomainasaheuristic

forouraccuracywiththatpropositionordomain.RolfReberandChristianUnkelbachhave

arguedthatfluencyheuristicsare,infact,oftenquiteuseful.ThroughaBayesiananalysis,

theyconcludethatfluencyisagoodheuristicwhentheuser’senvironmentcontainsmore

truepropositionsthanfalseones—andthebettertheratiooftruetofalsepropositionsin

theirenvironment,thebetterthefluencyheuristicwillwork(ReberandUnkelbach2010).

Butthatheuristiccanbegamed.10

Supposethattheusualfluencyheuristicisinplace.Howmightitbeexploited?Togame

the fluencyheuristic, amanipulatorwouldwant tooffer their targets ideas expressed in

somefamiliarmanner,byusingwell-wornpatternsofthoughtandformsofexpression.This

exploitativemethodologyshouldbequitefamiliar:itexplainstherhetoricalpowerofcliched

slogansandInternetmemes.

Suppose that theworldhasmany suchepistemicmanipulators in it, andhasbecome

chockfullofmisleadingideasthathavebeenengineeredtoseemfamiliar.Ourbeststrategy

toavoidmanipulationwouldbetoupdateourheuristicstocloseoffthiscognitivebackdoor.

AsReberandUnkelbachshowed,wearecapableofchangingandupdatingourheuristics

whenwereceivedevidence that theyhave leadusastray.Themanipulators, then,would

wanttomaskfromusanyevidencethatouruseofthefluencyheuristicwasleadingusastray.

10Troutmakesasimilarpointaboutfluencyandthesenseofunderstanding(Trout2017),althoughhisconcernislargelywithattackingotheraccountsofunderstanding,andnotprovidingafullpictureofexploitation.Itakemyselftobefillinginthedetailsofhissuggestion.

15

Thisis,however,easiertodoinsomedomainsthanothers.Someepistemicdomainshave

obviouslitmustests.Itiseasytocheckformistakenreasoninginthembecausesuccesses

andfailuresareobvioustoanyonlooker.Forexample,wecantellthatourtheoryofbridge-

buildinghasgonewrongcorruptedifournewbridgeskeepfallingdown.Butotherepistemic

domainshavenosucheasylitmustests—likethemoralandaestheticdomains.Ifone’srea-

soninghasbeensystematicallysubvertedinsuchasubtledomain,thereisnoobviouserror

resultthatcouldfunctionasacheck.11Soifmanipulatorswantedtogaincontrolviatheflu-

encyheuristic,onegoodstrategywouldbetoperformtheirfluency-manipulationsover,say,

claimsaboutmoralityandvalue.Alternatively,theymaywanttodevotetheirfluency-ma-

nipulationstocomplexanddiffusesocialphenomenaormoreesotericscientificphenome-

non.Someempiricalclaimscannotbestraightforwardlycheckedbythelayperson,suchas

scientificargumentsforclimatechangeorsociologicalclaimshowoppressionperpetuates.

Ifthemanipulators’targetshavebeengivenaseductivelyclearexplanationwhichdismisses,

say,sociologistsandclimatechangescientistsascorrupt,thoseexplanationswillbequite

hardtodislodge.Mosttargetswillbeunabletoseethattheyhavebeenledastray,andso

won’tupdatetheirheuristics(Nguyen2018b;Nguyen2018c).

Apingunderstanding

Perhapsitseemsimplausibletoyouthatsomebodywouldterminateareallyimportant

inquiryjustbecauseoffluency.Thereis,however,anothermuchmoresophisticatedformof

11Foranextensivediscussionoflitmustestsandexpert-vetting,seeNguyen(2018a).

16

epistemic seduction which will more plausibly trigger the thought-terminating function.

Hostileepistemicmanipulatorscantrytoimitate,notjustease,butafullfeelingofunder-

standing.Theycanpresentthephenomenaassociatedwithapositiveandrichexperienceof

clarity.

Inordertoseehowonemightfakethefeelingofunderstanding,let’sstartbythinking

aboutthenatureofgenuineunderstanding.Forthat,let’sturntoarecentdiscussionofthe

natureofunderstandinginthephilosophyofscience.Accordingtoarecentstrandofthink-

ing,knowledgeisn’tactuallytheprimarygoalofmuchofourepistemicefforts.Knowledgeis

usuallyconceivedofassomethinglikethepossessionoftruefacts.Havingknowledge,bythe

usualaccounts,doesn’trequireanyparticularintegrationofthosefacts.Butmanyofourin-

tellectual effortsareaimedat getting somethingmore than justknowing somedisparate

facts.Weaimatsomethingmoreholistic:understanding.Theprecisenatureofunderstand-

ingisstillundersomedebate,butwecanextractsomecommonandlargelyuncontroversial

ideas.12First,whenweunderstandsomething,wenotonlypossessalotofindependentfacts,

butweseehow those facts connect.Understanding isof a system; it involvesgraspinga

structureandnotjustindependentnodes.Second,whenweunderstandsomething,wepos-

sesssomeinternalmodeloraccountof itwhichwecanusetomakepredictions,conduct

12Muchofthedebateinthatliteraturehasturnedonwhatisconstitutiveofunderstanding,andwhatismerelytypicallyassociatedwithunderstanding.Forexample,accordingtoStevenGrimmandHenkdeRegt,theskillofpracticalapplicationispartiallyconstitutiveofunderstanding(Grimm2006;deRegt2009;Wilkenfeld2013,2017).MichaelStrevens,ontheotherhand,deniesthisconstitutiverelationship;skilltypicallyfollowsfromunderstanding,butisn’tconstitutiveofit(Strevens2013).Notethatwedon’tneedtoresolvedebateslikethisforthecurrentinquiry.Sincewe’reinterestedwhatsignsareassociatedwithunderstanding,wedon’treallyneedtodistinguishcarefullybetweenwhatisconstitutiveofunderstanding,andwhatfollowsfromit.Finally,KareemKhalifahasargumentthattheseaccountsofunderstandingcanbereducedtotheideaofknowinganexplanation(Khalifa2012).MyaccounthereshouldbecompatiblewithKhalifa’sview–though,inhislanguage,Iwouldbetalkingaboutfakingthefeelofknowinganexplanation.

17

furtherinvestigations,andcategorizenewphenomena.13

Thatisanaccountofwhatitmeanstoactuallyhaveanunderstanding.Sowhatarethe

experientialphenomenaassociatedwithunderstanding?Whatdoesitfeelliketounderstand

something?Thereareseveraldistinctphenomenatoconsiderhere.First,therearetheex-

periencesassociatedwithcomingtounderstand.AsCatherineElginputsit,whenwecome

to understand, ourway of looking at things suddenly shifts to accommodate new infor-

mation.Understanding,shesays,“comesnotthroughpassivelyabsorbingnewinformation,

butthroughincorporatingitintoasystemofthoughtthatisnot,asitstands,quitereadyto

receiveit”(Elgin2002,14).Whenwecometounderstand,oursystemofthoughtchanges

and pieces of information thatwe could not accommodate before suddenly find a place.

Kvanvigoffersasimilaraccount:tounderstand,hesays,istograspacoherencerelationship.

Itistobeawareofhowtheinformationfitstogether(Kvanvig2003,202).Theexperienceof

comingtounderstand,then,involvesanexperienceofgraspinganewandimprovedcoher-

ence.Letuscallthisthephenomenonofcognitiveepiphany.And,asGopnickpointsout,cog-

nitiveepiphaniesareincrediblypleasurable.

Next,therearephenomenaassociatedwithhavinganunderstanding.Understandingin-

volvesacertainfacilitywiththeterrain.AsKvanvigputsit,

…Tohavemasteredsuchexplanatoryrelationshipsisvaluablenotonlybecauseitinvolvesthefinding

ofnewtruthsbutalsobecausefindingsuchrelationshipsorganizesandsystematizesourthinkingon

asubjectmatterinawaybeyondthemereadditionofmoretruebeliefsorevenjustifiedtruebeliefs.

Suchorganizationispragmaticallyusefulbecauseitallowsustoreasonfromonebitofinformationto

13Thisdiscussionconstitutesafast-growingliterature.IamparticularlyinfluencedbyCatherineElgin’sac-count,StephenGrimm’susefulsurvey,andMichaelStrevens’andMichaelPatrickLynch’sdiscussions(Elgin2002,2017;Grimm2012,Strevens2013;Lynch2018).

18

anotherrelatedinformationthatisusefulasabasisforaction,whereunorganizedthinkingprovides

nosuchbasisforinference.Moreover,suchorganizedelementsofthoughtprovideintrinsicallysatis-

fyingclosuretotheprocessofinquiry,yieldingasenseorfeelingofcompletenesstoourgraspofa

particularsubjectmatter.(202)

Whenweunderstandacognitiveterrain,wecanmovebetweenitsnodesmorequickly

andeasily.Wecanuseourunderstandingtoeasilyandpowerfullygeneraterelevantexpla-

nations.Andifourunderstandingisfecund,thesenewexplanationswillservetocreateeven

moreusefulconnections.And,asMichaelStrevenssays,havinganunderstandingalsoin-

volveshavingthecapacitytocommunicatethatunderstanding—toexplaintohowthecon-

nectionswork(Strevens2013).Let’scallallthesethephenomenaofcognitivefacility.14And,

atleastinmyownexperience,thepleasureofclarityliesnotonlyinGopnick’smomentof

comingtounderstand,butalso in thecontinuing joysofapparent facilityand intellectual

power. It feels incrediblygoodtobeabletoswiftlyexplaincomplexphenomena. It is the

pleasureofengagingourskillsandcapacitiestopowerfuleffect.15

Let’senterintothemindsetofthehostileepistemicmanipulator.Ourgoalistoseduce

withapparentclarity—togameotherpeople’scognitiveprocessesandheuristicssothat

theywillacceptourpreferredsystemofthought.We’llwanttoengineerthatsystem,then,

tocreatethefeelingofcognitiveepiphany.We’llwanttomaximize,foroursystem’sadopters,

thesensethatunexplainedinformationisslidingintoplace,thefeelingofnewfoundcoher-

ency.Sowe’llwanttogivethesystemeasy-to-applycategorizationswhicharereadilycon-

nectedintoacoherentnetwork.And,oncethatsystemhasbeenadopted,we’llwant itto

create the feeling of cognitive facility.We’ll want to engineer it so that, once somebody

14IowemyframingtoLauraCallahan’s(2018,442)usefuldiscussionofunderstanding.15Formoreontheaestheticpleasureofone’sownskillfulaction,seeNguyen(2020,101-120).

19

adoptsthesystem,thinkinginitsterrainwillseemdistinctlyeasierandmoreeffectivethan

before.We’llwant it togiveadoptersaheightenedsensationof formingconnectionsand

movingeasilybetweenthem.We’llwantittocreatetheimpressionofexplanatorypower,

quicklyandeasilyexplaininganynewphenomenathatcomeup.Andwewouldwanttodo

allthatwhilesimultaneouslymaskingitsepistemicfaults.

Thismightseemlikeanoverwhelminglydifficulttaskfortheaspiringmanipulator.We

manipulators,however,havesomeverysignificantadvantages.First,wedon’tneedtosuc-

cessfullyimitateunderstandingallthewaydown.Wesimplyneedforoursystemtotrigger

the clarityheuristic early enough, before its adopters stumble across anyof the flaws. If

you’rebuildingaPotemkinvillage,youdon’tneedtoactuallybuildanyactualhouses.You

justneedtobuildthefacades—solongasthosefacadesconvincepeoplenottotryandenter

thebuildings.Wemanipulators,then,canhideoursystem’sweaknessandinferiorperfor-

mancebehindaveilofapparentclarity.16

Butourmostsignificantadvantageisthatweareunburdenedbytheconstraintsoftruth

inengineeringourextra-tastysystemofthought.Epistemicallysinceresystems—thatis,

systemsofthoughtgeneratedforthesakeofrealknowledgeandgenuineunderstanding—

areheavilyconstrainedbytheirallegiancetogettingthingsright.17Wemanipulatorsareun-

boundbyanysuchobligations.Wearefreetotweakoursystemtomaximizeitsappealing

clarity.Thisissimilar,inaway,tohowunhealthyrestaurantsarefreetoappealmoredirectly

tooursenseofdeliciousness,becausetheyarefreedfromconsiderationsofhealth.(Or,at

16Thisstrategyexploitsacognitiveerrorofover-weightingearlyevidence.Foradiscussionofwhythisisacognitiveerror,seeKelly(2008).Foranapplicationofthatdiscussiontoconspiracytheoriesandechocham-bers,seeNguyen(2018b).17Elgin(2017)defendstheuseofidealizationsandnon-truthsaspartsofthemodelsthathelpustounderstand.However,thechoiceofmodelsisstilldrivenbyanorientationtowardsgettingtheworldright,inamoreholisticway.

20

least,that’showmymothersawit.)Wemanipulators,then,canoptimizeoursystemtooffer

thesenseofeasilymadeconnectionandexplanations.Wecanbuildacartoonofunderstand-

ing.Andthatcartoonwillhaveacompetitiveadvantageinthecognitivemarketplace.Itcan

beengineeredforthesakeofpleasure,anditwillcarrywithitasignalthatinquiryisfinished,

andthatweshouldlookelsewhere.

Twosystemsofcognitiveseduction

Let’slookattwocasestudiesoftheseductionsofclarity:echochambersandinstitutional

quantification.Thefirstcasestudyofechochamberswillstrikemany,Isuspect,asaplausi-

bleandfamiliarcaseoftheseductionsofclarity.Thediscussionofquantificationmayprove

moresurprising.AndIhopethatthedifferencesbetweenthesetwocasestudieswillhelpus

tohoneinonthephenomenon’smoregeneralqualities.

Let’sstartwithechochambers.Mostsocialscientistsandjournalistsusetheterms“echo

chamber”and“epistemicbubble”synonymously.But,asI’veargued,ifwelookattheoriginal

sourcesoftheseterms,wefindtwoverydifferentphenomena.Anepistemicbubbleisaso-

cialphenomenonofsimpleomission.It’sbadconnectionsinyourinformationnetwork–like

ifallyourfriendsonFacebookshareyourpolitics,andyousimplyneverrunacrossthear-

gumentspresentedbytheotherside.Anechochamber,ontheotherhand,isasocialstruc-

turewhichdiscreditsalloutsiders.Whenyouareinabubble,youdon’theartheotherside.

Whenyou’reinanechochamber,youdon’ttrusttheotherside.Echochambersdon’tcutoff

21

linesofcommunicationfromtheoutsideworld;rather,theyisolatetheirmembersbyma-

nipulatingtheirmembers’trust(Nguyen2018b).

Whatmattersforthepresentstudyistheparticularcontentofthesystemsofthought

whichechochambersusetomanipulatetrust.I’mdrawinghereonKathleenJamiesonand

JosephCappella’sempiricalanalysisoftheechochamberaroundRushLimbaughandtheFox

Newsecosystem(JamiesonandCappella2010).AccordingtoJamiesonandCappella,Rush

Limbaughoffersaworld-viewwithsomeverydistinctivefeatures.First,Limbaughpresents

aworldofsharplydividedforceslockedinalife-or-deathstruggle.Therearenoonlookers

orreasonablemoderates.Eitheryou’reaLimbaughfollower—andsoonthesideofright—

oryouareoneofthemalevolentforcesouttounderminethesideofright.Limbaughthen

offersanexplanatorysysteminwhichmostmoralandpoliticalactioncanbeunderstoodin

termsofthatall-consumingstruggle.DisagreementwithLimbaugh’sworldviewcanberead-

ilyexplainedastheproductofsomeorganized,malevolentactiontoblockthesideofright.

Mostimportantly,forourpresentpurposes,theunderminingfunctionandtheexplanatory

functionareoftenaccomplishedwiththehelpofconspiracytheories,whichprovideaready

explanationfordisagreementfromoutsiders.Theliberalmediaisinthegripofanefarious

networkofelites,asareuniversities,andtheacademicsciences.Theseconspiracytheories

offertoexplaincomplexfeaturesoftheworldintermsofasinglecoherentnarrative.

Thisisanobviousdeploymentoftheseductionsofclarity.First,Limbaugh’sworld-view

offersthesensationsofepiphany.Oncehisworld-viewisaccepted,difficult-to-categorizeac-

tionssuddenlybecomeeasilycategorized.Previouslyhard-to-explainfacts—liketheexist-

enceofsubstantivemoraldisagreementbetweenapparentlysincerepeople—suddenlybe-

comeeasilyexplicableintermsofasecretwarbetweengoodandevil.Second,theworld-

22

viewoffers the sensations of cognitive facility. The conspiracy theory offers a ready and

neatlyunifiedexplanationforallsortsofbehavior.Andthoseexplanationsareeasytocreate.

Theworldsuddenlybecomesmoreintellectuallymanageable.This isparticularlyvivid in

someofcommunitiesaroundthewilderconspiracytheories.CNNrecentlyconductedsome

quite telling interviewswithsomemembersof the fast-growingcommunityofFlatEarth

conspiracytheorists.ManytheoristsdescribethesatisfactionsofbeingaFlatEarththeorist

inintermsofcognitivefacility.AsFlatEarththeoristandfilmmakerMarkSargentputsit,

"Youfeellikeyou'vegotabetterhandleonlifeandtheuniverse.It'snowmoremanageable.”

AndFlatEarththeoristDavidWeisssays, “WhenyoufindouttheEarth is flat…thenyou

becomeempowered”(Picheta2019).

Furthermore,well-designedechochamberstypicallyhavesystemsofbeliefwhichcan

reinterpretincomingevidenceinordertoavoidrefutation.Forexample,manyechocham-

bers include sweeping scientific claims, such as denying the existence of climate change.

Echochambermembersmayhaveadoptedbeliefsystemswiththehelpoftheclarityheuris-

tic.But,onemightthink,heuristicsaredefeasible—andcontraryscientificevidenceshould

surelybringmemberstoabandontheirsettledacceptanceoftheirbeliefsystem.However,a

cleverechochambercanpreemptivelydefusesuchcontraryevidence.Awell-designedecho

chambercaninclude,initsbeliefsystem,aconspiracytheoryabouthowthemediaandthe

institutionsofsciencewereentirelycorruptandinthegripofavastmaliciousconspiracy.

Thisexplanationperformsakindofintellectualjudo.AsEndreBegby(2020)pointsout,such

abeliefsystemtransformsapparentlycontraryevidenceintoconfirmationsofthebeliefsys-

tem—aprocesswhichhecalls“evidentialpre-emption”.IfLimbaughpredictsthatthelib-

23

eralmediawillaccusehimoffalsifyinginformation,thenwhenhisfollowershearsuchaccu-

sationsfromtheliberalmedia,theywillhavereasontoincreasetheirtrustinLimbaugh—

sincehispredictionshavebeenfulfilled!Butnoticethatthereisasecondaryeffect,beyond

thesimpleconfirmationBegbydescribes—aneffectthatarisesfromtheseductionsofclar-

ity.Thebeliefsystemmakesiteasytocreateanexplanationforincomingcontraryevidence

andtoprovideexplanationsthatunifyandconnectthateventwithmanyothers.Thispro-

videsanexperienceofcognitivefacility—whichshouldtriggertheclarityheuristic.Thisis

anextremelywell-designedepistemictrap,inwhichcontraryevidencetriggerstwodifferent

defensemechanisms.First,theconspiracytheorypreemptivelypredictsthepresenceofcon-

traryevidence,andsoconfirmsitself intheprocessofdismissingthatcontraryevidence.

Second,theeasewithwhichtheconspiracytheoryperformsthatpredictionanddismissalis

anexperienceofcognitivefacility—whichcreatesthesenseofclarity,which,inturn,trig-

gersthethought-terminatingheuristic.

Suchdefensiveconspiracytheoriesareanobviouscaseoftheseductive,manipulative

useofclarity.Let’snowturntoalessobviouscase.Considertheappealofquantifiedsystems.

Consider,especially,thewayinwhichlarge-scaleinstitutionstrytoreducecomplex,value-

ladenqualitiestosimplemetricsandmeasures.InTrustinNumbers,ahistoryoftheculture

ofquantification,TheodorePorternotesthatquantifiedsystemsarepowerfullyattractive.

Thisiswhy,hesays,politiciansandbureaucratslovetocitetheauthorityofquantifiedsys-

temsofanalysis.Numbers,hesays,smellofscience.Theyhavetheringofobjectivity,andso

theywillbeusedininappropriatecircumstancesinattemptstogainpoliticalcontrol(Porter

1996,8).IthinkPorterisentirelyrightaboutthecredibilityadvantageofnumbersandtheir

scientificfeel—butIdon’tthinkthisisthewholestory.Thedetailsofhisstudyofferusthe

24

opportunitytobuildasecondaccountoftheappealofnumbers,alongsidehiscredibilityac-

count,intermsoftheseductionsofclarity.

Thereare,saysPorter,qualitativewaysofknowingandquantitativewaysofknowing.

Porterisnotheremakingthecrudeclaimthatquantitativewaysofknowingareinherently

bad.Rather,he is interested in therelativeadvantagesanddisadvantagesofeachwayof

knowing.Qualitativewaysofknowing,hesays,aretypicallynuanced,sensitive,andrichin

contextualdetail,buttheyarenotportableoraggregatable.Whenwetransitiontofromqual-

itativetoquantitativewaysofknowing,westripoutmuchofthenuanceandmanyofthe

contextualdetails. Inreturn for this lossof informationalrichness,weget toexpressour

knowledge inneatpackages: in the formof numbers,whosemeanings areportable, and

whichcanbeeasilyaggregatedwithothernumericalresults.Thiscanbeveryvaluable.Ob-

viously,quantificationisvitalformodernscience.Andtherearemanyadministrativefunc-

tionswhichquantificationmakesfarmoreefficient.But,saysPorter,contemporaryculture

seemstohavelostsightofthedistinctivevalueofqualitativewaysofknowing.Wetendto

reachforquantitativewaysofknowingcompulsively,evenwhentheyaren’tmostappropri-

ateforthetaskathand.

InTheSeductionsofQuantification,MerryappliesPorter’sanalysistotherecentriseof

quantifiedmetrics in international governance. She is interested in indicators— simple,

quantifiedrepresentationsofcomplexglobalphenomena.OneindicatoristheUN’sHuman

DevelopmentIndex,whichgivescountriesasinglescorefortheirperformanceinsupporting

thequalityoflifeoftheircitizens.AnotherindicatoristheUSStateDepartment’sTrafficking

inPersonsReports,whichgivescountriesascoreontheirperformanceinreducingsextraf-

25

ficking. Indicatorspresent themselves in the formofa single, easy-to-use,easy-to-under-

standnumericalscore.These indicators,shesays,hidethecomplexityandsubjectivityof

theirmanufacture.And thatconcealment ismuchof thepoint. Theirpower, saysMerry,

comesinsignificantpartfromtheirappearanceofunambiguity.Andoncetheseindicators

havebeenmanufactured,theyinvariablybecomecentralinvariousgovernments’andpoli-

ticians’decision-makingprocesses.Theveryqualitieswhichmakethemsopowerfulalso

makethembluntinstruments,missinginmuchsubtletyanddetail.But,saysMerry,theyare

incrediblyhardtodislodgefromthemindsofthepublicandofpolicy-makers(Merry2016,

1-43,112-60).

Whyarequantificationssosticky?Theseductionsofclarityofferanexplanation.Quanti-

fiedsystemsare,bydesign,highlyusableandeasilymanipulable.Theyprovideapowerful

experienceofcognitivefacility.Itismucheasiertodothingswithgradesandrubricsthanit

iswithqualitativedescriptions.Wecanofferjustifications(“Iaverageditaccordingtothe

syllabus’directives”;“Iappliedtherubric”).Wecangenerategraphsandquantifiedsummar-

ies.And thesenseof facility isevenstronger in large-scale institutions,where theuseof

numbershasbeenstringently regularized.Becauseof theportabilityofnumbersand the

constancy and enforced regularity of typical institutional deliberation procedures, inside

suchinstitutions,itisvastlyeasiertousenumberstoproducepowerfulandeffectivecom-

munications.Andtheyarecommunicationsintermswhichweknowwillbeunderstoodand

actedupon—becausethemeaningsandusesoftheseinstitutionaltermshasbeensoag-

gressivelyregularized.

InauniversityforwhichIonceworked,alldepartmentshadtoproduceyearlyassess-

mentdatawhichwassupposedtodemonstrate,inquantitativeform,thequalityofeducation

26

thatourstudentshadreceived.Ourassessmentsresultshadtobecodedaccordingtocertain

institutionally specifiedEducational LearningOutcomes (ELOs). So, the fact thatour stu-

dentsscoredwellthisyearintheircriticalthinkingmultiplechoicetestsgetscodedanden-

teredintothesystem.Thosescoresnowsupportourclaimthataparticularclasssucceeds

insupportingcertainuniversity-widelearning:theCriticalThinkingELO,theWritingSkills

ELO,theMoralReflectionELOandtheMathematicalReasoningELO.Andthedataforeach

particularclass,inturn,isusedtosupporttheclaimthatourdepartmentasawholesupports

theuniversity-widelearningoutcomes.Andthatclaim,inturn,isusedassupporttheclaim

that theUniversity issucceeding in itsmission,andachieving itsstatedCoreValues: like

Communication,Community,andEngagement.Andthewayinwhichclass,departmental,

anduniversityELO’s linkuparecodedexplicitly intoourdatabasingsystem,sothatnew

datacantravelautomaticallyupthechain.WhenIenterthelatestbatchofscoresfrommy

students,itproducesanimmediateeffectintothesystem:allthereportedELOsupthechain

willchange.AndthisispossiblepreciselybecausethedataI’veenteredhasbeenrendered

portableandbecauseouroutcomesreportingsystemhasbeensetuptoautomaticallytake

advantageofthatportability.

Noticethatallthisgivesmetheexperienceofanenormousamountofapparentlyeffec-

tivecognitiveandcommunicativeactivity.Ihaveasenseofgraspingconnections.Icansee

exactlyhowmyclass’sELOssupportmydepartment’sELOs,whichinturnsupportmycol-

lege’sELOs,which inturnsupporttheuniversity’sELOsand, inturn, theUniversityCore

Values.Andmygraspofthissystemcangivemeacertainsenseofcognitivefacility.Ican

easilygenerateexplanationsofcoursecontentandgenerateevidenceofteachingsuccess.

AndIcanknowthat theywillbeunderstood,since theyhavebeenexpressed in thepre-

27

prepared,standardized,andexplicitlyinterconnectedlanguageoftheinstitution.Iknowthat

myjustificationswillbeincorporatedintolargerinstitutionalaggregates,becausemyjusti-

ficationsoccurinthoseintentionallystabilizedterms.AndIknowthatwhenIgivejustifica-

tionsinthosedesignatedterms,theywillusuallygeneratepre-specifiedsortsofactions—

oneswhichIcanusuallypredictwithsomesuccess.Astabilized,explicitsystemofquantified

andsystemizedinstitutionalvalueisdesignedsothatitsuserscanmakethemselveseasily

understoodandtheirpronouncementsquicklyintegratedintoinstitutionalsystemsofinfor-

mationprocessinganddecision-making. In short,byusing theprovided termsof institu-

tionaldiscourseinsidetheinstitution,myspeechandthinkingwillseemclear,preciselybe-

causetheyfitsowellintoapre-establishednetworkofcommunicationandjustification.That

pre-engineeredfitcreatesasenseofcognitivefacility,withallitsassociatedpleasures.And

the ring of clarity can trigger the thought-terminating heuristic in otherswho have also

boughtintotheprovidedsystemofinstitutionaldiscourse–endinginquiryintotheappar-

entlyclearclaim.

Ofcourse,I’llhavegenuinecognitivefacilityifmyvariousmentaleffortsactuallytrack

realelements intheworldandprocesstheminsomeepistemicallyvaluableway.And,as

CharlesPerrowandPaulDuGayhaveargued,bureaucraciescertainlyneedregularmethods

andquantifiedsystemsinordertofunctionandtoadministratefairly(DuGay2000;Perrow

2014).Theworry,though,isthatwemightsetupsystemsthatareusefulforcertainvery

specificdata-collectionandmanagerialfunction—butthatcanalsoexertamagneticpull

onourthinkinginnearbydomains.Forexample:GPAsandcitationsratesmightbeuseful

forcertainparticulartasksofbureaucraticadministration.But,becausetheyaresoseduc-

tive,studentsandscholarsmaystartusing themas theprimary lens throughwhich they

28

evaluatetheirowneducationandoutput.18AndsurelyGPAsarenotperfectindicatorsofa

goodeducation,andcitationsratesarenotperfectindicatorsofgoodscholarship.Aparticu-

larquantificationcangetanexcessgriponourreasoning,evenincontextswhenitisless

appropriate, by presenting an appealing sense of clarity. And we will fail to investigate

whetherthisquantifiedmetricisthemostappropriateformofevaluationtouse,precisely

becauseitsclarityterminatesourinvestigationsintoitsappropriateness.

Sofar,we’vebeenconcentratingonsystemsofthoughtwhosecontentsthemselvesare

seductivelyclear.Buttheseductionsofclaritycanalsoaffectourjudgmentsoftheexpertise

andauthorityofthesourcesofthosecontents.Theseductionsofclaritycangetustoaccept

asystembymakingitsusersandauthorsseemmorecredibleorexpert,preciselybecause

theyseemmoreclear.Recallthatoneofstandardsignalsofexpertiseiscommunicativefa-

cility.Non-experts trust purported expertswhen those experts are able to communicate

theirunderstanding—whenthepurportedexpertscanexplaintotheiraudiencesthecon-

nectionsbetweennodes,generatejustifications,andthelike.Butconsiderwhathappensto

theappearanceofcommunicativefacilityinsideabureaucratizedsystemofeducationalas-

sessment.Thoseuserswillingtoexpressthemselvesinthedesignatedtermsofthatsystem

haveaconsiderableadvantageindisplayingcommunicativefacility.Theycaneasilygenerate

justifications.Theycaneasilymaketheirreasonsandrequestsunderstoodandactedupon

ininstitutionalsettings.Theywillseemclearbecausetheircommunicationwillbereadily

takenupandactedupon.Theirapparentfacilitywillseemespeciallyimpressivetooutsiders,

whoareoutofcontactwiththesubtlervaluesinvolvedwitheducation.Thisis,obviously,a

18Iofferafullerdiscussionofhowsimplifiedandquantifiedsystemsofvaluecangivetheiradoptersthegame-likepleasuresofvalueclarityinNguyen(2020).

29

formofepistemicinjustice(Fricker2007).Here,itisaformofepistemicinjusticewhichgives

asignificantcredibilityadvantagetoanybodywillingtospeakinthetermsprovidedbybu-

reaucracies and institutions, which provide regularized systems of justification and lan-

guagesofevaluation.Andsincetheabilitytocreateanddisseminatesuchsystemsisusually

heldbythosealreadyinpower,thebureaucratizationoflanguagewilltypicallyservetoam-

plifypowerdifferentialsbygrantingmorecredibilitytothosewhoacceptthosebureaucratic

termsofdiscourse.

ToputitinKristieDotson’sterms,epistemicoppressionoccurswhenagentsaredenied

theopportunitytousesharedepistemicresourcestoparticipateinknowledgeproduction

(Dotson2014).Bureaucraticandinstitutionalizedlanguagecanenableaparticularkindof

epistemicoppression. Ideas thatcanbeeasilyexpressed in the institutional languageare

readilyenteredintothesharedknowledgebase.Butthestandardizationoflanguageputsa

special oppressive power in the hands ofwhoever creates the standardization.Once the

standardizationisinplaceandwidelyaccepted,anybodywhousesitwilldemonstratecog-

nitivefacilityanddemonstratecommunicativefacility.Theywillseemclearpreciselybecause

theyareusinglanguageforwhichasystemofreceptionhasbeenpre-prepared.

Thesenseofclarityisaterminatorforinquiry,andideasexpressedinthatregularized

institutionallanguagewillbearthatsenseofclarity.Soideasexpressedinthatlanguageare

morelikelytobeacceptedwithoutquestion.Informationthatisn’tplacedintoinstitutional

language,ontheotherhand,willtendtobedisappear.Suchrecalcitrantexpressionswillbe

lesslikelytobeaccepted,transmittedandrememberedwithinthesystem.Attheveryleast,

sincetheyseemconfusingratherthanclear,thoserecalcitrantexpressionswillbesubjectto

constantquestioningandinquiry,ratherthanquicklyaccepted.Inastandardizedsystem,

30

non-standardized informationwillbesubject to incredible friction.Thiscreatesa further

competitive disadvantage. By the very fact that such information transmits slowly and

poorly,theinformationanditsauthorswillseemtohavelesscommunicativefacilityandso

seemlesscredible.Thosewhoseideasdon’tfitcomfortablyintotheregularizedinstitutional

languageareatasignificantdisadvantageinparticipatingintheproductionanddissemina-

tionofknowledge.

Nuanceandclosure

Thepointhereisnottoclaimthatquantifiedsystemsandconspiracytheoriesarealways

bad.Scienceandbureaucracyneedquantification,andwecertainlyshouldacceptconspiracy

theorieswhenthereareactuallyconspiracies.19Thepointis,rather,thatthesesortsofideas

andmethodologiesareamongthechoicesttoolsforepistemicsubversion.Aruthlessepis-

temicmanipulator, freed from the constraintsof genuine inquiry, can re-formulate these

sortsofsystemstomaximizetheirpotentialforseductiveness.

Andthisalsooffersusinsightsintounintentionalcognitiveseduction.Bureaucraciesand

institutionshaveverygoodreasontodevelopinternallyconsistentandquantifiedsystems

ofevaluation.Suchsystemsmaketheadministrationofcomplexorganizationspossible.But

insofarassuchsystemsshareasignificantnumberofthetraitsandeffectsasthosesystems

19Thereisaveryusefuldiscussionoftheoccasionalusefulnessofconspiracytheoriesin(Coady2012,110-137;Dentith2018,2019).

31

madeforintentionalmanipulation—andespeciallyinsofarassuchsystemsperpetuatebe-

causeoftheirseductiveeffects—thensuchsystemsalsofunctionasseductivelyclear.

Thissuggestsanotherreasontoresisttheseductionsofclarity.Sometimes,weneedto

dwellinunclearsystemsofthoughtbecausewehavenotyetearnedtherighttoclarity.In

herstudyofmetaphors,ElizabethCamp(2006)suggeststhatmetaphorsaremostappropri-

atewhenwearestillintheprocessofcomingtounderstand.Metaphorsareunclearbyde-

sign.Theyare,saysCamp,aspecialwayofpointingtotheworld.Wedefinesimplenouns

throughsimpler formsofpointing. “Red”wedefineas looking like that.Metaphors letus

pointwitharough,wavinggesture.

Thereasonwemightwanttodoso,saysCamp,isthatsuchpointingletsusaccessthe

richnessoftheworldinourtalk.WhenIsay,“Idon’tunderstandwhat’sgoingonwithRobert

verymuch,buthisneurosisseemsalotlikeLiza’s,”I’mnotusingsomewell-definedabstract

predicatetodescribeRobert.IampointingtoLizaandtoalltherichfeaturesofrealitythat

areboundupwithher.IamsayingthatIdon’tknowwhatitisaboutLizathatmatters,ex-

actly,butit’ssomethingoverthere,where“there”isagestureinthedirectionofalltherich-

nessofLiza’sactualself.Andthissortofvaguegestureisespeciallyuseful,saysCamp,when

wearetryingtograpplewiththingswedonotyetadequatelyunderstand.Withmetaphors,

shesays,wearegesturingvaguelyatparttheworlds.

Intentionallyandopenlyvagueformsofcommunicationareveryimportant.Theyremind

usthatourthinking—ourconcepts,ourinquiries,ourunderstanding—isnotyetfinished.

Clarityiscompelling,butsignalsustoendourinquiries.Seductivelyclearsystemsmaskthe

factthatweshould,infact,beconfused,andshouldbepressingonwithourinquiries.They

32

presentthemselvesasfinalized.Ontheotherhand,metaphorsandtheirkinweartheirun-

finishednessplainlyontheirfaces.Theyarehardtouse,andthatdifficultyremindsusthat

thereismoreworktobedone.Theyleavethebasementdooropen,soweknowthereismore

toexploredownthere.Whenclarityseduces,itcanpreventusfrompushingon,fromfinding

anddwellingonourconfusions.Seductiveclaritypresentsuswithafalsefloorforourinves-

tigationsintotheworld.

Howdoweresisttheseductionsofclarity?Onepossibledefensivestrategyistodevelop

newcounter-heuristics,designedtosniffouttheseductivemanipulationofouroriginalheu-

ristics.Here’saroughanalogy:acertainkindofculinaryyumminesswasonceadecentheu-

risticfornutritiouseating.Butournutritiveenvironmentchanged,especiallywhenvarious

corporateforcesfiguredoutourheuristicsandtendenciesandstartedtoaggressivelygame

them.Inresponse,wehavehadtoadaptourheuristics.Wehaveneededtobecomesuspi-

ciousof toomuchyumminess.Manyofushave already trainedourselves tonoticewhen

thingsarejustalittletoodelicious.Thecrunchy,sweet,saltystuffthathitsusjustso—we

havelearnedtotasteinthemtheengineer’smanipulativetouch.Wehavedevelopedanin-

tuitivefeelfordesignedcraveability.Thisisacounter-heuristic,designedtotriggerinre-

sponsetosignalsthatoutsideforcesaretryingtomanipulateourmoreprimitiveheuristics.

Sweetness,crunchiness,saltiness—ourcounter-heuristicmakesasimmediatelysuspicious

whenwefindtheseinplenty.

Infightingtheseductionsofclarity,weneedtodevelopnewcounter-heuristicsinasim-

ilarkey.Thesenseofclarityissomethinglikecognitivesugar.Onceuponatime,usingour

senseofclarityasasignaltoterminateourinquiriesmighthavebeenagoodandusefulheu-

ristic.Butnowwe live inanenvironmentwherewearesurroundedbyseductiveclarity,

33

muchofitdesignedtoexploitourheuristics.Wenowneedtotrainourselvestobecomesus-

piciousofideasandsystemsthatgodownjustalittletoosweetly—thatarepleasurableand

effortless and explain everything so wonderfully. Systems of thought that feel too clear

shouldmakeusstepupourinvestigativeeffortsinsteadofendingthem.Weneedtolearnto

recognize,byfeel,theseductionsofclarity.20

Bibliography

Battaly,Heather.2018.“Closed-MindednessandDogmatism.”Episteme15(3):261–282.

Begby, Endre. 2020. “Evidential Preemption.” Philosophy and Phenomenological Research.

https://doi.org/10.1111/phpr.12654

Callahan,LauraFrances.2018.“MoralTestimony:ARe-ConceivedUnderstandingExplana-

tion.”PhilosophicalQuarterly68(272):437–459.

Camp,Elisabeth.2006.“MetaphorandThatCertain‘JeNeSaisQuoi.’”PhilosophicalStudies

129(1):1–25.

Cassam,Quassim.2016.“ViceEpistemology.”TheMonist99(2):159–180.

Chabris,Christopher,andDanielSimons.2011.TheInvisibleGorilla:HowOurIntuitionsDe-

ceiveUs.Reprintedition.Harmony.

20I’dliketothank,foralltheirhelpwiththispaper,AndrewBuskell,JoshDiPaolo,A.W.Eaton,CaitlinDolan,JonEllis,MelindaFagan,KerenGorodeisky,ArataHamakawa,RobHopkins,JennyJudge,SamanthaMatherne,JayMiller,StephaniePatridge,AntoniaPeacocke,GeoffPynn,NickRiggle,DavidSpurrett,MadelaineRansom,JonahSchupbach,TimSundell,andMattStrohl.

34

Coady,David.2012.WhattoBelieveNow:ApplyingEpistemologytoContemporaryIssues.Ho-

boken:Wiley-Blackwell.

Dallmann,Justin.2017.“WhenObstinacyIsaBetterPolicy.”Philosophers’Imprint17.

Dentith,MatthewR.X.2018.“TheProblemofConspiracism.”Argumenta3(2):327–343.

Dotson,Kristie.2014.“ConceptualizingEpistemicOppression.”SocialEpistemology28(2):

115–38.

Eaton,A.W.2020.“ArtifactsandTheirFunctions.”InOxfordHandbookofHistoryandMaterial

Culture. Edited by Ivan Gaskell and Sarah Anne Carter. Oxford: Oxford University

Press.

Elgin,CatherineZ.2002.“CreationasReconfiguration:ArtintheAdvancementofScience.”

InternationalStudiesinthePhilosophyofScience16(1):13–25.

Espeland,WendyNelsonandMichaelSauder.2016.EnginesofAnxiety:AcademicRankings,

Reputation,andAccountability.NewYork:RussellSageFoundation.

Fricker,Miranda.2007.EpistemicInjustice:PowerandtheEthicsofKnowing.ClarendonPress.

Gay,DrPauldu.2000.InPraiseofBureaucracy:Weber-Organization-Ethics.London:SAGE

PublicationsLtd.

Gigerenzer,Gerd,andDanielG.Goldstein.1996.“ReasoningtheFastandFrugalWay:Models

ofBoundedRationality.”PsychologicalReview103(4):650–69.

Gopnik,Alison.1998.“ExplanationasOrgasm.”MindsandMachines8(1):101–118.

Grimm,StephenR.2006.“IsUnderstandingaSpeciesofKnowledge?”BritishJournalforthe

PhilosophyofScience57(3):515–535.

———.2012.“TheValueofUnderstanding.”PhilosophyCompass7(2):103–117.

Jamieson,KathleenHall,andJosephCappella.2010.EchoChamber:RushLimbaughandthe

35

ConservativeMediaEstablishment.Oxford:OxfordUniversityPress.

Kahan,DanM.,andDonaldBraman.2006.“CulturalCognitionandPublicPolicy.”YaleLaw&

PolicyReview24:147–70.

Kahneman,Daniel.2013.Thinking,FastandSlow.1stedition.NewYork:Farrar,Strausand

Giroux.

Kelly,Thomas.2008.“Disagreement,Dogmatism,andBeliefPolarization.”JournalofPhiloso-

phy105(10):611–633.

Khalifa,Kareem.2012.“InauguratingUnderstandingorRepackagingExplanation?”Philoso-

phyofScience79(1):15–37.

Kvanvig,JonathanL.2003.TheValueofKnowledgeandthePursuitofUnderstanding.Cam-

bridge:CambridgeUniversityPress.

———.2011.“II—JonathanL.Kvanvig:MillarontheValueofKnowledge.”AristotelianSoci-

etySupplementaryVolume85(1):83–99.

Lupton,Deborah.2016.TheQuantifiedSelf.Cambridge:Polity.

Lynch,Michael.2018.“UnderstandingandComingtoUnderstand.” InMakingSenseof the

World:NewEssaysonthePhilosophyofUnderstanding,editedbyStephenGrimm,194–

208.Oxford:OxfordUniversityPress.

McGonigal, Jane. 2011.Reality Is Broken:Why GamesMake Us Better and How They Can

ChangetheWorld.NewYork:PenguinBooks.

Medina,Jose.2012.TheEpistemologyofResistance:GenderandRacialOppression,Epistemic

Injustice,andResistantImaginations.Oxford,NewYork:OxfordUniversityPress.

Merry,SallyEngle.2016.TheSeductionsofQuantification:MeasuringHumanRights,Gender

Violence,andSexTrafficking.Chicago:UniversityofChicagoPress.

36

Millar,Alan.2011.“WhyKnowledgeMatters.”AristotelianSocietySupplementaryVolume85

(1):63–81.

Millgram,Elijah.1997.PracticalInduction.Cambridge,Mass.:HarvardUniversityPress.

———.2004.“OnBeingBoredOutofYourMind.”ProceedingsoftheAristotelianSociety104

(2):163–184.

Millikan,RuthGarrett.1984.“Language,ThoughtandOtherBiologicalCategories:NewFoun-

dationsforRealism.”PhilosophyofScience52(3):477–478.

Nguyen,C.Thi.2018a.“CognitiveIslandsandRunawayEchoChambers:ProblemsforEpis-

temic Dependence on Experts.” Synthese. https://doi.org/10.1007/s11229-018-

1692-0.

———. 2018b. “Echo Chambers and Epistemic Bubbles.” Episteme.

https://doi.org/10.1017/epi.2018.32

———.2018c.“ExpertiseandtheFragmentationofIntellectualAutonomy.”PhilosophicalIn-

quiries6(2):107-124.

———.2020.Games:AgencyasArt.NewYork:OxfordUniversityPress.

———.Forthcoming.“HowTwittergamifiescommunication.”.AppliedEpistemology,ed.Jen-

niferLackey.NewYork:OxfordUniversityPress.

Oppenheimer,DanielM.2008.“TheSecretLifeofFluency.”TrendsinCognitiveSciences12

(6):237–41.

Perrow,Charles.2014.ComplexOrganizations:ACriticalEssay.Thirdedition.Brattleboro,

Vermont:EchoPointBooks&Media.

Porter,Theodore.1996.TrustinNumbers.Princeton:PrincetonUniversityPress.

Proctor,Robert,andLondaL.Schiebinger.2008.Agnotology:TheMakingandUnmakingof

37

Ignorance.StanfordUniversityPress.

Reber,Rolf,andChristianUnkelbach.2010.“TheEpistemicStatusofProcessingFluencyas

SourceforJudgmentsofTruth.”ReviewofPhilosophyandPsychology1(4):563–581.

Regt,HenkW.de.2009.“TheEpistemicValueofUnderstanding.”PhilosophyofScience76(5):

585–597.

Rini,Regina.2017.“FakeNewsandPartisanEpistemology.”KennedyInstituteofEthicsJour-

nal27(S2):43–64.h

Scott,JamesC.1998.SeeingLikeaState:HowCertainSchemestoImprovetheHumanCondi-

tionHaveFailed.NewHaven:YaleUniversityPress.

Simon,Herbert.1956.“RationalChoiceandtheStructureoftheEnvironment.”Psychological

Review63(2):129–38.

Stanley,Jason.2016.HowPropagandaWorks.Princeton:PrincetonUniversityPress.

Strevens,Michael.2013.“NoUnderstandingWithoutExplanation.”StudiesinHistoryandPhi-

losophyofSciencePartA44(3):510–515.

Sullivan, Shannon, and Nancy Tuana. 2007. Race and Epistemologies of Ignorance. SUNY

Press.

Sunstein,Cass.2017.#Republic.Princeton:PrincetonUniversityPress.

Trout,J.D.2002.“ScientificExplanationandtheSenseofUnderstanding.”PhilosophyofSci-

ence69(2):212–233.

———.2017.“UnderstandingandFluency”.InMakingSenseoftheWorld:NewEssaysonthe

Philosophy of Understanding. Edited by Stephen Grimm. Oxford: Oxford University

Press.

Wilkenfeld,DanielA.2013.“UnderstandingasRepresentationManipulability.”Synthese190

38

(6):997–1016.

———.2017.“MUDdyUnderstanding.”Synthese194(4):1273–1293.

Wimsatt,WilliamC.2007.Re-EngineeringPhilosophyforLimitedBeings:PiecewiseApproxi-

mationstoReality.Cambridge,Mass:HarvardUniversityPress.

Zagzebski,Linda.2001.“RecoveringUnderstanding.”InKnowledge,Truth,andDuty:Essays

onEpistemicJustification,Responsibility,andVirtue,editedbyM.Steup.Oxford:Oxford

UniversityPress.

Zimmerman,Eric,IanBogost,ConorLinehan,BenKirman,BryanRoche,MarkPesce,Scott

Rigby,etal.2015.TheGamefulWorld:Approaches,Issues,Applications.EditedbyStef-

fenP.WalzandSebastianDeterding.Cambridge,Mass.:TheMITPress.