extending act-r with jsegmanto perform a long task ...acs.ist.psu.edu/papers/ritterts20.pdf · the...
TRANSCRIPT
ExtendingACT-RwithJSegMan toPerformaLongTask:ImplicationsforPerception,MotorOutput,Fatigue,and
MotivationDavidM.Schwartz,Farnaz Tehranchi,andFrankE.Ritter
([email protected],[email protected],[email protected])6/4/2020
SlidespresentedatVISCA2020.
ACT-R
• Cognitivearchitecture• Modulestoimplementthefixedmechanismsofcognition
2
Core
Productions,if-thenrules,
actionable memory
Chunks,factualmemory,slot-valuepairs
Imagefrom:Ritter, F.E.,Tehranchi, F.,&Oury,J.D.(2019).ACT-R:Acognitive architecture formodeling cognition.WileyInterdisciplinaryReviews:CognitiveScience, 10(3), Papere1488.http://acs.ist.psu.edu/papers/ritterTOip.pdf
SegMan
• Sensors(eyes)andeffectors(hands)foragents• Capturesthescreenasanimageandsegmentsitintogroups
• Groupscanbecombinedtocaptureobjectsontheinterface
• Cansimulateuserinput• Keypresses,mousemovements,andclicks
• WorkedforWindows95/98/2000/XP• Becamehardtomaintain,onlyforWindows
3
Imagesfrom:St.Amant,R.,Riedl, M.O.,Ritter, F.E.,&Reifers,A.(2005).Imageprocessing incognitive modelswith SegMan.InProceedings ofHCIInternational '05, Volume4- TheoriesModelsandProcesses inHCI.Paper#1869. Mahwah,NJ:Erlbaum.http://acs.ist.psu.edu/papers/stamantRRR05.pdf
JSegMan
4
BiasedCoinTaskDismalTask ExcelTask
Visual patterns are small images that represent visual objects in cognitive architectures¾visual objects in ACT-R.
Headbutton Tail button
JSegMan
5
https://sites.psu.edu/ftehranchi/projects/& Tehranchi, F., & Ritter, F. E. (2018). Modeling visual search in interactive graphic interfaces: Adding visual
pattern matching algorithms to ACT-R. In Proceedings of 16th international conference on cognitive modeling (pp. 162-167). Madison, WI: University of Wisconsin. http://acs.ist.psu.edu/papers/tehranchiR18a.pdf
& Tehranchi, F., & Ritter, F. E. (2018). Using Java to provide cognitive models with a universal way to interact with graphic interfaces. In Proceedings of the International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Washington DC, USA. http://acs.ist.psu.edu/papers/tehranchiR18b.pdf
Task• DesertBus• DrivefromTucson,AZtoLasVegas,NV• Cangoamaximumof45mph• ”w”togoforward,“a”togoleft• Gamerunsinreal-time
• Nopausing,buspullsrightandslowsdown
6
Original
Free remakebyDinosaurGames
URLhere
Model
• Simplemodel(5DMs,7rules)• Exposesissueswiththearchitecture
• Need:Abilitytoholddownkeys• Need:Parallelmotoractions
• Theseissuesprevent• Stayingattopspeed• Acceleratingwhilesteering/realigning
• Issueslookingaroundtheenvironment• Relatedtosensitivity
7
Demonstration
8
Demonstration (zoomedin)
9
Discussion
• Perception• UseJSegMan todetectvisualtargetsintheapplication
• Limitationsofmotoroutput• Nosupportforholdingdownkeysnorparallelmotoroutput• Issueswithfirstpersongames
• Fatigue• Modeldrivesendlessly,butpeoplecannot—whatroledoesfatigueplay?
• Motivation• Howdoesaplayermanagetrade-offofplayingversusotherresponsibilities?
10
Conclusions
• Canbethelongestrunningcognitivemodel• Interactswiththetaskinterface
• Discoversissuesmodelinginteractioninfirstpersongames
• Exposesissueswithmodelingvisionandmotoroutput• Canbeusedtoexploreeffectsoffatigueandmotivation
11
(ImagefromtheSteampage forDesertBusVR)
References&AcknowledgmentsAnderson,J.R.(2007).Howcanthehumanmindexistinthephysicaluniverse?NewYork,NY:OxfordUniversityPress.Anderson,J.R.,Bothell,D.,Byrne,M.D.,Douglass,S.,Lebiere,C.,&Qin,Y.(2004).Anintegratedtheoryofthemind.PsychologicalReview,111(4),1036-1060.Ritter,F.E.,Tehranchi,F.,&Oury,J.D.(2019).ACT-R:Acognitivearchitectureformodelingcognition.WileyInterdisciplinaryReviews:CognitiveScience,10(3),Papere1488.Tehranchi,F. (forthcoming). AnEyesandHandsModel:ExtendingVisualandMotorModulesforCognitiveArchitectures,DepartmentofComputerScienceandEngineering, ThePennsylvaniaStateUniversity.Tehranchi,F.,&Ritter,F.E.(2018).Modelingvisualsearchininteractivegraphicinterfaces:AddingvisualpatternmatchingalgorithmstoACT-R.InProceedingsof16thinternationalconferenceoncognitivemodeling(pp.162-167).Madison,WI:UniversityofWisconsin.Tehranchi,F.,&Ritter,F.E.(2018).UsingJavatoprovidecognitivemodelswithauniversalwaytointeractwithgraphicinterfaces.InTheProceedingsoftheInternationalConferenceonSocialComputing,Behavioral-CulturalModelingandPredictionandBehaviorRepresentationinModelingandSimulation.WashingtonDC,USA.Schwartz,D.M.,Tehranchi,F.,Ritter,F.E.(underreview)<DesertBuspaper>.St.Amant,R.,Riedl,M.O.,Ritter,F.E.,&Reifers,A.(2005).ImageprocessingincognitivemodelswithSegMan.InProceedingsofHCIInternational'05,Volume4- TheoriesModelsandProcessesinHCI.Paper#1869. Mahwah,NJ:Erlbaum.
SupportforthisworkwasprovidedbyONRgrantsN00014-03-1-0248andN00014-18-C-7015,andaSMARTfellowshiptothefirstauthorfromAFRL/Rome.
12StopTheSpread.health