extending act-r with jsegmanto perform a long task ...acs.ist.psu.edu/papers/ritterts20.pdf · the...

ExtendingACT-RwithJSegMan toPerformaLongTask:ImplicationsforPerception,MotorOutput,Fatigue,and

MotivationDavidM.Schwartz,Farnaz Tehranchi,andFrankE.Ritter

(dms7225@psu.edu,farnaz.tehranchi@psu.edu,frank.ritter@psu.edu)6/4/2020

SlidespresentedatVISCA2020.

ACT-R

• Cognitivearchitecture• Modulestoimplementthefixedmechanismsofcognition

2

Core

Productions,if-thenrules,

actionable memory

Chunks,factualmemory,slot-valuepairs

Imagefrom:Ritter, F.E.,Tehranchi, F.,&Oury,J.D.(2019).ACT-R:Acognitive architecture formodeling cognition.WileyInterdisciplinaryReviews:CognitiveScience, 10(3), Papere1488.http://acs.ist.psu.edu/papers/ritterTOip.pdf

SegMan

• Sensors(eyes)andeffectors(hands)foragents• Capturesthescreenasanimageandsegmentsitintogroups

• Groupscanbecombinedtocaptureobjectsontheinterface

• Cansimulateuserinput• Keypresses,mousemovements,andclicks

• WorkedforWindows95/98/2000/XP• Becamehardtomaintain,onlyforWindows

3

Imagesfrom:St.Amant,R.,Riedl, M.O.,Ritter, F.E.,&Reifers,A.(2005).Imageprocessing incognitive modelswith SegMan.InProceedings ofHCIInternational '05, Volume4- TheoriesModelsandProcesses inHCI.Paper#1869. Mahwah,NJ:Erlbaum.http://acs.ist.psu.edu/papers/stamantRRR05.pdf

JSegMan

4

BiasedCoinTaskDismalTask ExcelTask

Visual patterns are small images that represent visual objects in cognitive architectures¾visual objects in ACT-R.

Headbutton Tail button

JSegMan

5

https://sites.psu.edu/ftehranchi/projects/& Tehranchi, F., & Ritter, F. E. (2018). Modeling visual search in interactive graphic interfaces: Adding visual

pattern matching algorithms to ACT-R. In Proceedings of 16th international conference on cognitive modeling (pp. 162-167). Madison, WI: University of Wisconsin. http://acs.ist.psu.edu/papers/tehranchiR18a.pdf

& Tehranchi, F., & Ritter, F. E. (2018). Using Java to provide cognitive models with a universal way to interact with graphic interfaces. In Proceedings of the International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Washington DC, USA. http://acs.ist.psu.edu/papers/tehranchiR18b.pdf

Task• DesertBus• DrivefromTucson,AZtoLasVegas,NV• Cangoamaximumof45mph• ”w”togoforward,“a”togoleft• Gamerunsinreal-time

• Nopausing,buspullsrightandslowsdown

6

Original

Free remakebyDinosaurGames

URLhere

Model

• Simplemodel(5DMs,7rules)• Exposesissueswiththearchitecture

• Need:Abilitytoholddownkeys• Need:Parallelmotoractions

• Theseissuesprevent• Stayingattopspeed• Acceleratingwhilesteering/realigning

• Issueslookingaroundtheenvironment• Relatedtosensitivity

7

Demonstration

8

Demonstration (zoomedin)

9

Discussion

• Perception• UseJSegMan todetectvisualtargetsintheapplication

• Limitationsofmotoroutput• Nosupportforholdingdownkeysnorparallelmotoroutput• Issueswithfirstpersongames

• Fatigue• Modeldrivesendlessly,butpeoplecannot—whatroledoesfatigueplay?

• Motivation• Howdoesaplayermanagetrade-offofplayingversusotherresponsibilities?

10

Conclusions

• Canbethelongestrunningcognitivemodel• Interactswiththetaskinterface

• Discoversissuesmodelinginteractioninfirstpersongames

• Exposesissueswithmodelingvisionandmotoroutput• Canbeusedtoexploreeffectsoffatigueandmotivation

11

(ImagefromtheSteampage forDesertBusVR)

References&AcknowledgmentsAnderson,J.R.(2007).Howcanthehumanmindexistinthephysicaluniverse?NewYork,NY:OxfordUniversityPress.Anderson,J.R.,Bothell,D.,Byrne,M.D.,Douglass,S.,Lebiere,C.,&Qin,Y.(2004).Anintegratedtheoryofthemind.PsychologicalReview,111(4),1036-1060.Ritter,F.E.,Tehranchi,F.,&Oury,J.D.(2019).ACT-R:Acognitivearchitectureformodelingcognition.WileyInterdisciplinaryReviews:CognitiveScience,10(3),Papere1488.Tehranchi,F. (forthcoming). AnEyesandHandsModel:ExtendingVisualandMotorModulesforCognitiveArchitectures,DepartmentofComputerScienceandEngineering, ThePennsylvaniaStateUniversity.Tehranchi,F.,&Ritter,F.E.(2018).Modelingvisualsearchininteractivegraphicinterfaces:AddingvisualpatternmatchingalgorithmstoACT-R.InProceedingsof16thinternationalconferenceoncognitivemodeling(pp.162-167).Madison,WI:UniversityofWisconsin.Tehranchi,F.,&Ritter,F.E.(2018).UsingJavatoprovidecognitivemodelswithauniversalwaytointeractwithgraphicinterfaces.InTheProceedingsoftheInternationalConferenceonSocialComputing,Behavioral-CulturalModelingandPredictionandBehaviorRepresentationinModelingandSimulation.WashingtonDC,USA.Schwartz,D.M.,Tehranchi,F.,Ritter,F.E.(underreview)<DesertBuspaper>.St.Amant,R.,Riedl,M.O.,Ritter,F.E.,&Reifers,A.(2005).ImageprocessingincognitivemodelswithSegMan.InProceedingsofHCIInternational'05,Volume4- TheoriesModelsandProcessesinHCI.Paper#1869. Mahwah,NJ:Erlbaum.

SupportforthisworkwasprovidedbyONRgrantsN00014-03-1-0248andN00014-18-C-7015,andaSMARTfellowshiptothefirstauthorfromAFRL/Rome.

12StopTheSpread.health