joschabach implementing emotion and motivation in ai

Joscha Bach

ImplementingEmotionandMotivationinAIArchitectures

TutorialAAAI2018

FromComputationtoEmotion

Howcanacomputationalsystemexperienceemotion?

Whatisemotion?Whatisexperience?Relationshipbetweenexperiencerandexperience?

AAAI 2018 Emotion and Motivation Tutorial 2

Emotionalstatesascognitiveconfigurations

Emotionsascognitiveconfigurations

Overview

• AGIperspectiveonminds• Basicarchitecturalcomponents• Modelingmotivation(MicroPsi model)• Modelsofemotion• EmotioninthePsitheory• Modelingpersonality• Emotion,selfandprosociality

BasicperspectiveongeneralAI

• Mindasmachine

• Machine=computationalsystem• computation=regularstatechange

• universalcomputation:setofcomputablefunctionsthatcancomputeallcomputablefunctions(whengivenunboundedresources)

• Accesstouniverseviadiscernibledifferences® Information

• Meaningofinformation:relationshiptochangesinotherinformation

• Intelligence:abilitytomodel• Modelingisfunctionapproximation• Purposeofmodelingisregulation,tomaximizerewards

• ClassicalAI:directmodelingofcognitivefunctionality® “firstorderAI”

• DeepLearning:systemsthatmodelfunctionalitythemselves;compositionalfunctionapproximation® “secondorderAI”

• MetaLearning:systemsthatlearnhowtobuildlearningsystems® “thirdorderAI”

BasicperspectiveongeneralAI:openquestions

• Arehumansmetalearningsystems?• Isevolutiona(slowandineffective)searchformetalearners?

• Isthereaclassofuniversalfunctionapproximatorsthatcanapproximateanyfunctionthatcanbeapproximatedbyacomputer(whengivenunboundedresources),anddoesitcontainitselfandus?

AIperspectiveonthemind

• Constructedarchitectures(Minsky,Simon,Newell)— classicalcognitivearchitectures

• Generatedarchitectures(Schmidhuber,Hutter,…):generalrecursivefunctionapproximation+rewardsystem

• Hybridperspective:mostlygenerated,butwithcomplexprerequisitesandbiases

LayersofCognition

Reflective

Deliberative

Reactive

ColumnsofCognition

Perception CognitiveProcessing Action

CognitiveGrid

ReflexivePerception Meta-

ManagementManagement

Action

DeliberativePerception

Planning,Reasoning

DeliberativeAction

ReactivePerception Reflexes Reflexive

Action

ConceptualAnalysis:HCogAff(Sloman2001)

Emotion and Motivation TutorialAAAI 2018 15

Artificial GeneralIntelligence

Methodsshouldfocusoncomponentsandperformancesnecessaryforintelligence:

• Whole,testable architectures• UniversalRepresentations:

Dynamicmodel of environment,possible worlds,and agent• (Semi-)UniversalProblemSolving:

Learning,Planning,Reasoning,Analogies,ActionControl,Reflection ...

• UniversalMotivation:Polythematic,adaptivegoal identification

• Emotionand affect

• Universalmentalrepresentations(compositional +distributedà neurosymbolic)

Componentsfor Cognitive AI

• (Semi-)Generalproblem solving:Operations over theserepresentations

(neural learning,categorization,planning,reflection,consolidation,...)

Op3 Op4Op2Op1 Opn...

• Perceptual grounding

Perception

• Perceptual grounding and action

Perception Action

• Perceptual grounding and action

Perception Action

• Modelof current situation,and protocol of past situations

Perception Action

Situationmodel

Protocol memory

• Modelof self

Perception Action

Situationmodel

Protocol memory

Self model

• Abstractions of objects,episodes and types

Perception Action

Self model

Declarative memory

Procedural memory Frame

• Anticipation of future developments

Perception Action

Self model

Declarative memory

Procedural memory Frame Expect.

• Actionselection and executive control

Perception Action

Longterm memory Worldmodel

Mentalstage

• Actionselection and executive control

Perception Action

Mentalstage

Action selection

• Universalmotivation:autonomous identification of goals

Perception Action

Mentalstage

Action selection

Motivational system:Motive Selection

and Decision making

• Emotionalmodulation and affect

Perception Action

Memory

Action selection

and Decision making

Urges/drives

Resolution

Arousal

Selectionthreshold

Securing rate

CognitiveArchitectures

SoarACT-R

“ClassicalCognitiveArchitectures” tendtofocusoncognitionasanisolatedproblemsolvingcapability.

AAAI 2018 Emotion and Motivation Tutorial

Perception Action

Memory

Action selection

and Decision making

Urges/drives

Resolution

Arousal

Selectionthreshold

Securing rate

MicroPsi architecture

PSItheoryPrinciples of Synthetic Intelligence

(Dörner1999;Bach2003,2009)

MicroPsiArchitecture–simplified

Short TermMemory/

Local PerceptualSpace

Long Term Memory(LTM)

Body Parameters

MicroPSI Agent

UrgeSensor PerceptSensorSensors/

Modulators

MemoryMaintenance

Behaviour Script Space /Execution Space

Motivation Execution

Internal Behaviours

Perception

Meta-Management

ExternalBehaviors

Action

AgentFunctionality

• Episodiclearning• Goal-directedbehavior,motivationalsystem• Emotionalmodulation• Hypothesisbasedperception• Simpleplanning• Executionofhierarchicalplans

AAAI 2018 35

Implementation:MicroPsi (Bach03,05,04,06)

Node Net Editor

Net Simulator/Agent

Execution

World Editor

Monitoring ConsoleApplication

World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Emotion and Motivation Tutorial

AAAI 2018 36

Node Net Editor

Net Simulator/Agent

Execution

World Editor

World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Implementation:MicroPsi(Bach03,05,04,06)

Low-levelperception

AAAI 2018 37

Node Net Editor

Net Simulator/Agent

Execution

World Editor

World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Low-levelperception

ControlandsimulationEmotion and Motivation Tutorial

AAAI 2018 38

Node Net Editor

Net Simulator/Agent

Execution

World Editor

World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Low-levelperception

ControlandsimulationMulti-agentinteractionEmotion and Motivation Tutorial

Node Net Editor

Net Simulator/Agent

Execution

World Editor

World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Low-levelperception

ControlandsimulationMulti-agentinteraction

Robotcontrol

Differentsimulationenvironments

AAAI 2018 41

Implementation:MicroPsi 2(Bach,Welland,Vuine,Herger12,14,…)

Cognitive Artificial Intelligence

• UniversalRepresentations:Dynamicmodel of environment,possible worlds,and agent

• (Semi-)UniversalProblemSolving:Learning,Planning,Reasoning,Analogies,ActionControl,Reflection ...

• Emotionand affect• Whole,testable architectures

ModelingMotivation

ModelingMotivationinaCognitiveArchitecture

• GeneralintelligenceneedsGeneralMotivation• Motivationalsystemstructurescognition• Motivationaldynamics:physiological,socialandcognitivedrives

• Intentionselectionandactioncontrol• Motivationvs.affect

Motivationvs.emotion

• Motivation:– reflectsneeds– givesrisetogoalsanddirectedbehavior– doesnothavetobeassociatedwithemotions

• Emotion:– modulatesperception,cognition,action– receivesvalencefrommotivation– receivesobjectsfromcognition– leadstoaffectiveexpression

MotivationalSystem

• Allgoals attempt to satisfy a(hard-wired)demandà flexiblegoals,but(evolutionary)suitable behavior

Intention

Motive selection

Urges/drives

liatio

MotivationalSystem

• Drivescorrespond to set of demands of the agent

Physiological

liatio

Social Cognitive

PhysiologicalDrives

• if autonomous regulation of body processes failsà actively managephysiology (seek food,water,healing,shelter,rest,warmth,...)à escape perilious situationsà implicitly seek physical survival

Social Drives

• Affiliation:structuresocialinteractionbeyondrationalutility

• increasedby‘legitimacysignals’,decreasedby‘antilegitimacysignals’ (andadaptivelyovertime);allowsfornon-materialrewardandpunishment

• external legitimacy:groupacceptance• internal legitimacy:“honor”,conformancetointernalizedsocialnorms

Cognitive Drives

• Competence– epistemic (problem specific)– general (ability to satisfy demands)– effect oriented

• Uncertainty reduction– novelty seeking

• Aesthetics– evolutionary preferences (stimulus oriented)– abstract (representation oriented)

MotivationalSystem

Allpossible goals correspond to (at leastone)demand

Physiological

liatio

Social Cognitive

FromNeedstoBehavior

NeedsUrge

Signals

Memory

Perception

Action

Priming + Modulation

Learning

Decision Making

Rewardsignals

Pleasureanddistress:– Change ofademandisreflectedinpleasureor distresssignal

– Strengthisproportional toamountofchange

– Pleasureanddistresssignalsdeliverreinforcementvaluesforbehavioralproceduresandepisodicsequencesanddefineappetitive andaversive goals.

• drive =demand +urge indicator

MotivationalSystem

ertarget

currentlevel urge indicator

swater

• motive =urge +goal situation

MotivationalLearning

ertarget

swater

• motive =urge +goal situation

ertarget

swater

goal situation

aversive situation

• Goal:situationoractionthataffordstosatisfyaneed

• Aversivegoal:situationoractionthatfrustrateaneed

• Allbehaviorisdirectedonsatisfyinganappetitivegoaloravoidinganaversivegoal

• Needsarepredefined,goalsarelearned

Physiologicalneeds

• Thirst• Hunger• Rest• Warmth• Libido• …

à Survivalasemergentproperty

Socialneeds

• Affiliation(Attentionfromothers,externallegitimacy)

• Internallegitimacy• Nurturing(caringforothers)• Affection• Dominance

Cognitiveneeds

• Competence:– Skillacquisition(epistemiccompetence)– Coping/controlability(generalcompetence)– Effectgeneration

• Uncertaintyreduction:– Exploration

• Aesthetics:– Stimulusoriented– Structureoriented(abstractaesthetics)

Needsandurges

• association by learning:

urgeindicator

si goalsituation

aversivesituation

changeindicator

valence associator

demand

• retrogradient reinforcement

urgeindicator

changeindicator

valence associator

demand

protocol chaingoal

Motivator:

valence associator

autonomousregulation

situations leading up to goal =plan

Intention:

autonomousregulation

goal 1

goal 2

goal 3

goal n

Motivationallearning

Motiveselection

Needparameters

• Strength:relativeimportance• Decay:rateofreplenishment• Gain:effectofsatisfaction• Loss:effectoffrustration

• differentconfigurationofneedparameters=differentpersonalitytraits

Anticipatedneeds

• Actualrewarddeterminesreinforcement(striatum,basalganglia)

• Anticipatedrewarddeterminesaction(prefrontaldopamine)

• Valenced reactionsarenotonlycausedbypresentrewards,butalsobyimagined/anticipatedrewards(expectations,memories)

Implementationofmotivation

𝑁𝑒𝑒𝑑 ≔

𝑡𝑦𝑝𝑒 ∈ 𝑝ℎ𝑦𝑠𝑖𝑜𝑙𝑜𝑔𝑖𝑐𝑎𝑙, 𝑠𝑜𝑐𝑖𝑎𝑙, 𝑐𝑜𝑔𝑛𝑖𝑡𝑖𝑣𝑒 ,𝑣𝑎𝑙𝑢𝑒𝑡 ∈ 0, 1 ,𝑣𝑎𝑙𝑢𝑒𝑡0 ∈ 0, 1 ,𝑢𝑟𝑔𝑒𝑡 ∈ 0, 1 ,

𝑢𝑟𝑔𝑒𝑛𝑐𝑦𝑡 ∈ 0, 1 ,𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑡 ∈ 0, 1 ,𝑝𝑎𝑖𝑛𝑡 ∈ 0, 1 ,𝑤𝑒𝑖𝑔ℎ𝑡 ∈ ℝ+,

𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ −1, 1 ,𝑔𝑎𝑖𝑛 ∈ 0, 1 ,𝑙𝑜𝑠𝑠 ∈ 0, 1 ,

𝑠𝑎𝑡𝑖𝑠𝑓𝑎𝑐𝑡𝑖𝑜𝑛𝑖𝑚𝑔 ∈ 0, 1 ,𝑓𝑟𝑢𝑠𝑡𝑟𝑎𝑡𝑖𝑜𝑛𝑖𝑚𝑔 ∈ 0, 1 ,

𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 ∈ ℝ+,𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 ∈ ℝ+,

𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ ℝ+,𝑝𝑎𝑖𝑛𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ ℝ+,

𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦𝑖𝑚𝑔 ∈ 0, 1 ,𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦𝑖𝑚𝑔 ∈ 0, 1

Urgestrengthandurgency

• 𝑢𝑟𝑔𝑒𝑡 = 𝑤𝑒𝑖𝑔ℎ𝑡 1 − 𝑣𝑎𝑙𝑢𝑒𝑡−1 012

• 𝑢𝑟𝑔𝑒𝑛𝑐𝑦𝑡 = 𝑤𝑒𝑖𝑔ℎ𝑡 𝑘−𝑟𝑒𝑚𝑎𝑖𝑛𝑖𝑛𝑔𝑡𝑖𝑚𝑒𝑡𝑘 0

Valueanddecayofaneed

• 𝑣𝑎𝑙𝑢𝑒𝑡 =

decay 𝑣𝑎𝑙𝑢𝑒𝑡−1+𝑔𝑎𝑖𝑛×𝛿𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑒

+𝑙𝑜𝑠𝑠×𝛿𝑡𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

+𝑔𝑎𝑖𝑛×𝑠𝑎𝑡𝑖𝑠𝑓𝑎𝑐𝑡𝑖𝑜𝑛𝑖𝑚𝑔𝛿𝑡𝑖𝑚𝑔𝑐𝑜𝑛𝑠𝑢𝑚𝑒

+𝑙𝑜𝑠𝑠×𝑓𝑟𝑢𝑠𝑡𝑟𝑎𝑡𝑖𝑜𝑛𝑖𝑚𝑔𝛿𝑡𝑖𝑚𝑔𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

• decay 𝑣𝑡−1 ≔ 𝜎 𝜎−1 𝑣𝑡−1 + duration 𝑡,𝑡−1𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒

Pleasureandpainassociatedwithaneed

• 𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑡 =decay 𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑡−1, 𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒+𝑔𝑎𝑖𝑛×𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦×𝛿𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑒

+𝑔𝑎𝑖𝑛×𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠.𝑖𝑚𝑔×𝛿𝑡𝑖𝑚𝑔𝑐𝑜𝑛𝑠𝑢𝑚𝑒

• 𝑝𝑎𝑖𝑛𝑡 =

decay 𝑝𝑎𝑖𝑛𝑡−1, 𝑝𝑎𝑖𝑛𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒+𝑙𝑜𝑠𝑠×𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦×𝛿𝑡𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

+𝑙𝑜𝑠𝑠×𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦𝑖𝑚𝑔×𝛿𝑡𝑖𝑚𝑔𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

+painfromdepletion(value𝑡−1) 0

• painfromdepletion 𝑣𝑎𝑙𝑢𝑒 ≔ 1 − 𝑣𝑎𝑙𝑢𝑒𝜃 0

Eventsandconsumptions

• 𝐸𝑣𝑒𝑛𝑡 ≔

𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 ∈ 𝐶𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛𝑠,𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑𝑟𝑒𝑤𝑎𝑟𝑑 ∈ −1, 1 ,

𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦 ∈ 0, 1 ,𝑠𝑘𝑖𝑙𝑙 ∈ 0, 1 ,

𝑟𝑒𝑚𝑎𝑖𝑛𝑖𝑛𝑔𝑡𝑖𝑚𝑒𝑡 ∈ ℝ+

• 𝐶𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 ≔

𝑛𝑒𝑒𝑑 ∈ 𝑁𝑒𝑒𝑑𝑠,𝑟𝑒𝑤𝑎𝑟𝑑𝑡 ∈ ℝ,

𝑡𝑜𝑡𝑎𝑙𝑟𝑒𝑤𝑎𝑟𝑑 ∈ ℝ,𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 ∈ ℝ+,

𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑 ∈ ℝ,𝑑𝑖𝑠𝑐𝑜𝑢𝑛𝑡 ∈ 0, 1

Rewardsignalandrewardsummation

• 𝑠𝑖𝑔𝑛𝑎𝑙 𝑡 : = 𝑡𝑒−12𝑡2

• 𝑡1 = 𝑡 − 𝑡𝑜𝑛𝑠𝑒𝑡 ]

𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛duration(t, t − 1)

• 𝑡1 = 𝑡 − 𝑡𝑜𝑛𝑠𝑒𝑡 ]

𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛duration(t, t − 1)

• 𝑟𝑒𝑤𝑎𝑟𝑑𝑡 =]×𝑡𝑜𝑡𝑎𝑙𝑟𝑒𝑤𝑎𝑟𝑑𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 ∫ 𝑡𝑒−

12𝑡2𝑡2

𝑡1 −𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

= ]×𝑡𝑜𝑡𝑎𝑙𝑟𝑒𝑤𝑎𝑟𝑑𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛

𝑒−12𝑡12− 𝑒−

12𝑡22

−𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

Actualizedrewardschangevaluesofneeds

• 𝛿𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑒 = 𝑟𝑒𝑤𝑎𝑟𝑑𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

• 𝛿𝑡𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒 = 𝑟𝑒𝑤𝑎𝑟𝑑𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

−∞

Dealingwithanticipatedrewards

• 𝛿𝑡𝑖𝑚𝑔𝑐𝑜𝑛𝑠𝑢𝑚𝑒 = 𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦×𝑠𝑘𝑖𝑙𝑙× 𝑟𝑒𝑤𝑎𝑟𝑑𝑡

𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

1+𝑑×𝑟𝑒𝑚𝑎𝑖𝑛.𝑡𝑖𝑚𝑒𝑡 0

• 𝛿𝑡𝑖𝑚𝑔𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒 = 𝑐𝑒𝑟𝑡.×(1 − 𝑠𝑘𝑖𝑙𝑙)× 𝑟𝑒𝑤𝑎𝑟𝑑𝑡

𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

1+𝑑×𝑟𝑒𝑚𝑎𝑖𝑛.𝑡𝑖𝑚𝑒𝑡 −∞

(d isdiscount;hyperbolicdiscounting)

Cognitive Artificial Intelligence

• UniversalRepresentations:Dynamicmodel of environment,possible worlds,and agent

• (Semi-)UniversalProblemSolving:Learning,Planning,Reasoning,Analogies,ActionControl,Reflection ...

• Emotionand affect• Whole,testable architectures

ModelingEmotion

Emotionalexpression

• PaulEkman:FacialActionCoding

Emotionalexpression

Affectivecomputing

Recognize,process,simulate,expresshumanaffects

Appraisalmodels

• MagdaArnold,RichardLazarus:– Emotionascognitiveappraisalsofrelations,motivation,cognition

• KlausScherer:– Stimulusevaluationchecks– innate(sensorymotor)® learned(schemas)® deliberate– relevance® implication® copingpotential® normativesignific.

Conceptualanalysis:Ortony,Clore,Collins1988:

AffectivedimensionsinthePSItheory(Dörner1999)

Valence(appentence/aversion)

Arousal(unspecificsympathicussyndrome)

Selectionthreshold(motivedominance)

Compare:Affectivedimensions(Wundt1910)

AffectivedimensionsinthePSItheory

Valence(appentence/aversion)

Arousal(unspecificsympathicussyndrome)

Selectionthreshold(motivedominance)

Resolutionlevel(focus)

Securingrate

Goaldirectedness

ThePsitheoryaboutemotion

• Affectisseenasaconfiguration ofacognitivesystem

• Modulatorsofcognition:– arousal,selectionthreshold,securingthreshold,resolutionlevel

– estimateofcompetenceandcertainty– pleasure/distresssignals® valence

• Affectivestateisemergentpropertyofmodulation

• Directedaffects(higher-levelemotions)emergebyassociationofdemandwithappetive oraversiveobjects/situations

Purposeofemotionalmodulation

• Controlwidth,depthandbiasofoperationsonmentalrepresentationsoftheagent®modifyperception,memory,planningandactionselection

• Reducecomplexityofcognitiveprocesses

ModulationinPSI/MicroPsi

Primarymodulators

Attentional modulators

Modulatordynamics

Modulatorparameters

• Baseline• Range• Volatility• Duration

• Differentmodulatorparameterconfigurations=differenttemperaments

Emotions as directed affect +Modulation

Examples:

Fear:anticipation of aversive events (à neg.valence)+arousalAnxiety:uncertainty (à neg.valence)+low competence +arousal,highsecuring behavior (frequent background checks)

Examples:

Anger:Perceived obstacle (usually agent)manifestly prevented reachingof anactive,motivationally relevantgoal (àneg.valence),sanctioningbehavior tendency (à goal relevance is re-directed to sanctioning ofobstacle),arousal,low resolution level,highaction readyness,highselection threshold

Sadness:Manifestprevention from all conceived ways of reachingactive,relevantgoal,without relevantobstacle (à neg.valence),support-seeking behavior (by increased demand for affiliation),low arousal,inhibition of active goalà decreased action readyness

Examples:

• Pride:highcompetence (à low securing rate),highinternal legitimacy,likely coincidence with highexternallegitimacy

• Joy:higharousal +highperceived reward signal fromsatisfying ademand

• Bliss:low arousal +highperceived reward signal fromsatisfying ademand (since physiological demands ofteninvolve higharousal,mostly related to cognitive demands,suchas aesthetics)

Implementationofaffectivemodulation

• 𝑀𝑜𝑑𝑢𝑙𝑎𝑡𝑜𝑟 ≔

𝑚𝑖𝑛 ∈ ℝ𝑚𝑎𝑥 ∈ ℝ,

𝑙𝑒𝑣𝑒𝑙𝑡 ∈ [𝑚𝑖𝑛,𝑚𝑎𝑥],𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 ∈ [𝑚𝑖𝑛,𝑚𝑎𝑥],

𝑣𝑜𝑙𝑎𝑡𝑖𝑙𝑖𝑡𝑦 ∈ ℝ+,𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ ℝ+

• 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 = d𝑚𝑎𝑥 − 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒, if𝑙𝑒𝑣𝑒𝑙𝑡−1 > 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 − 𝑚𝑖𝑛, else

• 𝛿𝑡 = 𝑡𝑎𝑟𝑔𝑒𝑡×𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 + 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 − 𝑙𝑒𝑣𝑒𝑙𝑡−1 𝑣𝑜𝑙𝑎𝑡𝑖𝑙𝑖𝑡𝑦

Implementationofvalence

• marginalsum 𝑉, 𝑙𝑖𝑚𝑖𝑡 : = ∑ 𝑆𝑛|𝑉|𝑛=0 l 𝑆𝑛 ≔

𝑙𝑖𝑚𝑖𝑡−𝑆𝑛−1𝑙𝑖𝑚𝑖𝑡

𝑣𝑛• 𝑙𝑖𝑚𝑖𝑡 = max 𝜔|𝜔 ∈ 𝑤𝑒𝑖𝑔ℎ𝑡𝑛𝑒𝑒𝑑𝑠

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑎𝑖𝑛 = marginalsum weightneed×𝑝𝑎𝑖𝑛𝑛𝑒𝑒𝑑

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒 = marginalsum weightneed×𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑛𝑒𝑒𝑑

• 𝑡𝑎𝑟𝑔𝑒𝑡𝑣𝑎𝑙𝑒𝑛𝑐𝑒 = 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒−𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑎𝑖𝑛𝑙𝑖𝑚𝑖𝑡

Implementationofarousal

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒 = marginalsum weightneed×𝑢𝑟𝑔𝑒𝑛𝑒𝑒𝑑𝑙𝑖𝑚𝑖𝑡

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒𝑛𝑐𝑦 = marginalsum weightneed×𝑢𝑟𝑔𝑒𝑛𝑐𝑦𝑛𝑒𝑒𝑑𝑙𝑖𝑚𝑖𝑡

• 𝑡𝑎𝑟𝑔𝑒𝑡𝑎𝑟𝑜𝑢𝑠𝑎𝑙 = 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒 + 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒𝑛𝑐𝑦 − 1

Implementationofaggression/regression

• 𝑒𝑝𝑖𝑠𝑡𝑒𝑚𝑖𝑐𝑐𝑜𝑚𝑝𝑒𝑡𝑒𝑛𝑐𝑒 = 𝑠𝑘𝑖𝑙𝑙𝑐𝑢𝑟𝑟𝑒𝑛𝑡𝑔𝑜𝑎𝑙𝑒𝑣𝑒𝑛𝑡

• 𝑔𝑒𝑛𝑒𝑟𝑎𝑙𝑐𝑜𝑚𝑝𝑒𝑡𝑒𝑛𝑐𝑒 = valuecompetence×𝑒𝑝𝑖𝑠𝑡𝑒𝑚𝑖𝑐𝑐𝑜𝑚𝑝.�

• 𝑡𝑎𝑟𝑔𝑒𝑡𝑎𝑔𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 = 𝑔𝑒𝑛𝑒𝑟𝑎𝑙𝑐𝑜𝑚𝑝.+𝑒𝑝𝑖𝑠𝑡𝑒𝑚𝑖𝑐𝑐𝑜𝑚𝑝.−1

Emotionviewer

Personalitymodeling

• Motivationparameters:Personalityproperties• Modulationparameters:Temperament

IndividualVariations by Parameterizing

Possible grounding of personality properties (FFM):• Openness:appreciation of art and new ideas,curiousity

• Conscientiousness:rulefollowing vs.chaotic• Extraversion:tendency to seek stimulation byenvironment and others

• Agreeableness:tendency for cooperativeness andcompassion

• Neuroticism:emotionalstability,effect of failureto self-confidence

NeedsandBigFive

Example:Mappingto FFM(BigFive)

Demanddynamics:

Physiological

Food Water Integrity Affiliation Internal Legitimacy

Gen./Epis.Competence

UncertaintyReduction Aesthetics

Social Cognitive

Demanddynamics:

Physiological

Affiliation Internal Legitimacy

UncertaintyReduction Aesthetics

Social Cognitive

Food Water Integrity

Demanddynamics:

UncertaintyReduction

Aesthetics

Valence:Pleasure/Pain signals

Aesthetics

Neuroticism:strongerexperienceofnegativeemotions,loweremotionalstability(strongnegativereward/strongerlossofcompetence,certainty;strongerdecay->morefrequentneedforreplenishment,possiblylowergain),pronenesstoanxietyduetolossofcertainty

Extraversion:surgency,activity insocial relations,expressivity(stronggain for affiliation and competence,highdecay of affiliation)

Aesthetics

Openness:desire for novelty,intellectual independence,non-conservatism,appreciation for art and new ideas(stronggain for uncertainty reduction,highepistemic competence foruncertainty reduction,probably strongrewards for alldemands)

AestheticsV+

Agreeableness:(strongpositiveand negativereward for affiliation,lowergains for competence)

Aesthetics

Conscientousness,Rigidity:(highloss incompetence,highselection threshold,possibly low effect of demands onarousal and resolutionlevel)

Aesthetics

• Why notone free variableperFFMdimension?

FFMdoes nottell the complete story

Shyness !=Introversion(highloss of affiliation,low competence)

Aesthetics

Howcanweevaluateamodelofmotivation?

• Games!

Emotion and Motivation TutorialAAAI 2018 118

SpaceInvaders(1978,TomohiroNishikado)

WorldofWarcraft (RobPardo,JeffKaplanetal.2004)

Playerpersonalitytypes

RichardBartle(1996):“Hearts,Clubs,Diamonds,Spades:PlayersWhosuitMUDs”

Motivationandpersonality

• Personalitypropertiescanbemodeledasmotivationalvariability

Affiliation

CompetenceUncertaintyreduction

Needsandplayertypes(withS.Tekovsky)

EmotionandtheSelf

Neocortexasamodelingsystem

• corticalmodelingisapredictivemodelofsensorypatterns,asadynamicsimulationofworldandagent

• Primarymodel:environmentandagent• Secondarymodel:interactionofagentwithenvironment

• Tertiarymodel:functioningofsecondarymodelingàSelf

• Consciousnessasamodelofattention

Integrationofemotionandmotivation

• Controlandselfregulationdependsonlearnedfunctionalrepresentationsincorticalstructure

• Attentionalbiases

• Attentionismechanismfordirectedlearning• Experientialaccessviaattentionalprotocol

• Structureofselfdeterminesexperienceofemotion

Socialemotions

• Objectofemotionrequiresrepresentationandmotivationalrelevance

• Socialurges:affiliation,romanticaffect,libido,dominancearetransactional

• Love:non-transactionalemotion

• Loverequiressharedpurpose,vialegitimacy

Acknowledgements

WorkonMicroPsi2iscollaborativeeffort:- RonnieVuine,DominikWelland,PriskaHerger,JonasKemperarecontributorstothecurrentversion

- Architecture/conceptshavebeeninspiredbyDietrichDörner,AaronSloman,MarvinMinsky,StanFranklinandmanyothers

- SupportfromHumboldtUniversityofBerlin,UniversityofOsnabrück(InstituteforCognitiveScience),BerlinSchoolofMindandBrain,HarvardProgramofEvolutionaryDynamics,MITMediaLab

Thank you!

Interesting questions:

• Is recursive function approximation plusmotivationalsystem sufficient for generalintelligence?

• Could we functionally recreate human-likemindswith our model?

• How does amotivated/emotionalsystem evolvewhen it can modify itself?

joschabach implementing emotion and motivation in ai

Documents

motivation & emotion - ivcc

motivation & emotion pertemuan 10

motivation and emotion chapter12

motivation and emotion

motivation , conflict, emotion

lecture 15: emotion and motivation - mit opencourseware ·...

motivation and emotion

personality, motivation and emotion

motivation & emotion

emotion and motivation

chapter9 motivation and emotion

motivation drives emotion stress emotion, motivation and...

motivation & emotion - wikimedia...applying principles of...

motivation, emotion, & stress

psychology: motivation and emotion

emotion,motivation et apprentissage

unit 8a: motivation and emotion: motivation

jeopardy motivation emotion stress