embodied compositional semantics ellen dodge edodge@berkeley.edu

Embodied Compositional Semantics

Ellen Dodgeedodge@berkeley.edu

Questions

• What is the nature of compositionality in the Neural Theory of Language?

• How can it be best represented using Embodied Construction Grammar?

Examples

• He bit the apple

• He was bitten (by a toddler)

• He bit into the apple

• His white teeth bit into the apple.

• He shattered the window

• The window was shattered

• The window shattered

Outline

• Compositionality• Neural Theory of Language and ECG

– Assumptions– Overview

• Examples:– Representation of constructions and meaning– Simulation

• Concluding Remarks

Compositionality

• Put the parts together to create the meaning of the whole.

Compositionality

• Put the parts together to create the meaning of the whole.

• Questions:– what is the nature of the parts?– How and why do they combine with one another?– What meaning is associated with this composition?

Short answers

• Parts = constructions, schemas

• Combination = binding, unification

• Meaning of the whole = simulation of unified parts

Constructions

Construction Grammar• Constructions are form-meaning pairings • A given utterance instantiates many different constructions

Embodied Construction Grammar• Construction meaning is represented using schemas• Meaning is embodied

Key assumptions of NTL

• Language understanding is simulation

• Simulation involves activation of neural structures

Comments

• Language understanding

• Understanding process is dynamic

• “Redundancy” is okay

Conceptual structure

• Embodied

• Schematic

• (Potentially) language-independent

• Highly interconnected

Simulation parameters

• Constructions unify to create semantic specification that supports a simulation

• Two types of simulation parameters for event descriptions:– Event content– Event construal

Putting the parts together

• Bindings

• Unification

“Pre-existing” structure

Cxn

schema

schema schema

schema

Cxn

Cxn

schema

Unification

Cxn

schema

schema schema

schema

Cxn

Cxn

schema

Summary

• Parts = constructions, schemas

• Combination = binding, unification

• Meaning of the whole = simulation of the combined parts

First example

• He bit the apple.

schema MotorControl subcase of Process roles Actor ↔ Protagonist

Effector Effort

Routine constraints Actor ← animate

Schemas

schema ForceApplication subcase of MotorControl evokes ForceTransfer as FT roles

Actor ↔ FT.Supplier ↔ Protagonist Acted Upon↔ FT.Recipient Effector

Routine Effort ↔ FT.Force.amount

schema ForceTransfer evokes Conact as C roles

Supplier ↔ C.entity1 Recipient ↔ C.entity2 Force

schema MotorControl subcase of Process roles Actor ↔ Protagonist

Effector Effort

Routine constraints Actor ← animate

schema Contact subcase of SpatialRelation roles Entity1: entity Entity2: entity

Schema networks 

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

Agentive Impact

SelfMotion

SelfMotionPath

MotionPath

Construction BITE1 subcase of Verb form: bite meaning: ForceApplication constraints: Effector ← teeth Routine ← bite // close mouth 

Verb Constructions

schema ForceApplication subcase of MotorControl evokes ForceTransfer as FT roles Actor ↔ FT.Supplier ↔ Protagonist Acted Upon ↔ FT.Recipient Effector Routine Effort ↔ FT.Force.amount

Verb Constructions

schema ForceApplication subcase of MotorControl

schema Agentive Impact subcase of ForceApplication

cxn BITE meaning: ForceApplication

schema MotorControl

cxn GRASP meaning: ForceApplicationcxn PUSH meaning: ForceApplicationcxn SLAP meaning: AgentiveImpactcxn KICK meaning: AgentiveImpactcxn HIT meaning: AgentiveImpact

Argument Structure Construction

construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Affected ↔ NPm

Argument Structure Construction

construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Affected ↔ NPm

CauseEffect schema

schema CauseEffect subcase of ForceApplication; Process roles

Causer ↔ Actor Affected ↔ ActedUpon ↔ Process.Protagonist Instrument ↔ Effector

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Network

Argument Structure Construction

construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes: EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Affected ↔ NPm

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Network

Important points

Compositionality does not require that each component contain different information.

Shared semantic structure is not viewed as an undesirable redundancy

Argument Structure Construction

construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Affected ↔ NPm

schema EventDescriptor

roles

EventType: Process

ProfiledProcess: Process

ProfiledParticipant: Entity

ProfiledState(s): State

SpatialSetting

TemporalSetting  

Event Descriptor schema

Argument Structure Construction

Construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Affected ↔ NPm

construction NPVP1 constituents: Subj: NP VP : VPform Constraints Subj f before VPf

meaning: EventDescriptor ProfiledParticipant ↔ Subjm   

Bindings with other cxnsconstruction ActiveTransitiveAction2 subcase of VP constituents: V ; NP form: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

Affected ↔ NPm

Construction NPVP1 constituents: Subj: NP VP : VPform constraints Subj f before VPf

meaning: EventDescriptor ProfiledParticipant ↔ Subjm   

Bindings with other cxnsconstruction ActiveTransitiveAction2 subcase of VP constituents: V ; NP form: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

Affected ↔ NPm

schema EventDescriptor roles EventType ProfiledProcess ProfiledParticipant ProfiledState(s) SpatialSetting TemporalSetting

Bindings with other cxns

schema EventDescriptor roles EventType ProfiledProcess ProfiledParticipant ProfiledState(s) SpatialSetting TemporalSetting

construction NPVP1 constituents: Subj: NP VP : VPform Constraints Subj f before VPf

meaning: EventDescriptor ProfiledParticipant ↔ Subjm   

construction ActiveTransitiveAction2 subcase of VP constituents: V ; NP form: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

Affected ↔ NPm

Unification

CauseEffect causer affected

ForceApplication actor actedupon

EventDescriptor EventType ProfiledProcess ProfiledParticipant

BITE

TransitiveAction2

HE

NP1

NPVP1

THE APPLE

NP2ReferentDescriptor

ReferentDescriptor

Meaning Constructions

Unification

CauseEffect causer affected

ForceApplication actor actedupon

EventDescriptor EventType ProfiledProcess ProfiledParticipant

BITE

TransitiveAction2

HE

NP1

NPVP1

THE APPLE

NP2ReferentDescriptor

ReferentDescriptor resolved referent

Meaning Constructions

Unification

CauseEffect causer affected

ForceApplication actor actedupon

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

BITE

TransitiveAction2 Verb

HE

NP1

NPVP1

THE APPLE

NP2ReferentDescriptor

ReferentDescriptor resolved referent

Meaning Constructions

Unification

CauseEffect causer affected

ForceApplication actor actedupon

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

BITE

TransitiveAction2

HE

NP1

NPVP1 subj

THE APPLE

NP2ReferentDescriptor

ReferentDescriptor

Meaning Constructions

Unification

CauseEffect causer affected

ForceApplication actor actedupon

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

BITE

TransitiveAction2 NP

HE

NP1

NPVP1

THE APPLE

NP2ReferentDescriptor

ReferentDescriptor

Meaning Constructions

Semantic SpecificationHe bit the apple

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

CauseEffect causer affected

ForceApplication actor actedupon routine bite effector teeth

RD55category

Person

Apple

RD27category

Process

Simulation - He bit the apple

CauseEffect

ForceApplication

Protagonist = Causer ↔ Actor

Protagonist = Affected ↔ ActedUpon

Process

Simulation - He bit the apple

CauseEffect

ForceApplication

Protagonist = Causer ↔ Actor

Protagonist = Affected ↔ ActedUpon

Passive voice

He was bitten (by a toddler)

Argument Structure ConstructionHe was bitten (by a toddler)

construction PassiveTransitiveAction2 subcase of VP constituents: V : PassiveVerb (PP: agentivePP) form constraints: VF before PPF

meaning: CauseEffectAction evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Affected ↔ ED.ProfiledParticipant FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Causer ↔ PP.NPm

Semantic SpecificationHe was bitten (by a toddler)

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

CauseEffect causer affected

ForceApplication actor actedupon routine bite effector teeth

RD48category

Person

Person

RD27category

Effect = Process

Simulation - He was bitten (by a toddler)

CauseEffect

Action = BiteProtagonist = Causer ↔ Actor

Protagonist = Affected ↔ ActedUpon

Variations on a theme

• He shattered the window

• The window was shattered

• The window shattered

Construction SHATTER1 subcase of Verb form: shatter meaning: StateChange constraints: Initial :: Undergoer.state ← whole Final :: Undergoer.state ← shards 

Verb Construction -- shatter

schema StateChange subcase of Process roles Undergoer ↔ Protagonist

Argument Structure ConstructionHe shattered the window

construction ActiveTransitiveAction3 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes: EventDescriptor as ED; StateChange as SC constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

SC ↔ Vm

Affected ↔ SC.Undergoer Affected ↔ NPm

Semantic SpecificationHe shattered the window

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

CauseEffect causer affected

StateChange Undergoer state “wholeness”

RD189category

Person

Window

RD27category

Process

Simulation - He shattered the window

CauseEffect

Action Protagonist = Causer

Protagonist = Affected ↔ Undergoer

Argument Structure ConstructionThe window was shattered

construction PassiveTransitiveAction3 subcase of VP constituents: V : PassiveVerb (PP: agentivePP) form constraints: VF before NPF

meaning: CauseEffect evokes: EventDescriptor as ED; StateChange as SC constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Affected ↔ ED.ProfiledParticipant

SC ↔ Vm

Affected ↔ SC.Undergoer Causer ↔ PP.NPm

Semantic SpecificationThe window was shattered

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

CauseEffect causer affected

StateChange Undergoer state “wholeness”

RD175category

Window

Process

Simulation - The window was shattered

CauseEffect

Action Protagonist = Causer

Protagonist = Affected ↔ Undergoer

Argument Structure ConstructionThe window shattered

construction ActiveIntransitiveAction1 subcase of VP constituents: V : verb form meaning: Process evokes: EventDescriptor as ED; StateChange as SC constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Protagonist ↔ ED.ProfiledParticipant

SC ↔ Vm

Protagonist ↔ SC.Undergoer

Semantic SpecificationThe window shattered

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

Process protagonist

StateChange Undergoer state “wholeness”

RD177categoryWindow

Process

Simulation - The window shattered

Process

Protagonist = Undergoer

Some more variations on a theme

• He bit the apple

• He bit into the apple

• His white teeth bit into the apple.

Argument Structure ConstructionHe bit into the apple

construction ActiveEffectorMotionPath2 subcase of VP constituents: V : verb PP: Spatial-PP form constraints:

VF before PPF

meaning: EffectorMotionPath evokes; EventDescriptor as ED; ForceApplication as FA constraints:

{Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Actor ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.ActorEffector ↔ FA.Effector // INITarget ↔ FA.ActedUpon SPG ↔ PPm

Target ↔ PPm .Prep.LM

Schema

schema EffectorMotionPath subcase of EffectorMotion subcase of SPG // or evokes SPG roles Actor ↔ MotorControl.protagonist

Effector ↔ SPG.Tr ↔ M.Mover ↔ Motion.protagonistTarget ↔ SPG.Lm

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Network

Argument Structure Construction He bit into the apple

construction ActiveEffectorMotionPath2 subcase of VP constituents: V : verb PP: Spatial-PP form constraints:

VF before PPF

meaning: EffectorMotionPath evokes: EventDescriptor as ED; ForceApplication as FA constraints:

{Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Actor ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.ActorEffector ↔ FA.Effector // INITarget ↔ FA.ActedUpon SPG ↔ PPm

Target ↔ PPm .Prep.LM

EffectorMotionPath

Action

Source

Path Goal

Effector Motion

Protagonist = Actor

Protagonist = Effector

Argument Structure Construction He bit into the apple

construction ActiveEffectorMotionPath2 subcase of VP constituents: V : verb PP: Spatial-PP form constraints:

VF before PPF

meaning: EffectorMotionPath evokes; EventDescriptor as ED; ForceApplication as FA constraints:

{Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Actor ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.ActorEffector ↔ FA.Effector // INITarget ↔ FA.ActedUpon SPG ↔ PPm

Target ↔ PPm .Prep.LM

Simulation: He bit into the apple

Action

Source

Path Goal

Effector Motion

Protagonist = Actor

Protagonist = Effector

Argument Structure ConstructionHis white teeth bit into the apple

construction ActiveEffectorMotionPath3 subcase of VP constituents: V : verb PP: Spatial-PP form constraints:

VF before PPF

meaning: EffectorMotionPath evokes; EventDescriptor as ED; ForceApplication as FA constraints:

{Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Effector ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.Actor // INIEffector ↔ FA.EffectorTarget ↔ FA.ActedUpon SPG ↔ PPm

Target ↔ PPm .Prep.LM

Simulation: His white teeth bit into the apple

Action

Source

Path Goal

Effector Motion

Protagonist = Actor

Protagonist = Effector

Non-agentive biting

• He landed on his feet, hitting the narrow pavement outside the yard with such jarring impact that his teeth bit into the edge of his tongue. [BNC]

• The studs bit into Trent's hand. [BNC]

• His chest burned savagely as the ropes bit into his skin. [BNC]

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Network

Simulation: His teeth bit his tongue

Source

Path Goal

MotionProtagonist = Mover

Summary

• Small set of constructions and schemas

• Composed in different ways

• Unification produces specification of parameters of simulation

• Sentence understanding is simulation

• Different meanings = different simulations