lectures i. overview 2. simulation semantics 3. ecg and best-fit analysis 4. compositionality 5....

LecturesI. Overview2. Simulation Semantics3. ECG and Best-fit Analysis4. Compositionality5. Simulation, Counterfactuals, and Inference

Constructions

Simulation

Utterance Discourse & Situational Context

Semantic Specification:image schemas, bindings,

action schemas

Analyzer:

incremental,competition-based,

psychologically plausibleA

Introduction: NTL• NTL’s main tenets

– direct neural realization, and– continuity of thought and language

• both of which entail a commitment to parallel processing and spreading activation

– existence of language communities• conventional beliefs, grammars

– simulation semantics• language understanding involves some of the brain circuitry

involved in perception, motion, and emotion– best-fit process

• underlying learning, understanding, and production of language

Levels in a Neural Theory of Language

The Neural Observation Level: Discoveries made via experimental neuroscience. The Neural Computation Level: A hypothesized (connectionist) account of what “Neural Computation” is and how the brain uses it to function.

The Formal Level: The use of a single formal notation linking the Neural Computational and Cognitive Linguistics levels.

In Embodied Construction Grammar (ECG), the notation is used in standard forms of computation, both to model the functionality of various aspects of the brain and for use in automatic language analysis. The Cognitive Linguistics Level: The analysis of language and thought using ideas that fit empirical results from the cognitive and brain sciences. The Cognitive and Linguistic Observation Level: Empirical observations about language and thought.

Introduction: ECG

• Embodied Construction Grammar– part of the Construction Grammar tradition (Croft

2001, Fillmore 1998, Fried & Boas 2005)– adds embodied semantics– Designed as a tool to formally explore the NTL

principles• in a tractable, expressive way• not the only way to formalize NTL; cannot directly

describe some of its aspects (e.g., spreading activation)

Embodied Construction GrammarECG

(Formalizing Cognitive Linguistics)

1. Community Grammar and Core Concepts2. Deep Grammatical Analysis3. Computational Implementation

a. Test Grammars b. Applied Projects – Question Answering

4. Map to Connectionist Models, Brain5. Models of Grammar Acquisition

ECG for linguistic analysis

• ECG unifies insights from construction grammars and cognitive linguistics

• ECG is not just about representation:– A computationally precise model makes it possible to build

systems for linguistic analysis and interpretation• Some history:

– Jurafsky (1996) first used construction grammar in a model of interpretation

– Bryant (2003): robust child-language interpretation– Steels and de Beule (2006): language learning over populations– Ball (2007): psychologically plausible language interpretation


• Constructional Analyzer– fits into the unified cognitive science (Feldman

2006)– and builds on

• cognitive linguistics• construction grammar• psycholinguistics• simulation-based language inference (Narayanan 1997)• Natural Language Processing techniques


• Constructional Analyzer (Bryant 2008)– Input:

• Grammar• Utterance• Context Model

– Output• Semantic Specification,

or SemSpec

Simplifying grammar by exploiting the understanding process

Mok and Bryant, BLS 2006

• Omission of arguments in Mandarin Chinese• Construction grammar framework• Model of language understanding• Our best-fit approach

• Mother (I) give you this (a toy).

CHILDES Beijing Corpus (Tardiff, 1993; Tardiff, 1996)

ma1+ma gei3 ni3 zhei4+g

emother give 2PS this+CLS

• You give auntie [the peach].

• Oh (go on)! You give [auntie] [that].

Productive Argument Omission (in Mandarin)1

2

3

ni3 gei3 yi2

2PS give auntieao ni3 gei3 ya

EMP 2PS give EMP

4 gei3

give

• [I] give [you] [some peach].

Arguments are omitted with different probabilities

All arguments omitted: 30.6%No arguments omitted: 6.1%

% elided (98 total utterances)

Giver

Recipient

Theme

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%60.00%

70.00%

80.00%

90.00%

100.00%

Problem: Proliferation of constructionsSubj Verb Obj1 Obj2

↓ ↓ ↓ ↓

Giver Transfer Recipient Theme

Verb Obj1 Obj2↓ ↓ ↓

Transfer Recipient Theme

…

Subj Verb Obj2↓ ↓ ↓

Giver Transfer Theme

Subj Verb Obj1↓ ↓ ↓

Giver Transfer Recipient

If the analysis process is smart, then...

• The grammar needs only state one construction• Omission of constituents is flexibly allowed• The analysis process figures out what was omitted

Subj Verb Obj1 Obj2↓ ↓ ↓ ↓


physics lowest energy state

chemistry molecular fit

biology fitness, MEU Neuroeconomics

vision threats, friends

language errors, NTL, OT

Constrained Best Fit in Natureinanimate animate

society, politicsframing, compromise

Competition-based analyzer finds the best analysis

• An analysis is made up of:– A constructional tree– A set of resolutions– A semantic specification

The best fit has the highest combined score

Combined score that determines best-fit

• Syntactic Fit:– Constituency relations– Combine with preferences on non-local elements– Conditioned on syntactic context

• Antecedent Fit:– Ability to find referents in the context– Conditioned on syntactic information, feature agreement

• Semantic Fit:– Semantic bindings for frame roles– Frame roles’ fillers are scored

Analyzing ni3 gei3 yi2 (You give auntie)

• Syntactic Fit: – P(Theme omitted | ditransitive cxn) = 0.65– P(Recipient omitted | ditransitive cxn) = 0.42

Two of the competing analyses:

ni3 gei3 yi2 omitted↓ ↓ ↓ ↓


Theme

ni3 gei3 omitted yi2↓ ↓ ↓ ↓


(1-0.78)*(1-0.42)*0.65 = 0.08 (1-0.78)*(1-0.65)*0.42 = 0.03

Using frame and lexical information to restrict type of reference

Lexical Unit gei3Giver (DNI)Recipient (DNI)Theme (DNI)

The Transfer FrameGiverRecipientTheme

MannerMeansPlace

PurposeReasonTime

Can the omitted arg be recovered from context?

• Antecedent Fit:ni3 gei3 yi2 omitted↓ ↓ ↓ ↓




Discourse & Situational Context

child motherpeach auntietable

?

How good of a theme is a peach? How about an aunt?

The Transfer FrameGiver (usually animate)Recipient (usually animate)Theme (usually inanimate)

ni3 gei3 yi2 omitted↓ ↓ ↓ ↓

Giver TransferRecipient

Theme



• Semantic Fit:ni3 gei3 yi2 omitted↓ ↓ ↓ ↓

Giver TransferRecipient

Theme

The argument omission patterns shown earlier can be covered with ONE construction

• Each construction is annotated with probabilities of omission • Language-specific default probability can be learned

Subj Verb Obj1 Obj2↓ ↓ ↓ ↓


Theme0.78 0.42 0.65P(omitted|cxn):

% elided (98 total utterances)

Giver

Recipient

Theme

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

Leverage processing to simplify representation

• The processing model is complementary to the theory of grammar

• By using a competition-based analysis process, we can:– Find the best-fit analysis with respect to constituency

structure, context, and semantics– Eliminate the need to enumerate allowable patterns of

argument omission in grammar• This is currently being applied in models of language

understanding and grammar learning.

ECG for linguistic analysis• Workbench by Luca Gilardi

– wraps the Constructional Analyzer

– two different uses• simplifies creation

and revising of grammars

• helps testing grammars


• ECG: the notation– two basic primitives:

• schemas• constructions

– organized in subcase lattices• i.e., hierarchical inheritance structures

with (possibly) multiple parents– Ex.:

• SlidePast is a subcase of Verb,• which is a subcase of Word,• which in turn is a subcase of RootType

(not shown)


• Workbench– single window

• simple!– lattices on the

left– editing area in

center– grammar file

view on the right– top, center:

input utterance


• Workbench– one adds new

schemas and constructions in the central pane

– they are shown automatically in the lattice representation

ECG for linguistic analysis• ECG: the notation

– we’ll see what’s needed for analyzing a simple sentence• he slid

– we need some notation first• keyword are in bold

– ECG is a Construction Grammar• two poles: form and meaning

– constructions:• pair form and meaning

– schemas• represent the meaning constraint of a construction

– subcase of• introduces an inheritance relation in a construction or a schema

– other features:• role: introduces a part (or feature) in the structure• evokes: an associated structure that’s neither a part nor a subcase

– bindings: • ECG is also a unification grammar• specified by double arrows: <-->


– the semantics of he slid• TrajectorLandmark, SPG

– conventional image schemas– related by inheritance

• SPG inherits all TL’s roles:– trajector, landmark, profiledArea

• MotionAlongAPath– actions involving a protagonist– the path is represented by the evoked

SPG• evokes introduces a new role (spg in

this case) – the mover is bound to the trajector of

the evoked SPG

schema TrajectorLandmarkroles

trajector landmarkprofiledArea

schema SPGsubcase of

TrajectorLandmarkroles

sourcepathgoal

schema MotionAlongAPath subcase of Motion evokes SPG as spg constraints mover ↔ spg.trajector


– the semantics of he slid• Motion

– a subcase of Process– the mover and the protagonist are bound

together by the double arrows• i.e., the mover is the primary participant in a

Motion action– the x-net role is typed (via the “:”) to be of the

x-schematic type motion– @process is in external ontology

• x-schemas– fine-grained process structure representations

• e.g. walking, pushing, sliding can all be represented as x-schematic structures (Narayanan 1997)

schema Process roles protagonist x-net: @process

schema Motion subcase of Process roles mover: @entity speed // scale heading // place x-net: @motion // modifiedconstraints mover ↔ protagonist

Schema Lattice

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

Agentive Impact

SelfMotion

SelfMotionPath

MotionPath

Verb Constructions

schema ForceApplication subcase of MotorControl

schema Agentive Impact subcase of ForceApplication

cxn BITE meaning: ForceApplication

schema MotorControl

cxn GRASP meaning: ForceApplicationcxn PUSH meaning: ForceApplicationcxn SLAP meaning: AgentiveImpactcxn KICK meaning: AgentiveImpactcxn HIT meaning: AgentiveImpact


– the semantics of he slid• Just two more schemas

– EventDescriptor (or ED)• the meaning of an entire scene• the verbal argument structure is

typically bound to the eventType role

• the verb’s meaning is usually bound to profiledProcess

– ReferentDescriptor (or RD)• typically represents constraints

associated with referents of nominal and pronominal constructions

schema EventDescriptor roles

eventType: Process

profiledProcess: Process

profiledParticipantprofiledState

spatialSetting

temporalSettingschema RD

roles ontological-category givenness referent number


– the analysis of he slid• Now for the constructions

– pair form and meaning• cname.f refers to the form pole of the

construction cname• cname.m refers to its meaning pole

• Verb– Word

• gives a Verb an orthographic form– HasVerbFeatures

• verbal agreement features (number and person)

– its meaning is a Process• SlidePast

– a Verb with an orthographic form– and an x-schematic motor program– its meaning is MotionAlongAPath

general construction Verb subcase of Word, HasVerbFeatures meaning: Process

construction SlidePast subcase of Verb form constraints self.f.orth ← "slid" meaning: MotionAlongAPath constraints self.m.x-net ← @slide


– the analysis of he slid• Clause-level construction

– Declarative: brings together• a subject (an NP constituent),

– the construction for He is a subcase of NP

• and a finite verb phrase, fin, of type VerbPlusArguments

• IntransitiveArgumentStructure is a subcase of this

(green marks the inherited structure)

construction Declarativesubcase of S-With-Subj

constructional constituents subj: NP

fin: VerbPlusArguments

formconstraints

subj.f before fin.f

meaning constraints

subj.m.referent ↔ self.m.profiledParticipantself.m ↔

fin.ed

self.m.speechAct ← "Declarative”


– the analysis of he slid• NP

– construction He is one of its subcases

– NominalFeatures: agreement features of nominals (number, case, gender, ...)

– meaning: a Referent Descriptor

general construction NP subcase of RootType constructional: NominalFeatures meaning: RD


– the analysis of he slid• VerbPlusArguments

– an ancestor of IntransitiveArgumentStrucure

– also a subcase of ArgumentStructure

– meaning: a Process(in green the inherited structure)

general construction ArgumentStructuresubcase of HasVerbFeatures

meaning: Process evokes EventDescriptor as ed constraints self.m ↔ ed.eventTypegeneral construction VerbPlusArguments

subcase of ArgumentStructure constructional constituents v: Verb constraints self.features ↔ v.features meaning: Process

constraints v.m ↔ ed.profiledProcess

evokes EventDescriptor as ed

self.m ↔ ed.eventType


– the analysis of he slid• SemSpec synthesis

– after the best-fit process has terminated– the VerbPlusArgument construction

• binds the Verb’s meaning pole with the profiledProcess role of the ED• bind its own meaning pole with the ED’s eventType role

– the Declarative cxn• binds that same ED to its meaning pole• constrains the subject’s referent to be the same as its meaning pole’s

profiledParticipant• in the form block, simply constrains the subject to appear before the verb



– after the best-fit process has terminated

general construction VerbPlusArgumentssubcase of ArgumentStructure

constructional constituents v: Verb constraints self.features ↔ v.features meaning: Process

constraints v.m ↔ ed.profiledProcess

evokes EventDescriptor as edself.m ↔ ed.eventType

construction Declarativesubcase of S-With-Subj

constructional constituents subj: NP

fin: VerbPlusArguments

formconstraints

subj.f before fin.f

meaning constraints

subj.m.referent ↔ self.m.profiledParticipant

self.m ↔ fin.ed

self.m.speechAct ← "Declarative”



– last piece of analysis: the argument structure chosen by the best-fit process

– IAS binds its meaning pole with the Verb’s

– constrains the protagonist of the action to be the same as the evoked ED’s profiledParticipant

• together with the constraint described above for VerbPlusArguments, implies that the event described by the intransitive argument structure is the same as the one described by its verb constituent.

• Goldberg (1995) describes for cases in which the meaning of the verb and argument structure constructions do not unify.

(inherited structure in green)

construction IntransitiveArgumentStructure subcase of VerbPlusArguments

constructional constituents v: Verb constraints self.features ↔ v.features

self.features.verbform ← FiniteOrGerund meaning: Process constraints

evokes EventDescriptor as ed

self.m ↔ ed.eventType self.m.protagonist ↔ ed.profiledParticipant self.m ↔ v.m

ECG for psycholinguistic modeling • The best-fit process in the Analyzer

– inspired by cognitive science, psychology, computer science

– algorithm is cognitively plausible• scans and incorporates in an interpretation one word at

a time• can only entertain a limited number of interpretations• approximates spreading activation with probabilities• combines syntactic and semantic evidence to rank

competing interpretations– such process is what we call the best-fit heuristic

ECG for psycholinguistic modeling • The best-fit process in the Analyzer

– best fit heuristic: cognitive motivation– psychology and psycholinguistics

• constraint-based (or interactionist) paradigm– [...] constraint-based models assume that multiple syntactic

alternatives are evaluated using both linguistic and non-linguistic sources of constraint. The comprehension system continuously integrates all the relevant and available information in order to compute the interpretation that best satisfies those constraints. (McRae, Spivey-Knowlton, & Tannenhaus, 1998)

– models that fit the constraint-based paradigm• Narayanan & Jurafsky (1998)• McRae et al. (1998) • Pado (2007)

ECG for psycholinguistic modeling

• The best-fit process in the Analyzer– best fit heuristic: cognitive motivation

• Connectionist models– best-fit models that use

• spreading activation to combine multiple domains • competition between the connectionist model’s units to model

competing hypotheses– Examples:

• Lane & Henderson (1998): connectionist network for syntactic parsing

• Feldman (2006): reduction of language interpretation to connectionist models



• Construction grammars– defines grammaticality in terms of formal

properties (syntax) and function (semantic and pragmatic constraints)



• Natural Language Processing (CS)– joint models of lexicalized PCFGs can be seen as

best-fit models– they use lexical dependency as a proxy for direct

semantic information


• Analyzer: modeling reading times– the best-fit machinery has been tested with real

psycholinguistic data• McRae, Spivey, Tannenhaus (McRae at el., 1998)

– self-paced reading paradigm with pairs of reduced relative sentences:

1. The cop arrested by the detective was guilty2. The crook arrested by the detective was guilty

– Sentences differed on whether the subject was a good agent of the p.p. (cop) or a good patient (crook)• sentence 1 is initially easier at the p.p.• harder at the prepositional phrase and main verb


• Analyzer: modeling reading times– words presented two at a time– semantic fit affects reading time– explanation:

• consequence of violation of semantic expectationsThe cop arrested by the detective was guilty– the cop arrested is biased towards the cop doing the arresting– by the detective violates such expectation


• Analyzer: modeling reading times– data from Penn TreeBank, Propbank, original data

from McRea et al. to approximate constituent filler probabilities

– simple grammar– 40 reduces samples from McRea et al.– 40 unreduced samples as baseline

ECG for psycholinguistic modeling • Analyzer: modeling reading times

– some discrepancies due to best-fit heuristic chosen– results qualitatively accurate nonetheless

NTL and ECGhttp://ecgweb.pbworks.com/

An introduction

LecturesI. Overview2. Simulation Semantics3. ECG and Best-fit Analysis4. Compositionality5. Simulation, Counterfactuals, and Inference

Constructions

Simulation


Semantic Specification:image schemas, bindings,

action schemas

Analyzer:

incremental,competition-based,

psychologically plausible

Schema Lattice

MotorControl

Motion

SPG

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

Agentive Impact

SelfMotion

SelfMotionPath

MotionPath

Verb Constructions

schema ForceApplication subcase of MotorControl

schema Agentive Impact subcase of ForceApplication

cxn BITE meaning: ForceApplication

schema MotorControl

cxn GRASP meaning: ForceApplicationcxn PUSH meaning: ForceApplicationcxn SLAP meaning: AgentiveImpactcxn KICK meaning: AgentiveImpactcxn HIT meaning: AgentiveImpact

Introduction: ECG• More specifically, ECG serves:

1. as a technical tool for linguistic analysis2. to specify shared grammar, conceptual conventions of a

linguistic community3. as a computer specification for implementing linguistic

theories4. as a representation for models and theories of language

acquisition5. as a front-end system for applied language-understanding tasks6. as a high-level functional description for biological and

behavioral experiments

Introduction: ECG

• NTL assumptions can lead to the formulation of questions and experiments not obvious from other perspectives

• To facilitate that, a precise notation is needed: ECG– Embodied Construction Grammar

Best-fit analysis reduces burden on the grammar representation

Constructions

Simulation


Semantic Specification:image schemas, frames,

action schemas

Analyzer:

incremental,competition-based, psycholinguistically

plausible

lectures i. overview 2. simulation semantics 3. ecg and best-fit analysis 4. compositionality 5....

Documents

analysis of language

language learning

neural theory of language

used construction grammar

automatic language analysis

neural computation level

neural observation level

construction grammars