language understanding and unified cognitive science
DESCRIPTION
Language Understanding and Unified Cognitive Science. Jerome Feldman International Computer Science Institute U. California at Berkeley Berkeley, CA [email protected]. Unified Cognitive Science. Neurobiology Psychology Computer Science Linguistics - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/1.jpg)
Language Understanding and Unified Cognitive Science
Jerome FeldmanInternational Computer Science Institute
U. California at BerkeleyBerkeley, CA
![Page 2: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/2.jpg)
Unified Cognitive Science
Neurobiology
Psychology
Computer Science
Linguistics
Philosophy
Social Sciences
Experience
Take all the Findings and Constraints Seriously
![Page 3: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/3.jpg)
Functionalism
In fact, the belief that neurophysiology is even relevant to the functioning of the mind is just a hypothesis. Who knows if we’re looking at the right aspects of the brain at all. Maybe there are other aspects of the brain that nobody has even dreamt of looking at yet. That’s often happened in the history of science. When people say that the mental is just the neurophysiological at a higher level, they’re being radically unscientific. We know a lot about the mental from a scientific point of view. We have explanatory theories that account for a lot of things. The belief that neurophysiology is implicated in these things could be true, but we have very little evidence for it. So, it’s just a kind of hope; look around and you see neurons: maybe they’re implicated.
Noam Chomsky 1993, p.85
![Page 4: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/4.jpg)
Embodiment Of all of these fields, the learning of languages would be the most impressive, since it is the most human of these activities. This field, however, seems to depend rather too much on the sense organs and locomotion to be feasible.
Alan Turing (Intelligent Machines,1948) Continuity Principle of the American Pragmatists
![Page 5: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/5.jpg)
LecturesI. Overview2. Simulation Semantics3. ECG and Best-fit Analysis4. Compositionality5. Simulation, Counterfactuals, and Inference
Constructions
Simulation
Utterance Discourse & Situational Context
Semantic Specification:
image schemas, bindings, action schemas
Analyzer:
incremental,competition-based,
psychologically plausible
![Page 6: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/6.jpg)
Psycholinguistic evidence• Embodied language impairs action/perception
– Sentences with visual components to their meaning can interfere with performance of visual tasks
(Richardson et al. 2003)
– Sentences describing motion can interfere with performance of incompatible motor actions
(Glenberg and Kashak 2002)
– Sentences describing incompatible visual imagery impedes decision task (Zwaan et al. 2002)
• Simulation effects from fictive motion sentences– Fictive motion sentences describing paths that require
longer time, span a greater distance, or involve more obstacles impede decision task (Matlock 2000, Matlock et al. 2003)
![Page 7: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/7.jpg)
Neural evidence: Mirror neurons• Gallese et al. (1996) found “mirror” neurons in
the monkey motor cortex, activated when– an action was carried out– the same action (or a similar one) was seen.
• Mirror neuron circuits found in humans (Porro et al. 1996)
• Mirror neurons activated when someone:– imagines an action being carried out (Wheeler et al. 2000)
– watches an action being carried out (with or without object) (Buccino et al. 2000)
![Page 8: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/8.jpg)
The Mirror System
The mirror system, like the motor system, is somatotopically organized.
Foot actions Hand actions Mouth actions
Buccino et al., 2001
humans watching videos of actions without objects
humans watching same actions with objects
mouthhandfoot
![Page 9: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/9.jpg)
Fast Brain ~ Slow Neurons
Mental Connections are Active Neural Connections
There is No Erasing in the Brain
![Page 10: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/10.jpg)
Movement vs. ActionsPulvermueller Lab
![Page 11: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/11.jpg)
Brains ~ Computers• 1000 operations/sec• 100,000,000,000 units• 10,000 connections/• graded, stochastic• embodied• fault tolerant• evolves• learns
• 1,000,000,000 ops/sec• 1-100 processors• ~ 4 connections• binary, deterministic• abstract, disembodied• crashes frequently• explicitly designed • is programmed
![Page 12: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/12.jpg)
The ICSI/BerkeleyNeural Theory of Language Project
Learning early constructions (Chang, Mok)
ECG
![Page 13: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/13.jpg)
Active representations• Many inferences about actions derive from what
we know about executing them• Representation based on stochastic Petri nets
captures dynamic, parameterized nature of actions
Walking:
bound to a specific walker with a direction or goal
consumes resources (e.g., energy)may have termination condition
(e.g., walker at goal) ongoing, iterative action
walker=Harry
goal=home
energy
walker at goal
![Page 14: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/14.jpg)
Learning Verb MeaningsDavid Bailey
A model of children learning their first verbs.Assumes parent labels child’s actions.Child knows parameters of action, associates with wordProgram learns well enough to: 1) Label novel actions correctly 2) Obey commands using new words (simulation)System works across languagesMechanisms are neurally plausible.
![Page 15: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/15.jpg)
System Overview
![Page 16: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/16.jpg)
Learning Two Senses of PUSH
Model merging based on Bayesian MDL
![Page 17: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/17.jpg)
The ICSI/BerkeleyNeural Theory of Language Project
Learning early constructions (Chang, Mok)
ECG
![Page 18: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/18.jpg)
The Binding Problem
Massively Parallel Brain
Unitary Conscious Experience
Many Variations and Proposals
Our focus: The Variable Binding Problem
![Page 19: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/19.jpg)
SHRUTI• SHRUTI does inference
by connections between simple computation nodes
• Nodes are small groups of neurons
• Nodes firing in sync reference the same object
![Page 20: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/20.jpg)
![Page 21: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/21.jpg)
![Page 22: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/22.jpg)
Proposed Alternative Solution
• Indirect references– Pass short signatures, “fluents”
• Functionally similar to SHRUTI's time slices
– Central “binder” maps fluents to objects• In SHRUTI, the objects fired in that time slice
– Connections need to be more complicated than in SHRUTI
• Fluents are passed through at least 3 bits• But temporal synchrony is not required
![Page 23: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/23.jpg)
LecturesI. Overview2. Simulation Semantics3. ECG and Best-fit Analysis4. Compositionality5. Simulation, Counterfactuals, and Inference
Constructions
Simulation
Utterance Discourse & Situational Context
Semantic Specification:
image schemas, bindings, action schemas
Analyzer:
incremental,competition-based,
psychologically plausible
![Page 24: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/24.jpg)
Ideas from Cognitive Linguistics
• Embodied Semantics (Lakoff, Johnson, Sweetser, Talmy
• Radial categories (Rosch 1973, 1978; Lakoff 1985)
– mother: birth / adoptive / surrogate / genetic, …
• Profiling (Langacker 1989, 1991; cf. Fillmore XX)
– hypotenuse, buy/sell (Commercial Event frame)
• Metaphor and metonymy (Lakoff & Johnson 1980, …)
– ARGUMENT IS WAR, MORE IS UP– The ham sandwich wants his check.
• Mental spaces (Fauconnier 1994)
– The girl with blue eyes in the painting really has green eyes.
• Conceptual blending (Fauconnier & Turner 2002, inter alia)
– workaholic, information highway, fake guns– “Does the name Pavlov ring a bell?” (from a talk on ‘dognition’!)
![Page 25: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/25.jpg)
Image schemas
• Trajector / Landmark (asymmetric)– The bike is near the house – ? The house is near the bike
• Boundary / Bounded Region – a bounded region has a closed boundary
• Topological Relations– Separation, Contact, Overlap, Inclusion, Surround
• Orientation– Vertical (up/down), Horizontal (left/right, front/back)– Absolute (E, S, W, N)
LMTR
bounded region
boundary
![Page 26: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/26.jpg)
Schema FormalismSCHEMA <name>
SUBCASE OF <schema>
EVOKES <schema> AS <local name>
ROLES < self role name>: <role restriction>
< self role name> <-> <role name>
CONSTRAINTS <role name> <- <value>
<role name> <-> <role name>
![Page 27: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/27.jpg)
A Simple Example
SCHEMA hypotenuse
SUBCASE OF line-segment
EVOKES right-triangle AS rt
ROLES Comment inherited from line-segment
CONSTRAINTS
SELF <-> rt.long-side
![Page 28: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/28.jpg)
Language understanding: analysis & simulation
“Harry walked into the cafe.”
Analysis Process
SemanticSpecification
Utterance
Constructions
General Knowledge
Belief State
CAFE Simulation
construction WALKEDform
selff.phon [wakt]meaning : Walk-Action constraints
selfm.time before Context.speech-time selfm..aspect encapsulated
![Page 29: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/29.jpg)
Semantic specification
The analysis process produces a semantic specification that
•includes image-schematic, motor control and conceptual structures
•provides parameters for a mental simulation
![Page 30: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/30.jpg)
Task: Interpret simple discourse fragments/ blurbs
France fell into recession. Pulled out by Germany
US Economy on the verge of falling back into recession after moving forward on an anemic recovery.
Indian Government stumbling in implementing Liberalization plan.
Moving forward on all fronts, we are going to be ongoing and relentless as we tighten the net of justice.
The Government is taking bold new steps. We are loosening the stranglehold on business, slashing tariffs and removing obstacles to international trade.
![Page 31: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/31.jpg)
![Page 32: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/32.jpg)
Results• Model was implemented and tested on discourse fragments from a
database of 50 newspaper stories in international economics from standard sources such as WSJ, NYT, and the Economist.
• Results show that motion terms are often the most effective method to provide the following types of information about abstract plans and actions.– Information about uncertain events and dynamic changes in goals
and resources. (sluggish, fall, off-track, no steam)– Information about evaluations of policies and economic actors and
communicative intent (strangle-hold, bleed).– Communicating complex, context-sensitive and dynamic
economic scenarios (stumble, slide, slippery slope).– Commincating complex event structure and aspectual information
(on the verge of, sidestep, giant leap, small steps, ready, set out, back on track).
• ALL THESE BINDINGS RESULT FROM REFLEX, AUTOMATIC INFERENCES PROVIDED BY X-SCHEMA BASED INFERENCES.
![Page 33: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/33.jpg)
Embodied Construction Grammar
• Embodied representations– active perceptual and motor schemas
– situational and discourse context
• Construction Grammar– Linguistic units relate form and meaning/function.
– Both constituency and (lexical) dependencies allowed.
• Constraint-based (Unification)– based on feature structures (as in HPSG)
– Diverse factors can flexibly interact.
![Page 34: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/34.jpg)
Embodied Construction GrammarECG
(Formalizing Cognitive Linguistics)
1. Community Grammar and Core Concepts
2. Deep Grammatical Analysis
3. Computational Implementationa. Test Grammars
b. Applied Projects – Question Answering
4. Map to Connectionist Models, Brain
5. Models of Grammar Acquisition
![Page 35: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/35.jpg)
Construction BITE1 subcase of Verb form: bite meaning: ForceApplication constraints: Effector ← teeth Routine ← bite // close mouth
Verb Constructions
schema ForceApplication subcase of MotorControl evokes ForceTransfer as FT roles Actor ↔ FT.Supplier ↔ Protagonist Acted Upon ↔ FT.Recipient Effector Routine Effort ↔ FT.Force.amount
![Page 36: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/36.jpg)
Semantic SpecificationHe bit the apple
EventDescriptor eventtype ProfiledProcess ProfiledParticipant
CauseEffect causer affected
ForceApplication actor actedupon routine bite effector teeth
RD55category
Person
Apple
RD27category
![Page 37: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/37.jpg)
Modeling context for language understanding and learning
• Linguistic structure reflects experiential structure
– Discourse participants and entities
– Embodied schemas:• action, perception, emotion, attention, perspective
– Semantic and pragmatic relations: • spatial, social, ontological, causal
• ‘Contextual bootstrapping’ for grammar learning
![Page 38: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/38.jpg)
physics lowest energy state
chemistry molecular
minima
biology fitness, MEU
Neuroeconomics
vision threats,
friends
language errors,
NTL
Constrained Best Fit in Natureinanimate animate
![Page 39: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/39.jpg)
Two perspectives on language learning
Computational models• Grammatical induction
– language identification– context-free grammars,
unification-based grammars– statistical NLP
• Word learning models– semantic representations
• logical forms• discrete representations• continuous representations
– statistical models
Developmental evidence
• Prior knowledge– concepts– event-based knowledge– social cognition– lexical items
• Data-driven learning– basic scenes– lexically specific patterns– usage-based learning
![Page 40: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/40.jpg)
Language Acquisition• Opulence of the substrate
– Prelinguistic children already have rich sensorimotor representations and sophisticated social knowledge
– intention inference, reference resolution– language-specific event conceptualizations
(Bloom 2000, Tomasello 1995, Bowerman & Choi, Slobin, et al.)
• Children are sensitive to statistical information– Phonological transitional probabilities– Most frequent items in adult input learned earliest
(Saffran et al. 1998, Tomasello 2000)
![Page 41: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/41.jpg)
Experiment: learning verb islands
• Given: initial lexicon and ontology
• Data: child-directed language annotated with contextual information
Form:
Participants : Mother, Naomi, Ball
Scene :
Discourse :
text : throw the ballintonation : falling
Throwthrower : Naomithrowee : Ball
speaker :Motheraddressee Naomispeech act : imperativeactivity : playjoint attention : Ball
• Question:– Can the proposed construction learning model
acquire English item-based motion constructions? (Tomasello 1992)
![Page 42: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/42.jpg)
The intuition behind learning a new form-meaning pairing from context
construction Put
construction Coat
construction Here
Put-Actionput-agentput-themelocation
Coat
Sofa
before
before
![Page 43: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/43.jpg)
The learner learns a new lexically-specific construction from the form-meaning pair
construction Put-Coat-Here
constituentsv: Puto: Coatp: Here
formvf before of before pf
meaning: Caused-Motion-Sceneselfm.means vm
selfm.mover om
selfm.path pm
![Page 44: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/44.jpg)
Experiment: learning verb islands
Subset of the CHILDES database of parent-child interactions (MacWhinney 1991; Slobin )
• coded by developmental psychologists for– form: particles, deictics, pronouns, locative phrases, etc.
– meaning: temporality, person, pragmatic function,type of motion (self-movement vs. caused movement; animate being vs. inanimate object, etc.)
• crosslinguistic (English, French, Italian, Spanish)– English motion utterances: 829 parent, 690 child utterances
– English all utterances: 3160 adult, 5408 child
– age span is 1;2 to 2;6
![Page 45: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/45.jpg)
A quantitative measure: coverage
• Goal: incrementally improving comprehension– At each stage in testing, use current grammar to analyze test set
• Coverage = % role bindings analyzed
• Example:
– Grammar: throw-ball, throw-block, you-throw– Test sentence: throw the ball.
• Bindings: scene=Throw, thrower=Nomi, throwee=ball• Parsed bindings: scene=Throw, throwee=ball
– Score test grammar on sentence: 2/3 = 66.7%
![Page 46: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/46.jpg)
Learning to comprehend
![Page 47: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/47.jpg)
Usage-based learning,comprehension, and production
reinforcement(usage)
reinformcent(correction)
reinforcement(usage)
hypothesize constructions& reorganize
reinforcement(correction)
constructicon
world knowledge
discourse & situational context
simulation
analysis
utterance
analyze &
resolve
utterance
response
comm. intent
generate
![Page 48: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/48.jpg)
Unified Cognitive Science
Neurobiology
Psychology
Computer Science
Linguistics
Philosophy
Social Sciences
Experience
Take all the Findings and Constraints Seriously
![Page 49: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/49.jpg)
The ICSI/BerkeleyNeural Theory of Language Project
• Principal investigators Jerome Feldman (UCB,ICSI) George Lakoff (UCB Ling) Srini Narayanan (UCB,ICSI) Lokendra Shastri (now India)
• Affiliated faculty Chuck Fillmore (ICSI) Eve Sweetser (UCB Ling) Rich Ivry (UCB Psych) Lisa Aziz-Zadeh (USC)
Graduate Students *Ellen Dodge (Ling) Michael Ellsworth (Ling) Joshua Marker (Ling) Shweta Narayan (Ling)
Alumni Terry Regier (UCB Ling,
CogSci) Johno Bryant (Ask) David Bailey (Google) Leon Barrett (Google) Nancy Chang (Sony Paris) Joe Makin (UCSF) Eva Mok (U. Chicago) Andreas Stolcke (ICSI, SRI) Dan Jurafsky (Stanford Ling) Olya Gurevich (Powerset) Benjamin Bergen (UCSD) Carter Wendelken (UCB) Srini Narayanan (ICSI, UCB) Steve Sinha (US Govt.) Gloria Yang (UCSF) Luca Gilardi (ICSI)
![Page 50: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/50.jpg)
![Page 51: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/51.jpg)
Source-Path-Goal
SCHEMA: spg
ROLES:
source: Place
path: Directed Curve
goal: Place
trajector: Entity
![Page 52: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/52.jpg)
Translational Motion
SCHEMA translational motion
SUBCASE OF motion
EVOKES spg AS s
ROLES
mover <-> s.trajector
source <-> s.source
goal <-> s.goal
CONSTRAINTS
before:: mover.location <-> source
after:: mover.location <-> goal
![Page 53: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/53.jpg)
Event Structure for semantic QASrini Narayanan
• Reasoning about dynamics– Complex event structure
• Multiple stages, interruptions, resources, framing– Evolving events
• Conditional events, presuppositions.– Nested temporal and aspectual references
• Past, future event references– Metaphoric references
• Use of motion domain to describe complex events.• Reasoning with Uncertainty
– Combining Evidence from Multiple, unreliable sources– Non-monotonic inference
• Retracting previous assertions• Conditioning on partial evidence
![Page 54: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/54.jpg)
Components of the System
• Object references– Fluents– Binder
• Short term storage– Predicate state
• Long term storage– Facts, mediators, what predicates exist
• Inference– Mediators
• Types– Ontology
![Page 55: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/55.jpg)
Simulation-based language understanding
“Harry walked to the cafe.”
Schema Trajector Goalwalk Harry cafe
Analysis Process
Simulation Specification
Utterance
SimulationCafe
Constructions
General Knowledge
Belief State
![Page 56: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/56.jpg)
The meaning pole may evoke schemas (e.g., image schemas) with a local alias. The meaning pole may include constraints on the schemas (e.g., identification constraints ).
construction TO
form selff.phon /thuw/
meaning evokes
Trajector-Landmark as tl Source-Path-Goal as spg constraints:
tl.trajectorspg.trajectortl.landmarkspg.goal
construction TO
form selff.phon /thuw/
meaning evokes
Trajector-Landmark as tl Source-Path-Goal as spg constraints:
tl.trajectorspg.trajectortl.landmarkspg.goal
Representing constructions: TO
local alias
identification constraint
![Page 57: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/57.jpg)
TO vs. INTO:INTO adds a Container schema and appropriate bindings.
The INTO construction construction INTO
form selff.phon /Inthuw/
meaning evokes
Trajector-Landmark as tl Source-Path-Goal as spg
Container as cont constraints:
tl.trajectorspg.trajectortl.landmarkcontcont.interiorspg.goalcont.exteriorspg.source
![Page 58: Language Understanding and Unified Cognitive Science](https://reader034.vdocument.in/reader034/viewer/2022052401/56814786550346895db4b60a/html5/thumbnails/58.jpg)
An ECG analysis with THROW-TRANSITIVE