modelling the evolution of language for modellers and non-modellers ijcai-05 1 modelling the...

Modelling the evolution of language for modellers and non-modellersIJCAI-05

1

ModellingModellingthe evolution of languagethe evolution of languagefor modellers and non-modellersfor modellers and non-modellers

Introduction and techniques


2

Today’s presentersToday’s presenters• Paul Vogt

– Language evolution and computation research unit, University of Edinburgh, UK

– Induction of linguistic knowledge group, Tilburg University, The Netherlands

• Tony Belpaeme (not here)– Center for Interactive Intelligent Systems

Univeristy of Plymouth, UK

• Bart de Boer– Department of artificial intelligence

Rijksuniversiteit Groningen, the Netherlands

• All alumni of the Brussels AI-lab• All AI-researchers with strong “cognitive” focus


3

Why this Tutorial?Why this Tutorial?

• AI in the sense of using computer models to understand human intelligence benefits from interaction with other disciplines– Evolution of language is such a discipline

• The study of language evolution deals with systems that are so complex that they need to be modeled with computers– Linguists/paleontologists appreciate modeling, but

are usually no good with computers themselves


4

Our aimsOur aims

• To introduce the field of language evolution to the AI community

• To present examples of possible models– Based on our own work

• To explain how to communicate outside the field of AI– Linguists/paleontologists read AI papers in a

different way than AI researchers


5

Organisation of the tutorialOrganisation of the tutorial

Theory14:00–14:45 Introduction and techniques (Bart)14:45–15:30 Topics of research (Paul)

BREAK16:00–16:45 Communication and caveats (Paul/Tony)

Practical Examples16:45–17:00 Vowel systems (Bart)17:00–17:15 Talking Heads simulator (Paul)17:15–17:30 Hands-on demonstration (Bart/ Tony)


6

Language EvolutionLanguage Evolution

Bart de Boer


7


8

Early Scientific ExperimentsEarly Scientific Experiments

• Pharaoh Psamtik I• Frederick II von

Hohenstaufen• James IV of Scotland


9

And speculation…And speculation…

• Jespersen’s critique– Bow-wow theory– Pooh-pooh theory– Ding-dong theory– Yo-he-ho theory

• But his own theory:– La-la theory


10

As a result:As a result:

• Also: Chomsky considered it impossible (and uninteresting) to study language evolution


11

Can we do better today?Can we do better today?

• 1990: Pinker & Bloom “Natural Language and Natural Selection”

• Since 1996 biannual Evolang Conference– Palaeontology– Archaeology– Anthropology– Linguistics– Biology– Ethology– Etc…

– And of course: Computer modelling


12

Why language?Why language?

• Interesting and difficult question– Many factors play a role (including chance),

complex dynamics– Possibilities for modelling


13

Communication Communication in Animalsin AnimalsHumans are not the only ones Humans are not the only ones with complex communicationwith complex communication


14

Relation with primatesRelation with primates

• Chimpanzees are very smart– But do not learn how to speak– And only learn sign language with

difficulty

• Apes do communicate vocally– But more comparable with involun-

tary human cries of pain, joy, laughter etc.

• Neural structures in ape and monkey brains for manipulation and vocalization are analogues of human brain structures for speech


15

What evolved?What evolved?

• Very specific mechanisms?– “Universal Grammar”– Principles and Parameters

• More general learning mechanisms, some specialised for communication?

• Completely general mechanisms– Language itself evolved culturally

• “Nature versus nurture”


16

Which of these had language?Which of these had language?

Australopithecusafricanus

Homoerectus

Homoneanderthalensis


17

When did language emerge?When did language emerge?

• Two extremes:– Late emergence (~30 000 years ago)– Early emergence (A. africanus)

• But there is no direct archaeological evidence


18

The argument for late emergenceThe argument for late emergence• Symbolic explosion

– About 30 000 years ago humans started to produce art– Problem: European bias, nowadays earlier and earlier

finds– Appears to have emerged and disappeared repeatedly


19

Against late emergenceAgainst late emergence

• How can complex language evolve so quickly?

• How does one explain biological adaptations to language?

• Homo sapiens started to spread much earlier than 50–70 000 years ago– Would language have emerged in different

places?


20

Fossil evidenceFossil evidence• Hypoglossal canal (Kay,

Cartmill & Balow)– For tongue control– Not enlarged in Homo

erectus, but in early sapiens and Neanderthal (>400 000 years)

• Thoracic vertebral canal (MacLarnon & Hewitt)– For diaphragm control– Not enlarged in Homo

ergaster, but in Neanderthal and modern man


21

Modern LanguageModern Language

• Are there primitive languages?– No, not if native– No data on language evolution– But data on possibilities of

language

• But: pidgin-languages– Jargons, second language etc.– And creolisation

• Or emergence of new language– Nicaraguan sign language

• Idea: proto-language– Bickerton


22

JackendoffJackendoffPre-existing primate conceptual structure

Use of symbols in non-situation-specific fashion

Use of an open, unlimited class of symbols Concatenation of symbols

Development of a phonological combinatorial system to enlarge open, unlimited class of symbols

Use of symbol position to convey basic semantic relations

(Protolanguage about here)

Hierarchical phrase structure

Symbols that explicitly encodeabstract semantic relations Grammatical categories

System of inflectionsto convey semanticrelations

System of grammaticalfunctions to convey semantic relations

(Modern language


23

Deep history of languageDeep history of language

• Historical linguistics can reconstruct older forms of a language (e.g. indo-european)– Traditional linguistics up to ~8000 years ago

• But:– Ruhlen claims proto-world– Very unlikely– Human expansion started about 150 000 yrs ago– After this time all similarities are gone


24

LanguageLanguageandandGenesGenes


25

ConclusionConclusion

• Language >400 000 years old• No primitive languages exist, and we know little

about how they spread very long ago– But we can observe emergence of new languages– That all have certain special properties– We can also observe incomplete languages

• Language probably emerged as primitive proto-language– Why is an interesting, but hard-to-answer question

• How did it spread, how did it emerge?– Cannot be reconstructed from fossils– But possible to model


26

TechniquesTechniques

• Bart de Boer


27

The process of modellingThe process of modelling

• Choose a cognitive/linguistic problem• Gather data and theories on which to base a model• Implement the theory as a computer model

– Make abstractions of reality– Make mappings abstractions reality– Make measures of performance of the model– Implement the model using abstractions and tradeoffs

• Run tests for different parameter settings• Check whether predictions of your model conform with

reality

• Communicate your results…


28

About compromisesAbout compromises

• Reality is too complex to model completely– Especially true for models of systems involving humans

• Computing power is limited • Our knowledge of the underlying systems is limited

– Especially of the cognitive aspects

• One is forced to make compromises/tradeoffs– Identify the bottlenecks: it is no use to make one part of

the system extremely realistic if other parts are not

• Always communicate your compromises

Realism Computation


29

About abstractionsAbout abstractions• Making abstractions is one form of increasing

performance– Do not model certain aspects of the system and consider

them a “black box”

• Abstractions are perfectly acceptable– Ensure to not abstract away the baby with the bathwater– Describe and defend your abstractions– Keep in mind how the abstractions map onto reality– Do not use the model for something you abstracted out

Speaker Hearer

Errorless wordsNoisy

Speech

Errorless words

Not modelled


30

About implementationAbout implementation

• Choose a language and a programming environment in which you are comfortable– Describe your model independent from your chosen

programming language– No single programming language is best

• Models are often on the edge of what is computationally feasible– (Evolution of) language is complex, so you want to

model as much of it as possible– Optimization of implementation becomes crucial

• Preferably choose algorithms that are cognitively plausible– No exponential complexity– No global knowledge


31

TechniquesTechniques

• Optimization• Genetic Algorithms• Agent-based models

• Mathematical analysis


32

Agent-based models (1)Agent-based models (1)

• Two aspects of language/linguistics– Individual

• Psycholinguistics/language acquisition/speech errors…• “performance”, “parole”

– Collective• Historical linguistics, general linguistics• “competence”, “langue”

• These aspects influence each other– Individual performance based on group conventions– Collective behaviour caused by individuals

• This link is difficult to investigate– Complex feedback– Non-linear behaviour (influences are not separable)– Difficult to understand with “pen-and-paper”


33

Agent-based models (2)Agent-based models (2)

• Computer simulations have no problems with complex systems– Ideal to investigate interaction of individual and

collective levels

• Model a population of individuals– Individual: learning behavior, language behavior– Population: Interactions, population dynamics

“Langue”

Speaker

Speaker

Speaker

SpeakerSpeaker


34

Architecture of an individualArchitecture of an individual

…

Languagebehaviour

Perception Production

LanguageLearning

Social behaviour

speech speech

chromosomes

Age


35

The populationThe population

Agent

Agent

Agent

Agent

Remove frompopulation

Add to population

Languageinteraction

Agent Agent

mating

Spatial structure

Agent

Agent

Socialinteraction


36

Agent-based paradigmsAgent-based paradigms

• Iterated learning model (Edinburgh)– Vertical transmission (Transfer over generations)– Small populations

• Language game model (Brussels)– Horizontal transmission (Transfer within generations)– Larger population

• Sometimes the differences are accentuated, but we would like to stress the similarities

Populationsize

Horizontal Verticaltransmission

Small

LargeLanguage

Games

IteratedLearning


37

On measuring performanceOn measuring performance

• Important to define measures of performance– Optimization needs quality function– GA needs a fitness function– Agent-based models need to be monitored

• The whole model contains too much data(too many degrees of freedom)– Especially true in agent-based models

• Large complex systems can sometimes be described by a smaller number of parameters– Compare statistical mechanics: properties of a gas

are temperature, pressure and specific gravity


38

MeasuresMeasures

• Examples– “Energy” (Liljencrants and Lindblom model)– Communicative success– Number of words– Coherence in the population– Productivity of a grammar– …

• Important to clearly define and describe your measures

• Important to explain how the measures map from the simulation onto real language– Measures are abstractions from an already abstract

model


39

About mathematical analysisAbout mathematical analysis

• Mathematical analysis can also be used to gain understanding of complex linguistic phenomena– Has been used successfully in a number of cases (i.e.

Nowak et al.)– Comparable to mathematical biology– Is considered more exact than computational models:

insight in the why, not just the how

• In order to do mathematical analysis, models must be made even more simple and abstract– Complex, non-linear models are often not solvable

• Complementary to computational models– Perhaps design models with analysis in mind– Use analysis to gain deeper understanding of model’s

dynamics


40

What have we seen?What have we seen?

• Steps in modeling– Design an abstract model of a linguistic phenomenon– Specify mappings from the abstraction to reality– Choose a technique for implementing your model– Make decisions about computational simplifications– Design measures on your system– Do mathematical analysis

• Techniques for modeling– Optimization– Genetic Algorithms– Agent-based models (Language Games and Iterated

Learning)– (mathematical analysis)

modelling the evolution of language for modellers and non-modellers ijcai-05 1 modelling the...

Documents

nonmodellers ijcai

language evolution bart

field of language evolution

nonmodellers introduction

study language evolution

theory slide

computer modelling slide

boer slide