kivik 2013nlp. corpus processing, ontologies1 the contribution of nlp corpus processing ontologies...

Kivik 2013 NLP. Corpus processing, Ontologies 1

The contribution of NLPCorpus processing

Ontologies and terminologies


What is NLP?

• Natural Language Processing– natural language vs. computer languages

• Other names – Computational Linguistics

• emphasizes scientific not technological

– Language Engineering – Language Technology


NLP and linguistics

LING

NLP

supply ideasinterpret results

test theoriesexpose gaps

plus turn into technology


Example: regular morphology

LINGUISTICS: – Rules: stems -> inflected forms

NLP: – program the rules

– apply rules to a lexicon of stems

– Is the output correct? Errors?

LINGUISTICS:– refine the theory

Needed for: web search, spell-checkers, machine translation, speech recognition systems etc.


Applications

• web search– Basic search– Filtering results

• spelling and grammar checking • machine translation (MT) • talk to computers

– speech processing as well

• information extraction– finding facts in a database of documents– answering questions


How can NLP make better dictionaries?

By pre-processing a corpus:

• tokenization

• sentence splitting

• lemmatization

• POS-tagging

• parsing

Each step builds on predecessors


Tokenization

“identifying the words”

from:he didn't arrive.

to: Hedidn’t arrive.


Automatic tokenization

• Western writing systems – easy! space is separator

• Chinese, Japanese, some other writing systems– do not use word-separator

– hard • like POS-tagging (below)


Why isn't space=separator enough (even for English)?

• what is a space– linebreaks, paragraph breaks, tabs

• Punctuation– characters do not form parts of words but may

be attached to words (with no spaces)

• brackets, quotation marks

• Hyphenation– is co-op one word or two? is well-managed?


Sentence splitting

“identifying the sentences”

from:he didn't arrive. to: Hedidn’t arrive.

to:<s> Hedidn’t arrive.</s>


Lemmatization

Mapping from text-word to lemma help (verb)

text-word to lemmahelp help (v)helps help (v)helping help (v)helped help (v)

.


Lemmatization

Mapping from text-word to lemma help (verb) help (noun), helping (noun)

text-word to lemmahelp help (v), help (n)helps help (v), helps (n)**helping help (v), helping (n)helped help (v) helpings helping (n)

**help (n): usually a mass noun, but part of compound home help which is a count noun, taking the "s" ending.

.


Lemmatization

Dictionary entries are for lemmas

Match between text-word and dictionary-word

lemmatization


Lemmatization

• Searching by lemma – English: little inflection

– French: 36 forms per verb

– Finno-Ugric: 2000.

• Not always wanted:– English royalty

• singular: kings and queens

• plural royalties: payments to authors


Automatic lemmatization• Write rules:

– if word ends in "ing", delete "ing"; – if the remainder is verb lemma, add to list of possible lemmas

• If detailed grammar available, use it• full lemma list is also required

– Often available from dictionary companies


Part-of-speech (POS) tagging

“identifying parts of speech”

from:he didn't arrive. to: …

.

to:<s> He PNP pers pronoun

did VVD past tense verb

n’t XNOT not

arrive VV base form of verb

. C punctuation

</s>


Tagsets

• The set of part-of-speech tags to choose between– Basic: noun, verb, pronoun …– Advanced: examples - CLAWS English

tagset• NN2 plural noun• VVG -ing form of lexical verb

• Based on linguistics of the language.


POS-tagging: why?

• Use grammar when searching– Nouns modified by buckle– Verbs that buckle is object of


POS-tagging: how?

• Big topic for computational linguistics – well understood – taggers available for major languages

• Some taggers use lemmatized input, others do not • Methods

– constraint-based: set of rules of the form if previous word is "the" and VERB is one of the

possibilities, delete VERB – Statistical:

• Machine learning from tagged corpus• Various methods

• Ref: Manning and Schutze, Foundations of Statistical Natural Language Processing, MIT Press 1999.


Parsing

• Find the structure:– Phrase structure (trees)

The cat sat on the mat– Dependency structure (links)

– The cat sat on the mat


Automatic parsing

• Big topic – see Jurafsky and Martin or other NLP

textbook

• Many methods too slow for large corpora

• Sketch Engine usually uses “shallow parsing”– Patterns of POS-tags– Regular expressions


Summary

• What is NLP?

• How can it help?– Tokenizing– Sentence splitting– Lemmatizing– POS-tagging– Parsing

Ontologies and Terminology

and how they relate to lexicography

Kivik 2013 23NLP. Corpus processing, Ontologies


Terminology

• Contains terms – for the objects and concepts in a domain – organized according to relations between

objects– Different language

• Same objects, so• Same organization• Different terms


Ontology• Artificial Intelligence• Like terminology with reasoning

• Tweety is-a swallow• A swallow is-a bird• Birds flyInference-----------------------• Tweety flies

the rationalist dream of automated reasoning

Bird• flies

swallow robin …

Tweety


Ontology

• Chris is-a dentist• Chris has-practice in Lancing• Chris works 9am-3pm Mon-Fri• …• You live-near Lancing• You want-to-visit dentist• You are-available …Inference---------------------------------------------------------Appointment, you, Chris, Lancing, 10am, Thursday


Items in an ontology

• Defined by relations in ontology

• Labelled (only) by words/phrases in various languages

X1EN: birdFR: oiseau

X2EN: swallowFR: hirondelle

…

•Ontology/things: language independent


Mismatches and gaps

Y1EN: body partsSP: …

Y2SP: dedo

Y5EN: armSP: bras

Y3EN: finger

Y4EN: toe


Thesaurus (eg Roget)

• Looks like a simple ontology – hierarchy only– supports inference?

• usually fudged

• Language independent?


WordNet

• Princeton Univ project, from ca 1990

• Thesaurus– Synonym sets or synsets– Hyponyms/hyperonyms, antonyms, part-of,

other lexical relations

• Free, online and available for download– Very widely used– Replicated for many languages, Global WN

Assn


Lexicon/dictionary

• About words

• Organized by words

• Language specific


Rationalists Empiricists

• Structure• Depth• Logic• Semantic Web

• Terminology

• Data• Breadth• Statistics• Google

• Lexicography


Terminology Lexicography

• What is the thing called– in languages x, y, z

• What kind of thing is it – Is-a link

– Its place in ontology

• Well-structured hierarchy

• How does the word behave?– what does it denote?

• Where does it occur?


Synthesis• Thesis

– Ontology, terminology, taxonomical lexicography• Semantic web, Roget, WordNets

• Antithesis– Corpus lexicography

• Synthesis: integrating• language-independent structure• language-specific word/phrase behaviour

– Corpus-based terminology– FrameNet


Summary

words

Lexicon

Thesaurus/Terminology

Ontology

things

kivik 2013nlp. corpus processing, ontologies1 the contribution of nlp corpus processing ontologies...

Documents

ontologies3 nlp

lemma help verb help

lemma help verb textword

computers speech processing

stems inflected forms

compound home help

noun textword

v helpingshelping n