w ord s ense d isambiguation by mahmood soltani tehran university 2009/12/24 1

WORD SENSE

DISAMBIGUATION

By Mahmood Soltani

Tehran University

2009/12/24

1

Introduction Knowledge source

• Lexical• Learned World

WSD Approaches• Knowledge-base• Curpos-base

• Unsupervised Approaches• Supervised Approaches

Some approach in detail

AGENDA

2

WSD 12/24/09 Tehran University

Defining word sense : The phenomenon of lexical ambiguity is traditionally

subdivided into polysemy and homonymy Polysemy: one work having several related meaning. Homonymy: two words have the same lexical form but

different etymoloies and unrelated meaning. In dictionaries, each word is listed with a number of discrete

senses and subsenses. possibly dierent from dictionary to dictionary

The first step involved in the task of WSD is the determination of dierent senses for all words in the text to be disambiguated, the sense inventory

3

INTRODUCTION


INTRODUCTION (CONT)

What is word sense disambiguation? WSD refers to a task that automatically assigns a sense,

selected from a set of pre-defined word senses to an instance of a polysemous word in a particular context.

identify the correct sense of an ambiguous word in a sentence.

determine which of the senses of an ambiguous word is invoked in a particular use of the word.

…

4


INTRODUCTION (CONT)

word sense disambiguation step:

Step 1: pre-defines sense A list of sense such as those found in everyday dictionaries A group of feature, catagories, or associated words An entry in transfer dictionary which includes translations in

another language Step 2 : assignment of words to senses.

The context of the word to be disambiguation External knowledge

5


INTRODUCTION (CONT)

Where is the use of WSD?

Sense disambiguation is an“intermediate task” ,which is not an end in itself, but rather is necessary at one level or another to accomplish most natural language processing tasks.

Machine translation Information retrieval Speech processing Text processing (spelling detection)

6


INTRODUCTION (CONT)

Conceptual Model of WSD WSD is the matching of sense knowledge and word

context. Sense knowledge can either be lexical knowledge

defined in dictionaries, or world knowledge learned from training corpora.

7


8

Lexical Knowledge Lexical knowledge is usually released with a dictionary. It

can be either symbolic, or empirical. It is the foundation of unsupervised WSD approaches.

Learned World Knowledge World knowledge is too complex or trivial to be

verbalized completely. So it is a smart strategy to automatically acquire world knowledge from the context of training corpora on demand by machine learning techniques

Trend Use the interaction of multiple knowledge sources to

approach WSD.

KNOWLEDGE SOURCES


LEXICAL KNOWLEDGE

Sense frequency Usage frequency of each sense of a word.

Sense gloss Sense definition and examples By counting common words

between the gloss and the context of the target word, we can nai

Concept Tree Represent the related concepts of the target in the form of

semantic network as is done by WordNet.

9


LEXICAL KNOWLEDGE (CONT)

Selectional Restrictions Syntactic and semantic restric

Subject Code Refers to the category to which one sense of the target word

belongs. Part of Speech (POS)

POS is associated with a subset of the word senses in both WordNet and LDOCE. That is, given the POS of the target, we may fully or partially disambiguate its sense (Stevenson & Wilks, 2001).

10


LEARND WORLD KNOWLEDGE

Domain-specific Knowledge Like selectional restrictions, it is the semantic restriction placed on

the use of each sense of the target word. The restriction is more specific.

Parallel Corpora Also called bilingual corpora, one serving as primary language, and

the other as a secondary language. Using some third-party software packages, we can align the major

words (verb and noun) between two languages. Because the translation process implies that aligned pair words share

the same sense or concept, we can use this information to sense the major words in the primary language (Bhattacharya et al. 2004).

11


EARLY WSD WORK IN MT The first attempts at automated sense disambiguation

were made in the context of machine translation Weaver (1949) : outlines the basis of an approach to WSD. Reifler’s (1955) “semantic coincidences” between a word and

its context,the role of syntactic relations, was also recognized Weaver’s (1949) Memorandum discusses the role of the

domain in sense disambiguation Oswald,1952, 1957; Oettinger, 1955; etc : micro-glossaries

contain only the meaning of a given word relevant for texts in a particular domain

Several researchers attempted to devise an “interlingua” based on logical and mathematical principles

…. without large-scale resources most of these ideas remained

untested and to large extent, forgotten until several decades later.

12


WSD APPROACHES

there are three ways to approach the problem of WSD: a knowledge-based approach, which uses an explicit

lexicon (machine readable dictionary (MRD), thesaurus) or ontology (e.g. WordNet).

Corpus-based disambiguation, where the relevant information about word senses is gathered from training on a large corpus.

Hybrid approach combining aspects of both of the forementioned methodologies

13


KNOWLEDGE-BASED APPROACH

WSD systems building on the information contained in MRDs use the available material in various ways: Lesk (1986) was the first to use dictionary denitions to WSD:

counts overlapping content words in the sense denitions & in the denitions of context words occurring nearby

Yarowsky (1992), the sense of a word is dened as its category in Roget's International Thesaurus Yarowsky (1992), thesense of a word is dened as its category in Roget's International Thesaurus

WordNet includes various potential sources of information Leacock et al. (1998) employ WordNet to counter data

sparseness.

14


CORPUS-BASED APPROACH

A corpus-based approach extracts information on word senses from a large annotated data collection. Distributional information about an ambiguous word refers to the

frequency distribution of its senses collocational or co-occurrence information part-of-speech …

15


CORPUS-BASED APPROACH(CONT)

There are two possible approaches to corpus-based WSD systems:

Supervised approaches use annotated training data and basically amount to a classication task

Unsupervised algorithms are applied to raw text material and annotated data is only needed for evaluation They correspond to a clustering task rather than a classication.

Bootstrapping , looks like supervised approaches, but it needs only a few seeds instead of a large number of training examples

16



Supervised exemplar-based: the k-nearest neighbor technique has been

employed most

Rule-base :use algorithms, e.g. decision lists, which search for discriminatory features in the training data

Probabilistic-base : use of dierent probabilistic classiers. Despite its relative simplicity, naive Bayes has been frequently applied

17



Unsupervised cluster the contexts of an ambiguous word into a

number of groups and discriminate between them without labeling them. A clear disadvantage is that, so far, the performance of unsupervised systems lies a lot lower than that of supervised systems

18


NAIVE BAYES CLASSIFIER

19

Test

Training


K-NN

20

Test

Training


Any Question?

21


w ord s ense d isambiguation by mahmood soltani tehran university 2009/12/24 1

Documents

word context

matching of sense knowledge

task of wsd

target word

ambiguous word

use of wsd

sense inventory

correct sense