combining relational and attributionalsimilarity for...

Combining Relational and Attributional Similarity

for Semantic Relation Classification

Preslav Nakov, National University of Singapore

Zornitsa Kozareva, University of Southern California

RANLP 2011

RANLP: Hissar, Bulgaria: September 14, 2011

2

What causes tumors to shrink?

The period of tumor shrinkage after radiation

therapy is often long and varied.

CAUSE-EFFECT

How do we build a system to classify the relation between two nouns?


3Nakov & Kozareva: Combining Relational and Attributional Similarity ... 3

Relation Extraction (between Nouns)

• Given a pair of nouns, identify the semantic

relation(s) between them

malaria mosquito

effect-cause

content-container

orange basket


4Nakov & Kozareva: Combining Relational and Attributional Similarity ...

Semantic Relations: Applications

� Help real applications:

�information extraction

�document summarization

�machine translation

�construction of thesauri and semantic networks

� Facilitate auxiliary tasks:

�word sense disambiguation

�language modeling

�paraphrasing

�recognizing textual entailment



� PART-WHOLE (Winston, Chaffin, and Hermann 1987)

� Component-Integral object

� cup-handle, kitchen-apartment, wing-bird

� Member-Collection

� soldier-army, professor-faculty, tree-forest

� Portion-Mass

� slice-pie, meter-kilometer

� Stuff-Object

� silk-dress, steel-car, and alcohol-wine

� Feature-Activity

� paying-shopping, chewing-eating

� Place-Area

� Geographic�oasis-desert, county-state, path-forest

� Geometric�the end (of a stick) is part of (that) stick

�the surface (of a lake) is part of (that) lake

�the side (of a building) is part of (that) building

Semantic Relations:Can Be Heterogeneous



� Thus, often handled using instance-based classifiers

� We need to measure SIMILARITY8

Semantic Relations:Can Be Heterogeneous



Two Kinds of SimilarityTurney (2006)

Rel1(X1,Y1) vs. Rel2(X2,Y2)

� Attributional Similarity

�Correspondence between attributes

�X1::X2 (mason :: carpenter)

�Y1::Y2 (stone :: wood)

� Relational Similarity

�Correspondence between relations

�Rel1 :: Rel2 (mason:stone :: carpenter:wood)

mason:stone

(a) teacher:chalk

(b) carpenter:wood(c) soldier:gun

(d) photograph:camera

(e) book:word



Measuring Relational SimilarityTurney (2006)

� Attributional similarity can be used to measure relational

similarity, e.g.

Fine for near analogies:

mason:stone

(a) teacher:chalk

(b) carpenter:wood(c) soldier:gun

(d) photograph:camera

(e) book:word

Bad for far analogies:

traffic:street

(a) ship:gangplank

(b) crop:harvest

(c) car:garage

(d) pedestrians:feet

(e) water:riverbed



Semantic Relation Extraction

Going back to semantic relations8

� Similar split between two lines of research:

� relational: rely on patterns that can connect the arguments (Hearst,92;Turney,05; Turney&Littman,05; Turney,06; Kim&Baldwin,06; Pantel&Pennacchiotti,06; O’Seaghdha&Copestake,07; Davidov et al.07; Davidov&Rappoport,08; Nakov& Hearst,08; Katrenko et al.,10)

� attributional: generalize the arguments using a lexical hierarchy (Rosario&Hearst,01; Rosario et al.,02; Girju et al.,05; Kim&Baldwin,07; O’Seaghdha,09)



Pattern-based Relation Extraction

� Pattern learning (relational)

� for context-dependent, episodic relations?

� e.g., CAUSE-EFFECT: My Friday’s exam causes me anxiety.

� Argument generalization (attributional)

� for context-independent, permanent relations?

� e.g., PART-WHOLE: door-car

� can benefit from pre-existing resources like WordNet

� BUT limited coverage, need for WSD, etc.



Our Objectives

� Combine relational and attributional similarity

� and study the relative importance for different relations

� Use no pre-existing lexical resources

� in the real-world, these resources have limited coverage

� Human-readable, explicit semantic representation

� ideallyC



Method



Our Web-based Approach at a Glance

• Learn from the Web

- paraphrasing verbs, prepositions, coordinating conjunctions

- hypernyms and co-hyponyms for each argument

• Compute vector similarity for kNN

malaria mosquito

carried by

transmitted via

spreadscauses

insectdisease fly



Learning Relational Patterns

� For “noun1 noun2”, query:

"noun1 THAT? * noun2"

"noun2 THAT? * noun1"

� Extract:

� V: verbs

� P: prepositions

� C: coordinating conjunctions

committee -- member



Learning Relational Patterns (cont.)

coffee -- guy



Sample of harvested patterns for

coffee -- guy

Learned Relational Patterns



Generalizing Arguments

• Doubly-anchored pattern has anchoring through

conjunctions or terms

- “ relation <seed> and * ”

- “ <seed> and * relation * ”

- “ * relation * and <seed> ”

• can mine both hypernyms and co-hyponyms

• achieves higher accuracy than (Etzioni,05; Pasca,04)

• easy to implement

“ making it vulnerable to predators such as jaguar and puma ”

“ big cats such as jaguar and puma. Similar animal

”“ car brands such as jaguar and land rover were known to be “


� Instantiate the noun arguments of each sentence with the

doubly-anchored lexico-syntactic pattern

� Submit each pattern as a query to Yahoo! Boss

� Harvested hypernyms and co-hyponyms

� Build a hypernym, co-hyponym vector


Argument Harvesting Procedure

coffee andsuch as

beveragedrink

food

teachocolate

cocoaproduct



Example: Learned (Co-)Hypernyms

“ * such as coffee and* ”

“* such as guy and * ”



Dataset



SemEval-1 task 4:

Classification of Semantic Relations between Nominals

Follow-up: SemEval-2 task 8

Multi-Way Classification of Semantic Relations Between Pairs of Nominals

Dataset


Dataset: SemEval-1 Task#4




Experiments and Evaluation



Classifier, Weights, Similarity

� Classifier: 1NN

� Weighting functions

�Frequencies

�TF.IDF

�TF.IDF w/ add-one smoothing for IDF

� Similarity Measures

�cosine

�Dice

�Lin’s measure



Features

� Relational

� verbs

� prepositions

� coordinating conjunctions

� Attributional

� nouns (arguments, attributes)

� hypernyms

� co-hyponyms



Combining Relational and Attributional Similarity

� Linear combination for relational and attributional similarity

� Sm: argument 1

� Sh: argument 2

� Sr: the relation

� Weights tuned on the training set with cross-validation

Also tried:

(a) Whyp = 1

(b) Wcoh = 1


Results: Overall Micro-averaged

� Combining attributional and relational similarity outperforms:

� all baselines by 0.5-19.5% absolute

(statistically significant in 15 out of 21 cases)

� the best system at SemEval-1 Task 4 (66.0%): by 5.3% absolute

� the state-of-the-art (Davidov&Rapport, 2008), who had 70.1%: by 1.2%

When using manually annotated WordNet senses was not allowed.



Results: Relational Similarity


� Instrument-Agency (laser--printer) and Product-Producer(honey--bee) are better characterized by patterns

� For Instrument-Agency

� the head (argument 2) is also important

� hypernyms are more important than co-hyponyms


Results: Attributional Similarity


� Theme-Tool (copyright--law) and Origin-Entity (olive--oil) are

best characterized by the properties of the arguments

� For Theme-Tool

� the head (argument 2) is most critical

� co-hyponyms are more important than hypernyms


Results: Both Similarities


� For Cause-Effect (growth -- hormone) both the modifier and the

relation are important

� For Cause-Effect

� the modifier (argument 1) is most critical

� again, co-hyponyms are more important than hypernyms


Results: Comparing to WordNet

� For some relations, the results outpeform those at SemEval-1

task 4, even when manual WordNet senses are allowed

� Origin-Entity: 77.8% vs. 72.8% (stat. significant)

� Theme-Tool: 74.7% vs. 74.6%




Conclusion



Conclusion

� Presented:

� Web-based approach to relation extraction

� Uses no pre-existing lexical resources

� Based on human-readable, explicit semantic representation

� paraphrases: verbs, prepositions, coordinating conjunctions

� hypernyms and co-hyponyms

� Studied the combination of relational and attributional similarity

� for various relations

� Achieved

� sizable improvements over a strong baseline

� small improvements over the state-of-the-art



Future Work

� We plan to

� use other relation inventories, e.g., SemEval-2 task 8

� model sentence context

� jointly generalize arguments and paraphrases

� use bootstrapping to mine more examples

� try explicit paraphrases beyond verbs/preps/conj.



Paraphrasing Semantics: Beyond Verbs

“onion tears”

tears from onions

tears due to cutting onion

tears induced when cutting onions

tears that onions induce

tears that come from chopping onions

tears that sometimes flow when onions are chopped

tears that raw onions give you

SemEval-2013 task 24C. Butnariu, I. Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. Ó Séaghdha, S. Szpakowicz, T. Veale



Future Work

� We plan to

� use other relation inventories, e.g., SemEval-2 task 8

� model sentence context

� jointly generalize arguments and paraphrases

� try explicit paraphrases beyond verbs/preps/conj.

� use bootstrapping to mine more examples

Thank you!



Detailed Results (1)

combining relational and attributionalsimilarity for...

Documents