extraction of opinions on the web richard johansson presentation at the lk summer school august 31,...
Post on 17-Dec-2015
214 Views
Preview:
TRANSCRIPT
Extraction of Opinions on the Web
Richard Johansson
Presentation at the LK summer school
August 31, 2011
Computer Science and Engineering Department
University of Trento
Email: johansson@disi.unitn.it
Funded by EU FP7: LivingKnowledge and EternalS
Personal Background
Defended doctoral dissertation in December
2008 at Lund University, Sweden
I now work as a postdoctoral researcher at the
University of Trento, Italy
PhD work focused on NLP tasks such as
syntactic parsing and shallow-semantic
extraction
Postdoc work on the applications of these
methods in areas such as opinion extraction
Overview
Introduction
Coarse-grained methods
Fine-grained methods
Resources
Advanced topics: recent research from LK
Introduction
Extraction of opinions expressed on the web is a
task with many practical applications“give me all positive opinions expressed by Sarkozy last week”
“what is the overall perception (positive/negative) on the New Start treaty?”
“Vaclav Klaus expressed his [disapproval] of
the treaty while French Prime Minister
Sarkozy [supported] it.”
Direct applications
Consumer informationQuickly surveying evaluations from other consumers
Conversely, companies may survey what customers think
Social and political sciencesSurveying popular opinion on contentious issues
Track the development of opinion over time
Measure the effect of some event on opinions
Indirect applications
Retrieval systemsgiven a topic, identify documents that express attitudes toward this topic
Question-answering systemsObvious: What does X think about Y?
Also: Filtering out opinionated text before returning answers
A note on terminology
Opinion extraction/analysis/mining etc
Sentiment analysis/extraction
Subjectivity analysis/extraction
Etc etc etc
Coarse-grained Opinion Extraction
Classification of fairly large units of text (e.g.
documents)
Examples:Distinguish editorials from “objective” news text
Given a review (product, movie, restaurant, …), predict the number of stars
Lexicon-based Methods
Simplest solution: count “positive” and “negative”
words listed in some lexiconAlso weighted
Lexicons may be generic or domain-specific
Example (with SentiWordNet, first sense):“This movie is awful with really boring actors”
awful: 0.875 negative
really: 0.625 positive
boring; 0.25 negative
Classification using machine learning
Coarse-grained opinion extraction is a type of
text categorization
Categorize the textAs factual or opinionated
As positive or negative (or the number of stars)
We may then obviously apply classical text
categorization methods (Pang and Lee, 2002)
Classification using machine learning
Represent a document using a bag of words
representation (i.e. a histogram)
Optionally, add extra features for words that
appear in some lexicon
Apply some machine learning method to learn to
separate the documents into classes (e.g. SVM,
MaxEnt, Naïve Bayes, …)
But the context…
“The price is high – I saw many cheaper options
elsewhere”
In practice, expressions of opinion are highly
context-sensitive: Unigram (BOW or lexicon)
models may run into difficulties
Possible solutions:Bigrams, trigrams, …
Syntax-based representations
Very large feature spaces: feature selection needed
Domain Adaptation
Problem: an opinion classifier trained on one
collection (e.g. reviews of hotels) may not
perform well on a collection from a different
domain (e.g. reviews of cars)
We may apply domain adaptation methods
(Blitzer et al., 2007, inter alia)
Similar methods may be applied for lexicon-
based opinion classifiers (Jijkoun et al., 2010)
Structural Correspondence Learning (Blitzer et al., 2007)
Idea:Some pivot features generalize across domains (e.g. “good”, “awful”)
Some features are completely domain-specific (“plastic”, “noisy”, “dark”)
Find correlations between pivot and domain-specific
Example experiment:DVD movies -> kitchen appliances
Baseline 0.74, upper bound 0.88
With domain adaptation: 0.81
Fine-grained Opinion Extraction
We may want to pose more complex queries:“give me all positive opinions expressed by Sarkozy last week”
“what is the overall perception (positive/negative) on the New Start treaty?”
“what is good and what is bad about the new Canon camera?”
“Vaclav Klaus expressed his [disapproval] of
the treaty while French Prime Minister
Sarkozy [supported] it.”
Common subtasks
Mark up opinion expressions in the text
Label expressions with polarity values
Find opinion holders for the opinions
Find the topics (targets) of the opinions
Opinion Expressions
An opinion expression is a piece of text that allows us to conclude that some entity has some opinion – a private state
The MPQA corpus (Wiebe et al., 2005) defines two main types of expressions:
Direct-subjective: typically emotion, communication, and categorization verbs
Expressive subjective: typically qualitative adjectives and “loaded language”
Examples of opinion expressions
I [love]DSE this [fantastic]ESE conference.
[However]ESE, it is becoming [rather
fashionable]ESE to [exchange harsh words]DSE
with each other [like kids]ESE.The software is [not so easy]ESE to use.
Opinion Holders
For every opinion expression, there is an associated opinion holder.
Also annotated in the MPQA
Our system finds three types of holders:Explicitly mentioned holders in the same sentence
The writer of the text
Implicit holder, such as in passive sentences (“he was widely condemned”)
Examples of opinion holders
Explicitly mentioned holder: I [love]DSE this [fantastic]ESE conference.
Writer (red) and implicit (green): [However]ESE, it is becoming [rather fashionable]ESE to [exchange harsh words]DSE with each other [like kids]ESE.
Nested structure of opinion scopes
Sharon [insinuated]ESE+DSE that Arafat [hated]DSE Israel.
Writer: negative opinion on Sharon
Sharon: negative opinion on Arafat
Arafat: negative opinion on Israel
The MPQA corpus annotates the nested structure of opinion/holder scopes
Our system does not take the nesting into account
Opinion polarities
Every opinion expression has a polarity: positive, negative, or neutral (for non-evaluative opinions)
I [love] this [fantastic] conference.
[However], it is becoming [rather fashionable] to [exchange harsh words] with each other [like kids].
The software is [not so easy] to use.
Tagging Opinion Expressions
The obvious approach – which we used as a
baseline – would be a standard sequence
labeler with Viterbi decoding.
Sequence labeler using word, POS tag, and
lemma features in a sliding window
Can also use prior polarity/intensity features
derived from the MPQA subjectivity lexicon.
This was the approach by Breck et al. (2007)
Extracting Opinion Holders
For opinion holder extraction, we trained a
classifier based on techniques common in
semantic role labeling
Applies to the noun phrases in a sentence
A separate classifier detects implicit and writer
opinion holders
At prediction time, the opinion holder candidate
with the maximal score is selected
Syntactic structure and semantic roles
We used the LTH syntactic/semantic parser to extract features (Johansson and Nugues, 2008)
Outputs dependency parse trees and semantic role structures
Classifying Expression Polarity
Given an opinion expression, assign a polarity
label (Positive, Neutral, Negative)
SVM classifier with BOW representation of the
expression and its context, lexicon features
Resources: Collections
Pang: Movie reviews (pos/neg)http://www.cs.cornell.edu/people/pabo/movie-review-data
Liu: Product featureshttp://www.cs.uic.edu/~liub/FBS/CustomerReviewData.zip
Dredze: Multi-domain product reviews (pos/neg)http://www.cs.jhu.edu/~mdredze/datasets/sentiment
MPQA: Fine-grained annotation: expressions, holder, polarities, intensities, holder coreference
http://www.cs.pitt.edu/mpqa/databaserelease
Resources: Lexicons
MPQA lexiconhttp://www.cs.pitt.edu/mpqa/lexiconrelease/collectinfo1.html
SentiWordNethttp://sentiwordnet.isti.cnr.it
Advanced topic 1: Opinion extraction with an interaction model
Previous work used bracketing methods with local features and Viterbi decoding
In a sequence labeler using local features only, the model can’t take into account the interactions between opinion expressions
Opinions tend to be structurally close in the sentence, and occur in patterns, for instance
Verb of categorization dominating evaluation:
He denounced as a human rights violation …
Discourse connections:
Zürich is beautiful but its restaurants are expensive
Interaction (opinion holders)
For verbs of evaluation/categorization, opinion holder extraction is fairly easy (basically SRL)
They may help us find the holder of other opinions expressed in the sentence:
He denounced as a human rights violation …
This is a human rights violation …
Linguistic structure may be useful to determine whether two opinions have the same holder
Interaction (polarity)
The relation between opinion expressions may influence polarity:
He denounced as a human rights violation …
Discourse relations are also important:Expansion:
Zürich is beautiful and its restaurants are good
Contrast:
Zürich is beautiful but its restaurants are expensive
Learning the Interaction model
We need a new model based on interactions between opinions
We use a standard linear model:
We decompose the feature representation:
But: Exact inference in a model with interactions is intractable (can be reduced to weighted CSP)
Approximate inference
Apply a standard Viterbi-based sequence labeler
based on local context features but no structural
interaction features.
Generate a small candidate set of size k.
Generate opinion holders/polarities for every
proposed opinion expression.
Apply a reranker using interaction features –
which can be arbitrarily complex – to pick the top
candidate from the candidate set.
Evaluation
(Johansson and Moschitti 2010a, 2010b, 2011)
Opinion markup F-measure
Baseline 53.8
Reranked 58.5
Holder identification F-measure
Baseline 50.8
Extended 54.2
Markup + polarity F-measure
Baseline 45.7
Extended 49.7
Advanced topic 2: Extraction of Feature Evaluations
Extraction of evaluations of product features
(Hu and Liu, 2004)
“This player boasts a decent size and weight, a
relatively-intuitive navigational system that
categorizes based on id3 tags, and excellent
sound”size +2, weight +2, navigational system +2, sound +2
We used only the signs (positive/negative)
Extraction of Feature Evaluations
We built a system that used features derived from the MPQA-style opinion expressions
We compared with two baselines:Simple baseline using local features only
Stronger baseline using sentiment lexicon
References
E. Breck, Y. Choi, C. Cardie. Identifying expressions of opinion in context. Proc.
IJCAI 2007.
J. Blitzer, M. Dredze, F. Pereira. Biographies, Bollywood, Boom-boxes and
Blenders: Domain adaptation for sentiment classification. Proc. ACL 2007.
Y. Choi, C. Cardie. Hierarchical sequential learning for extracting opinions and
their attributes. Proc. ACL 2010.
M. Hu, B. Liu. Mining opinion features in customer reviews. Proc. AAAI-2004.
V. Jijkoun, M. de Rijke, W. Weerkamp. Generating focused topic-specific
sentiment lexicons. Proc. ACL-2010.
R. Johansson, A. Moschitti. Syntactic and semantic structure for opinion
expression detection. Proc. CoNLL-2010.
R. Johansson, A. Moschitti. Reranking models in fine-grained opinion analysis.
Proc. Coling-2010.
References
R. Johansson, A. Moschitti. Extracting opinion expressions and their polarities –
exploration of pipelines and joint models. Proc. ACL-2011.
R. Johansson, P. Nugues. Dependency-based syntactic–semantic analysis with
PropBank and NomBank. Proc. CoNLL-2008.
B. Pang, L. Lee. A sentimental education: Sentiment analysis using subjectivity
summarization based on minimum cuts. Proc. ACL-2004.
S. Somasundaran, G. Namata, J. Wiebe, L. Getoor. Supervised and
unsupervised methods in employing discourse relations for improving
opinion polarity classification. Proc. EMNLP-2009.
J. Wiebe, T. Wilson, C. Cardie. Annotating expressions of opinions and
emotions in language. LRE, 39(2-3), 2005.
Acknowledgements
We have received funding from the European
Community’s Seventh Framework Programme
(FP7/2007-2013) under the following grants:Grant 231126: LivingKnowledge – Facts, Opinions and Bias in Time,
Grant 247758:Trustworthy Eternal Systems via Evolving Software, Data and Knowledge (EternalS).
We would also like to thank Eric Breck and Yejin
Choi for explaining their results and experimental
setup.
top related