Download - Easy-First Coreference Resolution Veselin Stoyanov and Jason Eisner Johns Hopkins University
Coreference Resolution
• Given a peace of text, identify all noun phrases that refer to the same real-world entity
Coreference Resolution
• Given a peace of text, identify all noun phrases that refer to the same real-world entity
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Coreference Resolution
• Given a peace of text, identify all noun phrases that refer to the same real-world entity
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Coreference Resolution
• Given a peace of text, identify all noun phrases that refer to the same real-world entity
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Coreference Resolution
• Typical pairwise approaches• But, inherently global task
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Coreference Resolution
• Inherently global task
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Coreference Resolution
• Inherently global task
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Global Approaches to Coreference Resolution
• A difficult search problem– Exponentially many possible clusterings
• Recent approaches have been successful:– [Culotta et al., 2007] – train a global coherence
measure; use greedy inference at test time– [Poon and Domingos, 2008] – use an unsupervised
approach based on MLNs to enforce global properties
A Different Approach
• A Multi-Pass Sieve [Raghunathan et al., 2010]– Build coreference clusters incrementally– Applying sieves (hand-specified rules) of
decreasing precision and increasing recall– Winner of the CoNLL-2011 shared task
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match2. Precise Constructs
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match2. Precise Constructs
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match2. Precise Constructs3. Strict Head match
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match2. Precise Constructs3. Strict Head match
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match2. Precise Constructs3. Strict Head match………….
7. Pronoun MatchingFormer Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010]
• Sieves:1. Exact match2. Precise Constructs3. Strict Head match………….
7. Pronoun MatchingFormer Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Easy-First Coreference Resolution
• Use the same idea:– Greedily build clusters– Using more information for the less confident
decisions• But learn automatically what constitutes an
“easy” (confident) decision– Leads to empirical improvement over the Multi-
Pass Sieve approach and a competitive baseline
Easy-First Coreference Resolution
• Incrementally and greedily build clusters– Using machine learning
• Learn policy for the test time inference procedure
Easy-First Coreference: Talk Overview
• Test-Time Inference• Training Algorithm• Experimental results
Test-Time Inference
• Begin with each NP in a separate cluster
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
Cluster 3SemType: personGender: maleNumber: singleNames: NIL
Cluster 3SemType: personGender: maleNumber: singleNames: NIL
• Begin with each NP in a separate cluster
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
SemTypeSame = False
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
SemTypeSame = FalseSemTypeComp = True
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
SemTypeSame = FalseSemTypeComp = TrueNumberSame = True
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
SemTypeSame = FalseSemTypeComp = TrueNumberSame = TrueRoleAppositive = True
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
SemTypeSame = FalseSemTypeComp = TrueNumberSame = TrueRoleAppositive = TrueSentDistance = 0
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
SemTypeSame = FalseSemTypeComp = TrueNumberSame = TrueRoleAppositive = TrueSentDistance = 0
Globalfeatures
Localfeatures
• Compute a feature vector for each pair of clusters
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
f(joinij)
• Compute a weight for each join action
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
w.f(joinij)
• Compute a weight for each join action
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
w.f(joinij)weightij=
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
w.f(joinij)aij=argmaxij
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 1SemType: uknwnGender: uknwnNumber: singleNames: NIL
Cluster 2SemType: personGender: maleNumber: singleNames: Mike Tyson
w.f(joinij)a12=argmaxij
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Cluster 15SemType: personGender: maleNumber: singleNames: Mike Tyson
w.f(joinij)a12=argmaxij
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Perform the action with the highest weight
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
• Until the action with the highest weight is the special action HALT
Test-Time Inference
Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens.In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.
Time complexity
• Let n = the number of mentions in a document• At most n clusters at any time• s = number of joins performed by the
algorithm• s<=n at most n joins before HALT (usually
much less)
Time complexity
• Each step requires:– Popping the top action -- O(logn) (priority queue)– Performing the join – O(1)– Deleting O(n) old actions – O(nlogn)– Computing scores for O(n) new actions -- O(n)– Inserting those -- O(nlogn) or O(n)
• Total runtime = O(n2logn)+O(snlogn)
Learning
• Learn weight vector w that leads the Easy-First algorithm to good clusterings
• At each step we are given a state s and a set of possible actions {joinij}∪{HALT}
• We want to learn what is “the best” action in each state
• But the current state s depends on the action taken so far
Reinforcement Learning
• A classic reinforcement learning setting• Act in the world and learn “good” actions in
each state• We tried several reinforcement learning
algorithms– Policy gradient [Sutton et al., 2000]– Q-learning [Watkins and Dayan, 1992]– No success: distant supervision; too many possible
states
Perceptron-Style Training
• A version of the early update structured perceptron [Collins and Roark, 2004]– Inspired by the success of easy-first parsing
[Goldberg and Elhadad, 2010]• The algorithm:
– Keep acting in the world until the next action will result in an error (reduction in score)
– Update weights away from the top action and towards the highest scoring positive action
Training algorithm
• Perform N iterations:– For each document d in the training set do:
• Initialize C so that each mention in its own cluster• Initialize A = {joinij}∪{HALT}
• repeat until amax != HALT:– for k steps do:
» amax =A.peekMax()
» if amax.isPositive() break
» apos =A.peekMaxPositive()
» w+=(feat(apos,C)-feat(amax ,C))» Recompute scores in A using new w
update weight
Training algorithm
• Perform N iterations:– For each document d in the training set do:
• Initialize C so that each mention in its own cluster• Initialize A = {joinij}∩{HALT}
• repeat until amax != HALT:– Update weights– amax =A.popMax()
– if amax is a join
» perform amax
Experimental Setup
• Five datasets:– MUC6, MUC7, ACE03, ACE04, ACE05
• Two evaluation metrics: – MUC score– B^3 score
• Implemented in the Reconcile research platform [Stoyanov et al. 2010]– Code available upon request
• Two settings:– Automatic mentions– Gold (manual) mentions
Baselines
• A pairwise baseline [Stoyanov et al. 2009]• Reimplementation of the pairwise baseline using
different features• Easy-First coreference resolution
– Easy-Firstpercep – weights from the pairwise baseline
– Easy-Firststruct – weights from our training algorithm
• Only for the gold mentions data– Entity-centered approach [Haghighi and Klein, 2009]– Multi-pass sieve [Raghunathan et al., 2010]
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2Easy-First percep 71.5 68.4 64.4 58.1 82.8 74.1 80.3 68.4 76.0 70.0
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2Easy-First percep 71.5 68.4 64.4 58.1 82.8 74.1 80.3 68.4 76.0 70.0Easy-First struct 72.1 68.2 64.6 57.7 83.1 74.5 80.9 68.8 75.9 70.9
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2Easy-First percep 71.5 68.4 64.4 58.1 82.8 74.1 80.3 68.4 76.0 70.0Easy-First struct 72.1 68.2 64.6 57.7 83.1 74.5 80.9 68.8 75.9 70.9
Results: Automatic Mentions
MUC6 MUC7 ACE03 ACE04 ACE05 B^3 MUC B^3 MUCB^3 MUC B^3 MUC B^3 MUC Reconcile 70.9 68.5 65.9 62.8 79.4 67.9 76.5 62.0 73.7 67.4Pairwise baseline 70.7 67.9 62.2 57.9 81.7 73.5 79.4 68.0 75.7 70.2Easy-First percep 71.5 68.4 64.4 58.1 82.8 74.1 80.3 68.4 76.0 70.0Easy-First struct 72.1 68.2 64.6 57.7 83.1 74.5 80.9 68.8 75.9 70.9
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Pairwise baseline 72.7 88.2 75.9 78.9
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Pairwise baseline 72.7 88.2 75.9 78.9
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Pairwise baseline 72.7 88.2 75.9 78.9Easy-First percep 73.3 88.2 78.4 79.8
Results: Gold Mentions
MUC6 ACE04
B^3 MUC B^3 MUC
Haghighi and Klein 75.0 81.9 76.9 76.5
Raghunathan et al. 73.2 77.7 78.9 78.1
Stoyanov et al. 76.1 88.2 77.8 76.2
Pairwise baseline 72.7 88.2 75.9 78.9Easy-First percep 73.3 88.2 78.4 79.8Easy-First struct 77.5 88.2 81.8 80.1
Conclusions
• A new approach to coreference using greedy incremental clustering– More accurate clustering by utilizing global
information as it becomes available for less confident decisions
• Learning policy for the test-time inference procedure leads to improvements