graph-based dependency parsingsswayamd/talks/cle.pdf · chu-liu-edmonds - intuition every non-root...
TRANSCRIPT
![Page 1: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/1.jpg)
Graph-based Dependency ParsingChu-Liu-Edmonds and Camerini (k-best)
Swabha Swayamdipta Sam Thomson
Carnegie Mellon University
November 13, 2014
![Page 2: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/2.jpg)
Dependency Parsing
TurboParser output fromhttp://demo.ark.cs.cmu.edu/parse?sentence=I%20ate%20the%20fish%20with%20a%20fork.
![Page 3: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/3.jpg)
Dependency Parsing - Output Structure
A parse is an arborescence (aka directed rooted tree):
I Directed [Labeled] Graph
I Acyclic
I Single Root
I Connected and Spanning: ∃ directed path from root to everyother word
![Page 4: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/4.jpg)
Arc-Factored Model
Every possible labeled directed edge e between every pair of nodesgets a score, score(e).
G = 〈V ,E 〉 =
(O(n2) edges)
Example from Non-projective Dependency Parsing using Spanning Tree Algorithms McDonald et al., EMNLP ’05
![Page 5: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/5.jpg)
Arc-Factored Model
Every possible labeled directed edge e between every pair of nodesgets a score, score(e).
G = 〈V ,E 〉 =
(O(n2) edges)
Example from Non-projective Dependency Parsing using Spanning Tree Algorithms McDonald et al., EMNLP ’05
![Page 6: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/6.jpg)
Arc-Factored Model
Best parse is:
A(1) = arg maxA⊆G
s.t. A an arborescence
∑e∈A
score(e)
The Chu-Liu-Edmonds algorithm finds this argmax.
Example from Non-projective Dependency Parsing using Spanning Tree Algorithms McDonald et al., EMNLP ’05
![Page 7: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/7.jpg)
Arc-Factored Model
Best parse is:
A(1) = arg maxA⊆G
s.t. A an arborescence
∑e∈A
score(e)
The Chu-Liu-Edmonds algorithm finds this argmax.
Example from Non-projective Dependency Parsing using Spanning Tree Algorithms McDonald et al., EMNLP ’05
![Page 8: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/8.jpg)
Arc-Factored Model
Best parse is:
A(1) = arg maxA⊆G
s.t. A an arborescence
∑e∈A
score(e)
The Chu-Liu-Edmonds algorithm finds this argmax.
Example from Non-projective Dependency Parsing using Spanning Tree Algorithms McDonald et al., EMNLP ’05
![Page 9: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/9.jpg)
Projective / Non-projective
I Some parses are projective: edges don’t cross
I Most English sentences are projective, but non-projectivity iscommon in other languages (e.g. Czech, Hindi)
Non-projective sentence in English:
and Czech:
Examples from Non-projective Dependency Parsing using Spanning Tree Algorithms McDonald et al., EMNLP ’05
![Page 10: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/10.jpg)
Dependency Parsing Approaches
I Chart (Eisner, CKY)I Only produces projective parsesI O(n3)
I Shift-reduceI “Pseudo-projective” trick can capture some non-projectivityI O(n) (fast!), but inexact
I Graph-based (MST)I Can produce projective and non-projective parsesI O(n2) for arc-factored
![Page 11: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/11.jpg)
Dependency Parsing Approaches
I Chart (Eisner, CKY)I Only produces projective parsesI O(n3)
I Shift-reduceI “Pseudo-projective” trick can capture some non-projectivityI O(n) (fast!), but inexact
I Graph-based (MST)I Can produce projective and non-projective parsesI O(n2) for arc-factored
![Page 12: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/12.jpg)
Dependency Parsing Approaches
I Chart (Eisner, CKY)I Only produces projective parsesI O(n3)
I Shift-reduceI “Pseudo-projective” trick can capture some non-projectivityI O(n) (fast!), but inexact
I Graph-based (MST)I Can produce projective and non-projective parsesI O(n2) for arc-factored
![Page 13: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/13.jpg)
Chu-Liu-Edmonds
Chu and Liu ’65, On the Shortest Arborescence of a Directed Graph, ScienceSinica
Edmonds ’67, Optimum Branchings, JRNBS
![Page 14: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/14.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edge
In fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 15: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/15.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 16: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/16.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 17: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/17.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 18: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/18.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 19: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/19.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 20: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/20.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 21: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/21.jpg)
Chu-Liu-Edmonds - Intuition
Every non-ROOT node needs exactly 1 incoming edgeIn fact, every connected component needs exactly 1 incoming edge
I Greedily pick an incoming edge for each node.
I If this forms an arborescence, great!
I Otherwise, it will contain a cycle C .
I Arborescences can’t have cycles, so we can’t keep every edgein C . One edge in C must get kicked out.
I C also needs an incoming edge.
I Choosing an incoming edge for C determines which edge tokick out
![Page 22: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/22.jpg)
Chu-Liu-Edmonds
Consists of two stages:
I Contracting
I Expanding
![Page 23: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/23.jpg)
Chu-Liu-Edmonds - Contracting Stage
I For each non-ROOT node v , set bestInEdge[v ] to be itshighest scoring incoming edge.
I If a cycle C is ever formed:I contract the nodes in C into a new node vCI edges incoming to any node in C now get destination vCI edges outgoing from any node in C now get source vCI For each node u in C , and for each edge e incoming to u from
outside of C :I add bestInEdge[u] to kicksOut[e], andI set the score of e to be score[e]− score[bestInEdge[u]].
I Repeat until every non-ROOT node has an incoming edge andno cycles are formed
![Page 24: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/24.jpg)
An Example - Contracting Stage
V1
ROOT
V3V2
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
bestInEdge
V1V2V3
kicksOutabcdefghi
![Page 25: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/25.jpg)
An Example - Contracting Stage
V1
ROOT
V3V2
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
bestInEdge
V1 gV2V3
kicksOutabcdefghi
![Page 26: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/26.jpg)
An Example - Contracting Stage
V1
ROOT
V3V2
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
bestInEdge
V1 gV2 dV3
kicksOutabcdefghi
![Page 27: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/27.jpg)
An Example - Contracting Stage
V1
ROOT
V3V2
a : 5− 10 b : 1− 11 c : 1
f : 5d : 11
h : 9− 10
e : 4
i : 8− 11g : 10
V4
bestInEdge
V1 gV2 dV3
kicksOuta gb dcdefgh gi d
![Page 28: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/28.jpg)
An Example - Contracting Stage
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
bestInEdge
V1 gV2 dV3V4
kicksOut
a gb dcdefgh gi d
![Page 29: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/29.jpg)
An Example - Contracting Stage
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
bestInEdge
V1 gV2 dV3 fV4
kicksOut
a gb dcdefgh gi d
![Page 30: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/30.jpg)
An Example - Contracting Stage
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
bestInEdge
V1 gV2 dV3 fV4 h
kicksOut
a gb dcdefgh gi d
![Page 31: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/31.jpg)
An Example - Contracting Stage
V4
ROOT
V3
b : −10−−1 c : 1− 5
f : 5
a : −5−−1
h : −1
e : 4
i : −3
V5
bestInEdge
V1 gV2 dV3 fV4 hV5
kicksOut
a g, hb d, hc fdefgh gi d
![Page 32: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/32.jpg)
An Example - Contracting Stage
V5
ROOT
b : −9
a : −4 c : −4
bestInEdge
V1 gV2 dV3 fV4 hV5
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 33: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/33.jpg)
An Example - Contracting Stage
V5
ROOT
b : −9
a : −4 c : −4
bestInEdge
V1 gV2 dV3 fV4 hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 34: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/34.jpg)
Chu-Liu-Edmonds - Expanding Stage
After the contracting stage, every contracted node will haveexactly one bestInEdge. This edge will kick out one edge insidethe contracted node, breaking the cycle.
I Go through each bestInEdge e in the reverse order that weadded them
I lock down e, and remove every edge in kicksOut(e) frombestInEdge.
![Page 35: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/35.jpg)
An Example - Expanding Stage
V5
ROOT
b : −9
a : −4 c : −4
bestInEdge
V1 gV2 dV3 fV4 hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 36: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/36.jpg)
An Example - Expanding Stage
V5
ROOT
b : −9
a : −4 c : −4
bestInEdge
V1 a �gV2 dV3 f
V4 a �hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 37: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/37.jpg)
An Example - Expanding Stage
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
bestInEdge
V1 a �gV2 dV3 f
V4 a �hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 38: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/38.jpg)
An Example - Expanding Stage
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
bestInEdge
V1 a �gV2 dV3 f
V4 a �hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 39: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/39.jpg)
An Example - Expanding Stage
V1
ROOT
V3V2
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
bestInEdge
V1 a �gV2 dV3 f
V4 a �hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 40: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/40.jpg)
An Example - Expanding Stage
V1
ROOT
V3V2
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
bestInEdge
V1 a �gV2 dV3 f
V4 a �hV5 a
kicksOut
a g, hb d, hc fde ffgh gi d
![Page 41: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/41.jpg)
Chu-Liu-Edmonds - Recursive Definition
def Get1Best(〈V , E〉, ROOT ):””” returns best arborescence as a map from each node to its parent ”””for v in V \ ROOT:
bestInEdge[v ]← arg maxu∈V score[(u, v)]
if bestInEdge contains a cycle C :# build a new graph in which C is contracted into a single nodevC ← new Node
V ′ ← V ∪ {vC} \ CE ′ ← ∅for e = (t, u) in E :
if t 6∈ C and u 6∈ C :e′ ← e
elif t ∈ C and u 6∈ C :e′ ← new Edge (vC , u)
score[e′]← score[e]
elif u ∈ C and t 6∈ C :e′ ← new Edge (t, vC )
kicksOut[e′]← bestInEdge[u]
score[e′]← score[e]− score[kicksOut[e′]]
real[e′]← e # remember the original
E ′ ← E ′ ∪ {e′}A← Get1Best(〈V ′, E ′〉, ROOT )return {real[e′] | e′ ∈ A} ∪ (CE \ {kicksOut[A[vC ]]})
return bestInEdge
![Page 42: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/42.jpg)
Chu-Liu-Edmonds - Notes
I Efficient implementation:Tarjan ’77, Finding Optimum Branchings, Networks
Not recursive. Uses a union-find (a.k.a. disjoint-set) datastructure to keep track of collapsed nodes.
I Even more efficient:Gabow et al. ’86, Efficient Algorithms for Finding Minimum Spanning
Trees in Undirected and Directed Graphs, Combinatorica
Uses a Fibonacci heap to keep incoming edges sorted.Describes how to constrain ROOT to have only one outgoingedge
I There is a version where you don’t have to specify ROOT
![Page 43: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/43.jpg)
Chu-Liu-Edmonds - Notes
I Efficient (wrong) implementation:Tarjan ’77, Finding Optimum Branchings*, Networks
*corrected in Camerini et al. ’79, A note on finding optimum branchings,
Networks
Not recursive. Uses a union-find (a.k.a. disjoint-set) datastructure to keep track of collapsed nodes.
I Even more efficient:Gabow et al. ’86, Efficient Algorithms for Finding Minimum Spanning
Trees in Undirected and Directed Graphs, Combinatorica
Uses a Fibonacci heap to keep incoming edges sorted.Describes how to constrain ROOT to have only one outgoingedge
I There is a version where you don’t have to specify ROOT
![Page 44: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/44.jpg)
Chu-Liu-Edmonds - Notes
I Efficient (wrong) implementation:Tarjan ’77, Finding Optimum Branchings*, Networks
*corrected in Camerini et al. ’79, A note on finding optimum branchings,
Networks
Not recursive. Uses a union-find (a.k.a. disjoint-set) datastructure to keep track of collapsed nodes.
I Even more efficient:Gabow et al. ’86, Efficient Algorithms for Finding Minimum Spanning
Trees in Undirected and Directed Graphs, Combinatorica
Uses a Fibonacci heap to keep incoming edges sorted.Describes how to constrain ROOT to have only one outgoingedge
I There is a version where you don’t have to specify ROOT
![Page 45: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/45.jpg)
Chu-Liu-Edmonds - Notes
I Efficient (wrong) implementation:Tarjan ’77, Finding Optimum Branchings*, Networks
*corrected in Camerini et al. ’79, A note on finding optimum branchings,
Networks
Not recursive. Uses a union-find (a.k.a. disjoint-set) datastructure to keep track of collapsed nodes.
I Even more efficient:Gabow et al. ’86, Efficient Algorithms for Finding Minimum Spanning
Trees in Undirected and Directed Graphs, Combinatorica
Uses a Fibonacci heap to keep incoming edges sorted.Describes how to constrain ROOT to have only one outgoingedge
I There is a version where you don’t have to specify ROOT
![Page 46: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/46.jpg)
Camerini
![Page 47: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/47.jpg)
The Goal
Find exact k-best parses of a sentence given the weights of thegraph
But why?
I Model might not be correct, rerank k-best parses
I Constrained models (think global features)
![Page 48: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/48.jpg)
The Goal
Find exact k-best parses of a sentence given the weights of thegraph
But why?
I Model might not be correct, rerank k-best parses
I Constrained models (think global features)
![Page 49: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/49.jpg)
The Goal
Find exact k-best parses of a sentence given the weights of thegraph
But why?
I Model might not be correct, rerank k-best parses
I Constrained models (think global features)
![Page 50: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/50.jpg)
State of the art
I MSTParser and MaltParser produce an approximate k-best list
I TurboParser has no k-best feature
![Page 51: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/51.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 52: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/52.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 53: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/53.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 54: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/54.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.
We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 55: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/55.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 56: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/56.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 57: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/57.jpg)
Central Idea
1. We know how to get A(1), the 1-best arborescence.
2. There is at least one edge in A(1), which should not be in the2nd best arborescence.
3. Let us call this maximum impact edge, say e.We have an algorithm to find e.
4. Now consider two possibilities:I e is banned (this includes the 2nd best solution)I e is required (this includes the 1st best solution, A)
5. Partition the whole search space into two smaller subspaces.
Partition the solution space
Let reqd = set of edges that must be includedand banned = set of edges that must be excluded.
![Page 58: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/58.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
![Page 59: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/59.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
![Page 60: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/60.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
reqd = ∅banned = {e0}
reqd = {e0}banned = ∅
bane 0
reqe
0
![Page 61: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/61.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
reqd = ∅banned = {e0}
reqd = {e0}banned = ∅
bane 0
reqe
0
![Page 62: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/62.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
reqd = ∅banned = {e0}
reqd = {e0}banned = ∅
reqd = ∅banned = {e0, e1}
reqd = {e1}banned = {e0}
reqd = {e0}banned = {e2}
reqd = {e0, e2}banned = ∅
bane 0
reqe
0
bane 1
reqe1
bane 2
req e2
![Page 63: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/63.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
reqd = ∅banned = {e0}
reqd = {e0}banned = ∅
reqd = ∅banned = {e0, e1}
reqd = {e1}banned = {e0}
reqd = {e0}banned = {e2}
reqd = {e0, e2}banned = ∅
bane 0
reqe
0
bane 1
reqe1
bane 2
req e2
![Page 64: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/64.jpg)
Partitioning the solution space
reqd = ∅banned = ∅
reqd = ∅banned = {e0}
reqd = {e0}banned = ∅
reqd = ∅banned = {e0, e1}
reqd = {e1}banned = {e0}
reqd = {e0}banned = {e2}
reqd = {e0, e2}banned = ∅
bane 0
reqe
0
bane 1
reqe1
bane 2
req e2
ban
req
ban
req
banreq
ban
req
![Page 65: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/65.jpg)
Outline of the rest of the talk
I Find best arborescence A s.t. reqd ⊆ A ⊆ E \ bannedAlgorithm GetConstrained1Best(G, ROOT, reqd, banned)
I Find an edge e ∈ A \ reqd that defines the next partition.Algorithm FindEdgeToBan(G, ROOT,A, reqd, banned)
I Smart way to search the subspace of solutionsAlgorithm GetKBest(G, ROOT, k)
![Page 66: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/66.jpg)
Algorithm GetConstrained1Best(G, ROOT, reqd, banned)
Throw out edges before you feed the graph into Get1Best:
I Throw out every edge in banned
I Throw out every edge that competes with any edge in reqd
I Run Get1Best
RuntimeO(n2)
![Page 67: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/67.jpg)
Outline of the rest of the talk
I Find best arborescence A s.t. reqd ⊆ A ⊆ E \ bannedAlgorithm GetConstrained1Best(G, ROOT, reqd, banned)
I Find an edge e ∈ A \ reqd that defines the next partition.Algorithm FindEdgeToBan(G, ROOT,A, reqd, banned)
I Smart way to search the subspace of solutionsAlgorithm GetKBest(G, ROOT, k)
![Page 68: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/68.jpg)
Algorithm FindEdgeToBan(G, ROOT,A, reqd, banned)
I Input (A, reqd, banned),I For every edge e in A \ reqd, find the next best alternative
edge, alt(e)I this alternative cannot be in bannedI the source of this alternative must not be lower down in the
tree A
I Return eBan, the edge e in A \ reqd with the highest scoringalternative
I Return diff = score(eBan)− alt(eBan)
Return variables eBan, diff
RuntimeO(n2)
![Page 69: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/69.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
diff = +∞, eBan = ∅
![Page 70: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/70.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
diff = +∞, eBan = ∅
![Page 71: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/71.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
diff = +∞, eBan = ∅
![Page 72: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/72.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
alt(d) = b
diff = 10, eBan = d
![Page 73: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/73.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
V4
alt(d) = b
diff = 10, eBan = d
![Page 74: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/74.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
alt(d) = b
diff = 10, eBan = d
![Page 75: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/75.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
alt(d) = b
diff = 10, eBan = d
![Page 76: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/76.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
alt(f ) = e
diff = 1, eBan = f
![Page 77: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/77.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
alt(f ) = e
diff = 1, eBan = f
![Page 78: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/78.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V4
ROOT
V3
b : −10 c : 1
f : 5
a : −5
h : −1
e : 4
i : −3
V5
alt(f ) = e
diff = 1, eBan = f
![Page 79: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/79.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V5
ROOT
b : −9
a : −4 c : −4
alt(f ) = e
diff = 1, eBan = f
![Page 80: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/80.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V5
ROOT
b : −9
a : −4 c : −4
alt(a) = c
diff = 0, eBan = a
![Page 81: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/81.jpg)
Example run FindEdgeToBan
FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
V5
ROOT
b : −9
a : −4 c : −4
alt(a) = c
diff = 0, eBan = a
![Page 82: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/82.jpg)
Outline of the rest of the talk
I Find best arborescence A s.t. reqd ⊆ A ⊆ E \ bannedAlgorithm GetConstrained1Best(G, ROOT, reqd, banned)
I Find an edge e ∈ A \ reqd that defines the next partition.Algorithm FindEdgeToBan(G, ROOT,A, reqd, banned)
I Smart way to search the subspace of solutionsAlgorithm GetKBest(G, ROOT, k)
![Page 83: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/83.jpg)
Revisit partitioning
reqd = ∅banned = ∅
reqd = ∅banned = {e0}
reqd = {e0}banned = ∅
reqd = ∅banned = {e0, e1}
reqd = {e1}banned = {e0}
reqd = {e0}banned = {e2}
reqd = {e0, e2}banned = ∅
bane 0
reqe
0
bane 1
reqe1
bane 2
req e2
ban
req
ban
req
banreq
ban
req
![Page 84: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/84.jpg)
Algorithm GetKBest(G, ROOT, k)
I For every partition, save the following tuple:(wt, eBan, A, reqd, banned)
I A = GetConstrained1Best(G, ROOT, reqd, banned)corresponds to the best solution in the partition
I diff, eBan = FindEdgeToBan(G, ROOT,A, reqd, banned)
I wt = score(A)− diff
I Maintain a priority queue, Q containing all tuples sorted by wt
I Q determines which path to traverse in the search space
![Page 85: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/85.jpg)
GetKBest
def GetKBest(G, ROOT, k):””” returns k-best arborescences ”””reqd← ∅ banned← ∅A(1) ← Get1Best(〈G.V , G.E〉, ROOT)
diff, eBan← FindEdgeToBan(G, ROOT,A(1), reqd, banned)
Q.push((score(A(1))− diff, eBan,A(1), reqd, banned))for j in 2 . . . k:
(wt, eBan, A, reqd, banned)← Q.pop()if wt== −∞:
return A(1), . . . ,A(j−1)
¯reqd← reqd ∪ {eBan}¯banned← banned ∪ {eBan}
A(j) ← GetConstrained1Best(G, ROOT, reqd, banned′)
diff, eBan ← FindEdgeToBan(G, ROOT, A, ¯reqd, banned)
Q.push((score(A)− diff, eBan, A, ¯reqd, banned))
diff, eBan ← FindEdgeToBan(G, ROOT, A, reqd, ¯banned)
Q.push((wt− diff, eBan, A, reqd, ¯banned))
return A(1), . . . ,A(k)
RuntimeO(kn2)
![Page 86: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/86.jpg)
GetKBest example : 1-best
A(1) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
Q
![Page 87: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/87.jpg)
GetKBest example : 1-best
A(1) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
Q
(diff = 0, eBan = a)← FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
![Page 88: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/88.jpg)
GetKBest example : 1-best
A(1) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = ∅)
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10
Q
(21, a,A(1), ∅, ∅)
(diff = 0, eBan = a)← FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = ∅)
![Page 89: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/89.jpg)
GetKBest example: 2-best
A(2) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = {a})
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10Q
(21, a,A(1), ∅, ∅)
![Page 90: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/90.jpg)
GetKBest example: 2-best
A(2) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = {a})
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10Q
(21, a,A(1), ∅, ∅)
(diff = 1, eBan = f )← FindEdgeToBan(G, ROOT,A(1), reqd = {a}, banned = ∅)
![Page 91: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/91.jpg)
GetKBest example: 2-best
A(2) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = {a})
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10Q
(21, a,A(1), ∅, ∅)(20, f ,A(1), {a}, ∅)
(diff = 1, eBan = f )← FindEdgeToBan(G, ROOT,A(1), reqd = {a}, banned = ∅)
![Page 92: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/92.jpg)
GetKBest example: 2-best
A(2) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = {a})
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10Q
(21, a,A(1), ∅, ∅)(20, f ,A(1), {a}, ∅)
(diff = 1, eBan = f )← FindEdgeToBan(G, ROOT,A(1), reqd = {a}, banned = ∅)
(diff = 2, eBan = h)← FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = {a})
![Page 93: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/93.jpg)
GetKBest example: 2-best
A(2) ← GetConstrained1Best(G, ROOT, reqd = ∅, banned = {a})
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10Q
(21, a,A(1), ∅, ∅)(20, f ,A(1), {a}, ∅)(19, h,A(2), ∅, {a})
(diff = 1, eBan = f )← FindEdgeToBan(G, ROOT,A(1), reqd = {a}, banned = ∅)
(diff = 2, eBan = h)← FindEdgeToBan(G, ROOT,A(1), reqd = ∅, banned = {a})
![Page 94: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/94.jpg)
GetKBest example : 3-best
A(3) ← GetConstrained1Best(G, ROOT, reqd = {a}, banned = {f })
V1
ROOT
V2 V3
a : 5 b : 1 c : 1
f : 5d : 11
h : 9
e : 4
i : 8g : 10Q
(21, a,A(1), ∅, ∅)(20, f ,A(1), {a}, ∅)(19, h,A(2), ∅, {a})
![Page 95: Graph-based Dependency Parsingsswayamd/talks/cle.pdf · Chu-Liu-Edmonds - Intuition Every non-ROOT node needs exactly 1 incoming edge In fact, every connected component needs exactly](https://reader036.vdocument.in/reader036/viewer/2022070808/5f076e967e708231d41cf3e0/html5/thumbnails/95.jpg)
Conclusion
I Graph-based formulation for dependency parsing
I 1-best algorithm by Chu-Liu-Edmonds
I k-best algorithm by Camerini