programming for linguistics: evaluating parser output
DESCRIPTION
TRANSCRIPT
![Page 1: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/1.jpg)
Programming for Linguistics December 16, 2013
Evaluating parser output
![Page 2: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/2.jpg)
Outline of topics
1. Preprocessing (10 slides)2. Processing (6 slides)3. Evaluation (1 slide)4. Results (1 slide)5. Improvements (1 slide)
![Page 3: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/3.jpg)
Programming for Linguistics
1 Preprocessing
Penn Treebank trees as NLTK Tree objects
![Page 4: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/4.jpg)
Programming for Linguistics
1 Preprocessing
![Page 5: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/5.jpg)
Programming for Linguistics
1 Preprocessing
![Page 6: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/6.jpg)
Programming for Linguistics
1 Preprocessing
![Page 7: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/7.jpg)
Programming for Linguistics
1 Preprocessing
![Page 8: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/8.jpg)
Programming for Linguistics
1 Preprocessing
![Page 9: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/9.jpg)
Programming for Linguistics
1 Preprocessing
![Page 10: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/10.jpg)
Programming for Linguistics
1 Preprocessing
![Page 11: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/11.jpg)
Programming for Linguistics
2 Processing
D. Klein and C. Manning 2002. A generative constituent-context model for improved grammar induction. In Proceedings of the ACL.
![Page 12: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/12.jpg)
Programming for Linguistics
2 Processing
Does this work for original trees and processed trees?
![Page 13: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/13.jpg)
Programming for Linguistics
2 Processing
Spans represented as label-span tuples:(‘label’, start, end)
![Page 14: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/14.jpg)
Programming for Linguistics
2 Processing
![Page 15: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/15.jpg)
Programming for Linguistics
2 Processing
![Page 16: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/16.jpg)
Programming for Linguistics
2 Processing
![Page 17: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/17.jpg)
Programming for Linguistics
2 Processing
Now what can we do with this information?
![Page 18: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/18.jpg)
Programming for Linguistics
4 Evaluation
![Page 19: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/19.jpg)
Programming for Linguistics
4 Evaluation
![Page 20: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/20.jpg)
Programming for Linguistics
5 Results
Evaluation after removing all complex labels and empty categories.GL = gold labelsAL = auto labelsM = matches
![Page 21: Programming for linguistics: Evaluating Parser Output](https://reader034.vdocument.in/reader034/viewer/2022051323/549fc3e5ac795924768b4a74/html5/thumbnails/21.jpg)
Programming for Linguistics
5 Improvements
● Cross brackets● Remove only labels/categories● Consider context● Constituent types● Depends on task…
Thanks!