![Page 1: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/1.jpg)
Feature Driven!Question Answering
Benjamin Van Durme
![Page 2: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/2.jpg)
Question Answering (QA) Natural Language Interface to Databases Information Retrieval (+ Information Extraction)
![Page 3: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/3.jpg)
Question Answering (QA) Natural Language Interface to Databases Information Retrieval (+ Information Extraction)
![Page 4: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/4.jpg)
1960’s, 1970’s, … build semantic parsers by hand
![Page 5: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/5.jpg)
Baseball (Green et al., 1961) “Where did the Red Sox play on July 7?”
[Where] did [the Red Sox] play (on [July 7])?
[ANSWER]
![Page 6: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/6.jpg)
Lunar (Woods, 1977) “What is the concentration of silicon in S10046?”
(For The X / (Dataline S10046 Overall SIO2) : T ; (For Every Y / Thing : (Equal X Y) ; (Printout Y) ))
[ANSWER]
![Page 7: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/7.jpg)
[fast forward]
![Page 8: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/8.jpg)
Difference? 1960’s, 1970’s, …
build grammars by hand 1990’s, 2000’s
induce grammars from annotations
![Page 9: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/9.jpg)
Zelle and Mooney (1996) “What is the capital of the state with the largest population?” answer(C,(capital(S,C),
largest(P, (state(S), population(S,P)))))
[ANSWER]
![Page 10: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/10.jpg)
Zettlemoyer and Collins (2005) “What states border texas?”
λx.state(x) ∧ borders(x, texas)
[ANSWER]
![Page 11: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/11.jpg)
1960’s, 1970’s, … build grammars by hand
1990’s, 2000’s induce grammars from annotated questions
2010’s induce grammars based on questions + answers
![Page 12: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/12.jpg)
Clarke et al. (2010) “What is the largest state that borders Texas?”
New Mexico
???
![Page 13: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/13.jpg)
Clarke et al. (2010) “What is the largest state that borders Texas?”
New Mexico
largest(state(next_to(const(texas)))
![Page 14: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/14.jpg)
structured query parse logic
![Page 15: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/15.jpg)
parse logic structured query
structured query
structured query
… parse
parse
… logic
logic
…
![Page 16: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/16.jpg)
Question Answering (QA) Natural Language Interface to Databases Information Retrieval (+ Information Extraction)
![Page 17: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/17.jpg)
Question Answering (QA) Natural Language Interface to Databases Information Retrieval (+ Information Extraction)
![Page 18: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/18.jpg)
“What is the something something something?” Lambda x [constraint1(x), constraint2(x), …]
[ANSWER]
![Page 19: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/19.jpg)
“What is the something something something?” Lambda x [constraint1(x), constraint2(x), …]
![Page 20: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/20.jpg)
“What is the something something something?” Lambda x [constraint1(x), constraint2(x), …]
![Page 21: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/21.jpg)
“What is the something something something?” Lambda x [constraint1(x), constraint2(x), …]
![Page 22: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/22.jpg)
“What is the something something something?”
[ANSWER]
![Page 23: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/23.jpg)
NIST TREC-8 QA!(Voorhees 1999)
200 questions, each with “snippet” answers “How many calories in a Big Mac?”
There are 549 calories in 1 burger (7.6 oz) of McDonald's Big Mac Burger. You'd need to walk …
![Page 24: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/24.jpg)
features
parse logic structured query
simple query
structured query
structured query
… parse
parse
… logic
logic
…
candidate
candidate
candidate
…
![Page 25: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/25.jpg)
Chen and Van Durme (in-progress) Yao and Van Durme (2014) Yao et al. (2014) Yao et al. (2013a,b,c) Van Durme et al. (2003) Nyberg et al. (2003)
![Page 26: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/26.jpg)
Chen and Van Durme (in-progress) Yao and Van Durme (2014)
Yao et al. (2014)
Yao et al. (2013a,b,c) Van Durme et al. (2003) Nyberg et al. (2003)
Xuchen Yao
![Page 27: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/27.jpg)
Question Answering (QA) Information Retrieval (+ Information Extraction) Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 28: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/28.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 29: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/29.jpg)
“What sport does Jennifer Capriati play?”
[ANSWER]
![Page 30: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/30.jpg)
“What sport does Jennifer Capriati play?”
“Tennis player Jennifer Capriati is 23” “Jennifer Capriati enjoys sports” …
“Capriati hasn't played on the tour since …”
![Page 31: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/31.jpg)
“What sport does Jennifer Capriati play?”
“Tennis player Jennifer Capriati is 23” “Jennifer Capriati enjoys sports” …
“Capriati hasn't played on the tour since …”
![Page 32: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/32.jpg)
“What sport does Jennifer Capriati play?”
“Tennis player Jennifer Capriati is 23” “Jennifer Capriati enjoys sports” …
“Capriati hasn't played on the tour since …”
![Page 33: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/33.jpg)
“What sport does Jennifer Capriati play?”
Tennis player Jennifer Capriati is 23
Tennis player Jennifer Capriati is 23
![Page 34: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/34.jpg)
“BIO” tagging, 3 types of hidden states: B-ANS (beginning of answer) I-ANS (inside of answer) O (outside of answer, i.e., not an answer)
Tennis player Jennifer Capriati is 23
B-‐ANS O O O O O
“What sport does Jennifer Capriati play?”
![Page 35: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/35.jpg)
Answer Extraction as Sequence Tagging!using a linear-chain Conditional Random Field (CRF)
conditional model p(y|x) : f(yt,yt-1,xt) : feature functions θ : feature weights (to learn)
)},,(exp{)(
1)|( 111
tttk
K
kk
T
tyyf
Zp x
xxy −
==∑∏= θ
Yt-‐1 Yt Yt+1
Xt-‐1 Xt Xt+1
hidden states Y
observation X
![Page 36: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/36.jpg)
Answer Extraction as Sequence Tagging!using a linear-chain Conditional Random Field (CRF)
conditional model p(y|x) : f(yt,yt-1,xt) : feature functions θ : feature weights (to learn)
Yt-‐1 Yt Yt+1
Xt-‐1 Xt Xt+1
hidden states Y
observation X
![Page 37: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/37.jpg)
NLP is full of features does question starts with “what”, “where”, “who”, … ? what parts of speech? any named entities, like a PERSON or a LOCATION ? automatic alignments between question and passage (tree) edit distance between question and passage …
![Page 38: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/38.jpg)
Joint Features Question starts with (“who”) AND Passage contains entity of type (PERSON)
Question contains word (“born”) AND Question starts with (“where”) AND Passage contains word (“birthplace”)
![Page 39: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/39.jpg)
Ranking Sentence Candidates
![Page 40: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/40.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 41: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/41.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 42: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/42.jpg)
“When was Alaska purchased?”
![Page 43: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/43.jpg)
“When was Alaska purchased?”
![Page 44: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/44.jpg)
“When was Alaska purchased?”
![Page 45: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/45.jpg)
“When was Alaska purchased?”
Standard search query: ( alaska, purchased )
![Page 46: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/46.jpg)
“When was Alaska purchased?”
“Eventually Alaska Airlines will allow all travelers who have purchased electronic tickets through any means …”
![Page 47: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/47.jpg)
Expected Answer Type
![Page 48: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/48.jpg)
Expected Answer Type
![Page 49: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/49.jpg)
“When was Alaska purchased?”
Important learned QA feature question starts with (“when”) AND
passage contains entity of type(DATE)
![Page 50: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/50.jpg)
“When was Alaska purchased?”
Standard search query: ( alaska, purchased )
Improved query: ( alaska, purchased ) & contains( DATE )
![Page 51: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/51.jpg)
“When was Alaska purchased?”
Improved query: ( alaska, purchased ) & contains( DATE )
Results improved search (17% relative MRR) improved extraction (20% relative F1 for top candidate)
![Page 52: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/52.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 53: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/53.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 54: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/54.jpg)
parse logic structured query
structured query
structured query
… parse
parse
… logic
logic
…
![Page 55: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/55.jpg)
features
parse logic structured query
simple query
structured query
structured query
… parse
parse
… logic
logic
…
candidate
candidate
candidate
…
![Page 56: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/56.jpg)
features
parse logic structured query
simple query
structured query
structured query
… parse
parse
… logic
logic
…
candidate
candidate
candidate
…
![Page 57: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/57.jpg)
features
parse logic structured query
simple query
structured query
structured query
… parse
parse
… logic
logic
…
candidate
candidate
candidate
…
![Page 58: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/58.jpg)
features simple query
candidate
candidate
candidate
…
Question Answering on FreeBase
![Page 59: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/59.jpg)
who is the brother of Justin Bieber?
![Page 60: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/60.jpg)
who is the brother of Justin Bieber?
![Page 61: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/61.jpg)
who is the brother of Justin Bieber?
![Page 62: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/62.jpg)
who is the brother of Justin Bieber?
![Page 63: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/63.jpg)
features who is the brother of Justin Bieber?
![Page 64: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/64.jpg)
features simple query
Justin Bieber
![Page 65: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/65.jpg)
Freebase Topic Graph
Justin Bieber
![Page 66: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/66.jpg)
Freebase Topic Graph
![Page 67: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/67.jpg)
Freebase Topic Graph
![Page 68: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/68.jpg)
features simple query
London? Person? Male? Jaxon Bieber Jasmin Bieber? …
![Page 69: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/69.jpg)
Features on Graph!extract features for each node
Justin Bieber Jazmyn Bieber Jaxon Bieber
has:awards won has:sibling has:sibling
has:place of birth gender:female gender:male
type:person type:person type:person
... ... ...
![Page 70: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/70.jpg)
Features on Question
![Page 71: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/71.jpg)
Joint Features
on graph on question • has:awards_won • has:place_of_birth • has:sibling • type:person • …
Jus@n Bieber
• has:sibling • gender:female • type:person • …
Jazmyn Bieber
• has:sibling • gender:male • type:person • …
Jaxon Bieber
for every edge e(s,t), extract: s, t, s|t, and s|e|t
• qword=what • qfocus=name • qverb=be • qtopic=person • qword=what|cop|qverb=be • qword=what|nsubj|qfocus=name • brother|nn|qtopic=person • …
![Page 72: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/72.jpg)
Evaluation
Data: WebQuestions Berant, et. al. (2013) 5810 questions crawled from Google Suggest
![Page 73: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/73.jpg)
Evaluation
Data: WebQuestions 5810 questions crawled from Google Suggest Berant et al. (2013)
![Page 74: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/74.jpg)
Evaluation
which states does the connecticut river flow through? who does david james play for 2011? what date was john adams elected president? what kind of currency does cuba use? who owns the cleveland browns? …
Data: WebQuestions 5810 questions crawled from Google Suggest Berant et al. (2013)
![Page 75: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/75.jpg)
Model
L1 regularized Logistic Regression high-performance toolkit: Classias (Okazaki, 2009) Start with: 7 million feature types extracted Train for: 4 hours Result: 30 thousand features with non-zero weight
![Page 76: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/76.jpg)
Our Results
0 10 20 30 40 50 60
Berant et al. '13
Yao and Van Durme, '14
Berant and Liang, '14
Yih et al. '15
F1 Avg
Accuracy
![Page 77: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/77.jpg)
Stanford
0 10 20 30 40 50 60
Berant et al. '13
Yao and Van Durme, '14
Berant and Liang, '14
Yih et al. '15
F1 Avg
Accuracy
![Page 78: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/78.jpg)
Microsoft Research
0 10 20 30 40 50 60
Berant et al. '13
Yao and Van Durme, '14
Berant and Liang, '14
Yih et al. '15
F1 Avg
Accuracy
![Page 79: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/79.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 80: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/80.jpg)
Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB
![Page 81: Feature Driven Question Answering€¦ · Feature driven answer extraction Feature driven retrieval Feature driven QA on a KB. ... Answer Extraction as Sequence Tagging! using a linear-chain](https://reader034.vdocument.in/reader034/viewer/2022042518/5f87c3390cd6a675e9429e4f/html5/thumbnails/81.jpg)
This work supported by …