deep learning for information retrieval - hang li outline •introduction to huawei noah’s ark lab...
TRANSCRIPT
![Page 1: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/1.jpg)
Opportunities and Challenges in Deep Learning for Information Retrieval
Hang Li
Noah’s Ark Lab, Huawei Technologies
![Page 2: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/2.jpg)
Talk Outline
• Introduction to Huawei Noah’s Ark Lab • Deep Learning – New Opportunities for Information
Retrieval • Three Useful Deep Learning Tools • Information Retrieval Tasks
– Image Retrieval – Retrieval-based Question Answering – Generation-based Question Answering – Question Answering from Knowledge Base – Question Answering from Database
• Discussions and Concluding Remarks
![Page 3: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/3.jpg)
![Page 4: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/4.jpg)
Noah’s Ark Lab is Research Lab Working on
Intelligent Mobile Devices
Intelligent Telecommunication Networks
Intelligent Enterprise
Data Mining &
Artificial Intelligence
![Page 5: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/5.jpg)
Software-defined Network
Network Maintenance
Network Planning and Optimization
Intelligent Telecommunication Networks
![Page 6: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/6.jpg)
Information Recommendation
Personal Information
Management Machine
Translation
Information Extraction
Natural Language Dialogue(Question
Answering)
Intelligent Mobile Devices
![Page 7: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/7.jpg)
Intelligent Enterprise
Customer Relationship Management
Supply Chain Management
Information and Knowledge
Management Communication
Human Resources Management
![Page 8: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/8.jpg)
Talk Outline
• Introduction to Huawei Noah’s Ark Lab • Deep Learning – New Opportunities for Information
Retrieval • Three Useful Deep Learning Tools • Information Retrieval Tasks
– Image Retrieval – Retrieval-based Question Answering – Generation-based Question Answering – Question Answering from Database – Question Answering from Knowledge Base
• Discussions and Concluding Remarks
![Page 9: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/9.jpg)
Overview of Information Retrieval
Information and Knowledge Base
Information Retrieval System
Query
Relevant Result
Intent: Key words, Question
Content: Documents, Images, Knowledge
Key Questions: How to Represent Intent and Content, How to Match Intent and Content
![Page 10: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/10.jpg)
Key is Matching
intent content
matching
•Indexing: for efficient retrieval •Ranking: when there are multiple result •Generation: when only return single answer
![Page 11: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/11.jpg)
Approach in Traditional IR
Query: star wars the force awakens reviews
Star Wars: Episode VII Three decades after the defeat of the Galactic Empire, a new threat
arises.
||||||||
,),(
dq
dqdqfVSM
0
0
1
q
1
0
1
d
• Representing query and document as word vectors • calculating cosine similarity between them
),( dqf
Document:
![Page 12: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/12.jpg)
Approach in Modern IR
• Conducting query and document understanding • Representing query and document as multiple feature vectors • Calculating multiple matching scores between query and document • Training ranker with matching scores as features using learning to rank
Document: Query: star wars the force awakens reviews
Star Wars: Episode VII Three decades after the defeat of the Galactic Empire, a new threat
arises.
qm
q
v
v
q
1
dn
d
v
v
d
1),( dqf(star wars)
(the force awakens) (reviews)
![Page 13: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/13.jpg)
Examples of Query Document Mismatch
Query Document Term Matching
Semantic Matching
seattle best hotel seattle best hotels
no yes
pool schedule swimmingpool schedule
no yes
natural logarithm transformation
logarithm transformation
partial yes
china kong china hong kong partial no
why are windows so expensive
why are macs so expensive
partial no
13
![Page 14: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/14.jpg)
Hard Problems in IR and NLP
How far is sun from earth?
How tall is Yao Ming?
A dog catching a ball
Name Height Weight
Yao Ming 2.29m 134kg
Liu Xiang 1.89m 85kg
The average distance between the Sun and the Earth is about 92,935,700 miles.
Key Questions: How to Represent Intent and Content, How to Match Intent and Content
Image Retrieval
Question Answering
Question Answering from Relational Database
![Page 15: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/15.jpg)
Representation and Matching Are Key Problems in IR
intent content matching
Deep Learning
Recent Progress: Deep Learning Enables Representation Learning and Matching in IR
![Page 16: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/16.jpg)
Talk Outline
• Introduction to Huawei Noah’s Ark Lab • Deep Learning – New Opportunities for Information
Retrieval • Three Useful Deep Learning Tools • Information Retrieval Tasks
– Image Retrieval – Retrieval-based Question Answering – Generation-based Question Answering – Question Answering from Knowledge Base – Question Answering from Database
• Discussions and Concluding Remarks
![Page 17: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/17.jpg)
Representation of Sentence Meaning
John loves Mary
Mary is loved by John
Mary loves John
Using high-dimensional real-valued vectors to represent the meaning of sentences
New finding: This is possible
![Page 18: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/18.jpg)
Word Representation: Neural Word Embedding (Mikolov et al., 2013)
)()(
),(log
cPwP
cwP
10 1 5
2 1
2.5 1
1w
2w
3w
1c 2c 3c4c 5c
7 1 1
2 3
1 1.5 2
1w
2w
3w
1t 2t 3t
TWCM matrix factorization
word embedding or word2vec
W
M
![Page 19: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/19.jpg)
Convolutional Neural Network (CNN) (Hu et al., 2014)
the cat sat on the mat
Concatenation
……
the cat sat on the mat
cat sat
the cat sat
the cat
sat on
cat sat on
cat sat
on the
sat on the
sat on
the mat
on the mat
on the
sat on
the cat sat
the cat
the mat
on the mat
sat on
convolution
max pooling
![Page 20: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/20.jpg)
Recurrent Neural Network (RNN) (Mikolov et al. 2010)
the cat sat …. mat
• On sequence of words • Variable length • Long dependency: LSTM or GRU
the cat sat on the mat
),( 1 ttt xhfh
tx
1th
![Page 21: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/21.jpg)
Talk Outline
• Introduction to Huawei Noah’s Ark Lab • Deep Learning – New Opportunities for Information
Retrieval • Three Useful Deep Learning Tools • Information Retrieval Tasks
– Image Retrieval – Retrieval-based Question Answering – Generation-based Question Answering – Question Answering from Knowledge Base – Question Answering from Database
• Discussions and Concluding Remarks
![Page 22: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/22.jpg)
DL for NLP @Noah Lab
Zhengdong Lu
Lifeng Shang Lin Ma Zhaopeng Tu Xin Jiang
![Page 23: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/23.jpg)
Image Retrieval
![Page 24: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/24.jpg)
Image Retrieval
Find the picture that I had dinner with my friends at an Italian restaurant in Hong Kong
• Scenario
– Image search on smartphone
– Key: matching queries to images
• Technology
– Deep model for matching text and image
query representation
image representation
index of images
Matching model
![Page 25: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/25.jpg)
Deep Match Model for Image and Text • Represent text and image as vectors and then match
the two vectors
• Word-level matching, phrase-level matching, sentence-level Matching
• Our model (CNN) work better than state of the art models (RNN)
Ma et al., ICCV 2015
A dog is catching a ball
CNN CNN
MLP
![Page 26: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/26.jpg)
Word-level Matching Model
• Adding image vector to word vectors
……
a dog is catching a ball
CNN
MLP
![Page 27: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/27.jpg)
Sentence-level Matching
• Combing image vector with sentence vector
……
a dog is catching a ball
CNN
MLP
![Page 28: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/28.jpg)
Demo
![Page 29: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/29.jpg)
Experimental Result
Our CNN Model outperforms all existing models using RNN
Flickr 30K images
![Page 30: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/30.jpg)
Retrieval based Question Answering
![Page 31: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/31.jpg)
Retrieval-based Question Answering
U: How far is Huawei Headquarter from Hong Kong Science Park? P: About 30km U: How can I get there? P: You can first take MTR train to Lo Ma Chow and then take a taxi
Retrieval System
Repository of Question Answer Pairs
Learning System
![Page 32: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/32.jpg)
Question Answering System - Retrieval based Approach
index of questions and
answers
matching
ranking
question
retrieval
retrieved questions and answers
ranked answers
matching models
ranking model
online
offline
best answer
matched answers
![Page 33: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/33.jpg)
Deep Match CNN - Architecture I
MLP
……
……
• First represent two sentences as vectors, and then match the vectors
Hu et al., NIPS 2014
![Page 34: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/34.jpg)
Deep Match CNN - Architecture II
• Represent and match two sentences simultaneously
• Two dimensional model
34
MLP
Matching Degree
…
2D convolution
more 2D convolution & pooling
max-pooling
1D convolution
sentence X
sen
ten
ce Y
![Page 35: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/35.jpg)
Retrieval based Approach: Accuracy = 70%
上海今天好熱,堪比新加坡。 上海今天热的不一般。 想去武当山 有想同游的么? 我想跟帅哥同游~哈哈
It is very hot in Shanghai today, just like Singapore . It is unusually hot. I want to go to Mountain Wudang, it there anybody going together with me? Haha, I want to go with you, handsome boy
Using 5 million Weibo Data
![Page 36: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/36.jpg)
Generation based Question Answering
![Page 37: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/37.jpg)
Generation-based Question Answering
Generation System
Learning System
U: How far is Huawei Headquarter from Hong Kong Science Park? P: About 30km U: How can I get there? P: You can first take MTR train to Lo Ma Chow and then take a taxi
![Page 38: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/38.jpg)
Natural Language Dialogue System - Generation based Approach
• Encoding questions to intermediate representations
• Decoding intermediate representations to answers
• Recurrent Neural Network (RNN)
question
answer
Encoder
Txxx 21x
Decoder
tyyy 21y
c
h
Context Generator
Shang et al., ACL 2015
![Page 39: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/39.jpg)
Encoder
Global Encoder Local Encoder
1x 2x txTx
1h 1th thTh
1ts
tc
… …
![Page 40: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/40.jpg)
Decoder
1c 2ctc Tc
1s 1ts tsTs
1y 1ty tyTy
… …
![Page 41: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/41.jpg)
Generation based Approach Accuracy = 76%
占中终于结束了。 下一个是陆家嘴吧? 我想买三星手机。 还是支持一下国产的吧。
Occupy Central is finally over. Will Lujiazui (finance district in Shanghai) be the next? I want to buy a Samsung phone Let us support our national brands
vs. Accuracy of translation approach = 26% Accuracy of retrieval based approach = 70%
![Page 42: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/42.jpg)
Demo
![Page 43: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/43.jpg)
Question Answering from Knowledge Base
![Page 44: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/44.jpg)
Question Answering from Knowledge Base
(Yao-Ming, spouse, Ye-Li) (Yao-Ming, born, Shanghai) (Yao-Ming, height, 2.29m) … … (Ludwig van Beethoven, place of birth, Germany) … …
Knowledge Base
Q: How tall is Yao Ming? A: He is 2.29m tall and is visible from space. (Yao Ming, height, 2.29m) Q: Which country was Beethoven from? A: He was born in what is now Germany. (Ludwig van Beethoven, place of birth, Germany)
Question Answering System
Q: How tall is Liu Xiang? A: He is 1.89m tall
Learning System
![Page 45: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/45.jpg)
GenQA
• Interpreter: creates representation of question using RNN
• Enquirer: retrieves top k triples with highest matching scores using CNN model
• Generator: generates answer based on question and retrieved triples using attention-based RNN
• Attention model: controls generation of answer
Short Term Memory
Long Term Memory
(Knowledge Base)
How tall is Yao Ming?
Interpreter
Enquirer
Generator
He is 2.29m tall
Attention Model
Key idea: • Generation of answer based on question and retrieved result • Combination of neural processing and symbolic processing
Yin et al. 2015
![Page 46: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/46.jpg)
Enquirer: Retrieval and Matching
• Retaining both symbolic representations and vector representations • Using question words to retrieve top k triples • Calculating matching scores between question and triples using CNN model • Finding best matched triples
(how, tall, is, liu, xiang)
< liu xiang, height, 1.90m> < yao ming, height, 2.26m> … … <liu xiang, birth place, shanghai>
retrieved top k triples and their embeddings
question and its embedding
Matching
![Page 47: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/47.jpg)
Generator: Answer Generation
• Generating answer using attention mechanism • At each position, a classifier decides whether to generate a word or use the object of top triple
2s 3s
3c
He is
03 z 13 z
2.29m tall
< yao ming, height, 2.29m>
3z
How tall is Yao Ming ?
3y3y2y
… o
…
…
…
![Page 48: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/48.jpg)
Experimental Results accuracy = 52%
60K triples and 700K QA pairs for training
![Page 49: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/49.jpg)
Question Answering from Relational Database
![Page 50: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/50.jpg)
Question Answering from Relational Database
Relational Database
Q: How many people participated in the game in Beijing? A: 4,200 SQL: select #_participants, where city=beijing Q: When was the latest game hosted? A: 2012 SQL: argmax(city, year)
Question Answering System
Q: Which city hosted the longest Olympic game before the game in Beijing?
A: Athens
Learning System
year city #_days #_medals
2000 Sydney 20 2,000
2004 Athens 35 1,500
2008 Beijing 30 2,500
2012 London 40 2,300
![Page 51: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/51.jpg)
Neural Enquirer
Yin et al. 2015
• Query Encoder: encoding query • Table Encoder: encoding entries in table • Five Executors: executing query against table
![Page 52: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/52.jpg)
Query Encoder and Table Encoder
• Creating query embedding using RNN • Creating entry table embedding for each entry using DNN
RNN
…
query representation
query
DNN
entry representation
field value
table representation
Query Encoder Table Encoder
![Page 53: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/53.jpg)
Executors
• Five layers, except last layer, each layer has reader, annotator, and memory • Reader fetches important representation for each row, e.g., city=beijing • Annotator encodes result representation for each row, e.g., row where city=beijing
Select #_participants where city = beijing
![Page 54: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/54.jpg)
Experimental Results
• Experiments on synthetic data • Outperforms Semantic Parser
![Page 55: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/55.jpg)
Talk Outline
• Introduction to Huawei Noah’s Ark Lab • Deep Learning – New Opportunities for Information
Retrieval • Three Useful Deep Learning Tools • Information Retrieval Tasks
– Image Retrieval – Retrieval-based Question Answering – Generation-based Question Answering – Question Answering from Knowledge Base – Question Answering from Database
• Discussions and Concluding Remarks
![Page 56: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/56.jpg)
Discussions
• Key is to combine symbolic processing and neural processing
• Advantage of symbolic processing: direct, effective, and easy to control
• Advantage of neural processing: flexible, robust, and completely data-driven
• Challenge: difficult to make the combination
![Page 57: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/57.jpg)
Summary
• Matching is key for Information Retrieval • Deep Learning provides new opportunities for IR • Can learn better representations for matching • Information Retrieval Tasks
– Image Retrieval – Retrieval-based Question Answering – Generation-based Question Answering – Question Answering from Knowledge Base – Question Answering from Database Key question: how to
combine symbolic processing and neural processing
• Future relies on combination of symbolic processing and neural processing
![Page 58: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/58.jpg)
References • Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen. Convolutional Neural
Network Architectures for Matching Natural Language Sentences. NIPS'14, 2042-2050, 2014.
• Lifeng Shang, Zhengdong Lu, Hang Li. Neural Responding Machine for Short Text Conversation. ACL-IJCNLP'15, 2015.
• Lin Ma, Zhengodng Lu, Lifeng Shang, Hang Li . Multimodal Convolutional Neural Networks for Matching Image and Sentence, ICCV’15, 2015.
• Pengcheng Yin, Zhengdong Lu, Hang Li, Ben Kao. Neural Enquirer: Learning to Query Tables. arXiv, 2015
• Jun Yin, Xin Jiang, Zhengdong Lu, Lifeng Shang, Hang Li, Xiaoming Li. Neural Generative Question Answering. arXiv, 2015.
![Page 59: Deep Learning for Information Retrieval - Hang Li Outline •Introduction to Huawei Noah’s Ark Lab •Deep Learning – New Opportunities for Information Retrieval •Three Useful](https://reader033.vdocument.in/reader033/viewer/2022051602/5afce1f17f8b9a8b4d8ccc2e/html5/thumbnails/59.jpg)
Thank you!