neural graph collaborative filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... ·...
TRANSCRIPT
![Page 1: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/1.jpg)
Neural Graph Collaborative Filtering
Advisor: Jia-Ling Koh
Presenter: You-Xiang Chen
Source: SIGIR ‘19
Data: 2019/12/20
1
![Page 2: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/2.jpg)
Content
Introduction
Method
Experiment
01
Conclusion
02
03
042
![Page 3: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/3.jpg)
Introduction
3
![Page 4: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/4.jpg)
Recommended System
4
![Page 5: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/5.jpg)
Collaborative Filtering
Matrix Factorization
Neural Collaborative Filtering
5
![Page 6: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/6.jpg)
Introduction
1. Embedding , which transforms users and items to
vectorized representations
2. Interaction modeling , which reconstructs historical
interactions based on the embeddings.
Components of learnable CF models
6
![Page 7: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/7.jpg)
Motivation
Inherent drawback of existing methods
• Most existing methods build the embedding
function with the descriptive features only.
• The embedding function lacks an explicit
encoding of the crucial collaborative signal .
7
![Page 8: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/8.jpg)
Goal
• Proposing a new recommendation framework
based on graph neural network .
• The work explicitly encodes the collaborative
signal in the form of high-order connectivities .
8
![Page 9: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/9.jpg)
Method
9
![Page 10: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/10.jpg)
Neural graph collaborative filtering
Embedding
Embedding Propagation Layer
Prediction Layer
10
![Page 11: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/11.jpg)
Embedding Layer
11
![Page 12: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/12.jpg)
Embedding Propagation LayerFirst-order Propagation
• Message Construction
• Message Aggregation
For a connected user-item pair (u,i) , we define
the interaction message as:
We aggregate the messages propagated
from 𝒖′𝒔 neighborhood as 𝑢′𝑠representation :
𝒆𝒖(𝒍)
𝒆𝒖(𝟏)
𝒆𝒖(𝟐)
12
![Page 13: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/13.jpg)
Embedding Propagation LayerFirst-order Propagation
• Message Construction𝒇 ∙ : 𝒆𝒏𝒄𝒐𝒅𝒊𝒏𝒈 𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏
𝒑𝒖𝒊: 𝒄𝒐𝒆𝒇𝒇𝒊𝒄𝒊𝒆𝒏𝒕 𝒕𝒐 𝒄𝒐𝒏𝒕𝒓𝒐𝒍 𝒅𝒆𝒄𝒂𝒚 𝒇𝒂𝒄𝒕𝒐𝒓
𝐖𝟏,𝐖𝟐 ∈ ℝ𝒅′×𝒅
• Message Aggregation
𝑮𝑪𝑵 𝒄𝒐𝒏𝒔𝒊𝒅𝒆𝒓 𝒆𝒊 𝒐𝒏𝒍𝒚
13
![Page 14: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/14.jpg)
Embedding Propagation LayerHigh-order Propagation
• Message Aggregation
𝐖𝟏𝒍 ,𝐖𝟐
𝒍 ∈ ℝ𝒅𝒍×𝒅𝒍−𝟏
14
![Page 15: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/15.jpg)
Embedding Propagation LayerHigh-order Propagation
• Propagation Rule in Matrix Form
It allows us to discard the node sampling procedure
Simultaneously updating the representations for all
users and items in a rather efficient way
(facilitate batch implementation)
15
![Page 16: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/16.jpg)
Embedding Propagation LayerHigh-order Propagation
• Matrix Form of layer-wise propagation rule
𝑹 ∈ ℝ𝑵×𝑴 :user-item interaction matrix
A :adjacency matrix
D :Diagonal degree matrix
𝑬(𝒍) ∈ ℝ 𝑵+𝑴 ×𝒅𝒍
Node degree
16
![Page 17: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/17.jpg)
Laplacian Matrix
1
3
4
2
5
ℒ = 𝐷 − 𝐴 =
−1 −1 0 0 0−1 2 −1 −1 00 −1 3 −1 −10 −1 −1 3 −10 0 −1 −1 2
𝐴 =
0 1 0 0 01 0 1 1 00 1 0 1 10 1 1 0 10 0 1 1 0
𝐷 =
1 0 0 0 00 2 0 0 00 0 3 0 00 0 0 3 00 0 0 0 2
• Adjacency Matrix & Diagonal Degree Matrix
• Laplacian Matrix
17
![Page 18: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/18.jpg)
Laplacian Matrix
ℒ 𝑠𝑦𝑚 = 𝐷−12 𝐿 𝐷−
12 = 𝐼 − 𝐷−
12 𝐴 𝐷−
12
ℒ𝑖,𝑗𝑠𝑦𝑚
• Symmetric normalized Laplacian
18
![Page 19: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/19.jpg)
Model Prediction
19
![Page 20: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/20.jpg)
OptimizationBayesian Personalized Ranking
Dislike? Missing value?
20
![Page 21: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/21.jpg)
OptimizationBayesian Personalized Ranking
𝑦𝑢𝑖𝑗 = 𝑦𝑢𝑖 − 𝑦𝑢𝑗
• Learning models with BPR
• Objective function
𝓡+: observed interactions𝓡−: unobserved interactions
𝐿2 𝑟𝑒𝑔𝑢𝑙𝑎𝑟𝑖𝑧𝑎𝑡𝑖𝑜𝑛
𝑓𝑜𝑟 𝑒𝑣𝑒𝑟𝑦 𝑟𝑎𝑛𝑑𝑜𝑚𝑙𝑦 𝑠𝑎𝑚𝑝𝑙𝑒𝑑 𝒕𝒓𝒊𝒑𝒍𝒆 𝒖, 𝒊, 𝒋 ∈ 𝓞
BPR
21
![Page 22: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/22.jpg)
Experiment
22
![Page 23: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/23.jpg)
Dataset
Dataset Interaction# Item# User# Density
Gowalla 29,858 40,981 1,027,370 0.00084
Yelp2018 31,831 40,841 1,666,869 0.00128
Amazon-Book
52,643 91,599 2,984,108 0.00062
• Statistics of the evaluation datasets
23
![Page 24: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/24.jpg)
Evaluation metrics
𝐷𝐶𝐺𝑘 =
𝑖=1
𝑘2𝑟𝑒𝑙𝑖 − 1
log2(𝑖 + 1)
𝑁𝐷𝐶𝐺@𝐾 =𝐷𝐶𝐺@𝐾
𝐼𝐷𝐶𝐺
• Normalized Discounted cumulative gain
24
![Page 25: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/25.jpg)
Comparison Baselines
• Model-based CF methods
MF
NeuMF
CMN
• Graph-based CF methods
HOP-Rec
• Graph Convolutional Network
PinSage
GC-MC
25
![Page 26: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/26.jpg)
Performance Comparison
26
![Page 27: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/27.jpg)
Comparison w.r.t. Sparsity Level
27
![Page 28: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/28.jpg)
Comparison w.r.t. Sparsity Level
28
![Page 29: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/29.jpg)
Study of NGCF
29
![Page 30: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/30.jpg)
Study of NGCF
30
![Page 31: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/31.jpg)
Conclusion
31
![Page 32: Neural Graph Collaborative Filtering184pc128.csie.ntnu.edu.tw/presentation/19-12-20... · Introduction Method Experiment 01 Conclusion 02 03 04 2. Introduction 3. Recommended System](https://reader030.vdocument.in/reader030/viewer/2022040307/5ecd53507b8a796bf06b957b/html5/thumbnails/32.jpg)
Conclusion
• Extensive results demonstrate the state-of-the-art performance
of NGCF and its effectiveness in improving the embedding
quality with neural embedding propagation.
• The research present the NGCF model, a new recommendation
framework based on graph neural network, which explicitly
encodes the collaborative signal in the form of high-order
connectivities by performing embedding propagation.
32