composition-based multi-relational graph convolutional ... · composition-based multi-relational...
TRANSCRIPT
Composition-based Multi-Relational Graph Convolutional Networks
1
Shikhar Vashishth*1,2
1Indian Institute of Science, 2Carnegie Mellon University, 3Columbia University
Soumya Sanyal*1
[email protected] Talukdar1
[email protected] Nitin1,3
/ 17
Multi-relational Graphs
● Graphs with directed-labeled edges
● Multi-relational graphs are pervasive, examples include...
Dependency Parse
2
ProteinsKnowledge Graphs
/ 17
Graph Convolutional Networks (GCNs)
Wx1
Wx2
Wx3
Wx4
GCN First-order approximation
(Kipf et. al. 2016)
3
v
● Most GCN formulations are for simple undirected graphs
● Naive extension of GCNs to Multi-relational graphs using relation-specific filter matrix (W)
○ Suffers from overparameterization
/ 17
Existing Multi-Relational GCN models
4
● Directed-GCN: Utilizes direction-specific filter matrix
● Weighted-GCN: Learns a scalar weight for each relation
● Relational-GCN: Relation-specific filters in terms of basis matrices
Although solve overparameterization to different degrees of
granularity, none of them learn relation embeddings
/ 17
Motivation
5
● Extensive research done on embedding Knowledge Graphs
where representations of both nodes and relations are jointly
learned.
● Can we develop a GCN framework that can leverage the advances
in KGE approaches to:
○ Learn both node and relation embeddings
○ Solve the issue of overparameterization
Wx1
Wx2
Wx3
Wx4
KG Embedding methods Graph ConvNets
/ 17
Contributions
● We propose CompGCN, a novel framework for incorporating
multi-relational information in GCNs which leverages a variety of
composition operations from KG embedding techniques.
● Unlike previous GCN methods, CompGCN jointly learns to embed
both nodes and relations in the graph
6
/ 17
CompGCN: Overview
7
/ 17
CompGCN: Overview
8
/ 17
CompGCN: Overview
9
/ 17
CompGCN: Overview
10
/ 17
CompGCN: Update Equation
11
Node Update: Composition Operation
Original edgesInverse EdgesSelf Loops
Summation over neighborhood of v
non-linearity
Composition Operation:
Relation Update:
v
Subtraction (TransE)
Multiplication (DistMult)
Circular-correlation (HolE)
/ 17
Evaluation Tasks
12
● Link Prediction in Knowledge Graph
spouse_of
parent_of
parent_of
Knowledge Graph Inferring missing links
● Node Classification
Sasha Obama
Barack Obama
Functional Group Classification
Michelle Obama
● Graph Classification
Molecule Classification
Carcinogenic
spouse_of
parent_of
parent_of
Sasha Obama
Barack Obama
Michelle Obama
Link Prediction
/ 17
CompGCN: Link Prediction Results
● Effect of different GCN models and composition operators
ConvE + CompGCN(Corr) gives best performance across all settings.
13
/ 17
CompGCN: Link Prediction Results
● Performance on Link Prediction
CompGCN provides a consistent improvement across all the datasets.
14
/ 17
CompGCN: Results
● Performance on Node Classification and Graph Classification
CompGCN outperforms or performs comparably to existing baselines.
15
Graph classification PerformanceNode classification Performance
/ 17
CompGCN: Scalability
● Effect of number of relation basis vectors and relations on FB15k-237
COMPGCN outperforms RGCN even with limited parameters.
16
/ 17
● Multi-relational graphs are prevalent in real-world problems.
● Current GCN approaches mainly focus on simple undirected
graphs.
● We propose CompGCN, a parameter efficient method for
embedding both nodes and relation types.
● We demonstrate the effectiveness of CompGCN for link
prediction, node and graph classification tasks.
Conclusion
17
Source Code:
github.com/malllabiisc/CompGCN
Thank you!
18
based implementation
Paper Link: Composition-Based Multi-Relational
Graph Convolutional Networks
Research Supported by: