cognitive graph in practice with their applications in ... · 10. anrl: attributed network...
TRANSCRIPT
Cognitive Graph in Practice with Their Applications in
Ecommerce Recommendation
Dr. Hongxia Yang
For Discovery, Adventure, Momentum and Outlook
Senior Staff Data Scientist
Part 1
01 Motivation
Part 2
02 Graph Neural Network in Practice
CONTENTS
Part 3
03 Cognitive Graph
Part 1
01 Motivation
Cognitive Recommendation
• What is recommendation based on cognitive reasoning? Moisturizing vs Girls born after 90s need moisturizing if staying up late
• Why cognitive reasoning? • Supermarket (Initial) -> Personalized Shelf (Thousand People Thousand Faces) ->
Personalized Shelf with AI Shopping Guide (Cognitive Reasoning)
Content Display Common Sense Knowledge Graph +
GNN As Reasoning Engine
Fashion Icon Articles to connect KG and
common sense
Recommendation based on
Cognitive Reasoning
Content
Commercialization Generate recommended reasons and
recall related videos
Video and Articles product detection
DEMO
Cognitive Intelligence Algorithm Platform
GNN Build Personalized Super Brain
Capable of inductive reasoning, explainable, better know consumers
than themselves
Alibaba Economy Big
Data Platform Lead Consumers Lives
Large Scale Graph Neural Network Oriented
Material, Entertainment and Health
Text, image and video influence the mind of
consumers
Part 2
02 Graph Neural Network in
Practice
Graph Embedding
Shallow Embedding Methods
Matrix Factorization Based Methods
• [Belkin et al. NIPS2002] • [Ahmed et al. WWW2013] • [Cao et al. KDD2015] • [Ou et al. KDD2016]
Random Walk Based Methods
• [Perozzi et al. KDD2014] • [Grover et al. KDD2016]
Deep Embedding Methods
Graph Autoencoder
• [Cao et al. AAAI2016] • [Wang et al. KDD2016]
Graph Convolutional Network
• [Bruna et al., ICLR2014] • [Duvenaud et al., NIPS2015] • [Kipf&Welling, ICLR2017]
Graph-Gated Recurrent Unit
• [Li et al. ICLR2018] • [Yan et al. AAAI’2018] • [Wu et al. IJCAI’2019] • [Guo et al. AAAI2019]
GE Family
No parameters sharing • Simply an embedding lookup based on arbitrary node id Fail to leverage node attributes • E.g., user profiles on a recommendation task Transductive model • The cold start problem
Difficult to deal with large scale graphs • The input dimension to the autoencoder is fixed
at |V| Cannot cope with unseen nodes • The structure and size of the autoencoder is fixed
Extremely Large Scale Attributed Heterogeneous Graphs with Billions of
Nodes and Trillions of Edges
Unified Graph Embeddings for Both Various Topological and
Attributed Spaces Practical Challenges and Our Current
Focuses
Sampling: Representative and
Negative Samples
Multiplex: Node and Edge Heterogeneities
Mixture Modes: Users with Different Latent Interest Categories
Hierarchical: Users’ community structures and
items’ categories
A GE Oriented Recommendation System in Practice
GNN Framework Overview
System Optimizations
Graph partitioning and clustering
Co-locate embedding variables with graph partitions
Reuse embedding if possible
Embedding h
TensorFlow Heterogeneous Graph
Computing CPU GPU MPI
RAW DATA PROCESS DISPATCH &
LOAD TRAINING INFERENCE OUTPUTS FEEDBACK
WORKER
PARAMETERS GRADIENTS
Extremely
Large
Scale
Graph
Represent-
ation
Platform
PARALLEL LEARNING AND INFERNCE EXTREMELY LARGE SCALE GRAPHS
Billions of nodes, trillions of edges
GNN Platform Overview
Node
prediction
Edge
prediction Community
detection
Dynamic
graph
Data Sources
Load
Store Graphs
Built in
Algo
Sample
Aggregate Merge
Apply
Pattern
Recognition
WORKER
WORKER
PS
PS
PS 1. Rich algorithm warehouse with improved accuracy measures (5%-90%) for practical
challenges;
2. Performs an order of magnitude faster in terms of graph building, e.g., 492.90 million
vertices, 6.82 billion edges and rich attributes, 2 minutes vs hours by other state-of-
the-arts;
3. 40%-50% faster with the novel caching strategy and demonstrates around 12 times
speed up with the improved runtime.
AliGraph: A Comprehensive Graph Neural Network Platform. 45th International Conference on Very Large Data Bases (VLDB), 2019.
Various GE/GNN Methods
Multiplex Representation Learning for Attributed Multiplex Heterogeneous Network, KDD 2019
• Transductive Model:
• Inductive Model:
Mixture GNN Is a Single Vector Enough? Exploring Node Polysemy for Network Embedding, KDD 2019
Hierarchical GNN Hierarchical Representation Learning for Bipartite Graphs, IJCAI 2019
Evolving GNN Large Scale Evolving Graphs with Burst Detection, IJCAI 2019
Bayesian GNN
• BEM is proposed to bridge KG and BG seamlessly, with the consideration of the behavior-specific bias. This framework provides a new perspective of making a reasoning mechanism (cognitive graph).
• As a method, BEM is generic and flexible in that it can use any KG embeddings to correct any BG embeddings. On the contrary, it is potentially able to help KG embeddings acquire novel knowledge from the BG embeddings that does not exist in the knowledge graph.
Refining Representation by Integrating Knowledge Graphs and Behavior-specific Networks, CIKM 2019
1. Representation Learning for Attributed Multiplex Heterogeneous Network. 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD),
2019.
2. Is a Single Vector Enough? Exploring Node Polysemy for Network Embedding. 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD),
2019.
3. Towards Knowledge-Based Personalized Product Description Generation in E-commerce. 25th ACM SIGKDD Conference on Knowledge Discovery and Data
Mining (KDD), 2019.
4. Sequential Scenario-Specific Meta Learner for Online Recommendation. 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD),
2019.
5. AliGraph: A Comprehensive Graph Neural Network Platform. 45th International Conference on Very Large Data Bases (VLDB), 2019.
6. Large Scale Evolving Graphs with Burst Detection. 28th International Joint Conference on Artificial Intelligence (IJCAI), 2019.
7. Hierarchical Representation Learning for Bipartite Graphs. 28th International Joint Conference on Artificial Intelligence (IJCAI), 2019.
8. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019.
9. Subgraph-augmented Path Embedding for Semantic User Search on Heterogeneous Social Network, WWW, 2018.
10. ANRL: Attributed Network Representation Learning via Deep Neural Networks, IJCAI, 2018.
11. Adversarial Detection with Model Interpretation. KDD, 2018.
12. SPARC: Self-Paced Network Representation for Few-Shot Rare Category Characterization. KDD, 2018.
13. Mobile access record resolution on large-scale identifier-linkage graphs. KDD, 2018.
14. Interactive Paths Embedding for Semantic Proximity Search on Heterogeneous Graphs. KDD, 2018.
15. PRRE: Personalized Relation Ranking Embedding for Attributed Network. 27th ACM International Conference on Information and Knowledge Management
(CIKM), 2018.
17. Heterogeneous Embedding Propagation for Large-scale E-Commerce User Alignment, 2018 IEEE International Conference on Data Mining (ICDM), 2018
18. Local Algorithm for User Action Prediction Towards Display Ads. 23rd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2017.
19. Hybrid Framework for Text Modeling with Convolutional RNN. 23rd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2017.
20. Bayesian Heteroscedastic Matrix Factorization for Conversion Rate Prediction. 26th ACM International Conference on Information and Knowledge
Management (CIKM), 2017.
21. Will Triadic Closure Strengthen Ties in Social Networks?, ACM Transactions on Knowledge Discovery from Data (TKDD), 2017.
Selected Publications
Part 3
03 Cognitive Graph
Cognitive GNN Towards Knowledge-Based Personalized Product Description Generation in E-commerce, KDD 2019
Cognitive GNN Cognitive Graph for Multi-Hop Reading Comprehension at Scale, ACL 2019
Causality/GNN/Disentanglement
Causality
GNN Disentangle Disentangled RepLearn on Graphs
• Representation learning (RepLearn):
• To infer 𝒁 that explains 𝑿, usually via a generative model 𝑃 𝑿 ∣ 𝒁 .
• Causal discovery:
• To infer the causal mechanism 𝑃 𝑿 ∣ 𝑑𝑜 𝒁 , when given 𝒁 and 𝑿.
• Disentangled RepLearn[Definition from Suter et al., 2019]:
• To infer (1) 𝒁 that causes 𝑿, and (2) the causal mechanism 𝑃 𝑿 ∣ 𝑑𝑜 𝒁 .
• Moreover, (3) 𝒁 should contain independent, direct causes of 𝑿.
RepLearn
Generative RepLearn
Causal Discovery
Disentangled RepLearn
Robustly Disentangled Causal Mechanisms. Suter et al., ICML 2019.
Causality vs Disentanglement
• Anti-causal learning: To infer the cause given the effect.
• E.g., to predict a user’s preference (cause) given his/her behavior (effect).
• E.g., to predict lung cancer (cause) given a lung image (effect).
On Causal and Anticausal Learning. Schölkopf et al., ICML 2012. Generalization in anti-causal learning. Kilbertus et al., arXiv 1812.00524.
𝒁 𝒁
𝑿
𝒇 𝒁 = 𝑿
Causality vs RepLearn
• All these representations can achieve the maximum likelihood.
• Disentangled representation I: 𝑃𝜃𝐴(𝐴) and 𝑃𝜃𝐵∣𝐴(𝐵 ∣ 𝐴) .
• Disentangled representation II: 𝑃𝜃𝐵(𝐵) and 𝑃𝜃𝐴∣𝐵(𝐴 ∣ 𝐵) .
• But if 𝐴 → 𝐵 is true, then 𝑃𝜃𝐴 𝐴 ⋅ 𝑃𝜃𝐵∣𝐴(𝐵 ∣ 𝐴) needs much fewer new samples to adapt to the new distribution once 𝑃(𝐴, 𝐵) shifts.
• Because we need to update only 𝜃𝐵|𝐴.
It adapts faster if correctly disentangled.
This is a new approach to infer the causal direction!
Causality vs RepLearn
Thanks