Deep Learning for GraphsTrends & Open Questions

Federico Errica

The What

2

● Node u → Entity

● Edge (u,v) → Relationship

● Generalizations: Multigraphs & Hypergraphs

The What (cont.)

3

● Representation Learning on graphs○ Vertex & Graph embeddings

● Supervised○ Vertex/Graph classification/regression

● Unsupervised○ Link Prediction○ Clustering○ Maximum Likelihood Estimation

● Generative○ Molecule generation

A visual example (Graph t-SNE)

4

Leow, Yao Yang, Thomas Laurent, and Xavier Bresson. GraphTSNE: A Visualization Technique for Graph-Structured Data. ICLR Workshop (2019).https://leowyy.github.io/graphtsne

The Why

5

RnxmHave → fun → with → Machine → Learning

Flat Sequences

Trees

Graphs

The Why (cont.)

6

● Handle cyclic structures ○ No recursion!

● Variable size

● Variable shape

● Efficiency

● No more feature engineering○ i.e. kernel methods

Me

The feature engineering guy

The How (in a nutshell)● Neighborhood Aggregation to the rescue● Use layering to spread context between vertices

● How can we aggregate neighbors?● How many layers do we need?

7

Resemblance to CNNs● Convolution as neighborhood aggregation

○ On regular grids

● Layers increases the local receptive field of each vertex

8

A review of some works

9

NN4G (Micheli, TNNLS 2009)● Constructive approach

○ Cascade Correlation

● Aggregation function○ Sum

● The first spatial DGN!

10

CGMM (Bacciu, Errica & Micheli, ICML 2018)● A deep stack of probabilistic layers

● Unsupervised constructive approach

● Switching Parent approximation○ Borrowed from

Hidden Tree Markov Models ● It works well

○ State-of-the-art accuracycompared to GNNs

● CGMM exploits layering

11

ℓ

GraphESN (Gallicchio & Micheli, IJCNN 2010)● Does not require training but for the output layer

● Let the Reservoir reach convergence

● Train a linear readout

12

DiffPool (Ying et al., 2018)● Differentiable Pooling technique

13

Robust Comparisons (Bacciu, Errica, Micheli & Podda, ICLR 2020)

● Popular model are not reproducible

● We tried to solve this problem

● Some optimized on test 👿

● We ran 47k experimentsto fix this

14

Y: yes N: no A: ambiguous - : not provided

Theses and Projects (for students that look for a challenge 🤓 )

15

● Unsupervised Criterion to automatically select layers● Unsupervised/Probabilistic pooling strategy● CGMM extensions

○ Supervised CGMM version○ Apply to node classification tasks → (Project or Thesis)○ Automatic selection of # of states○ Use more powerful aggregation functions

● Design a new GNN and compare it against sequence/tree/graph models○ e.g. in NLP

● Interpretable/explainable DGNs● Few-shot learning with DGNs● For Projects:

○ Implement a Graph Neural Network (using Pytorch Geometrics, Deep Graph Library).

Thank you!

16

You can reach out to me via:

Email: federico.errica@phd.unipi.it

Office: Room 328, Department of Computer Science

Website: http://pages.di.unipi.it/errica/

Interested? Check out “A Gentle Introduction to Deep Learning for Graphs” (pre-print on arXiv).

Powered by CIML

mailto:federico.errica@phd.unipi.ithttp://pages.di.unipi.it/errica/