deep learning for recommender systems - budapest recsys meetup
TRANSCRIPT
Deep Learning forRecommender Systems
Alexandros KaratzoglouSenior Research Scientist @ Telefonica Research
[email protected]@alexk_z
Telefonica Research
Machine Learning
HCI
Network & Systems
Mobile Computing
http://www.tid.es
Why Deep?
ImageNet challenge error rates (red line = human performance)
Why Deep?
Inspiration for Neural Learning
Early aviationattempts aimed atimitating birds, bats
Neural Model
Neuron a.k.a. Unit
Feedforward Multilayered Network
Learning
Stochastic Gradient Descent
Generalization of (Stochastic) Gradient Descent
Stochastic Gradient Descent
Stochastic Gradient Descent
Stochastic Gradient Descent
Feedforward Multilayered Network
Backpropagation
Backpropagation
Does not work well in plain a “normal”multilayer deep network
Vanishing Gradients
Slow Learning
SVM’s easier to train
2nd Neural Winter
Modern Deep Networks
Ingredients:
Rectified Linear Activation function a.k.a. ReLu
Modern Deep Networks
Ingredients:
Dropout:
Modern Deep Networks
Ingredients:
Mini-batches:
Stochastic Gradient Descent
Compute gradient over many (50 -100)
data points (minibatch) and update.
Modern Deep Networks
Ingredients:
Softmax output:
Modern Deep Networks
Ingredients:
Categorical cross entropy loss
Modern Feedforward Networks
Ingredients:
Batch Normalization
Modern Feedforward Networks
Ingredients:
Adagrad a.k.a. adaptive learning rates
Restricted Boltzmann Machines
Restricted Boltzmann Machines
Convolutional Networks
Convolutional Networks
[Krizhevsky 2012]
Convolutional Networks
[Faster R-CNN: Ren, He, Girshick, Sun 2015] [Farabet et al., 2012]
Convolutional Networks
[Faster R-CNN: Ren, He, Girshick, Sun 2015] [Farabet et al., 2012]
Convolutional Networks
Self Driving Cars
Convolutional example slides from Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 6 75
Convolutional Networks
Standford CS231n: Convolutional Neural Networks for Visual Recognition
Convolutional Networks
Convolutional Networks
Convolutional Networks
Convolutional Networks
Convolutional Networks
Convolutional Networks
Convolutional Networks
AlexNet [Krizhevsky et al 2014]
dd
D-tour → Matrix Factorization
2
5
4
5
4
4
1
5
5
4
1
2
5
2
4
1
Convolutional Networks forenhancing Collaborative Filtering
VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback He,etl AAAI 2015
Convolutional Networks for Musicfeature extraction
Deep Learning can be used to learn item profiles e.g. music
Map audio to lower dimensional space where it can be useddirectly for recommendation
Useful in recommending music from the long tail (not popular)
A solution to the cold start problem
Convolutional Networks for Musicfeature extraction
A. van den Oord, S. Dielmann, B. Schrauwen Deep content-based music recommendation NIPS 2014
Convolutional Networks
deepart.io
Recurrent Neural Networks
Recurrent Neural Networks
Long Short Term Memory
Recurrent Neural Networks
Recurrent Neural Networks
PANDARUS:Alas, I think he shall be come approached and the dayWhen little srain would be attain'd into being never fed,And who is but a chain and subjects of his death,I should not sleep.
Second Senator:They are away this miseries, produced upon my soul,Breaking and strongly should be buried, when I perishThe earth and thoughts of many states.
DUKE VINCENTIO:Well, your wit is in the care of side and that.
Second Lord:They would be ruled after this chamber, andmy fair nues begun out of the fact, to be conveyed,Whose noble souls I'll have the heart of the wars.
Clown:Come, sir, I will make did behold your worship.
VIOLA:I'll drink it.
Recurrent Neural Networks
Recurrent Neural Networks
Recurrent Neural Networks
Recurrent Neural Networks
Session-based recommendationwith Recurrent Neural Networks
RNN (GRU) with ranking loss functionICLR 2016 [B. Hidasi, et.al.]
Treat each user session as sequence of clicks
Session-based recommendationwith Recurrent Neural Networks
RNN (GRU) with ranking loss functionICLR 2016 [B. Hidasi, et.al.]
Treat each user session as sequence of clicks
Autoencoders
Autoencoders
Autoencoders
Personalized Autoencoders
Collaborative Denoising Auto-Encoders for Top-N Recommender Systems Wuet.al. WSDM 2016
(Some) Deep Learning Software
Theano: Python Library
TensorFlow: Python Library
Keras: High Level Python Library (Theano &TF)
MXNET: R, Python, Julia
Thanks
● Some slides or parts of slides are taken fromother excellent talks and papers on DeepLearning (e.g. Yan Lecun, Andrej Karpathy andother great deep learning researchers)