Download - Music Recommendation
-
7/31/2019 Music Recommendation
1/113
Music RecommendationListen to the music You like
Recuperao de Informao 1semestre 2011/2012Ricardo Dias, n55444
-
7/31/2019 Music Recommendation
2/113
Bibliography
Music Recommendation and Discovery: The Long
Tail, Long Fail, and Long Play
scar Celma, Springer 2010, Ch-1-3,5
Recommender SystemsPrem Melville, Vikas Sindhwani
Encyclopedia of Machine Learning, 2010
Handbook of Multimedia for Digital
Entertainment and Arts
Borko Furht (Ed.), Springer 2009
-
7/31/2019 Music Recommendation
3/113
MOTIVATION & CONTEXT
What is Recommendation? Do we need it? Why?
-
7/31/2019 Music Recommendation
4/113
Recommendation in our lives
-
7/31/2019 Music Recommendation
5/113
Restaurants
-
7/31/2019 Music Recommendation
6/113
Discos and Bars
-
7/31/2019 Music Recommendation
7/113
Food
-
7/31/2019 Music Recommendation
8/113
Books
-
7/31/2019 Music Recommendation
9/113
And Music!
-
7/31/2019 Music Recommendation
10/113
Do we need Music recommendation?
-
7/31/2019 Music Recommendation
11/113
Music Consumption Change
Time
Physical Stores Online Stores
-
7/31/2019 Music Recommendation
12/113
Music RecommendationDigital Era - Portability
Up to ~20 tracks Up to ~40.000 tracks
Time
-
7/31/2019 Music Recommendation
13/113
Music RecommendationDigital Era Online Services
-
7/31/2019 Music Recommendation
14/113
Music RecommendationDigital Era Online Services
Amazon
17 Million Songs
-
7/31/2019 Music Recommendation
15/113
Music Recommendation Problem
Overwhelming number of choices of whichmusic to listen to
Users feel:
Paralyzed
Doubtful
Need to provide personalized filters and
recommendations to easeusers decisions
-
7/31/2019 Music Recommendation
16/113
-
7/31/2019 Music Recommendation
17/113
Music RecommendationDigital Era
-
7/31/2019 Music Recommendation
18/113
Music Characteristics
Different from other types of media
Tracking users preferences can be implicit
Items can be consumed several times
(even repeatedly and continuously)
Instant feedback
Music consumption depends on context
(morning, work, afternoon, etc.)
-
7/31/2019 Music Recommendation
19/113
Music Recommendation Specificities
Current music recommendation algorithms tryto accurately predict what people will want tolisten
Making accurate predictions about a user could
listento, or buynext, independently of howusefulthe provided recommendations are tothe user
-
7/31/2019 Music Recommendation
20/113
THE RECOMMENDATION PROBLEM
Formalization, Use Cases, Profile Generation, Recommendation Methods
-
7/31/2019 Music Recommendation
21/113
Formalization
Recommendation Problem
Prediction problemestimationof the items
likeliness for a given user Recommend a list ofN items assuming that the
system can predict likeliness for yet unrated items
-
7/31/2019 Music Recommendation
22/113
Prediction Problem
= , , , the set ofUsers
= , , , items that can be
recommend
list of items a userj expressed his interests
Function ,j
predicted likeness of item
,
for the active user , where Usually represented by a rating < , , >
-
7/31/2019 Music Recommendation
23/113
Recommendation Problem
Find a list of N items that the user willlike the most
The ones with higher, The resultant list should not contain items from
the users interests
=
-
7/31/2019 Music Recommendation
24/113
Use Cases
Common usages of a recommender system:
1. Find good items
2. Find all good items3. Recommend sequence (e.g. playlist generation)
4. Just browsing
5. Find credible recommender6. Express self
7. Influence others
-
7/31/2019 Music Recommendation
25/113
General Model
Users and Items
Two types of
recommendations: Top-N predicted items
Top-N predicted
neighbors
-
7/31/2019 Music Recommendation
26/113
User Profile Generation
Two key elements:
Generation and Maintenance
Exploitation of the profile using arecommendation system
-
7/31/2019 Music Recommendation
27/113
-
7/31/2019 Music Recommendation
28/113
-
7/31/2019 Music Recommendation
29/113
-
7/31/2019 Music Recommendation
30/113
Recommendation Methods
Standard classification of recommendersystems:
1. Demographic Filtering2. Collaborative Filtering
3. Content-based Filtering
4. Context-based Filtering5. Hybrid Approaches
-
7/31/2019 Music Recommendation
31/113
Demographic Filtering
Used to identify the kind of users that like acertain item
Classifies user profiles in clusters based on: Personal data (age, gender, marital status, etc.)
Geographic data (city, country)
Psycographic data (interests, lifestyle, etc.)
-
7/31/2019 Music Recommendation
32/113
Advantages/Limitations
The simplest recommendation method
But
Recommendations are too general Requires effort from the user to generate the
profile
-
7/31/2019 Music Recommendation
33/113
Collaborative Filtering
Predict user preferences for items by learningpast user-item relationships
CF methods work by build a matrix M with nitems and m users, that contains theinteraction (e.g. ratings, plays, etc.) of the
users with the items.
-
7/31/2019 Music Recommendation
34/113
Collaborative Filtering
The value , represents the rating of the
user for the item
-
7/31/2019 Music Recommendation
35/113
Collaborative Filtering Approaches
Item-Based Neighborhood
User-Based Neighborhood
Matrix Factorization
-
7/31/2019 Music Recommendation
36/113
Item-Based Neighborhood
Only users that rated the items and , are
taken into account in the process
-
7/31/2019 Music Recommendation
37/113
Item-Based Neighborhood
Only users that rated the items and , are
taken into account in the process
-
7/31/2019 Music Recommendation
38/113
Item-Based Neighborhood
1. Compute the similarity between two items, iandj
1. Example: Adjusted cosine similarity
2. Predict to the target user, u, a value for theactive item, i
; - set of k neighbors of item I,that the user u has rated
-
7/31/2019 Music Recommendation
39/113
User-Based Neighborhood
Compute the predicted rating value of item i,for the active user u, taking into account those
users that are similar to u
- average rating for user u()- set of k neighbors for user u (the similar ones)
-
7/31/2019 Music Recommendation
40/113
Matrix Factorization
Useful when the M user-item matrix is sparse
Reduce dimensionality of the original matrix,
generating matrices U and V that approximate theoriginal one
Example: SVDSingular Value Decomposition
Computes matrices and , for a given
number k, such as:
diagonal matrix containing the singular values of M
-
7/31/2019 Music Recommendation
41/113
Matrix Factorization
After matrix reduction we can calculate thepredicted rating value for item i for a user u
,
-
7/31/2019 Music Recommendation
42/113
-
7/31/2019 Music Recommendation
43/113
Content-based Filtering
Uses information describing the items
Process of characterizing item data set can be:
Manual (annotations by domain experts) Automatic (extracting features by analyzing the
content)
Key component: Similarity Function
-
7/31/2019 Music Recommendation
44/113
Content-based Filtering
Similarity Functions
1. Euclidean
2. Manhattan
3. Chebychev
4. Mahalanobis
-
7/31/2019 Music Recommendation
45/113
Limitations
Cold-start problem (only to user)
Gray-sheep problem
Novelty (?)
Limitationof extracted automatic features
Subjective (personal opinions) not taken into
account
-
7/31/2019 Music Recommendation
46/113
Context-based Filtering
Uses context information to describe andcharacterize the items Context Information any information that can be
used to characterize a situation or an entity
Context != Content
Two main techniques: Web mining
Social Tagging
-
7/31/2019 Music Recommendation
47/113
-
7/31/2019 Music Recommendation
48/113
Social Tagging
Aims at annotating web content using tags
Tags are freely chosen keywords, not
constrained to a predefined vocabulary Recommender systems can derive social
tagging data to derive item (or user) similarity
-
7/31/2019 Music Recommendation
49/113
Social Tagging
When users tag items, we get tuples of :< , , >
These triples conform a 3-order matrix(tensor)
-
7/31/2019 Music Recommendation
50/113
Social Tagging
Two approaches to compute item (and user)similarity:
1. Unfold the 3-order tensor in threebidimensionalmatrices (user- tag, item-tag and user-itemmatrices)
2. Directly use the 3-order tensor
-
7/31/2019 Music Recommendation
51/113
Unfolding the 3-order tensor
User-Tag (U matrix) - ,contains the number
of times user i applied the tagj
Item-Tag (I matrix) - , contains the numberof times an item i has been tagged with tag j
User-Item (R binary matrix) - , denotes
whether the user i has tagged the itemj
-
7/31/2019 Music Recommendation
52/113
Unfolding the 3-order tensor
Item similarity (using I) or user similarity (usingU or I), can be computed using:
Cosine-based distance Dimensionality reduction techniques(SVD, NMF)
Then recommendations can be made by using:
R user-item matrix or, User profile obtained from U or I
-
7/31/2019 Music Recommendation
53/113
Using the 3-order tensor
The available techniques are (high-order)extensions ofSVD and NMF
HOSVD is a higher order generalization of matrixSVD for tensors,
Non-negative Tensor Factorization (NTF) is ageneralization of NMF
-
7/31/2019 Music Recommendation
54/113
Limitations
Coverage
Problems with tags:
Polysemy Synonymy
Usefulness of personal tags
Sparsity Attacks / Vandalism
-
7/31/2019 Music Recommendation
55/113
Hybrid Approaches
Goal
Achieve better recommendations by combining
some of the previous approaches Methods:
Weighted
Switching Mixed
Cascade
-
7/31/2019 Music Recommendation
56/113
Factors Affecting Recommendation
Novelty and Serendipity
Explainability (transparency)
Cold Start Problem Data Sparsity and High Dimensionality
Coverage
-
7/31/2019 Music Recommendation
57/113
Factors Affecting Recommendation
Trust
Attacks
Temporal Effects Understanding the Users
-
7/31/2019 Music Recommendation
58/113
MUSIC RECOMMENDATION
Use cases, User and Item Profiles Representation, Recommendation Examples
-
7/31/2019 Music Recommendation
59/113
Use Cases
Main task of a music recommendation
system:
Propose interesting music, consisting of a mix ofknown and unknown artists, as well as theavailable tracks, given a user profile
-
7/31/2019 Music Recommendation
60/113
Use Cases
Artist Recommendation
Playlist Generation
Shuffle, Random Playlists Personalized Playlists
Neighbor Recommendation
-
7/31/2019 Music Recommendation
61/113
How about other use cases?
-
7/31/2019 Music Recommendation
62/113
User Profile Representation
Extend user profile with music relatedinformation
Has not been largely investigated
Useful to:
Improve music recommendation Share with others your preferences
-
7/31/2019 Music Recommendation
63/113
Type of Listeners
Each type of listener needs different type ofrecommendations
User Profile Representation
-
7/31/2019 Music Recommendation
64/113
User Profile RepresentationProposals
Most relevant proposals are:
User modeling for Information Retrieval (UMIRL)
MPEG-7 standard Friend of a Friend (FOAF) initiative
User Modeling for Information
-
7/31/2019 Music Recommendation
65/113
User Modeling for InformationRetrieval
Allows one to describe perceptual andqualitative features of the music
-
7/31/2019 Music Recommendation
66/113
MPEG-7 User Preferences
User preferences in MPEG-7 includes:
Content filtering
Searching and browsingpreferences
Usage history
FOAF: User Profiling in the
-
7/31/2019 Music Recommendation
67/113
FOAF: User Profiling in theSemantic Web
Provides conventions and a language to tella machine the type of things a user says about
herself in her homepage
-
7/31/2019 Music Recommendation
68/113
Item Profile Representation
Music items:
Artists
Songs
-
7/31/2019 Music Recommendation
69/113
Music Information Plane
-
7/31/2019 Music Recommendation
70/113
Music Information Plane
Music knowledge management categories:
Editorial Metadata
Cultural Metadata Acoustic Metadata
-
7/31/2019 Music Recommendation
71/113
Editorial Metadata
-
7/31/2019 Music Recommendation
72/113
-
7/31/2019 Music Recommendation
73/113
Acoustic Metadata
-
7/31/2019 Music Recommendation
74/113
Music Description Facets
Low-level Timbre Descriptors
Spectral Centroid/Flateness/Skewness, MFCCs, etc.
Instrumentation
Rhythm
Harmony
Structure
Intensity
Genre
Mood
Recommendation Methods
-
7/31/2019 Music Recommendation
75/113
Recommendation Methods(examples and specificities)
Collaborative Filtering (CF)
Explicit/Implicit Feedback
Content-Based Filtering Context-Based Filtering
Hybrid Methods
-
7/31/2019 Music Recommendation
76/113
Collaborative Filtering
CF makes use of the editorial and culturalinformation
Explicit feedback based on ratings about songs /artists
Implicit feedback tracking user listening habits
-
7/31/2019 Music Recommendation
77/113
CF with Explicit Feedback
Examples:
Ringo 1st music recommender based on CF andexplicit feedback
Racofibased on CF and a set of logic rules basedon Horn clauses
Indiscover
Slope One CF method
-
7/31/2019 Music Recommendation
78/113
CF with Implicit Feedback
Main Drawbacks:
The value that a user assigns to an item is notalways in a predefined range (e.g. from 1..5 or likeit/hate it)
Cannot gather negativefeedback
Recommendations usually performed at artist
level, but listening habits are recorded at songlevelAggregation
d l
-
7/31/2019 Music Recommendation
79/113
Content-Based Filtering
Uses content extracted from music to providerecommendations
Compute similarity among songs, in order torecommend music to the user
Two ways to describe audio content:
Manually
Automatically
Manually Audio Content
-
7/31/2019 Music Recommendation
80/113
Manually Audio ContentDescription
Very time consuming
Scalability problems
But Annotations can be more accurate than automatic
Example: Pandora
Analysts annotate 400 parameters per song, usinga ten point scale per attribute
~ 15.000 songs analyzed per month
Automatic Audio Content
-
7/31/2019 Music Recommendation
81/113
Automatic Audio ContentDescription
Early work on audio similarity is based on low-level descriptors, such as Mel Frequency
Cepstral Coefficients ( MFCC) Foote proposed a music indexing system based on
MFCC histograms
Audio features are usually aggregated together
using mean and variance, or modeling it as aGaussian Mixture Model (GMM)
Automatic Audio Content
-
7/31/2019 Music Recommendation
82/113
Automatic Audio ContentDescription
Analyses audio signal and automaticallyextracts a set of features: Tzanetakis extracted a set of features representing
the spectrum, rhythm and harmony (chordstructure); then merged into a single vector, andwere used to determine song similarity
Cataltepe et al. presented a music
recommendation system based on audiosimilarity, where users listening history is takeninto account
C B d Fil i T h i
-
7/31/2019 Music Recommendation
83/113
Context-Based Filtering Techniques
Uses cultural information to compute artist orsong similarity
Mainly based on web mining techniques, ormining data from collaborative tagging
C B d Fil i T h i
-
7/31/2019 Music Recommendation
84/113
Context-Based Filtering Techniques
Example:
M3 (Music for My Mood), uses context (season,month, day of the week, weather, temperature)and Case-based Reasoning to recommend music
H b id M th d
-
7/31/2019 Music Recommendation
85/113
Hybrid Methods
Allows a system to minimize the issues that asolely method can have
How cascade approach works: One technique is applied first, obtaining a ranked list of
items. Then, a second technique refines or re-rank theresults obtained in the first step
H b id M th d
-
7/31/2019 Music Recommendation
86/113
Hybrid Method
Example:
Tiemann et al. investigate ensemble learningmethods for hybrid music recommenderalgorithms. Their approach combines social andcontent-based methods, where each oneproduces a weak learner. Then using a
combination rule, it unifies the output of the weaklearners.
-
7/31/2019 Music Recommendation
87/113
EVALUATION
System-centric, Network-centric, User-centric
E l ti
-
7/31/2019 Music Recommendation
88/113
Evaluation
Three different strategies
System-centric
Network-centric User-centric
S t t i E l ti
-
7/31/2019 Music Recommendation
89/113
System-centric Evaluation
Evaluation measures how accurate the systemcan predict the actual values that user havepreviously assigned
S t t i E l ti
-
7/31/2019 Music Recommendation
90/113
System-centric Evaluation
Most approaches based on the leave-n-outmethod Similar to the classic n-fold cross validation
Dataset divided into two (usually disjunct)sets: Training and Test
Accuracy evaluation based only on a usersdataset
The rest of the items of the catalog are ignored
S t t i E l ti
-
7/31/2019 Music Recommendation
91/113
System-centric Evaluation
Metrics:
Predictive accuracy
Mean Absolute Error, Root Mean Square Error
Decision based
Mean Average Precision, Recall, F-measure, Accuracy,ROC
Rank based Spearmans , Kandall, Half-life Utility, Discounted
Cumulative Gain
-
7/31/2019 Music Recommendation
92/113
-
7/31/2019 Music Recommendation
93/113
Network centric Evaluation
-
7/31/2019 Music Recommendation
94/113
Network-centric Evaluation
The similarity network is the basis to providethe recommendations
Important to analyze and understand theunderlying topology of the similarity network
Measures:
Coverage
Diversity of recommendations
Network centric Evaluation
-
7/31/2019 Music Recommendation
95/113
Network-centric Evaluation
In terms of: Navigation
Average Shortest Path, Giant Component
Connectivity Degree Distribution, Degree-Degree Correlation, Mixing
Patterns
Clustering
Local/Global Clustering Coefficient
Centrality
Degree, Closeness, Betweeness
Network centric Evaluation
-
7/31/2019 Music Recommendation
96/113
Network-centric Evaluation
Limitations:
Accuracy of the recommendations cannot bemeasured
Transparency (explainability) and trustworthiness(confidence) of the recommendations cannot bemeasured
The perceived quality (i.e. usefulness andeffectiveness) of the recommendations cannot bemeasured
User centric Evaluation
-
7/31/2019 Music Recommendation
97/113
User-centric Evaluation
Evaluation focuses on the users perceived
quality and usefulness of therecommendations
User centric Evaluation
-
7/31/2019 Music Recommendation
98/113
User-centric Evaluation
Copes with the limitations of both:
System- and Network-centric approaches
Evaluates: Novelty
Perceived Quality
-
7/31/2019 Music Recommendation
99/113
-
7/31/2019 Music Recommendation
100/113
Novelty
-
7/31/2019 Music Recommendation
101/113
Novelty
Ask users if they recognize the predicted itemsor not
Combining novelty and perceived quality wecan infer if:
User likes to receive and discover unknown items
Prefers more conservative and familiarrecommendations
A/B Testing
-
7/31/2019 Music Recommendation
102/113
A/B Testing
Present two different versions of an algorithm(or two algorithms)
Evaluate which one performs the best
Performance measured by the impact the
new algorithm has on the visitors behavior,compared to the baseline algorithm
User centric Evaluation
-
7/31/2019 Music Recommendation
103/113
User-centric Evaluation
Limitations:
Need ofuser intervention in the evaluationprocess
Gathering feedback from the user can be tedious forsome users
Time-consuming
Evaluation summary
-
7/31/2019 Music Recommendation
104/113
Evaluation summary
Combining the three methods we can coverall the facets of a recommender algorithm
-
7/31/2019 Music Recommendation
105/113
-
7/31/2019 Music Recommendation
106/113
Last fm Dataset 1K users
-
7/31/2019 Music Recommendation
107/113
Last.fm Dataset 1K users
Contains tuples
Represents the listening habits for ~1.000 users
Collected from Last.fmAPI
User.getRecentTracks()
Statistics: ~108,000 Artists with MusicBrainz ID
~70.000 Artists without MusicBrainz ID
Last fm Dataset 360K users
-
7/31/2019 Music Recommendation
108/113
Last.fm Dataset 360K users
Contains tuples from360.000 users
Collected from Last.fm API User.getTopArtists()
Statistics:
~190.000 Artists with MusicBrainz ID ~107.000 Artists without MusicBrainz ID
The Million Song Dataset
-
7/31/2019 Music Recommendation
109/113
The Million Song Dataset
One Million Songs!!! 280GB of data
~45.000 unique artists
~8.000 unique terms
> 2 Million asymmetric similarity relationships
Acoustic features
Pitch, Timbre, Loudness, etc. Links to other sources to obtain more information
Musicbrainz, 7digital, playme
NEXTONE PLAYER
-
7/31/2019 Music Recommendation
110/113
NEXTONE PLAYER
NEXTONE PLAYER: A
Music Recommendation
System Based on User
BehaviorYajie Hu and Mitsunori OgiharaISMIR 2011
Session-based CF for Music
-
7/31/2019 Music Recommendation
111/113
Recommendation
Session-based Collaborative
Filtering for Predicting the
Next SongSung Eun Park, Sangkeun Lee, Sang-goo LeeCNSI 2011
-
7/31/2019 Music Recommendation
112/113
-
7/31/2019 Music Recommendation
113/113