annotating music and lyrics
DESCRIPTION
Annotating Music and Lyrics. Kristine Monteith CS 652 - Research Project June 11, 2009. Project Goal. Find a suitable song for a given situation Applications Indexing songs by topic and mood Soundtracks for movie scenes Music therapy groups. What do we want to label?. Lyrics Themes - PowerPoint PPT PresentationTRANSCRIPT
Annotating Music and LyricsKristine Monteith
CS 652 - Research ProjectJune 11, 2009
Project GoalFind a suitable song for a
given situationApplications
Indexing songs by topic and mood
Soundtracks for movie scenes
Music therapy groups
What do we want to label? Lyrics
Themes Moods
Music Moods Energy Emotional evocativeness Genre
Song Ontology
How to Extract Features:Lyrics Current state: Keyword search
Find all occurrences of the search term in the song
Find all occurrences of search term synonyms (Using WordNet synsets)
Future work: Extend searches to phrases Determining mood
Supervised Learning
Classifier
Features of a Musical Selection
Prediction of target label
Labeled Training Data
Input Features Bag of Words
Words appearing on page and word counts
Looking for other methods to analyze documents and collect input features
Output LabelsDerived from questionnaireHand-labeled by researchers
How to Extract Features:Music Target labels to predict
Moods (labeled by subject or expert) Energy (determined by direction of
change in biofeedback responses) Emotional evocativeness (determined
by extent of change in biofeedback responses)
Genre (labeled by subject, expert, or clustering)
Input Features:Acoustic Properties
Spectral Centroid Spectral Rolloff Point Spectral Flux Compactness Spectral Variability Root Mean Square Fraction of Low Energy Windows Zero Crossings Strongest Beat Beat Sum Strength of Strongest Beat Strongest Frequency Via Spectral Centroid Strongest Frequency Via FFT Maximum
Input Features:Acoustic Properties
MFCC LPC Method of Moments Partial Based Spectral Centroid Partial Based Spectral Flux Peak Based Spectral Smoothness Relative Difference Functions Area Method of Moments
Input Features:Symbolic Features
Tempo Key Mode Musical form Rhythmic structure Vocalization Instrumentation Melodic contour Harmonic patterns
Output Labels:Questionnaire-based Responses
Output Labels:Physiological Responses Heart rate Breathing rate Perspiration Skin temperature
Each subject will listen to one minute segments separated by one minute of silence
Conclusion Demo: Music Therapist Assistant
Any Questions/Suggestions?