kristian nymoen, universitet i oslo...•recommendation systems, automated playlist generations...
TRANSCRIPT
![Page 1: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/1.jpg)
Music Information RetrievalKristian Nymoen,Universitet i Oslo
![Page 2: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/2.jpg)
RITMO
• RITMO is an interdisciplinary research centre focused on rhythm as a structuring mechanism for
the temporal dimensions of human life.
• Funded by the Research Council of Norway's Centres of Excellence scheme
• 13 permanent faculty members
• 23 PhD/Postdocs
• 10 more PhD/Postdoc positions recently announced
https://www.hf.uio.no/ritmo/english/about/vacancies/
• 4 administration / technical support staff members
Dept. of Psychology Dept. of Musicology Dept. of Informatics
![Page 3: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/3.jpg)
Outline
• Musical information
• First example: signal processing approach
• Machine learning applications in music information retrieval
• Understanding listeners
![Page 4: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/4.jpg)
Information in a sound/music signal
Intensity
Pitch
Attack time
Decay time
Duration
Sound source
LocationMotion
Harmony
Melody
Rhythm
Movement/Dance
Grouping in time
and frequencyType
Single events Affordances
Emotions
![Page 5: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/5.jpg)
First example:
Chord recognition
![Page 6: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/6.jpg)
![Page 7: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/7.jpg)
![Page 8: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/8.jpg)
![Page 9: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/9.jpg)
![Page 10: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/10.jpg)
![Page 11: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/11.jpg)
![Page 12: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/12.jpg)
![Page 13: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/13.jpg)
Chord prediction•Offline: Create lead sheets from audio
•Real time: Predict chords and have automated accompaniment
![Page 14: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/14.jpg)
https://www.youtube.com/watch?v=COPNciY510g
(Dorfer, Henkel & Widmer, 2018)
Score following•Offline: Create more advanced music databases
•Real time: Music education
![Page 15: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/15.jpg)
Music classification •Genre, artist, instrument, composer…
• Fill in missing metadata in music databases
•Recommendation systems, automated playlist generations
![Page 16: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/16.jpg)
Pattern matching and detection •Audio fingerprinting
•Protect against copyright violations
•Cover song detection
•Query by example (e.g. Shazam)
•Query by humming (e.g. SoundHound)
•Query by tapping (e.g. SongTapper)
•Query by gesture (e.g. SoundTracer)
![Page 17: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/17.jpg)
Sound source separation •Create music notation from audio files
•Music analysis, music education
Chandna, Miron, Janer, & Gomez (2017)
https://www.youtube.com/watch?v=71WwHyNaDfE
![Page 18: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/18.jpg)
![Page 19: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/19.jpg)
Listening patterns
Emotional response
Psychoacoustics
Behavioural response
![Page 20: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/20.jpg)
![Page 21: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/21.jpg)
![Page 22: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/22.jpg)
• MIRtoolbox function:
miremotion
![Page 23: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/23.jpg)
“Simple visualisation of Valence and Arousal in music. The model is based on work by
Tuomas Eerola, Petri Toiviainen and Olivier Lartillot (see e.g. Eerola et al., 2009)
and the real-time implementation was made by Petri Toiviainen in MAX/MSP. “
https://www.youtube.com/watch?v=JqYoA3OM-b4 https://www.youtube.com/watch?v=EJQw5XGK3tI
![Page 24: Kristian Nymoen, Universitet i Oslo...•Recommendation systems, automated playlist generations Pattern matching and detection •Audio fingerprinting •Protect against copyright](https://reader035.vdocument.in/reader035/viewer/2022071108/5fe2612ea23fdc14992671aa/html5/thumbnails/24.jpg)
More material:• MIR communities:
- The International Society of Music Information Retrieval:
www.ismir.net
- Music Information Retrieval Evaluation eXchange:
www.music-ir.org
- Sound and Music Computing Network:
http://www.smcnetwork.org
- Nordic Sound and Music Computing Network:
https://nordicsmc.create.aau.dk
• Open Source Toolboxes:
- essentia.upf.edu (C++/Python library)
- https://github.com/olivierlar/miningsuite/wiki (Matlab)
- https://www.jyu.fi/hytk/fi/laitokset/mutku/en/research/materials/mirtoolbox (Matlab)
https://www.audiolabs-
erlangen.de/fau/professor/mueller/bo
okFMP
Book: