music generation and manipulation
TRANSCRIPT
1
Music generation and
manipulation
Iris Yuping Ren, Anja VolkSound and Music Technology
11-01-2017
2
Overview
g Different contexts for automatic generationg computer as supporting tool for composerg computer as testing tool for researchersg computer as interactive musical instrumentg computer as instrument within educational contextg automatic generation for games
3
History
g Process of composing itself often almost algorithmicalg Example: canong Explained by: Justin Henry Rubin, University of Minnesota
Duluth
http://www.d.umn.edu/~jrubin1/JHR%20Canonic%20Examples2ab.htm
4
History
g Composing as constraint-based “programming”g Example: Palestrina counterpoint, described in Johann Joseph
Fux: Gradus ad Parnassum (1725)g Palestrina Pal (by Anna Huang, 2005): automatic rule checkerg Composing Fifth Species Counterpoint Music With A Variable
Neighborhood Search Algorithm, FuX (by Dorien Herremans, 2013)
g Exploring the Rules in Species Counterpoint (by Iris Ren, 2016)
g Constraint programming in music (Truchet & Assayag, 2011), IRCAM in Paris, for supporting composer
g more using genetic algorithms and neural networks...
5
History
g Musical dice gameg Johan Philipp Kirnberger, 1757
6
History
g Musical dice gameg Johan Philipp Kirnberger, 1757
https://www.youtube.com/watch?v=9Zdg6Ec4mVw
https://www.youtube.com/watch?v=fK2MCXpDWB4
7
History
g Musical dice gameg Johan Philipp Kirnberger, 1757
g Joris de Man, game music composer, ISMIR 2010
https://www.youtube.com/watch?v=9Zdg6Ec4mVw
https://www.youtube.com/watch?v=fK2MCXpDWB4
8
History
g Babbages’ Analytical engineg First fully-automatic calculating machineg conceived in 1834, designed to evaluate any mathematical
formula; part of the machine was completed before Babbage's death in 1871
g checkout Science Museum London
9
History
g Ada Lovelace on Babbages’ Analytical engine:
g Supposing, for instance, that the fundamental relations of pitched sound in the signs of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent”
g Lovelace’s notes became one of the critical documents to inspire Alan Turing’s work on the first modern computers in the 1940s. (http://findingada.com/about/who-was-ada/)
10
History
g Iannis Xenakisg “Formalized music”, 1963g Stochastic methods for composingg Not fully automated, computer has aided the composition
processg https://www.youtube.com/watch?v=qIGDMMN1lQI
g Mario Baroni & Carlo Jacoboni: Proposal for a Grammar of Melody: The Bach Chorales, 1978
g Kemal Ebcioglu: CHORAL (1986)g Rule-based approach to produce Chorals in the style of Bach
11
Recent examples
g Francois Pachet: Continuatorg Computer as interactive musical instrumentg combines techniques from interactive and automatic learning
systems. g learns and interactively plays with a user in the user's style. g based on a Markov model of musical styles
12
Recent examples
g Francois Pachet: Continuator
https://www.youtube.com/watch?v=ynPWOMzossI
https://youtu.be/cHKcq0D5EY4
13
Most recent example
g Franscois Pachet: The flow machineOrchestrations of Beethoven's Ode to Joy by Flow Machines
g Tutorial at this ISMIR 2017: http://www-desir.lip6.fr/~briot/documents/deep-learning-music-generation-tutorial-ismir-2017.pdf
g New Survey paper: Deep Learning Techniques for Music Generation-A Survey https://arxiv.org/abs/1709.01620
g first album!
14
Most recent example
g The BCD in Bach generation
BachBot | Creative AI for Classical Music
Counterpoint by convolution - ismir 2017
DeepBach: harmonization in the style of Bach generated using deep learning
15
And much more
Folk, Jazz, improvisation, style transfer, etc.https://www.theguardian.com/stage/2015/dec/01/beyond-th
e-fence-computer-generated-musical-greenham-common
https://medium.com/artists-and-machine-intelligence/neural-nets-for-generating-music-f46dffac21c0
“Vocaloids like Hatsune Miku have shown that a virtual persona backed by a vocal synthesizer can bring together millions of people in a massively crowdsourced act of composition and listening. Music is probably older than language, but we’re still discovering all the things music can be, and all the ways it might be crafted.”
16
What do you think?
Current status: Convincing Music from algorithms!
What do you think?
Will you listen to algorithmically generated music? Will you be able to tell the difference? Will you use such algorithms? Will it bother or help you?
◍ Jukedeck◍ Amper Music◍ Aiva◍ Brain.fm◍ Popgun◍ Mubert
Music Generation Landscape
Recap from Ryan Grove
◍ Google’s Magenta team◌ Recently released
a melody interpolator4
Music Generation Landscape
4https://nips2017creativity.github.io/doc/Hierarchical_Variational_Autoencoders_for_Music.pdf
◍ Spotify?
Music Generation Landscape
4https://www.musicbusinessworldwide.com/spotify-is-creating-its-own-recordings-and-putting-them-on-playlists/
◍ Spotify?◌ Was accused of creating ‘fake’ artists5
Music Generation Landscape
5https://www.musicbusinessworldwide.com/spotify-is-creating-its-own-recordings-and-putting-them-on-playlists/
◍ Spotify?◌ Was accused of creating ‘fake’ artists4
◌ Recently hired Francois Pachet
Music Generation Landscape
4https://www.musicbusinessworldwide.com/spotify-is-creating-its-own-recordings-and-putting-them-on-playlists/
◍ Niantic (Pokemon Go)◌ CEO says that, for augmented reality,
audio is critically important6
Music Generation Landscape
6https://techcrunch.com/2017/09/18/pokemon-go-creators-next-game-will-incorporate-audio-into-the-ar-experience/
23
Most recent example
g Dorien Herremans and Elaine Chew: MorpheuS (2016)g Dagstuhl stimulus talk, March 2016
24
MorpheuS
25
Constraints: tension and patterns
26
Tonal tension
https://vimeo.com/151142022
https://www.youtube.com/watch?v=bLNIiuCqeno
27
Tonal tension
28
Tonal tension
29
Tonal tension
30
Pattern
31
Generation
32
Generation
33
Generation
34
Generation
36
Student presentation
g Carles Roig, Lorenzo J. Tardón , Isabel Barbancho, Ana M. Barbancho, Automatic melody composition based on a probabilistic model of music style and harmonic rules
37
Student presentation
g Francois Pachet, Pierre Roy: Non-Conformant Harmonization: the Real Book in the Style of Take 6