automatic design of sound synthesizers as pure data patches using coevolutionary mixed-typed...
TRANSCRIPT
E N G A G I N G T H E W O R L D
Philippe Pasquier [email protected]
1
Matthieu Macret [email protected]
Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming
Wednesday, July 16, 2014
The Synthesis Calibration Problem
2
Target sound
Sound Synthesizer
i1
i2
...
in
=Synthesized sound
Input parameters
Wednesday, July 16, 2014
Automatic calibration of Sound synthesizers
3
Synthesis technique Op/miza/on technique Reference
Addi)ve HMM Heise (AES Conven)on 2009)
Substra)ve Par)cle Swarm Yoshimura (Speech Communica)on and Tech. 1999)
Substra)ve Neural Network Roth (AES Conven)on 2011)
FM Cellular Automata Serquera (Applica)ons of Evolu)onary Computa)on 2010)
FM GA Horner (Computer Music Journal 1993)
Addi)ve GA Horner (Computer Music Journal 1996)
Granular GA Fujinaga (ICMC 1994)
Wednesday, July 16, 2014
ModFM synthesizer (Macret SMC 2012)
4
Wednesday, July 16, 2014
ModFM synthesizer (Macret SMC 2012)
4
Target
Wednesday, July 16, 2014
ModFM synthesizer (Macret SMC 2012)
4
Target
Wednesday, July 16, 2014
ModFM synthesizer (Macret SMC 2012)
4
Wednesday, July 16, 2014
ModFM synthesizer (Macret SMC 2012)
4
Synthesized
Wednesday, July 16, 2014
ModFM synthesizer (Macret SMC 2012)
4
Synthesized
Wednesday, July 16, 2014
More complex synthesizer• VST Instruments with GA (Yee-King ICMC 2011)
5
Wednesday, July 16, 2014
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Wednesday, July 16, 2014
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Target
Wednesday, July 16, 2014
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Target
Wednesday, July 16, 2014
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Wednesday, July 16, 2014
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Synthesized
Wednesday, July 16, 2014
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Synthesized
Wednesday, July 16, 2014
General Sound Synthesis Problem
7
Target sound
Sound Synthesizer
i1
i2
...
in
=Synthesized sound
Input parameters
Wednesday, July 16, 2014
Genetic Programming - Garcia (DAFx 2001)
8
Produc)on rulesin had hoc synthesis system
Sound Synthesizer
i1
i2
...
in
=
Synthesized sound
Target sound
GPGA
Wednesday, July 16, 2014
Coevolutionary Mixed-typed Cartesian GP
• Coevolution of:–Mixed-typed Cartesian Genetic Programming•Evolve synthesizer architecture
–Genetic Algorithm•Optimize Input Parameters
9
Wednesday, July 16, 2014
Pure Data• Graphical programming environment for audio, video, and image processing• http://puredata.info/• Popular among musician, artist and sound designers
10
Wednesday, July 16, 2014
11
Wednesday, July 16, 2014
11
Wednesday, July 16, 2014
11
Wednesday, July 16, 2014
Mixed-typed Cartesian GP
12
• Harding (GECCO 2012)
• Characteristics:–Directed Acyclic Graph–Mixed-typed
Wednesday, July 16, 2014
13
0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1
CGP graph
Pd Patch
Wednesday, July 16, 2014
14
Gene Input 1 Input 2 Extra Param.
Descrip/on
Output Type
3 control control -‐ Division control
4 signal control -‐ Oscillator signal
Wednesday, July 16, 2014
15
0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1
CGP graph
Pd Patch
Wednesday, July 16, 2014
Fitness function• MFCC (Mel-Frequency Cepstral Coefficients) in windows of 23 ms
16
Euclidean distance between synthesized sound and target sound
Wednesday, July 16, 2014
Fitness function• MFCC (Mel-Frequency Cepstral Coefficients) in windows of 23 ms
16
Euclidean distance between synthesized sound and target sound
Wednesday, July 16, 2014
Evaluation
17
i1
i2
...
in
i1
i2
...
ini1
i2
...
in
i1
i2
...
in
GA popula)on (150 individuals) CGP Popula)on (5 individuals)
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Wednesday, July 16, 2014
Genetic Algorithm
18
• Binary encoding of chromosome• Binary tournament selection• 2-points crossover• Bitflip mutation
Wednesday, July 16, 2014
Mixed-typed Cartesian GP• Integer encoding of chromosome• Evolutionary Strategy 1+4• Mutation-only approach
19
Wednesday, July 16, 2014
20
Initialize CGP /GA population
Evaluate CGP / GA population
Variate GA population
Select best CGP individual
Variate CGP population
Termination criteria ?End
No
Yes
Wednesday, July 16, 2014
Clarinet sound
21
Wednesday, July 16, 2014
Clarinet sound
21
Wednesday, July 16, 2014
Clarinet sound
21
Wednesday, July 16, 2014
Clarinet sound
21
Target
Wednesday, July 16, 2014
Clarinet sound
21
Target
Wednesday, July 16, 2014
Clarinet sound
21
Wednesday, July 16, 2014
Clarinet sound
21
Synthesized
Wednesday, July 16, 2014
Clarinet sound
21
Synthesized
Wednesday, July 16, 2014
Clarinet sound
22
Wednesday, July 16, 2014
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
Stats
24
Wednesday, July 16, 2014
Conclusions• New approach to automate the design of sound synthesizers using Coevolutionary Mixed-typed Cartesian Programming• Synthesizers represented as reusable Pd patches
25
Wednesday, July 16, 2014
Future works• Our methods involves many parameters– Parameter sensitivity analysis
• Extend our system to reproduce not only one target sound but a set of target sounds• Improve our fitness function– Move to a multi-objective fitness function
• Quantitative/Qualitative comparison with previous similar systems
26
Wednesday, July 16, 2014