real-time corpus- based concatenative synthesis with catart
TRANSCRIPT
Real-Time Corpus-based Concatenative
Synthesis with CataRTDiemo Schwarz
IMTR Real-Time Music Interaction TeamIrcam–Centre Pompidou
http://imtr.ircam.fr CataRT releases
http://ftm.ircam.fr FTM for Max 5
Ircam Overview 24.6.2011
1
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Principle of CBCS
Corpus-basedsounds + segmentation + descriptors (timbre, metadata, classes) concatenative grains + overlap + chainingreal-time synthesis interactive + explorative (navigation)
2
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Motivation and Approachwork with all the nuances of real soundlarge sound databases exist, ready to use
new method → new sound creationdata-driven vs. rule-based approachgeneral superiority of data-driven approaches in many other domains
3
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Influences and LinksInspired by concatenative speech synthesisCBCS in real-time (as implemented in CataRT) can be seen as a content-aware extension to granular synthesis:
providing direct access to specific sound characteristicnot restricted to selection by position in a sound
4
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Applications of CBCS
Interactive exploration of sound databases (browsing)Resynthesis of an audio target (mosaicing)Texture synthesis (ambiences, soundtracks)High-level instrument synthesis (example: Synful)Expressive speech synthesis
5
CataRT — http://imtr.ircam.fr — Diemo Schwarz
A Short History of Data-Driven Synthesis
I.Prehistory (1948)Concrete and early electronic music
II.The Classics — revisited (1980, 1990)SamplingGranular SynthesisJohn Oswald
III.The Modern Times (2000)many research projects, often in mutual ignorance
6
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Tape splicing
soft attack or decay
combined attack and decay of two sounds
medium attack or decay
hard attack or abrupt finish
softer and less abrupt than D
[figures: Mikhail Malt]
Karlheinz Stockhausen: Étude des1000 collants (1952)
7
CataRT — http://imtr.ircam.fr — Diemo Schwarz
John Cage: Williams Mix (1953)
[http://www.medienkunstnetz.de/works/williams-mix]
8
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Related Work up to 2005
1948
19801990
1993
20002001
2001
20012002
2003
2003
2003
2003
2003
20042004
2004
20042005
2005
2005
Selection
Let them sing it for you
Granular
Samplers
Synful
Musescape
CataRT
MoSievius
stochastic SoundscapesSoundclustering
La legende des sieclesN.A.G.
MATConcat
targeted Directed Soundtracks
Soundmosaicing
Input driven resynthesisMPEG−7 Audio Mosaics
Soundspotter
automatic
Ringomatic
Musical Mosaicing
Caterpillar
Audio Analogies
manual segmental high−level
Musique Concretemanual
Plunderphonics
similarity descriptorssimilarityframe spectrum
Analysis
fixed
2005
2003
[Schwarz, JNMR 2006]
9
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Current Related WorkTimbreID (William Brent) is a collection of PD externals that allow descriptor analysis, corpus-based concatenative synthesis and CataRT-style 2D interactionExtending voice-driven synthesis to audio mosaicing (Jordi Janer, Maarten de Boer): a system for controlling audio mosaicing with a voice signalMused: Navigating the Personal Sample Library (Graham Coleman): a system for sample-based music compositionScrambledHacks (2006): performance system for spectral lookup alongside ganged music video concatenation
10
CataRT — http://imtr.ircam.fr — Diemo Schwarz
CataRT Introductionflexible testbed to experiment real-time corpus-based concatenative synthesis: CataRTrealisation of CBCS in real-time
patch for Max/MSP with FTM, Gabor, MnM by Norbert Schnell et al. from http://ftm.ircam.frreleased as Free Software under GPL at http://imtr.ircam.fr mailing list concat on http://list.ircam.frmodular MVC architecture:
11
CatArt (?)
CataRT
12
CataRT — http://imtr.ircam.fr — Diemo Schwarz
CataRT Overview
CorpusAnalysis
DB
Synthesis
Audio Input Control Input
13
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Analysis
Corpus
Soundfile+markers
+descriptors
Persistent Sound and Unit
Database
descriptor analysis
Soundfile+grain sizeor markers
RT audio input
Audio Input
onset detection
Soundfile
14
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Synthesis
Corpusnearest
neighbour selection
Target descriptor values Control
transformation + concatenation
descriptor analysis
RT audio input
onset detection
controller
15
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Architectureone catart.data per corpusone catart.import for segmentation and analysisany number of catart.data.proxy to access dataany number of catart.lcd for display and controlany number of catart.selection per corpusany number of catart.synthesis per selection or per corpus
16
Composers and Projects using CataRTSound design and compositionLuis Naõn, Hans Tutschku, Matthew Burtner, Sebastien Roux, Hector Parra, Roque Rivas, Ben Hackbarth
InstallationPlumage (project Enigmes about the navigable score, Roland Cahen), Pierre Jodlowski, Cécile Babiole, Franck Leibovici
Live electro-acoustic music Dai Fujikura, Stefano Gervasoni, Miguel Angel Ortiz-Pérez, Sam Britton, Aaron Einbond, Bruno Ruviaro, Christopher Trapani, Marc Vitoria, Eric Maestri, theconcatenator
Examples: myspace.com/catartsoftware
!"
!"
18
""
""
q = 60
q = 60
!"
!"
q = 102
q = 102
!#
!#
""
""
!"
!"
A. Fl.(smallgroup)
B. Cl.(smallgroup)
Tpt.(smallgroup)
1 Vln.(smallgroup)
1 Vc.(smallgroup)
electronics
Fl.
Ob. I
Ob. II
Cl. I
Cl. II
Bsn. I
Bsn. II
Hn. I
Hn. II
Hn. III
Hn. IV
Tpt.
Tbn. I
Tbn. II
Timp.
B. D.
Vln. I
Vln. II
Vla.
Vc.
Db.
!
"
!
"
!
"
!
"
!
"
!
"
!"
$SMALL GROUP
%&'%( % % %) % # % #
p
* + + ,ca. 20 - 21 sec.
, ,- , , , ,
$ % %( %&'(% % % #
p
%) * , , , ,- , , , ,
$ % % #
p
% #) . + , , , ,- , , , ,
$ %&'( #% % #
% % %p
+ / , , ,- , , , ,
0 $% %1 % % %p
% #) . , , , ,- , , , ,
$
section 3
*
4
%%%%%%%%%(
211( %%%%%%%
((
more and more dense and violent(grain size gets smaller and smaller)Harmony 3Harmony 2
gliss.
gliss.
+3 semitones,
-4 semitones,
%%%%%%%%%1
2222 %%%%%%%22(1 ,
CUT OFF
5
34 5 . * + /
$ , , , , ,- +marcato
*ff
$%'6 $%
'6$%
'6%7( %7($%
'6
%7 . . %7 . $%'
( 6$%
'6%7( %7 .$%
'6 %78 . * $%'6%78 . * .
$%'
6$%
'( 6
%7 .%7
*sffffz
$%'6%7'( % * * espress.
fff
. $%'6 %8 %( %
$%'6
%% #
$ , , , , ,- +marcato
*ff
$%'( 6 $%'
( 6%7 %71 * $%
'6 %78 . + $%'( 6 $%'6
%7 .%7(
.$%
'6$%'6%7 %7 * %
%')$%
'( 6
$%'6
%7 %7
. %7( *sffffz
%7'(8 . *
espress.
*fff
%) $%'6% %( # % %
$ , , , , ,- + marcato*ff
. %%'()
$%'6
%74 $%'6%7(
. . %7 .
$%'
( 6 %78 *$%
'6 %7(8 . * * $%'6
$%'6
%7 %7 . $%
'( 6 %7(8 * .
$%'6
sffffz
%78fff
%7'(. . % %8 . * + *
espress.
%6
$ , , , , ,- +marcato
*ff
%%') %74 .
%%
'
2 )%%')
$%'6
%7%7 .
%7
(.
%%')
%%
'
)%%
'
)%7( %7 %7* $%'( 6 $%
'6
%%
'
)
%7 %7(%%')
%%
'
)%7. %7 %7
. %%')
%%
'
)$%
'6
%7
( .%7( %7 .
sffffz fff
$%
'
( 6
%7'
(
%(. + *
espress.
%%')
%%(
%%'
()%
%
$ORCHESTRA
, , , , ,- +marcato
* .ff
%%')
$%'( 6
%74%%
'
2 )%7 .
%7. .
%%
'
) %%')%7 %7 . %
%')%%
'
) %%')
$%'
( 6
%7 %72. %7
$%'
( 6
%%')
%7( . . %71%%
'
)%72. %7
.%%
'
)%%')
sffffz
%7 %7fff
espress.
%%'
()$
%
'
(6
%7'% %( %
% #%
% #%
%%
'
)%
% %
9 , , , , ,- +marcato
*ff
$%'2 6%78 .$%
'6
$%'6%7 . %7 .
$%'
6$%
'6$%'2 6 $%
'6%7 %72 . %7 $%'6 $%'6
%%')
%7 %7 . %7%%')
$%'6
%7 . . %7$%
'6%7 .%7
.$%
'6 %78 .
sffffz fff
$%'
( 6 espress.
%7'
2%
%( . $%'
( 6%( %( % %) *
$%'6 % # %
9 , , , , ,- +marcato
*ff
%%') %74 . .
$%'6
$%'6%7
%7 . %
%'2) %%') $%
'6%7 %7 . %7 * %
%') %%')
$%'6
%7 %7$%
'6%7 . %7 . . %%') %74 .
$%'
6 %78 *sffffz
espress.
fff
%7' % %( % %
) * $%'
( 6 %( # % %) *
$ , , , , ,- + + marcato
ff
. $%'6
%74 * 9 . $%'6
$%'
2 6%7. %7 . $%
'6 $%'6%7 . %7 * . $%'6%78 * $%
'6 %78 . *sffffz
ff
$%'
( 6%7' * %7( * $%'6 %7(8 . + .$%'6 %78 *
$ , , , , ,- + 9+ marcato*ff
$%'2 6 %78 . .$%
'6 %728 *$%
'6%78 . *$%
'6%78 . * $%'6 %78 . * $%
'6 %728 .sffffz ff
$%'6%7'
. . %( . $%'( 6 %78 * $%'6
$%'
( 6%7 . . %7 + $
$ , , , , ,- + marcato9 *ff
$%'6%78 . .
$%'6%78 * * . $%
'6$%
'6%728 $%'6%7 . . %7 + * $%'
( 6 %78 . * $sffffz
%7'8 . *ff
*$%
'6 %7( 4 . * *
$%'
6%74 . * 9
$ , , , , ,- + 9*ff
marcato$%'2 6%78 . * $%'6%78 . . $%
'6%74 * * . %
%') %74 * . $%'6
$%'6
%74 %728 . * .$%
'6
sffffz
%78ff
%%')%7'( %7( * * $%
'( 6 %78 . * $ . $%'6 %7( 4 *
%%'
() %7( 4 .
$ , , , , ,- + + marcato
ff
$%'( 6%78 . * *$%
'6 %78 . .
$%
'6 %%
')%72
. %7. $%'( 6%78 * . $%
'6 %7(8 * *sffffz ff
$%'6%7' %7 * * . $%'6 %78 + .$%'6 %78 *
9 , , , , ,- , , ,(senza sord.)
sffffz
%7'2 4 . * + /
9 , , , , ,- , , ,straight mute
sffffz
%7'(8 . *super short staccato
+p
%7'( 7' 7' 7' 7'
5
gliss.
9 , , , , ,- , , ,p
soft sticks
%'( ' ' ' ' ' ' ' ' ' ' '3 3 3 3
gliss.
: , , , , ,- , , ,sffffz
%7'4 . * + /
$ , , , , ,- marcato+ *ff
$%'6 %7(8 .
$%'6
$%'
6%7( . . %7 * $%
'6 %7(8 .$%
'6 $%'6%7(
. .%7
* $%'6
%74 . . $%'
( 6 %7(8 *$%
'6 %74 .
sffffz
unis.
fff
$%'6 espress.%7'( %
%($%'6
% %% %( % %4 . *
$ , , , , ,-div.
marcato+ * .ff
$%'6 %7(8 * $%
'6 %78 . . $%'( 6 %7(8 * *
$%'6 %78 .
$%'6 $%
'( 6%7 . . %71 . $%
'( 6 $%
'6%%')%7(
.%7
. %7
$ , , , , ,-
div.
marcato
+ *ff
$%'( 6%78 .$%
'6$%
'6%7(. .
%7(*
$%
'6 %74 . %
%'()%%')
%7
( . . %7* . %
%') %78 .$%
'( 6
$%'
6sffffz
unis.%7* .
%7
2fff
$%'6 espress.
$%'
6%7'( %7 . %(
%) * /
$ , , , , ,-marcato
+ff
* .$%
'( 6
%718 * . $%'6 %7(8 .
$%'
( 6 %78 *
$%'6
$%'6%7( . %7 .
$%'6
$%'
6%7
. %7 .
$%'
6 %%')%72
. . %7
( *
0 , , , , ,-div.
marcato+ *ff
$%6%74 . $%
'2 6 %
%')%7 . . %7 * $%
'6 %78 . . $%'6 %72 4 *
$%'
( 6$%
'6%7 . %7 . %
%') %78 . $%'
2 6%74 .
$%'
6 %74 .
sffffz
unis.
fff
espress.%7' %( %$%
'6%
%( % %% #
% % *
0 , , , , ,- marcato+ff
* .$%( 6
%78 * $%'
2 6%74 . . $%'6 %72 4 *
$%'
( 6 %78 . *$%
'6$%
'6%7 . %7 . . $%
'6%78 * . %%') %72 4
9 , , , , ,- +marcato
*ff
$%'6
%74 . .$%
'6 %78 * $%'2 6$%
'6%7 . . %7 * $%
'6%74 . $%'6
%%')%7 . . %7 * .
$%'
6 %78 *
sffffz
unis.
%%'()
espress.
%7'%( .
fff
. %%')
% %($%
'( 6
% % % +
9 , , , , ,-div.
+marcato
*ff
.$%
'6%78 * $%
'6%74 . . $%
'6 %78 *$%
'6
$%'6%7
. %7 . $%'2 6
%%')%7 . . %7 +
$%'
6 %78 .
9 , , , , ,- +marcato
*ff
$%'6
%78 . $%'6$%
'6%7. %7 . .
$%'6
%78 * .$%
'2 6 %78 *
$%'6
%78 . * .$%
'6%78 +
sffffz
%7'%7( * *
fff
$%'6 %(
) + * %)
2
17
Installation GrainStick“Rainstick” gestural control metaphor
Demo of SAME (Sound And Music for Everyone Everyday Everywhere Every way) EU project
Composition of sounds and ambiences by Pierre Jodlowski
WFS spatialisation and Motion Tracking by Grace Leslie and the Room Acoustics team
Gesture analysis by Bruno Zamborlin and Diemo Schwarz
18
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Grainstick Example 1
19
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Grainstick Example 2
20
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Grainstick InstallationsPrototype: Agora Festival 2009, Ircam, ParisPublic installation: 6. June 2010, Cité des Sciences, Paris
including video projection, more collaborative elements, several universes
Supports et concept de diffusion
Structure tubulaire démontable autonome de 10 mètres de diamètre. Surface de projection en
toile de cinéma circulaire et système de diffusions vidéo et audio intégrées. Ouverture 1/6e de
la circonférence pour l'entrée du public.
21
Installation Xe-RocksTurning photocopiers into musical instruments, by video and sound artist Cécile Babiole
On show at the ECM Gantner, Bourogne
22
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Installation PlumageCollaboration with national design school ENSCI led by Roland Cahen, Yoan Ollivier, Benjamin Wulf (ENSCI), Christian Jacquemin, Rami Ajaj (LIMSI), Diemo Schwarz (Ircam)
within the project ENIGMES about the navigable score3D interface using Virtual Choreographer http://virchor.sourceforge.net
23
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Plumage
24
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Plumage
25
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Plumage
26
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Live Recording the CorpusCorpus-based improvisation with live sound input
Diemo Schwarz with Etienne Brunet
Live Algorithms for Music (LAM) concerts:Improvisation of George Lewis, Evan Parker, Sam Britton, Diemo Schwarz and othersRien du tout by Sam Britton, Diemo Schwarz
27
28
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Electro–Acoustic Music (1)
Whisper Not by Stefano Gervasoni for viola and electronics, computer music realization by Thomas Goepfer
“response” of CataRT to the instrumentmorphing between corpora of pizzicato viola and water-drops
Interaction with pre-recorded sound,
interpolation, rearranging, navigation
29
!"
!"
18
""
""
q = 60
q = 60
!"
!"
q = 102
q = 102
!#
!#
""
""
!"
!"
A. Fl.(smallgroup)
B. Cl.(smallgroup)
Tpt.(smallgroup)
1 Vln.(smallgroup)
1 Vc.(smallgroup)
electronics
Fl.
Ob. I
Ob. II
Cl. I
Cl. II
Bsn. I
Bsn. II
Hn. I
Hn. II
Hn. III
Hn. IV
Tpt.
Tbn. I
Tbn. II
Timp.
B. D.
Vln. I
Vln. II
Vla.
Vc.
Db.
!
"
!
"
!
"
!
"
!
"
!
"
!"
$SMALL GROUP
%&'%( % % %) % # % #
p
* + + ,ca. 20 - 21 sec.
, ,- , , , ,
$ % %( %&'(% % % #
p
%) * , , , ,- , , , ,
$ % % #
p
% #) . + , , , ,- , , , ,
$ %&'( #% % #
% % %p
+ / , , ,- , , , ,
0 $% %1 % % %p
% #) . , , , ,- , , , ,
$
section 3
*
4
%%%%%%%%%(
211( %%%%%%%
((
more and more dense and violent(grain size gets smaller and smaller)Harmony 3Harmony 2
gliss.
gliss.
+3 semitones,
-4 semitones,
%%%%%%%%%1
2222 %%%%%%%22(1 ,
CUT OFF
5
34 5 . * + /
$ , , , , ,- +marcato
*ff
$%'6 $%
'6$%
'6%7( %7($%
'6
%7 . . %7 . $%'
( 6$%
'6%7( %7 .$%
'6 %78 . * $%'6%78 . * .
$%'
6$%
'( 6
%7 .%7
*sffffz
$%'6%7'( % * * espress.
fff
. $%'6 %8 %( %
$%'6
%% #
$ , , , , ,- +marcato
*ff
$%'( 6 $%'
( 6%7 %71 * $%
'6 %78 . + $%'( 6 $%'6
%7 .%7(
.$%
'6$%'6%7 %7 * %
%')$%
'( 6
$%'6
%7 %7
. %7( *sffffz
%7'(8 . *
espress.
*fff
%) $%'6% %( # % %
$ , , , , ,- + marcato*ff
. %%'()
$%'6
%74 $%'6%7(
. . %7 .
$%'
( 6 %78 *$%
'6 %7(8 . * * $%'6
$%'6
%7 %7 . $%
'( 6 %7(8 * .
$%'6
sffffz
%78fff
%7'(. . % %8 . * + *
espress.
%6
$ , , , , ,- +marcato
*ff
%%') %74 .
%%
'
2 )%%')
$%'6
%7%7 .
%7
(.
%%')
%%
'
)%%
'
)%7( %7 %7* $%'( 6 $%
'6
%%
'
)
%7 %7(%%')
%%
'
)%7. %7 %7
. %%')
%%
'
)$%
'6
%7
( .%7( %7 .
sffffz fff
$%
'
( 6
%7'
(
%(. + *
espress.
%%')
%%(
%%'
()%
%
$ORCHESTRA
, , , , ,- +marcato
* .ff
%%')
$%'( 6
%74%%
'
2 )%7 .
%7. .
%%
'
) %%')%7 %7 . %
%')%%
'
) %%')
$%'
( 6
%7 %72. %7
$%'
( 6
%%')
%7( . . %71%%
'
)%72. %7
.%%
'
)%%')
sffffz
%7 %7fff
espress.
%%'
()$
%
'
(6
%7'% %( %
% #%
% #%
%%
'
)%
% %
9 , , , , ,- +marcato
*ff
$%'2 6%78 .$%
'6
$%'6%7 . %7 .
$%'
6$%
'6$%'2 6 $%
'6%7 %72 . %7 $%'6 $%'6
%%')
%7 %7 . %7%%')
$%'6
%7 . . %7$%
'6%7 .%7
.$%
'6 %78 .
sffffz fff
$%'
( 6 espress.
%7'
2%
%( . $%'
( 6%( %( % %) *
$%'6 % # %
9 , , , , ,- +marcato
*ff
%%') %74 . .
$%'6
$%'6%7
%7 . %
%'2) %%') $%
'6%7 %7 . %7 * %
%') %%')
$%'6
%7 %7$%
'6%7 . %7 . . %%') %74 .
$%'
6 %78 *sffffz
espress.
fff
%7' % %( % %
) * $%'
( 6 %( # % %) *
$ , , , , ,- + + marcato
ff
. $%'6
%74 * 9 . $%'6
$%'
2 6%7. %7 . $%
'6 $%'6%7 . %7 * . $%'6%78 * $%
'6 %78 . *sffffz
ff
$%'
( 6%7' * %7( * $%'6 %7(8 . + .$%'6 %78 *
$ , , , , ,- + 9+ marcato*ff
$%'2 6 %78 . .$%
'6 %728 *$%
'6%78 . *$%
'6%78 . * $%'6 %78 . * $%
'6 %728 .sffffz ff
$%'6%7'
. . %( . $%'( 6 %78 * $%'6
$%'
( 6%7 . . %7 + $
$ , , , , ,- + marcato9 *ff
$%'6%78 . .
$%'6%78 * * . $%
'6$%
'6%728 $%'6%7 . . %7 + * $%'
( 6 %78 . * $sffffz
%7'8 . *ff
*$%
'6 %7( 4 . * *
$%'
6%74 . * 9
$ , , , , ,- + 9*ff
marcato$%'2 6%78 . * $%'6%78 . . $%
'6%74 * * . %
%') %74 * . $%'6
$%'6
%74 %728 . * .$%
'6
sffffz
%78ff
%%')%7'( %7( * * $%
'( 6 %78 . * $ . $%'6 %7( 4 *
%%'
() %7( 4 .
$ , , , , ,- + + marcato
ff
$%'( 6%78 . * *$%
'6 %78 . .
$%
'6 %%
')%72
. %7. $%'( 6%78 * . $%
'6 %7(8 * *sffffz ff
$%'6%7' %7 * * . $%'6 %78 + .$%'6 %78 *
9 , , , , ,- , , ,(senza sord.)
sffffz
%7'2 4 . * + /
9 , , , , ,- , , ,straight mute
sffffz
%7'(8 . *super short staccato
+p
%7'( 7' 7' 7' 7'
5
gliss.
9 , , , , ,- , , ,p
soft sticks
%'( ' ' ' ' ' ' ' ' ' ' '3 3 3 3
gliss.
: , , , , ,- , , ,sffffz
%7'4 . * + /
$ , , , , ,- marcato+ *ff
$%'6 %7(8 .
$%'6
$%'
6%7( . . %7 * $%
'6 %7(8 .$%
'6 $%'6%7(
. .%7
* $%'6
%74 . . $%'
( 6 %7(8 *$%
'6 %74 .
sffffz
unis.
fff
$%'6 espress.%7'( %
%($%'6
% %% %( % %4 . *
$ , , , , ,-div.
marcato+ * .ff
$%'6 %7(8 * $%
'6 %78 . . $%'( 6 %7(8 * *
$%'6 %78 .
$%'6 $%
'( 6%7 . . %71 . $%
'( 6 $%
'6%%')%7(
.%7
. %7
$ , , , , ,-
div.
marcato
+ *ff
$%'( 6%78 .$%
'6$%
'6%7(. .
%7(*
$%
'6 %74 . %
%'()%%')
%7
( . . %7* . %
%') %78 .$%
'( 6
$%'
6sffffz
unis.%7* .
%7
2fff
$%'6 espress.
$%'
6%7'( %7 . %(
%) * /
$ , , , , ,-marcato
+ff
* .$%
'( 6
%718 * . $%'6 %7(8 .
$%'
( 6 %78 *
$%'6
$%'6%7( . %7 .
$%'6
$%'
6%7
. %7 .
$%'
6 %%')%72
. . %7
( *
0 , , , , ,-div.
marcato+ *ff
$%6%74 . $%
'2 6 %
%')%7 . . %7 * $%
'6 %78 . . $%'6 %72 4 *
$%'
( 6$%
'6%7 . %7 . %
%') %78 . $%'
2 6%74 .
$%'
6 %74 .
sffffz
unis.
fff
espress.%7' %( %$%
'6%
%( % %% #
% % *
0 , , , , ,- marcato+ff
* .$%( 6
%78 * $%'
2 6%74 . . $%'6 %72 4 *
$%'
( 6 %78 . *$%
'6$%
'6%7 . %7 . . $%
'6%78 * . %%') %72 4
9 , , , , ,- +marcato
*ff
$%'6
%74 . .$%
'6 %78 * $%'2 6$%
'6%7 . . %7 * $%
'6%74 . $%'6
%%')%7 . . %7 * .
$%'
6 %78 *
sffffz
unis.
%%'()
espress.
%7'%( .
fff
. %%')
% %($%
'( 6
% % % +
9 , , , , ,-div.
+marcato
*ff
.$%
'6%78 * $%
'6%74 . . $%
'6 %78 *$%
'6
$%'6%7
. %7 . $%'2 6
%%')%7 . . %7 +
$%'
6 %78 .
9 , , , , ,- +marcato
*ff
$%'6
%78 . $%'6$%
'6%7. %7 . .
$%'6
%78 * .$%
'2 6 %78 *
$%'6
%78 . * .$%
'6%78 +
sffffz
%7'%7( * *
fff
$%'6 %(
) + * %)
2
Electro–Acoustic Music (2)
swarming essence by Dai Fujikura for orchestra and electronics, computer music realization by Manuel Poletti
harmonic selection of specially composed small instrument group and extended playing style
corpus-based orchestration, separate treatment
Re-orchestration, rearranging, navigation
30
!"
!"
18
""
""
q = 60
q = 60
!"
!"
q = 102
q = 102
!#
!#
""
""
!"
!"
A. Fl.(smallgroup)
B. Cl.(smallgroup)
Tpt.(smallgroup)
1 Vln.(smallgroup)
1 Vc.(smallgroup)
electronics
Fl.
Ob. I
Ob. II
Cl. I
Cl. II
Bsn. I
Bsn. II
Hn. I
Hn. II
Hn. III
Hn. IV
Tpt.
Tbn. I
Tbn. II
Timp.
B. D.
Vln. I
Vln. II
Vla.
Vc.
Db.
!
"
!
"
!
"
!
"
!
"
!
"
!"
$SMALL GROUP
%&'%( % % %) % # % #
p
* + + ,ca. 20 - 21 sec.
, ,- , , , ,
$ % %( %&'(% % % #
p
%) * , , , ,- , , , ,
$ % % #
p
% #) . + , , , ,- , , , ,
$ %&'( #% % #
% % %p
+ / , , ,- , , , ,
0 $% %1 % % %p
% #) . , , , ,- , , , ,
$
section 3
*
4
%%%%%%%%%(
211( %%%%%%%
((
more and more dense and violent(grain size gets smaller and smaller)Harmony 3Harmony 2
gliss.
gliss.
+3 semitones,
-4 semitones,
%%%%%%%%%1
2222 %%%%%%%22(1 ,
CUT OFF
5
34 5 . * + /
$ , , , , ,- +marcato
*ff
$%'6 $%
'6$%
'6%7( %7($%
'6
%7 . . %7 . $%'
( 6$%
'6%7( %7 .$%
'6 %78 . * $%'6%78 . * .
$%'
6$%
'( 6
%7 .%7
*sffffz
$%'6%7'( % * * espress.
fff
. $%'6 %8 %( %
$%'6
%% #
$ , , , , ,- +marcato
*ff
$%'( 6 $%'
( 6%7 %71 * $%
'6 %78 . + $%'( 6 $%'6
%7 .%7(
.$%
'6$%'6%7 %7 * %
%')$%
'( 6
$%'6
%7 %7
. %7( *sffffz
%7'(8 . *
espress.
*fff
%) $%'6% %( # % %
$ , , , , ,- + marcato*ff
. %%'()
$%'6
%74 $%'6%7(
. . %7 .
$%'
( 6 %78 *$%
'6 %7(8 . * * $%'6
$%'6
%7 %7 . $%
'( 6 %7(8 * .
$%'6
sffffz
%78fff
%7'(. . % %8 . * + *
espress.
%6
$ , , , , ,- +marcato
*ff
%%') %74 .
%%
'
2 )%%')
$%'6
%7%7 .
%7
(.
%%')
%%
'
)%%
'
)%7( %7 %7* $%'( 6 $%
'6
%%
'
)
%7 %7(%%')
%%
'
)%7. %7 %7
. %%')
%%
'
)$%
'6
%7
( .%7( %7 .
sffffz fff
$%
'
( 6
%7'
(
%(. + *
espress.
%%')
%%(
%%'
()%
%
$ORCHESTRA
, , , , ,- +marcato
* .ff
%%')
$%'( 6
%74%%
'
2 )%7 .
%7. .
%%
'
) %%')%7 %7 . %
%')%%
'
) %%')
$%'
( 6
%7 %72. %7
$%'
( 6
%%')
%7( . . %71%%
'
)%72. %7
.%%
'
)%%')
sffffz
%7 %7fff
espress.
%%'
()$
%
'
(6
%7'% %( %
% #%
% #%
%%
'
)%
% %
9 , , , , ,- +marcato
*ff
$%'2 6%78 .$%
'6
$%'6%7 . %7 .
$%'
6$%
'6$%'2 6 $%
'6%7 %72 . %7 $%'6 $%'6
%%')
%7 %7 . %7%%')
$%'6
%7 . . %7$%
'6%7 .%7
.$%
'6 %78 .
sffffz fff
$%'
( 6 espress.
%7'
2%
%( . $%'
( 6%( %( % %) *
$%'6 % # %
9 , , , , ,- +marcato
*ff
%%') %74 . .
$%'6
$%'6%7
%7 . %
%'2) %%') $%
'6%7 %7 . %7 * %
%') %%')
$%'6
%7 %7$%
'6%7 . %7 . . %%') %74 .
$%'
6 %78 *sffffz
espress.
fff
%7' % %( % %
) * $%'
( 6 %( # % %) *
$ , , , , ,- + + marcato
ff
. $%'6
%74 * 9 . $%'6
$%'
2 6%7. %7 . $%
'6 $%'6%7 . %7 * . $%'6%78 * $%
'6 %78 . *sffffz
ff
$%'
( 6%7' * %7( * $%'6 %7(8 . + .$%'6 %78 *
$ , , , , ,- + 9+ marcato*ff
$%'2 6 %78 . .$%
'6 %728 *$%
'6%78 . *$%
'6%78 . * $%'6 %78 . * $%
'6 %728 .sffffz ff
$%'6%7'
. . %( . $%'( 6 %78 * $%'6
$%'
( 6%7 . . %7 + $
$ , , , , ,- + marcato9 *ff
$%'6%78 . .
$%'6%78 * * . $%
'6$%
'6%728 $%'6%7 . . %7 + * $%'
( 6 %78 . * $sffffz
%7'8 . *ff
*$%
'6 %7( 4 . * *
$%'
6%74 . * 9
$ , , , , ,- + 9*ff
marcato$%'2 6%78 . * $%'6%78 . . $%
'6%74 * * . %
%') %74 * . $%'6
$%'6
%74 %728 . * .$%
'6
sffffz
%78ff
%%')%7'( %7( * * $%
'( 6 %78 . * $ . $%'6 %7( 4 *
%%'
() %7( 4 .
$ , , , , ,- + + marcato
ff
$%'( 6%78 . * *$%
'6 %78 . .
$%
'6 %%
')%72
. %7. $%'( 6%78 * . $%
'6 %7(8 * *sffffz ff
$%'6%7' %7 * * . $%'6 %78 + .$%'6 %78 *
9 , , , , ,- , , ,(senza sord.)
sffffz
%7'2 4 . * + /
9 , , , , ,- , , ,straight mute
sffffz
%7'(8 . *super short staccato
+p
%7'( 7' 7' 7' 7'
5
gliss.
9 , , , , ,- , , ,p
soft sticks
%'( ' ' ' ' ' ' ' ' ' ' '3 3 3 3
gliss.
: , , , , ,- , , ,sffffz
%7'4 . * + /
$ , , , , ,- marcato+ *ff
$%'6 %7(8 .
$%'6
$%'
6%7( . . %7 * $%
'6 %7(8 .$%
'6 $%'6%7(
. .%7
* $%'6
%74 . . $%'
( 6 %7(8 *$%
'6 %74 .
sffffz
unis.
fff
$%'6 espress.%7'( %
%($%'6
% %% %( % %4 . *
$ , , , , ,-div.
marcato+ * .ff
$%'6 %7(8 * $%
'6 %78 . . $%'( 6 %7(8 * *
$%'6 %78 .
$%'6 $%
'( 6%7 . . %71 . $%
'( 6 $%
'6%%')%7(
.%7
. %7
$ , , , , ,-
div.
marcato
+ *ff
$%'( 6%78 .$%
'6$%
'6%7(. .
%7(*
$%
'6 %74 . %
%'()%%')
%7
( . . %7* . %
%') %78 .$%
'( 6
$%'
6sffffz
unis.%7* .
%7
2fff
$%'6 espress.
$%'
6%7'( %7 . %(
%) * /
$ , , , , ,-marcato
+ff
* .$%
'( 6
%718 * . $%'6 %7(8 .
$%'
( 6 %78 *
$%'6
$%'6%7( . %7 .
$%'6
$%'
6%7
. %7 .
$%'
6 %%')%72
. . %7
( *
0 , , , , ,-div.
marcato+ *ff
$%6%74 . $%
'2 6 %
%')%7 . . %7 * $%
'6 %78 . . $%'6 %72 4 *
$%'
( 6$%
'6%7 . %7 . %
%') %78 . $%'
2 6%74 .
$%'
6 %74 .
sffffz
unis.
fff
espress.%7' %( %$%
'6%
%( % %% #
% % *
0 , , , , ,- marcato+ff
* .$%( 6
%78 * $%'
2 6%74 . . $%'6 %72 4 *
$%'
( 6 %78 . *$%
'6$%
'6%7 . %7 . . $%
'6%78 * . %%') %72 4
9 , , , , ,- +marcato
*ff
$%'6
%74 . .$%
'6 %78 * $%'2 6$%
'6%7 . . %7 * $%
'6%74 . $%'6
%%')%7 . . %7 * .
$%'
6 %78 *
sffffz
unis.
%%'()
espress.
%7'%( .
fff
. %%')
% %($%
'( 6
% % % +
9 , , , , ,-div.
+marcato
*ff
.$%
'6%78 * $%
'6%74 . . $%
'6 %78 *$%
'6
$%'6%7
. %7 . $%'2 6
%%')%7 . . %7 +
$%'
6 %78 .
9 , , , , ,- +marcato
*ff
$%'6
%78 . $%'6$%
'6%7. %7 . .
$%'6
%78 * .$%
'2 6 %78 *
$%'6
%78 . * .$%
'6%78 +
sffffz
%7'%7( * *
fff
$%'6 %(
) + * %)
2
31
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Live Control and Composition Beside Oneself, Aaron Einbond, 2008 for viola and electronics:
real time control of CataRT from audio analysis of the viola
What the blind see 2009 for small ensemble and electronics
real time control of CataRT from audio analysis of the viola
corpus-based “orchestration” or transcription
texture example:1. Rain on leaves
2. CataRT resynthesis with extended technique instrument sounds
32
Aaron Einbond – What the blind see
B
&
&
ã
ã
t
&
ã
Alto
Cl.B.en Sib
Hp.
Pno.
Vib.
F
◊
√ √ √
C10
C10
C10
a tempo
a tempo
a tempo
1936
T T .Tπ
Œ Ù .T
∏T
∑6
≈ . ym
bruit des clefs : doigtés ad lib.
"f "yn y
7
TmflM
ßym y ym ym y Ù TM
flßy ym y y yb
7
® x xx x xm
œSF neige to concatenation
Œ
7
‰ ® jœb œbp
œœ œ œm œ œ
7
‰ œ œ> œ œ .œ
π≈
œétouffé avec papier sempre
p
œm
Ó
Ó ?
Œ ‰ jœb˝
‰ ≈6
˝superball
Vib.
p
3‰6˝ ‰
° sempre
ongles
∑
∑
6
® Tn -
pT JT Tb T
6
ym"f "
ym ym yy T- JT
∑
Jœ
≈
6 6 6 6 6>paume
ß
6
œ
pœ œ œm œb
œ œ
6æ j6æ
jyO>߉ 5≈ .
ym>
pplectre
y
˝
> . .y
yO
>
3
‰ œn ˝
® .
.œb˝
6≈̋
6
œn ˝
≈ 6˝
.6˝
7
7
7‰ ® yæ gliss.(main)
arco
πIVyæ 3
yb y. y. y. y.∑
6
Tym
ym y y y
6
yn ym ym yn ymT- JT Tb T .TbflM
ßTb Mflß
∑
6
® œm œ
.œœ œb œ œ
œ .œ>7
œ .œ œ œ
7
‰ ® œ œb7
∑
jyO
7
.yO
yb
π̋y
6
‰ ®6
6
6˝
œb ˝
6. ‰6
®œ˝
6>ß
28
1. snow melting on a metal roof 2. CataRT resynthesis with instrument sounds3. manually edited transcription 4. live reading by ensemble
33
Bruno Ruviaro – Intellectual Improperty 0.6
Using Catork, a CataRT-based system for electronic music composition for laptop orchestrashttp://www.brunoruviaro.com/catork
Premiered by the Stanford Laptop Orchestra (SLOrk) April, 2010 (chamber version with six laptops), and June (full ensemble version)
34
Gestural Control
35
CataRT — http://imtr.ircam.fr — Diemo Schwarz
New Concepts for Musical Applications
Re-arranging units by other rules than temporal order
Composition = Navigation through the sound space of heterogeneous sound databases
exploit richness of detail of recorded sound
efficient control by perceptually and musically meaningful descriptors
Interaction with self-recorded sound: sound of a musician is available for interaction beyond simple repetition
Cross-selection and interpolation: apply sound characteristics from one corpus to another, morphing between corpora
Orchestration and re-orchestration: matching a mass of sounds to a harmonic or sonic target while retaining precise control over the result
36
Current and Future DirectionsInterface
multi-dimensional scaling by mass–spring–damper model
uniformisation of density
Sound texture synthesis
Modeling of transitions and articulations
40 60 80
0.05
0.1
0.15
0.2
0.25
light rain
40 60 80
0.05
0.1
0.15
0.2
0.25
medium rain
40 60 80
0.05
0.1
0.15
0.2
0.25
heavy rain
37
CataRT — http://imtr.ircam.fr — Diemo Schwarz
AcknowledgementsCataRT is based on FTM&Co by Norbert Schnell et al.Thanks go to:
Alexis Baskind, Julien Bloit, Greg Beller, Miguel Angel Ortiz PérezRoland Cahen, Yoan Olliver, Benjamin Wulf (ENSCI),Christian Jacquemin, Rami Ajaj (LIMSI)
Norbert Schnell, Riccardo Borghesi, Fréderic Bevilacqua, Rémy Muller, Jean-Philippe Lambert (IMTR Team)the French National Agency of Research ANR and the projects Sample Orchestrator and Topophonie
Links:CataRT wiki on imtr.ircam.fr,
mailinglist [email protected] and [email protected] on http://lists.ircam.fr
38
CataRT — http://imtr.ircam.fr — Diemo Schwarz
Exercise SuggestionsAdapting CataRT
Integration in your concert patchAdapting to your musical needs/ideas
Improving CataRTCorpus Data
soundfile editor (list/delete sounds in corpus)
soundset editor (define sound sets, add/remove units)
descriptor range viewer/editor (e.g.: exclude units below threshold)
Analysissegmentation editor
Selectionconstraints: don’t repeat unit
Synthesisforced pitch/loudness
multi-channel output
Controlby MIDI notes
by envelopes, sequencer
by live sound analysis
39