Percussion Synthesis using Loopback Frequency Modulation Oscillators

Jennifer S. Hsu, Tamara SmythUniversity of California San Diego, Department of Music

jsh008@ucsd.edu, trsmyth@ucsd.edu

ABSTRACT

In this work, we apply recent research results in loop-back frequency modulation (FM) to real-time parametricsynthesis of percussion sounds. Loopback FM is a vari-ant of FM synthesis whereby the carrier oscillator “loopsback” to serve as a modulator of its own frequency. LikeFM, more spectral components emerge, but further, whenthe loopback coefficient is made time varying, frequencytrajectories that resemble the nonlinearities heard in acous-tic percussion instruments appear. Here, loopback FM isused to parametrically synthesize this effect in struck per-cussion instruments, known to exhibit frequency sweeps(among other nonlinear characteristics) due to modal cou-pling. While many percussion synthesis models incorpo-rate such nonlinear effects while aiming for acoustic accu-racy, computational efficiency is often sacrificed, prohibit-ing real-time use. This work seeks to develop a real-timepercussion synthesis model that creates a variety of novelsounds and captures the sonic qualities of nonlinear per-cussion instruments. A linear, modal synthesis percussionmodel is modified to use loopback FM oscillators, whichallows the model to create rich and abstract percussive hitsin real-time. Musically intuitive parameters for the percus-sion model are emphasized resulting in a usable percussionsound synthesizer.

1. INTRODUCTION

Synthesis of plates, membranes, and other percussioninstruments have been realized using several differentmodeling techniques including modal synthesis (MS) [1],the Functional Transformation Method (FTM) [2], finitedifference schemes (FDS) [3], and the digital waveguidemesh (DWM) [4]. When real percussion instruments arestruck with a large velocity excitation, nonlinear effectsoften result. Examples of these nonlinearities include thecascade of energy from low to high frequency componentsthat give cymbals their characteristic sound and pitchglides heard with gongs [5].

Many nonlinear percussion synthesis models are compu-tationally expensive and may exhibit stability issues thatrender them unsuitable for real-time synthesis on standardcomputers [1–3]. An exception to this is described in [6],where computationally heavy calculations are approx-imated so that a nonlinear membrane model is able to

Copyright: c© 2018 Jennifer S. Hsu et al. This is an open-access article distributedunder the terms of the Creative Commons Attribution 3.0 Unported License, which

permits unrestricted use, distribution, and reproduction in any medium, provided

the original author and source are credited.

*

+Modal Synthesis with Loopback FM

modal synthesis output

Commuted Synthesis

++

loopback FM oscillators amplitude envelopes

e(n)

y(n)output signal

excitation

*

acoustic resonator IR

m(n)

r(n)

w1(n)

w2(n)

w3(n)

p(n)

zc,1(n)

zc,3(n)

zc,2(n)

Figure 1. The loopback FM percussion synthesis method.

simulate up to 1000 modes in real-time at a sampling rateof 44.1kHz. For the percussion models in [7] and [2], themost computationally expensive component is the nonlin-ear calculation. Though the linear version of each modelcan be efficiently computed, they produce sounds that areless interesting than when nonlinearities are added. Withthis understanding, one may like to use a linear systemto synthesize percussive sounds and approximate thenonlinearities in an efficient and perceptually similar way.

For example, in [2], Avanzini and Marogna presenta sound synthesis simulation of a nonlinear, tension-modulated percussion membrane. The model consists of alinear portion and a nonlinear feedback section simulatingtension modulation. The linear model is computationallyefficient, but the nonlinear tension modulation requires afeedback calculation for every sample, a computationalcomplexity that makes it unable to run in real time. In afollowing paper [8], the nonlinear feedback calculationis replaced with an efficient approximation that can becalculated with the computational expense similar to thatof the linear model.

mailto:jsh0081@ucsd.edumailto:trsmyth@ucsd.eduhttp://creativecommons.org/licenses/by/3.0/

Instead of striving for an acoustically accurate simulationas some previous research has done, the aim here is to cre-ate a percussion synthesizer that creates a variety of novelsounds inspired by the dynamic and nonlinear phenomenaheard in percussion instruments. Similar to the strategyused in [9], in which the pitch glide capabilities of a Duff-ing oscillator are explored in the sound synthesis of a gong,the work presented herein employs the nonlinear effects ofloopback FM, a technique initially presented in [10] andfurther developed in [11]. Here, loopback FM oscillatorsare used to enhance a modal synthesis (MS) of percussionsound (see Figure 1). Loopback FM is a variant of FMsynthesis where the carrier signal is looped back to modu-late its own frequency, resulting in complex spectra (muchlike traditional FM), and interesting frequency trajectoriesthat resemble the nonlinearities observed in real percussioninstruments when the loopback coefficient is made timevarying.

In Section 2, we explore traditional MS and how it canbe used to synthesize percussive sounds. Section 3 re-views loopback FM equations relevant to the current con-text. Section 4 explains how traditional MS can be mod-ified with loopback FM oscillators to create a wide vari-ety of percussion sounds. Synthesis parameters are dis-cussed in Section 5. Section 6 presents synthesis examplesof a marimba, tom tom, and circular plate. Concludingthoughts and future research directions are considered inSection 7.

2. PERCUSSION SYNTHESIS USINGTRADITIONAL MODAL SYNTHESIS

MS is a technique that resynthesizes the sound of anacoustic object according to its acoustic modes or vibra-tional patterns. The resonant frequencies of an acousticobject arise through the sinusoidal motion of the object’smodes. With traditional MS, each mode is synthesizedusing a second-order resonating filter with a correspondingfrequency, initial amplitude, and decay [12].

To synthesize a percussive sound with MS, we begin witha list of Nf modal frequencies fi, the values of which canbe obtained from acoustic experiments, spectral analysisof recorded or physically modeled sounds (e.g. DWMs,FDSs, etc), or calculated using theoretical equations.

Though MS traditionally models each frequency modewith a second-order bandpass resonant filter with centerfrequency fi, in this work the filters are replaced by si-nusoidal oscillators of frequency (or center frequency iffrequency is time varying) fi. This allows for a straight-forward comparison with the loopback FM version (alsoimplemented here with oscillators) than if traditional MSbandpass filters had been used. A sinusoidal componentwith carrier frequency ωc,i = 2πfi is expressed as si(n) =sin(ωc,inT ) for time sample n and period T = 1/fs forsampling rate fs. Each sinusoidal component is multi-plied with an amplitude envelope wi(n). For percussivesounds, wi(n) are typically exponentially decreasing en-velopes with possibly different initial amplitudes and de-cay rates for different modes. For example, for naturalsounding results, higher-frequency modes should be madeto decay more rapidly. Enveloped sinusoidal components

Traditional modal synthesis spectrogram, fc=2000Hz

200 400 600 800

Time (ms)

0

2

4

6

Fre

quency (

kH

z)

-150

-100

-50

Pow

er/

frequency (

dB

/Hz)

Figure 2. Traditional MS for three modal frequencies.

are added together to form the MS output given by

ms(n) =

Nf−1∑i=0

wi(n)si(n), (1)

the spectrum of which is shown in Figure 2 for Nf = 3.MS is efficient and useful for recreating the sound of ob-

jects that consist of a small number of resonant frequencymodes. However, MS is a linear method and (1) is inca-pable of capturing nonlinear effects. A simple modifica-tion to the sinusoidal components of the MS frameworkallows the system to create complex and dynamic soundsreminiscent of nonlinear vibrations in percussion instru-ments. In this modification, each sinusoidal component islooped back to modulate its own carrier frequency, a syn-thesis technique coined by the authors as “Loopback FM.”

3. LOOPBACK FM

Loopback FM is a self-modulated form of FM wherethe oscillator loops back and modulates its own carrierfrequency according to a feedback coefficient. This differsfrom Feedback FM [13], in which the output is used tomodulate its own initial phase. Loopback FM with a staticfeedback coefficient, B, and feedback FM both createpeaks in the spectrum at integer multiples of a soundingfrequency. As described in [10], the difference betweenthe two synthesis methods is that with loopback FM, thefeedback coefficient B can be varied over time to createboth predictable pitch and spectral changes. Conversely,feedback FM preserves pitch (in some contexts a desirablefeature) and only introduces spectral changes. As shownin [11], loopback FM and its closed-form IIR approxima-tion, an expression that resembles the transfer function ofa “stretched” allpass filter [14] but for which only the realpart is used as a time-domain signal, can be used to createcomplex frequency spectra and pitch contours. Here, wepresent the equations for loopback FM and its closed formrepresentation with static pitch and timbre followed bytheir time-varying formulations, which can be used tomodulate timbre and sounding frequency.

3.1 Loopback FM Formulation

The loopback FM equation involves a carrier frequencyωc = 2πfc where ωc is the angular frequency and fc isthe frequency in Hz, and a feedback parameter B, whichcontrols the output’s timbre and fundamental frequency.

The loopback FM equation for static B and time samplen is

zc(n) = ejωcT (1+B

with the initial condition zc(0) = 1 causing oscillation.The output that we listen to is the real part of zc(n). Thefundamental frequency of this oscillator is not ωc but ratherω0 = 2πf0, where f0 is the sounding frequency in Hz. Therelationship between ω0 and ωc is described by

ω0 = ωc√

1−B2, −1 ≤ B ≤ 1, (3)

which shows that for ω0 to remain real, the value ofB mustbe within the interval (−1, 1).

3.2 An Alternate Representation of the Loopback FMOscillator

The loopback FM oscillator zc(n) given in (2) with staticpitch and timbre may also be represented by the closed-form representation

z0(n) =b0 + e

jω0nT

1 + b0ejω0nT, (4)

which is similar to the transfer function of a “stretched” all-pass filter used in [14]. In this synthesis context, (4) is usedas a time-domain signal that is a function of time samplen, where b0 influences spectrum, ω0 specifies the sound-ing frequency, and the sound is the real signal given by

where ∗ indicates convolution. Because there is no depen-dence between m(n), e(n), and r(n), m(n) can be com-muted with r(n). The excitation and resonator impulseresponse can be convolved to form an aggregate excitationa(n) = e(n) ∗ r(n).

Aggregate excitations can be stored for several excita-tion and resonator combinations. During run-time, a low-latency convolution method, such as the one described in[16], can be used to convolve a(n) with m(n) to form thefinal percussion model output

y(n) = a(n) ∗m(n). (13)

In our syntheses, we use a variety of resonator impulse re-sponses as presented in Section 6 along with two differenttypes of parametric excitations.

4.3 Excitations

The “Excitation” block in Figure 1 involves p(n), afunction that describes the vertical position of a drum-stick/mallet hitting a surface at time n. The excitationsignal e(n) = p(n) − p(n − 1), relates to the velocity ofthe drumstick/mallet and is convolved with the acousticresonator impulse response to form a(n). Here, we useraised cosine envelopes and filtered noise bursts for p(n).These signals are parametric and affect the resultingoutput timbre.

4.3.1 Raised Cosine Envelopes

The raised cosine envelope has a single parameter: thewindow length L. The equation for the excitation is

p(n) =

0.5(

1− cos(

2πn

L− 1

)), for 0 ≤ n < L

0, for n ≥ L(14)

4.3.2 Filtered Noise Bursts

The parameters for a filtered noise burst are noise burst du-ration td and low and high frequency cutoffs for a bandpassfilter flow and fhigh. Examples in this paper use white noisefiltered by a second-order Butterworth bandpass filter.

5. MUSICAL PARAMETERS FOR LOOPBACK FMPERCUSSION SYNTHESIS

Musical parameters for the loopback FM percussion syn-thesis method are presented here along with their corre-sponding variables and equations.

5.1 Timbre: Oscillators created with zc,i(n) or z0,i(n)

The MS oscillators can be synthesized using zc,i(n) orz0,i(n). To use z0,i(n) oscillators, replace zc,i(n) in (11)with z0,i(n). While both forms produce almost identicalresults from fc = 0 Hz to around fc = 2500 Hz at asampling rate of 44.1 kHz, when fc > 2500 Hz, the ver-sion that uses zc,i(n) becomes much noisier, due to alias-ing. If the sampling rate is increased, the output is thesame whether the oscillators are created using zc,i(n) orz0,i(n). Like FM, both zc,i(n) and z0,i(n) produce signals

Spectrogram of MS using zc, i

(n), fc=2000Hz

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

quency (

kH

z)

-150

-100

-50

Pow

er/

fre

qu

ency (

dB

/Hz)

Spectrogram of MS using z0,i

(n), fc=2000Hz

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

quency (

kH

z)

-150

-100

-50

Pow

er/

freq

ue

ncy (

dB

/Hz)

Figure 3. MS using zc,i(n) and z0,i(n) with low carrierfrequencies create almost identical results.

Spectrogram of MS using zc,i

(n) oscillators, fc=4000Hz

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

quency (

kH

z)

-120

-100

-80

-60

-40

Pow

er/

fre

quen

cy (

dB

/Hz)

Spectrogram of MS using z0,i

(n) oscillators, fc=4000Hz

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

quency (

kH

z)

-150

-100

-50

Pow

er/

frequency (

dB

/Hz)

Figure 4. MS with zc,i(n) creates noisier output than MSwith z0,i(n) for high carrier frequencies.

that are not bandlimited. With loopback FM, a large fcmeans a large feedback amount, which can mean increasedbandwidth and aliasing, similar to how a large index ofmodulation corresponds to a wider bandwidth in traditionalFM. Figure 3 shows that the zc,i(n) and z0,i(n) MS os-cillators produce similar spectrograms when the lowest of3 modal frequencies is set to a low carrier frequency offc = 2000Hz. Vastly different spectrograms are producedwhen the lowest of the 3 modal frequencies is set to ahigher carrier frequency of fc = 4000 Hz as shown in Fig-ure 4. The MS using zc,i(n) synthesizes a noisier outputand can be used to create cymbal- and crash-like sounds asshown in Section 6.3.

5.2 Timbre: B and b0

Loopback FM parameter B controls timbre in (2) whilez0(n) parameter b0 affects timbre in (4). For carrier fre-quencies below 2500Hz, the frequency components cre-ated using (2) or (4) are almost identical and are spaced atinteger multiples of f0. When B = 0 or b0 = 0, there areno sidebands and the output is a pure tone. As B and b0increase towards 1 (or decrease towards −1), more side-bands appear and the timbre brightens. The sidebands log-arithmically decrease in amplitude for each multiple of f0.

0 1000 2000 3000 4000 5000

Frequency (Hz)

-60

-40

-20

0A

mp

litu

de

(d

B)

Loopback FM magnitude spectrum for f0 = 300Hz

Figure 5. The loopback FM magnitude spectrum. Fre-quency components occur at integer multiples of thesounding frequency, 300Hz, and the amplitude of the com-ponents decreases logarithmically.

Figure 5 is a plot of the magnitude spectrum for a loopbackFM oscillator with static B = 0.9 and f0 = 300Hz. Thesounding frequency can be seen as a peak at 300Hz and thesidebands are spaced at integer multiples of 300Hz with alogarithmic decrease.

(5) explains the relationship between b0 and B, thoughas described in Section 5.1, at high carrier frequencies, theoutput from (2) will differ from that of (4).

5.3 Time-varying Timbre: B(n) and b0(n)

With (6), B(n) affects the time-varying timbre and sound-ing frequency. When using (9), b0(n) controls the time-varying timbre, independent of pitch. As in the static case,as B(n) and b0(n) near 0, the output approaches a puretone, while as B(n) and b0(n) approach 1 and −1, thenumber of sidebands created by the oscillators increasesand the timbre becomes brighter.

In Figure 6, B(n) = gn where g = 0.9999, b0(n) is ob-tained according to (8), and amplitude envelopes are thesame for all modal frequencies. The top and middle plotsin Figure 6 compare spectrograms for a static timbre ofb0 = −0.6312 with (4) and a time-varying timbre whereb0(n) is used with (9). The sidebands in the top plot arethe same over the course of the signal, but the higher fre-quency sidebands die out over time in the middle plot asb0(n) increases from −1 to 0. Time-varying timbre be-tween (6) and (9) can be compared using the middle andbottom plots. In the bottom plot, time-varying B(n) cre-ates timbre and pitch variation as n increases. In the mid-dle plot, b0(n) changes the timbre without affecting thefrequency trajectories.

5.4 Sounding Frequency: ω0

For Eqs. 2 and 4, the sounding frequency can be controlledwith ω0 = 2πf0. For a desired ω0 with (2), one would use(3) and either 1) set B to a desired value and solve for ωcor 2) set ωc and solve for B.

Because the modal frequencies for percussive instru-ments are often inharmonic, the sounding frequency forpercussion synthesis is not clearly defined. With MS usingz0,i(n) oscillators, ω0,i = 2πfi is used to set the soundingfrequencies of individual oscillators. For MS using zc,i(n)oscillators, the carrier frequencies can be set to the modalfrequencies: ωc,i = 2πfi or the sounding frequencies canbe set to the modal frequencies: ω0,i = 2πfi. According

MS using z0,i

(n) oscillators, static timbre, b0=-0.6312

200 400 600 800

Time (ms)

0

10

20

Fre

qu

en

cy (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

MS using z0,i

(n) oscillators, time-varying timbre, b0(n)

200 400 600 800

Time (ms)

0

10

20

Fre

qu

en

cy (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

MS using zc,i

(n) oscillators, time-varying timbre, B(n)

200 400 600 800

Time (ms)

0

10

20

Fre

qu

ency (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

Figure 6. Static and time-varying timbre with various MSoscillators. Middle: b0(n) with (9) modulates timbre in-dependently of pitch. Bottom: B(n) with (6) affects bothtimbre and pitch.

to (3), when B = 0, ω0 = ωc, and setting either to themodal frequencies would create the same output. When Bis large and close to 1 or −1, ω0 will be a lower frequencythan ωc. This means that using ωc,i = 2πfi produceslower sounding frequencies while setting ω0,i = 2πfiproduces higher sounding frequencies, which will mostlikely produce aliasing effects, especially with (2), asdescribed in Section 5.1. Figure 7 demonstrates that whenB is close to 1, ωc,i = 2πfi creates a toned output whileω0,i = 2πfi creates a noisy output as higher frequenciescontribute to extreme aliasing effects.

5.5 Pitch Glides: B(n) and ω0(n)

With (6), a pitch glide can be added by varying B(n) overtime. This also produces timbral changes. A pitch glidecan be created with (9) by varying ω0(n) over time as de-scribed in Section 3.4. To modify the pitch independentlyof timbre with (9), b0 should be held constant.

As described in Section 5.1, differences between (6) and(9) can be observed when ωc is high, and this effect oc-curs with pitch glides. Figure 8 shows the high carrier fre-quency difference for a pitch glide over three modal fre-quencies using MS with (6) and (9). The pitch glide iscreated with B(n) = 0.9999n, so timbre also changes. Athigher carrier frequencies, MS with Eq. 6 creates noise-like output for the first 100ms and more spectral compo-nents than MS with Eq. 9 from 100−250ms. From 300msthrough the remainder of the signal, the frequency compo-nents are more similar.

Pitch glides are constrained to use exponential and linearB(n) functions in the synthesis examples presented in this

Loopback FM modal synthesis with fc = f

i

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20F

requency (

kH

z)

-150

-100

-50

Pow

er/

fre

qu

ency (

dB

/Hz)

Loopback FM modal synthesis with f0 = f

i

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

quency (

kH

z)

-120

-100

-80

-60

-40

Pow

er/

freq

ue

ncy (

dB

/Hz)

Figure 7. Loopback FM MS spectrograms for ωc,i = 2πfi(top) and ω0,i = 2πfi (bottom). The top and bottom sig-nals are generated using the same 3 modal frequencies withB = 0.9.

Spectrogram of MS using zc,i

(n) oscillators with pitch glide, fc=4000Hz

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

qu

en

cy (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

Spectrogram of MS using z0,i

(n) oscillators with pitch glide, fc=4000Hz

100 200 300 400 500 600 700 800 900

Time (ms)

0

10

20

Fre

qu

en

cy (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

Figure 8. MS using (6) vs. (9) at high carrier frequencieswith a pitch glide produce different spectrograms for thefirst 250ms. The lowest oscillator frequency uses fc =4000Hz.

research. This restriction onB(n) creates natural soundingpitch glides and allows us to draw parallels between thezc(n) and z0(n) forms.

5.5.1 Pitch Glides with Exponential B(n)

For the exponential case, B(n) = gn can be used directlyin (6) to produce a pitch glide. When using (9) for a pitchglide, Θ0(n) can be found using B(n) = gn along withEquations 7 and 10 which, as shown in [11], is given by

Θ0(n) =ωc

log(g)(√

1− g2n − tanh−1(√

1− g2n)) + C

(15)where C is the constant of integration.

5.5.2 Pitch Glides with Linear B(n)

For the linear case, B(n) = kn + l produces a pitch glidewhen used with (6). (9) uses the instantaneous phase givenby

Θ0(n) =ωc2k

((kn+ l)√

1− (kn+ l)2

+ sin−1(kn+ l)) + C (16)

5.6 Decay Time: wi(n)

The decay time for the percussion signal can be con-trolled through the amplitude envelopes wi(n). Anatural sounding way to set these envelopes is to modelthem as exponentially decreasing envelopes over time:wi(n) = A0e

−n/τ , with different initial amplitude valuesA0, as shown in Figure 10, and/or different decay rates τ .

5.7 Commuted Synthesis Parameters

5.7.1 Attack Sharpness: Raised Cosine Envelopes

With raised cosines envelopes, small values of L createsharper sounding attacks, while longer values ofL increasethe presence of low frequencies in the output and resultin bass-heavy sounds. Intuitively, L is proportional to themass of a hammer or mallet used to excite a drum head: alonger L means a hammer/mallet with greater mass.

5.7.2 Attack Noisiness: Filtered Noise Bursts

For filtered noise burst excitations, a longer noise burst tdand higher bandpass frequency cutoff fhigh will create anoisier attack. flow and fhigh should be tuned to filter outundesired frequencies. For example, for a high pitched per-cussion sound, the lower frequencies could be filtered outfrom the noise burst by setting flow to a higher frequency.

5.7.3 Timbre: Acoustic Resonator Impulse Response r(n)

The acoustic resonator filters the synthesis output, so thetimbre can be further shaped by the frequencies present inr(n). For an expansive and large sound, a room impulse re-sponse with a long T60 may work well while for a shorter,tuned sound, the impulse response of a small, acoustic tubemodel could be used.

6. SYNTHESIS EXAMPLES

While the loopback FM percussion synthesis method iscapable of creating a variety of percussive sounds, thissection covers three sound synthesis examples that usemodal frequencies from [17]: the marimba, tom tom,and circular plate. Although these modal frequenciesare associated with real, physical instruments, the aim ofthis synthesis is not to recreate the naturally occurringsounds. Rather, we seek to synthesize many differenttypes of sounds with nonlinearities similar to those thatoccur in percussion instruments. For these examples,differences between percussion synthesis using tradi-tional and loopback FM MS are compared for the samemodal frequencies, decaying amplitude envelopes, andcommuted synthesis parameters. Sound examples can befound athttp://musicweb.ucsd.edu/˜trsmyth/other/percussionSynthesisLoopbackFM.html.

6.1 Marimba

Figure 9 compares the spectrograms of a marimba modeledas a bar with two free ends using traditional and loopbackFM MS. This example sets ω0,i to seven modal frequencies

http://musicweb.ucsd.edu/~trsmyth/other/percussionSynthesisLoopbackFM.htmlhttp://musicweb.ucsd.edu/~trsmyth/other/percussionSynthesisLoopbackFM.html

MS of marimba with raised cosine excitation

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Time (secs)

0

5

10

15F

requency (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

uency (

dB

/Hz)

Loopback FM MS of marimba with raised cosine excitation

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Time (secs)

0

5

10

15

Fre

quency (

kH

z)

-150

-100

-50

Pow

er/

fre

qu

ency (

dB

/Hz)

Figure 9. Traditional (top) vs. Loopback FM MS (bottom)using the modal frequencies of an ideal bar with two openends. The excitation is a raised cosine and the acousticresonator is an ideal tube synthesized using traditional MS.

0 0.5 1 1.5 2

Time (sec)

0

0.5

1

Am

plit

ud

e (

line

ar) Decaying exponential amplitude envelopes

Figure 10. The marimba synthesis amplitude envelopesare decaying exponentials. The initial amplitude of the en-velopes is inversely proportional to the modal frequency ofthe oscillator that is paired with the envelope.

calculated as

fi =

{440, for i = 0

440 (2i+3)2

3.0112 , otherwise(17)

The amplitude envelopes are decaying exponentials whereinitial amplitudes decrease exponentially from 1 for thefirst (lowest) modal frequency to 0.01 for the seventh(highest) modal frequency. Figure 10 is a plot of theamplitude envelopes used for this marimba example.This example is created using zc,i(n) oscillators withan 8-sample length raised cosine excitation and a pitchglide created by setting B(n) = 0.9999n. The acousticresonator is an ideal, open-closed tube synthesized usingtraditional MS. Compared to the signal generated usingtraditional MS, the signal created using loopback FM MShas more frequency components and a clearly increasingpitch glide.

6.2 Tom Tom

The spectrogram of a tom tom synthesized using tradi-tional vs. loopback FM MS is shown in Figure 11. Themodal frequencies used to synthesize the tom tom are

fi = 142 · [1, 2.15, 3.17, 3.42, 4.09, 4.80, 4.94] (18)

The amplitude envelopes are the same as those used forthe marimba as shown in Figure 10. The synthesis uses

MS of tom tom with filtered noise burst

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Time (secs)

0

2

4

Fre

qu

en

cy (

kH

z)

-150

-100

-50

0

Pow

er/

frequency (

dB

/Hz)

Loopback FM MS of tom tom with filtered noise burst

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Time (secs)

0

2

4

Fre

qu

en

cy (

kH

z)

-150

-100

-50

0

Pow

er/

frequency (

dB

/Hz)

Figure 11. Traditional (top) vs. Loopback FM MS (bot-tom) using the modal frequencies of a tom tom. The exci-tation is a filtered noise burst and the acoustic resonator isa taiko drum recording.

z0,i(n) oscillators, b0 = −0.9, and the pitch glide is cre-ated linearly increasing B from 0.55 to 0.91. The excita-tion is a 0.05 second long noise burst filtered with a 2nd-order Butterworth bandpass filter with frequency cutoffs at120Hz and 4000Hz. The acoustic resonator is a recordingof a taiko drum retrieved from freesound.org. In Fig-ure 11, there is more high frequency energy for the loop-back FM MS than for the traditional MS, especially in thebeginning of the signal.

6.3 Circular Plate

In Figure 12, loopback FM MS of a simply-supported cir-cular plate is compared to traditional MS of the same cir-cular plate. The modal frequencies used are

fi = f0·[1, 2.80, 5.15, 5.98, 9.75, 14.09, (19)14.91, 20.66, 26.99]

where f0 = 0.2287cL(h/a2) for plate thicknessh = 0.005m, plate radius a = 0.09m, and longitudinalwave speed cL =

√E/ρ(1− ν2) with Young’s modulus

E = 2 · 1011N/m2, plate density ρ = 7860kg/m3, andPoisson ratio ν = 0.3. The amplitude envelopes aredecaying exponentials over time. The initial amplitudeof these envelopes decreases exponentially as frequencyincreases from 1 for the lowest modal frequency to 0.5 forthe highest modal frequency. Using zc,i(n) oscillators, aslight upwards pitch glide is created by linearly changingB(n) from 0.91 to 0.90 over the course of the signal. Theexcitation is an 8-sample long raised cosine envelope andthe acoustic resonator is a room impulse response retrievedfrom echothief.com. In this example, the drasticaliasing effects in loopback FM MS are used to create anextremely “noisy” signal. Perceptually, the traditional MSoutput sounds like a clean bell sound, while the loopbackFM MS sounds more like a noisy, struck cymbal.

7. CONCLUSIONS

This work has presented a real-time method to synthesizenovel, abstract percussion sounds using MS with loopback

freesound.orgechothief.com

MS of circular plate with raised cosine excitation

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Time (secs)

0

5

10

15

Fre

quency (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

Loopback FM MS of circular plate with raised cosine excitation

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Time (secs)

0

5

10

15

Fre

quency (

kH

z)

-150

-100

-50

Po

we

r/fr

eq

ue

ncy (

dB

/Hz)

Figure 12. Traditional (top) vs. Loopback FM MS (bot-tom) using the modal frequencies of a simply-supportedcircular plate. The excitation is a raised cosine, and theresonator is a room impulse response.

FM oscillators. Loopback FM creates complex spectra andpitch glides similar to the nonlinear effects observed inexisting percussion instruments. The synthesis techniqueallows for parametric control of musical dimensions in-cluding sounding frequency, decay time, timbre, and pitchglide. Synthesis examples using the modal frequencies ofa marimba, tom tom, and circular plate are examined.

A future research direction involves investigating thealiasing that occurs with large carrier frequencies for bothloopback FM and its closed form expression. Anotherresearch interest is to explore other methods of creatingnonlinearities in oscillators and using these methods withloopback FM to create an even wider range of percussionsounds.

Acknowledgments

This research was inspired by a conversation with Profes-sor Miller Puckette when he shared with us his implemen-tation of Loopback FM in Pure Data (Pd). We are im-mensely grateful to Miller for the time and insight that heoffered to make this work possible.

8. REFERENCES

[1] M. Ducceschi and C. Touzé, “Modal approach for non-linear vibrations of damped impacted plates: Applica-tion to sound synthesis of gongs and cymbals,” Journalof Sound and Vibration, vol. 344, pp. 313–331, Jan.2015.

[2] F. Avanzini and R. Marogna, “A modular physicallybased approach to the sound synthesis of membranepercussion instruments,” IEEE Transactions on Audio,Speech, and Language Processing, vol. 18, no. 4, pp.891–902, May 2010.

[3] S. Bilbao, “A family of conservative finite differ-ence schemes for the dynamical von Karman plateequations,” Numerical Methods for Partial DifferentialEquations, vol. 24, no. 1, pp. 193–216, Jan. 2008.

[4] S. A. V. Duyne and J. O. Smith, “Physical model-ing with the 2-d digital waveguide mesh,” in Proceed-ings of the International Computer Music Conference,Tokyo, Japan, 1993, pp. 40–47.

[5] N. H. Fletcher, “The nonlinear physics of musical in-struments,” Reports on Progress in Physics, vol. 62, pp.723–764, 1999.

[6] S. Petrausch and R. Rabenstein, “Tension modulatednonlinear 2d models for digital sound synthesis withthe functional transformation method,” in Proceedingsof EUSIPCO-05, Thirteenth European Signal Process-ing Conference, Antalya, Turkey, Sept. 2005.

[7] S. Bilbao, “Sound synthesis for nonlinear plates,” inProceedings of the 8th International Conference onDigital Audio Effects (DAFx’05), Madrid, Spain, Sept.2005.

[8] R. Marogna, F. Avanzini, and B. Bank, “Energy basedsynthesis of tension modulation in membranes,” inProceedings of the 13th International Conference onDigital Audio Effects (DAFx-10), Graz, Austria, Sept.2010.

[9] M. Jossic, D. Roze, and T. Hélie, “Energy shapingof a softening duffing oscillator using the formalismof port-hamiltonian systems,” in Proceedings of the20th International Conference on Digital Audio Effects(DAFx’17), Edinburgh, UK, Sept. 2017.

[10] T. Smyth and J. Hsu, “On phase and pitch in loop-back frequency modulation with a time-varying feed-back coefficient,” in 26th International Congress onSound and Vibration, Montreal, Canada, July 2019.

[11] ——, “Representations of self-coupled oscillators withtime-varying frequency,” in Proceedings of the Soundand Music Computing Conference, Málaga, Spain,May 2019.

[12] J. M. Adrien, “The missing link: Modal synthesis,” inRepresentations of Musical Signals. The MIT Press,1991, pp. 269–295.

[13] N. Tomisawa, “Tone production method for an elec-tronic musical instrument,” U.S. Patent 4 249 447A,Feb 1998.

[14] V. Välimäki, J. S. Abel, and J. O. Smith, “Spectral de-lay filters,” Journal of the Audio Engineering Society,vol. 57, no. 7/8, pp. 521–531, July/Aug. 2009.

[15] J. O. Smith, “Efficient synthesis of stringed musical in-struments,” in Proceedings of the International Com-puter Music Conference, Tokyo, Japan, Sept. 1993.

[16] W. G. Gardner, “Efficient convolution without input-output delay,” Journal of the Audio Engineering Soci-ety, vol. 43, no. 3, pp. 127–136, March 1995.

[17] T. Rossing, Science of Percussion Instruments, ser. Se-ries in Popular Science. Singapore: World Scientific,2000, vol. 3.

1. Introduction 2. Percussion Synthesis using Traditional Modal Synthesis 3. Loopback FM3.1 Loopback FM Formulation3.2 An Alternate Representation of the Loopback FM Oscillator3.3 Time-varying B: Pitch and Timbre Modulation with zc(n)3.4 Time-varying b0 and 0: Pitch and Timbre Modulation with z0(n)

4. Percussion Synthesis with Loopback FM Oscillators4.1 Modal Synthesis with Loopback FM4.2 Commuted Synthesis4.3 Excitations4.3.1 Raised Cosine Envelopes4.3.2 Filtered Noise Bursts

5. Musical Parameters for Loopback FM Percussion Synthesis5.1 Timbre: Oscillators created with zc,i(n) or z0,i(n)5.2 Timbre: B and b05.3 Time-varying Timbre: B(n) and b0(n)5.4 Sounding Frequency: 05.5 Pitch Glides: B(n) and 0(n)5.5.1 Pitch Glides with Exponential B(n)5.5.2 Pitch Glides with Linear B(n)

5.6 Decay Time: wi(n)5.7 Commuted Synthesis Parameters5.7.1 Attack Sharpness: Raised Cosine Envelopes5.7.2 Attack Noisiness: Filtered Noise Bursts5.7.3 Timbre: Acoustic Resonator Impulse Response r(n)

6. Synthesis Examples6.1 Marimba6.2 Tom Tom6.3 Circular Plate

7. Conclusions 8. References