it don’t mean a thing - jazzomat · 2018-07-08 · it don’t mean a thing (if it ain’t got...

26
It Don’t Mean a Thing (If It Ain’t Got That Swing) Estimating Swing Ratios and Soloist Micro-timing from Jazz Recordings with Aligned Beat Grids Christian Dittmar [email protected]

Upload: others

Post on 18-Mar-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

It Don’t Mean a Thing

(If It Ain’t Got That Swing)

Estimating Swing Ratios and Soloist Micro-timing

from Jazz Recordings with Aligned Beat Grids

Christian Dittmar

[email protected]

© AudioLabs, 2014

Author

Title of Presentation

2

Overview

Introduction

Swing ratio

Ensemble timing

Automatic method

Swing ratio estimation

Log-lag ACF analysis

Preliminary results

Dataset / Comparison to prior work

Example cases

Log-lag ACF

Futher steps

© AudioLabs, 2014

Author

Title of Presentation

3

Introduction: Swing ratio

„Jazz music is supposed to swing“ (Friberg & Sundström

2002)

Lengthening odd (on-beat) 8th notes

Shortening even (off-beat) 8th notes

Ratio of long 8th to short 8th swing ratio

1 straight binary 2 tied-triplet

1 2 3 4 5 6 7 8

𝑠𝑟

© AudioLabs, 2014

Author

Title of Presentation

4

Introduction: Swing ratio

Jazz drummers often use typical cymbal pattern inducing

the swing (4th, long 8th, short 8th, 4th, long 8th, etc. …)

Relation between swing ratio and relative note durations:

𝑙𝑜𝑛𝑔 8𝑡ℎ = 𝑠𝑟

1 + 𝑠𝑟

𝑠ℎ𝑜𝑟𝑡 8𝑡ℎ = 1

1 + 𝑠𝑟

𝑠𝑟 = 𝑙𝑜𝑛𝑔 8𝑡ℎ

𝑠ℎ𝑜𝑟𝑡 8𝑡ℎ

𝑙𝑜𝑛𝑔 8𝑡ℎ + 𝑠ℎ𝑜𝑟𝑡 8𝑡ℎ = 1

© AudioLabs, 2014

Author

Title of Presentation

5

Introduction: Swing ratio

Illustrative example: Swing ratio 1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time in Seconds

Wavefo

rm /

Am

plit

ude E

nvelo

pe

Cymbal pattern at 120 BPM and swing ratio 1

𝑠𝑟 =0.25

0.25= 1

© AudioLabs, 2014

Author

Title of Presentation

6

Introduction: Swing ratio

Illustrative example: Swing ratio 2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time in Seconds

Wavefo

rm /

Am

plit

ude E

nvelo

pe

Cymbal pattern at 120 BPM and swing ratio 2

𝑠𝑟 =0.333

0.167= 2

© AudioLabs, 2014

Author

Title of Presentation

7

Introduction: Swing ratio

Illustrative example: Swing ratio 3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time in Seconds

Wavefo

rm /

Am

plit

ude E

nvelo

pe

Cymbal pattern at 120 BPM and swing ratio 3

𝑠𝑟 =0.375

0.125= 3

© AudioLabs, 2014

Author

Title of Presentation

8

Introduction: Swing ratio

Illustrative example: Swing ratio 4

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time in Seconds

Wavefo

rm /

Am

plit

ude E

nvelo

pe

Cymbal pattern at 120 BPM and swing ratio 4

𝑠𝑟 =0.4

0.1= 4

© AudioLabs, 2014

Author

Title of Presentation

9

Introduction: Ensemble timing

Friberg‘s observations:

Less pronounced swing of the soloist (wind instruments)

Large onset delay w.r.t. on-beat

Small onset delay w.r.t. off-beat (coincide with cymbal)

© AudioLabs, 2014

Author

Title of Presentation

10

Automatic method: Swing ratio estimation

Compute spectrogram from solo excerpts via STFT

Detection of percussive onsets candidates

Heuristics to retain only onsets in upper freq. band

Detection of onset ‚triplets‘ within one beat (plus tolerance)

Derivation of swing ratio

Matching of soloist onsets to the previously found ‚triplets‘

Derivation of swing ratio from matched ‚triplets‘

Measurement of onset delay w.r.t. on-beat and off-beat

© AudioLabs, 2014

Author

Title of Presentation

11

Automatic method: Log-lag ACF analysis

Ommit error-prone steps:

Cymbal onset detection

‚triplet‘ selection from onsets

Autocorrelation on onset detection function

Resample ACF to logarithmically-spaced lag-axis

Tempo differences turn into shifts (Gruhne & Dittmar 2009)

Common rhythmic patterns are retained

© AudioLabs, 2014

Author

Title of Presentation

12

Automatic method: Log-lag ACF analysis

Illustrative example: Swing ratio 1

294 243 200 165 136 112 93 76 63 520

0.1

0.2

0.3

0.4

0.5

Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 1

Log-lags in BPM

Peridic

ity s

alie

nce

120

BPM

Eighth

(240 BPM)

© AudioLabs, 2014

Author

Title of Presentation

13

Automatic method: Log-lag ACF analysis

Illustrative example: Swing ratio 2

294 243 200 165 136 112 93 76 63 520

0.1

0.2

0.3

0.4

0.5

Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 2

Log-lags in BPM

Peridic

ity s

alie

nce

120

BPM

Eighth

(240 BPM)

© AudioLabs, 2014

Author

Title of Presentation

14

Automatic method: Log-lag ACF analysis

Illustrative example: Swing ratio 3

294 243 200 165 136 112 93 76 63 520

0.1

0.2

0.3

0.4

0.5

Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 3

Log-lags in BPM

Peridic

ity s

alie

nce

120

BPM

Eighth

(240 BPM)

© AudioLabs, 2014

Author

Title of Presentation

15

Automatic method: Log-lag ACF analysis

Illustrative example: Swing ratio 4

294 243 200 165 136 112 93 76 63 520

0.1

0.2

0.3

0.4

0.5

Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 4

Log-lags in BPM

Peridic

ity s

alie

nce

120

BPM

Eighth

(240 BPM)

© AudioLabs, 2014

Author

Title of Presentation

16

Preliminary results: Dataset

Subset of the Jazzomat data

Rhythm feel: Swing

Manually aligned beat grids

available

Manually transcribed solo wind

instruments play 8th lines

Relatively steady drum beat

Only ‘reliable’ results shown here

Drummers: Soloists:

© AudioLabs, 2014

Author

Title of Presentation

17

50 100 150 200 250 300 3500.5

1

1.5

2

2.5

3

3.5

4

Tempo in BPM

Sw

ing r

atio

Drummers' swing ratio vs. tempo

Billy Higgins

Art Taylor

Tony Williams

Max Roach

Carl Allen

Preliminary results: Drummers‘ swing ratio

Friberg & Sundström 2002: Jazzomat @ AudioLabs:

© AudioLabs, 2014

Author

Title of Presentation

18

100 150 200 250 300 3500.5

1

1.5

2

2.5

Tempo in BPM

Sw

ing r

atio

Soloists' swing ratio vs. tempo

John Coltrane

Art Pepper

Wayne Shorter

Dizzy Gillespie

Sonny Rollins

Preliminary results: Soloists‘ swing ratio

Friberg & Sundström 2002: Jazzomat @ AudioLabs:

© AudioLabs, 2014

Author

Title of Presentation

19

100 150 200 250 300 350-40

-20

0

20

40

60

80

100

120

Tempo in BPM

On-b

eat

dela

y in m

s

Soloists' on-beat deleay vs. tempo

John Coltrane

Art Pepper

Wayne Shorter

Dizzy Gillespie

Sonny Rollins

Preliminary results: Soloists‘ on-beat delay

Friberg & Sundström 2002: Jazzomat @ AudioLabs:

© AudioLabs, 2014

Author

Title of Presentation

20

100 150 200 250 300 350-40

-20

0

20

40

60

80

100

120

Tempo in BPM

Off

-beat

dela

y in m

s

Soloists' off-beat deleay vs. tempo

John Coltrane

Art Pepper

Wayne Shorter

Dizzy Gillespie

Sonny Rollins

Preliminary results: Soloists‘ off-beat delay

Friberg & Sundström 2002: Jazzomat @ AudioLabs:

© AudioLabs, 2014

Author

Title of Presentation

21

50 100 150 200 250 300 3500.5

1

1.5

2

2.5

3

3.5

4

Tempo in BPM

Sw

ing r

atio

Drummers' swing ratio vs. tempo

Billy Higgins

Art Taylor

Tony Williams

Max Roach

Carl Allen

Preliminary results: Example cases

Art Pepper – Blues for Blanche:

28.5 29 29.5 30 30.5 31

Cymbal: Billy Higgins

Soloist: Art Pepper

Cymbal beat grid

Tapped beat grid

Time in Seconds

MelID: 2 - Art Pepper - Blues for Blanche

© AudioLabs, 2014

Author

Title of Presentation

22

50 100 150 200 250 300 3500.5

1

1.5

2

2.5

3

3.5

4

Tempo in BPM

Sw

ing r

atio

Drummers' swing ratio vs. tempo

Billy Higgins

Art Taylor

Tony Williams

Max Roach

Carl Allen

Preliminary results: Example cases

Steve Lacy – Let‘s Cool One:

85.5 86 86.5 87 87.5 88 88.5

Cymbal: Billy Higgins

Soloist: Steve Lacy

Cymbal beat grid

Tapped beat grid

Time in Seconds

MelID: 157 - Steve Lacy - Let's Cool One

© AudioLabs, 2014

Author

Title of Presentation

23

50 100 150 200 250 300 3500.5

1

1.5

2

2.5

3

3.5

4

Tempo in BPM

Sw

ing r

atio

Drummers' swing ratio vs. tempo

Billy Higgins

Art Taylor

Tony Williams

Max Roach

Carl Allen

Preliminary results: Example cases

Clifford Brown – George‘s Dilemma:

57 58 59 60 61 62 63 64

Cymbal: Max Roach

Soloist:

Cymbal beat grid

Tapped beat grid

Time in Seconds

MelID: 36 - Clifford Brown - George's Dilemma

© AudioLabs, 2014

Author

Title of Presentation

24

Preliminary results: log-lag ACF

Log-lag ACF shape correlates to swing ratio

Very noisy Normalized log-lag ACFs vs. swing ratio

Log-lags in BPM

Sw

ing

ratio

294 243 200 165 136 112 93 76 63 52

1.1811

1.3957

1.6192

1.8092

2.1355

2.553

© AudioLabs, 2014

Author

Title of Presentation

25

Further steps

More elaborate onset detection for cymbals

NMF-based drum transcription (Dittmar & Gärtner 2014)

Refine onsets using center of gravity in onset frame

Automatic detection of outliers

Systematically evaluate usage of log-lag ACF

Ingest automatically refined beat grid back to Jazzomat DB

Correct editorial metadata inconsistencies

© AudioLabs, 2014

Author

Title of Presentation

26

References

Anders Friberg & Andreas Sundström, „Swing Ratios and Ensemble Timing in

Jazz Performance: Evidence for a Common Rhythmic Pattern“, Music

Perception, 2002

Matthias Gruhne & Christian Dittmar, “Improving Rhythmic Pattern Features

Based on Logarithmic Preprocessing”, Proceedings AES International

Conference, 2009

Christian Dittmar & Daniel Gärtner, “Real-time transcription and separation of

drum recordings based on NMF decomposition”, Proceedings of the 17th

International Conference on Digital Audio Effects, 2014