extracting melodic contour using wavelet-based multi-resolution analysis

Extracting Melodic Countour Using Wavelet-based Multi-resolution Analysis

Tetsuro Kitahara (Nihon Univ., Japan) and Masaki Matsubara (Univ. of Tsukuba, Japan)

Introduction

To establish a theory of non-experts’ melody cognition

Non-experts don’t listen to individual notes separately

Our hypothesis

They grasp a whole melody as a single stream

We explore a melody representation that is:

Non-notewise Hierarchical

Our final goal

GTTM vs our approachGTTM Our approach

Pitch trajectory

...

Melody reduction means:Reducing less important notes

Reducing the resolution of the melody representation

2.5

-2.5

0.25 -0.125

7.5 2.5 7.5

DWT

IDWT

Pitch trajectory

Distance between contour trees

0.0

-2.5

0.20 -0.175

0.0 0.0 0.00.0

-2.5

0.25 -0.125

0.0 0.0 0.0

Root mean square of each element’s difference

But normalized by the num. of elements for each depth

Application 1: Repetetion detection1) Caclulate distances between subtrees

2) Detect low-distance subtree pairs

Sq. dist.＝4.27

Sq. dist.＝326.63

Sq. dist.＝68.41

Sq. dist.＝0.78 Sq. dist.＝9.13

and

Method

Target melody

Result

Squared distances of repeated phrases are small

How similar phrases are regarded as repetition can becontrolled by the fineness of the contour.

Application 2: Cognitive(?) melodic similarity

Piano sonata K.331 (first 8 measures)

Method Compare ours with GTTM-based method

Target melodies 12 Vars. on “Ah, vous dirai-je, maman”

Dist. between Theme and each Var.

Obtained contours

Apply rules

Thresholding

Time-spantree

0.0

-2.5

0.25 -0.125

0.0 0.0 0.0

Thresholding

Melodic contour

Contour tree

T1 T2

(Continued from buttom left)

(Dis)similarities

Distances (dissimilarities)

Similarities (-2.0 to 2.0)Higher but weak

MatsubaraICMC 2014

Hirata CMMR 2013

Mismatch. Sound like two streams

In the future...

Real-time analysis

Stream segregation

Integration with schema-based one

Use of RNN-based melody prediction

...and a lot