the effectiveness study of music information retrieval arbee l.p. chen national tsing hua university...
Post on 21-Dec-2015
220 views
TRANSCRIPT
![Page 1: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/1.jpg)
The Effectiveness Study of Music Information Retrieval
Arbee L.P. ChenNational Tsing Hua University
2002 ACM International CIKM Conference
![Page 2: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/2.jpg)
Outline
• Motivation• The Ultima Project
– The 1D-List approach
– The APS approach
• The Effectiveness Study– Estimating all relevant references
• Conclusion
![Page 3: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/3.jpg)
Motivation
• Various approaches were proposed to provide efficient and effective content-based retrieval of music objects– Music representations
• pitch, rhythm, contour, chord
– Index structures• tree-based index, list-based index, n-gram index
– Query processing methods• exact match, partial match, approximate match
![Page 4: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/4.jpg)
Motivation (Cont.)
• A platform is needed for the evaluation of various music information retrieval methods– Efficiency
• response time
– Effectiveness• recall-precision curve
• The Ultima project builds such a platform– Same data set and query set– Also serves as a testbed whenever new approaches
are proposed
![Page 5: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/5.jpg)
The Ultima Project (Cont.)
Architecture• Mediator
• Query processing module
• Report module
• Summarization module
• Query generation module
• Data store
Data Store (MS Access)
SMF
Med
iato
r
Query Processing Module
Table
to the InternetSummarization Module
1D-List APS APM
Report Module
Query Generation Module
![Page 6: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/6.jpg)
The Ultima Project (Cont.)
• Two approaches have been compared – 1D-List
– APS
Approach Representation Index structure
1D-List melody string list-based
APS sequence of music segments suffix tree-based
![Page 7: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/7.jpg)
The 1D-List Approach
• The 1D-List approach– Music objects are coded as melody strings
• “so-mi-mi-fa-re-re-do-re-mi-fa-so-so-so”
– Melody strings are organized as linked lists– Both exact and approximate matching can be handl
ed
• Exact link, insertion link, dropout link, transposition link
![Page 8: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/8.jpg)
The 1D-List Approach (Cont.)
1:7 1:11:41:21:5
2:1 1:31:6
2:9 2:71:91:8
2:82:22:5
2:32:6
2:4
2:10
2:11 2:12
1:10 1:11
1:12
1:13
do re mi fa so la si
1:7 1:21:5
2:1 1:31:6
2:9 1:91:8
2:22:5
2:62:10
2:11 2:12
do re mi
start end 1:7 1:21:5
2:1 1:31:6
2:9 1:91:8
2:22:5
2:62:10
2:11 2:12
do re mi
start end
(a) (b) (c)
![Page 9: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/9.jpg)
The APS Approach
• The APS approach– Music objects are coded as sequences of
music segments• four segment types to model the music contour• pitch and duration are considered
– Index structures• one-dimensional and two-dimensional augmented suffix
tree
– Both exact and approximate matching can be handled
![Page 10: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/10.jpg)
The APS Approach (Cont.)
• Representation
note number
beat
60
62
65
64
67
(B, 3, -3)
(A, 1, +1)
(D, 3, -3)
(B, 1, -2)
(C, 1, +2)
(C, 1, +2)
(C, 1, +1)
type A
type B
type C
type D
![Page 11: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/11.jpg)
The APS Approach (Cont.)
1 4
AB C
B C
C $
$
2 5
3
(a)
root
A
C
A
(b)
root
A<1,1>
C<7,8> C<1,3>
A<7,8> A<3,4>
N1
N2
The suffix tree of the string S=“ABCAB”
(a) An example of suffix tree(b) A 1-D augmented suffix tree
1
A
$
A
B
$B2
A
B
$3
![Page 12: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/12.jpg)
The APS Approach (Cont.)
• Similarity measure– Given a query sequence Q = (i1, j1, k1) (i2, j2, k2) ... (in, jn, kn),
and a candidate sequence from the database C = (i1, x1, y1) (i2, x2, y2) ... (in, xn, yn).
iip
n
i
ppitch
n
iiid
dduration
beatdurationpitchduration
pitchpitchdurationduration
ykMaxPitchifMaxPitch
if
nsimdis
xjnMaxDuratioifnMaxDuratio
if
nsimdis
wwww
simdiswsimdiswCQSIMDIS
,MIN and ,1
_)3(
,MIN and , 1
_)2(
1,0 and 1)1(where
__,_
1
2
1
![Page 13: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/13.jpg)
The Effectiveness Study
• Traditional measures of effectiveness are precision and recall
• However, the number of relevant references are usually unknown– it is unrealistic for the user to make relevant
judgments to all music objects in the database
retrievedarethatreferencesnumber of
relevantarethatreferencesretrievednumber ofprecision
referencesrelevantnumber of
relevantarethatreferencesretrievednumber ofrecall
![Page 14: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/14.jpg)
The Effectiveness Study (Cont.)
• How to estimate the number of relevant references NR?– ASx is the set of relevant objects from the top x ranked results
– RSx is the set of the top x ranked results retrieved by an approach
– Assumption 1: , ,• the number of the retrieved results is a function of the number of retrieve
d relevant objects
• Assumption 2: , where B is a positive integer
• Based on the two assumptions, NR can be derives as follows:
xx ASfRS DBx 1for 1 and 1 AS
1 BBRS xASx
)1)(BRS(log
))1B(DB(log AS
xB
Bx
NR
![Page 15: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/15.jpg)
The Effectiveness Study (Cont.)
rank relevance recall precision
1 Y 0.1 1
2 Y 0.2 1
3 Y 0.3 1
4 0.3 0.75
5 Y 0.4 0.80
6 0.4 0.67
7 Y 0.5 0.71
8 0.5 0.63
9 Y 0.6 0.67
10 Y 0.7 0.70
11 0.7 0.64
12 0.7 0.58
13 Y 0.8 0.62
14 0.8 0.57
15 0.8 0.53
16 Y 0.9 0.56
17 0.9 0.53
18 0.9 0.50
19 Y 1 0.53
20 1 0.50
Rank relevance recall precision
1 Y 0.1 1
2 Y 0.2 1
3 Y 0.3 1
4 0.3 0.75
5 Y 0.4 0.80
6 0.4 0.67
7 Y 0.5 0.71
8 0.5 0.63
|RSx| = 8, |ASx| = 5, |DB| = 20
![Page 16: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/16.jpg)
The Effectiveness Study (Cont.)
MethodFactor
APS
1D-List1-D AST (duration
)
1-D AST (pitch
)2-D AST
Number of music objects for generating queries
10 10
Is the query sample a refrain or an incipit?
refrain/incipit refrain/incipit
Length of query sample, denoted L
6/10 (segment) 8/12 (note)
Number of query samples per music object
4 4
Threshold setting of approximation for a query
sample
th_d = 0, 0.5, 1.0th_p = 0, 0.5, 1.0
K=0, 4, 7 (for L=8)K=0, 6, 11 (for L=12)
Total number of posing queries 120 120
Experiment setup
![Page 17: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/17.jpg)
The Effectiveness Study (Cont.)
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Recall
Pre
cisi
on
1D-AST_D0
1D-AST_P0
2D-AST_PD0
1D-List_K0
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Recall
Pre
cisi
on1D-AST_D0.51D-AST_P0.5
2D-AST_PD0.51D-List_K4,6
• Experiment results
![Page 18: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/18.jpg)
The Effectiveness Study (Cont.)
• Experiment results
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Recall
Pre
cisi
on
1D-AST_D_R
1D-AST_P_R
2D-AST_R
1D-List_R
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Recall
Pre
cisi
on
1D-AST_D_In
1D-AST_P_In
2D-AST_In
1D-List_In
![Page 19: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/19.jpg)
The Effectiveness Study (Cont.)
• Experiment results– 1D-list achieves a high precision in the limited
range of recall, while a moderate precision for the APS family can be obtained
– Comparing the APS family, the precision in a descending order is: 1D-AST (pitch), 2D-AST, and 1D-AST (duration)
– In average, the effectiveness of “incipit” queries is better than “refrain” queries
![Page 20: The Effectiveness Study of Music Information Retrieval Arbee L.P. Chen National Tsing Hua University 2002 ACM International CIKM Conference](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d635503460f94a45937/html5/thumbnails/20.jpg)
Conclusion
• The Ultima project builds a platform for evaluating the performance of various approaches of music information retrieval
• A new measure for estimating the number of relevant references is proposed
• Future work– Design and implement the summarization module
as well as the query generation module– Extend the project for evaluating polyphonic musi
c retrieval methods