Download - Music Analyser
Music AnalyserMusic Analyser
MSc in Information Engineering
Jean-Marie Froux
MSc Information Engineering 2003
Music Analyser - JM Froux City University - 2003
2
Abstract
Transcription of music : from the sound to the paper through computer.
The technologies used to develop :
C#, C++, DirectX
Music Analyser - JM Froux City University - 2003
3
• Introduction
• 1: Theory
• 2: Wave file analyser (in C#)
• 3: Music analyser (in C++)
• Conclusion
• Questions
Contents
Music Analyser - JM Froux City University - 2003
4
Introduction
• What is music transcription?
• Music and computers
• Why is it interesting ?
• The programs developed
Music Analyser - JM Froux City University - 2003
5
1/ Musical Theory
• 1.1 The physic of sound
• 1.2 The Fourier Analysis
Music Analyser - JM Froux City University - 2003
6
1.1 The physic of sound
• There are 4 parameters to describe a sound: – Duration, (s).– Intensity, (dB).– Height, (Hz).
Music Analyser - JM Froux City University - 2003
7
Frequency and notes (1)
Ear sensibility : 20 Hz to 20 kHz
represents 10 octaves
A flute can play between 130.81 Hz and 1046.5 Hz
Change octave by doubling frequency
1.1 The physic of sound
Music Analyser - JM Froux City University - 2003
8
Frequency and notes (2)
• 12 intervals in an octave • Tone is an interval of T = 62 • Semitone is an interval of T = 122
(=1.0594)
• We obtain thus the scale for Major C:
Note C D E F G A B C
Frequency f1 21/6f1 21/3f1 25/12f1 27/12f1 23/4f1 211/12f1 2f1
1.1 The physic of sound
Music Analyser - JM Froux City University - 2003
9
Frequency and notes (3)
All change possible with sharps and flats.
The basic (reference) note is “A3” 440Hz.
Other are deduced by multiplication with
T= 122
1.1 The physic of sound
Music Analyser - JM Froux City University - 2003
10
The physic of sound
• There are 4 parameters to describe a sound: – Duration, (s).– Intensity, (dB).– Height, (Hz).– Tone
Music Analyser - JM Froux City University - 2003
11
The tone (1)
• A sound is made of many frequencies :
–The fundamental
–Harmonics (multiples of the fundamental)
1.1 The physic of sound
Music Analyser - JM Froux City University - 2003
12
The tone (2)
Each instrument produces a specific spectrum :
2 4 6 8 Frequency
Magnitude
3 5 7 Frequency
Magnitude
A sound played by a clarinet
The same sound played by a oboe
1.1 The physic of sound
Music Analyser - JM Froux City University - 2003
13
1.2 Fourier transform
• Any pseudoperiodic signal can be broken up into an infinite sum of sinusoidal signals of frequency multiple of a basic frequency.
1
0 00)( 2sin2cosk
tfkbtfkaa kktx
Music Analyser - JM Froux City University - 2003
14
Interest
• The FFT allows us to see the frequencies and magnitudes of sinusoidal signals.
1.2 Fourier transform
frequency
Magnitude
time
Frequency
Music Analyser - JM Froux City University - 2003
15
2/ Wave file Analyser
• 2.1 Design
• 2.2 Implementation
• 2.3 Testing
Music Analyser - JM Froux City University - 2003
16
2.1 Design
Record data
2.1 Wave Analyser Design
Analyse data
Write score
Music Analyser - JM Froux City University - 2003
17
Record data2.1 Wave Analyser Design
Select capture device
Create capture buffer
Wave format tag (compression)
Average bytes per second
Samples per second
Bits per sample
Block Align
Channel(s)Start
Music Analyser - JM Froux City University - 2003
18
Analysis2.1 Wave Analyser Design
Open the file
Get the file size
Create an array of the same size
Copy the data in the array
Decode the header chunk
Analyse the data chunk
Music Analyser - JM Froux City University - 2003
19
Analysis (2)
• The program the should analyse the sample with an FFT and find the notes.
• We will see the algorithm with the second program.
• Once the notes found, they are stored in a file.
2.1 Wave Analyser Design
Music Analyser - JM Froux City University - 2003
20
Write notes
2.1 Wave Analyser Design
Open file
Read file
title
signature
notesDraw the score
Music Analyser - JM Froux City University - 2003
21
The .notes fileRules :
– First line: Title: Title– Second line : Signature1: Int– Third line : Signature: Int– Then the notes described by letters A to G.
• 3 octaves lower (add: l), medium (m=default), and higher (h)
• Sharp (s) and flat (f)• Timing: 16th (add x), 8th (add e), quarter is
default, half (a), three-quarter (t), whole (w).
2.1 Wave Analyser Design
Music Analyser - JM Froux City University - 2003
22
Testing • The software:
2.2 Wave Analyser Testing
Music Analyser - JM Froux City University - 2003
23
Testing • Capture devices and record:
2.2 Wave Analyser Testing
Music Analyser - JM Froux City University - 2003
24
Testing • Analysis
2.2 Wave Analyser Testing
Music Analyser - JM Froux City University - 2003
25
Testing • The score:
2.2 Wave Analyser Testing
Music Analyser - JM Froux City University - 2003
26
Conclusion
• C# is a very good programming language, but it needs the latest version of DirectX9.
• As it is very new, the help is poor.
• That’s why I have decided to change my way, and I passed to C++.
2.Wave Analyser
Music Analyser - JM Froux City University - 2003
27
3/ The sound analyser in real time
• 3.1 Sampling.
• 3.2 FFT.
• 3.3 Search for the note.
• 3.4 Display.
• 3.5 Save.
• 3.6 The recognition of rhythm.
Music Analyser - JM Froux City University - 2003
28
3.1 Sampling
• tell the soundcard to start sampling the input from the microphone
• store the samples in a client buffer • The client creates a separate thread
which is suspended waiting on an event• Once the buffer is full, notifies the client • The client thread does the necessary
work and waits for the next event.
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
29
3.2 FFT
• The FFT buffer is filled with samples in bit-reversed order
• The bit reversal is done, converts real samples into complex numbers (with the imaginary part set to zero)
• Then the FFT calculation is organized into bunches.
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
30
3.3 Find the note
• For each FFT
Recover the frequency and magnitude of the highest peak
if magnitude is above the noise, seek if there is a peak at frequency divided by 2.
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
31
3.3 Find the note (2)
• If yes:The peak of higher magnitude isn’t the
fundamental (but the first harmonic)
3/ The sound analyser in real time
Example with a guitar: spectrum of an A2:
In this case, the highest peak is the first harmonic, we obtain the fundamental by dividing by 2 this frequency
Music Analyser - JM Froux City University - 2003
32
3.3 Find the note (3)
• If not:
seek a peak at frequency multiplied by 2 (the first harmonic).
If there is one, the fundamental is the peak of higher magnitude.
Then find the note with the table of frequencies
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
33
3.3 Display the note (1)
• Problem: not display several times the same note played once.
• If for 2 consecutive FFT– The note found is the same– The magnitude of the second is not higher
It is the same note.• If the magnitude changes, it means that the
note is played again.
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
34
3.3 Display the note (2)• The notes are represented by an “o” and a
# if needed. To help the position of the note, I have adapted it to the octave:
3/ The sound analyser in real time
Frequency octave
f<131 Hz 1
131f<261 2
261f<523 3
523f<1046
4
f1046 5
•The notes go from the "G" of octave 1 until "A" of octave 3
•For the too high or too low notes the octave is modified in order to display the note on the scale.
Music Analyser - JM Froux City University - 2003
35
3.3 Save the notes (1)
• Important to allow musician to recover their work in another software.
• First step to MIDI file, write the data in a .txt file
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
36
3.3 Save the notes (2)• Example:
MThd | Format=0 | # of Tracks=1 | Division=96Track #0
**************************************** Time Event1: 1: 0 |On Note | chan= 1 | pitch=D 1 | vol=127 2: 0 |Off Note | chan= 1 | pitch=d 1 | vol=100 | On Note | chan= 1 | pitch=A 2 | vol=127 3: 0 |Off Note | chan= 1 | pitch=a 2 | vol=100 |End of track|
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
37
3.4 Rhythm recognition• Metronome
– Help the musician when recording– Can give the tempo of the piece
• Trace.txt: Save the tempo, the note, the value of the timer and the amplitude.
• E.g.:
3/ The sound analyser in real time
60.C. 0. 608.C#. 47. 1407.D. 94. 1212.D. 141. 1675.D. 188. 1880.
Next step is the analysis of this file.
Music Analyser - JM Froux City University - 2003
38
Testing• The software:
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
39
Testing• The software:
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
40
Conclusion
• With C++ I found complete help on the Internet
• It gives a program which works well.
• Some improvements can be done to continue this work.
3/ The sound analyser in real time
Music Analyser - JM Froux City University - 2003
41
Further Work
• Rhythm recognition
• Detection of tonality
• Polyphonic recognition?
Music Analyser - JM Froux City University - 2003
42
QUESTIONS??? ???
Music Analyser - JM Froux City University - 2003
43
Thank you for your attention
Music Analyser
Jean-Marie FROUX
MSc Project
City University
Supervisor: D J STYLES