digital audio and computer music · final remarks distinctions in this presentation are superficial...
TRANSCRIPT
![Page 1: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/1.jpg)
Digital audio and computer music
COS 116, Spring 2012 Guest lecture: Rebecca Fiebrink
![Page 2: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/2.jpg)
Overview
1. Physics & perception of sound & music 2. Representations of music 3. Analyzing music with computers 4. Creating music with computers
![Page 3: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/3.jpg)
1. Sound and music
![Page 4: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/4.jpg)
What is sound?
Discussion Time
“Pressure wave”
![Page 5: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/5.jpg)
What do we hear?
Pitch Loudness Timbre Location Meter, rhythm, harmony, melody, structure etc...
http://www.youtube.com/watch?v=EvxS_bJ0yOU http://www.youtube.com/watch?v=wY1EMwDeaBw http://www.youtube.com/watch?v=nIt9QF_5C_w
![Page 6: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/6.jpg)
Psychoacoustics Psychoacoustics: relationships between
physical phenomenon and our perception Frequency: pitch (20-20,000Hz)
Amplitude: loudness
Timbre: Identities and strengths of frequencies present
+ =
![Page 7: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/7.jpg)
Discussion Time
What is music?
“Organized sound”
• Psychoacoustics play an important role • Also dependence upon history, culture,
experience • Engages listeners’ psychological
mechanisms for expectation/reward
![Page 8: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/8.jpg)
2. Representations of sound and music
![Page 9: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/9.jpg)
Score:
Digital waveform
Spectrogram
How do you represent music?
![Page 10: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/10.jpg)
Digital representation of music
![Page 11: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/11.jpg)
Compression A “better” representation with fewer bits Why? Security, transmission, storage How?
Psychoacoustic principles MP3: Masking
Physical principles of sound production (uses models of sound source)
![Page 12: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/12.jpg)
Choosing a representation
Representations make compromises Standard representations are somewhat
arbitrary Appropriate choice is task-dependent
![Page 13: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/13.jpg)
3. Using technology to analyze sound and music
![Page 14: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/14.jpg)
Analyzing speech
Real-life apps: Customer service phone routing Voice recognition software
![Page 15: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/15.jpg)
Auditory Scene Analysis
Applications: Archival and retrieval, forensics, AI
![Page 16: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/16.jpg)
Music information retrieval
Analyzing musical data Query, recommend, visualize, transcribe,
detect plagiarism, follow along score Sites/apps you can try
midomi Themefinder.com Pandora.com (includes “human-powered”
algorithms) Shazaam
![Page 17: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/17.jpg)
Machine learning for analysis
![Page 18: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/18.jpg)
4. Using technology to create music and sound
![Page 19: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/19.jpg)
Creating music: Synthesis
![Page 20: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/20.jpg)
Four approaches to synthesis
1. Additive synthesis 1. Figure out proportions of various frequencies 2. Synthesize waves and superimpose them
3. Modify amplitude using an “envelope”:
+ + …=
![Page 21: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/21.jpg)
2. FM Synthesis
Modulate the frequency of one sine oscillator using the output of another oscillator
![Page 22: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/22.jpg)
3. Physical Models
1. Start with knowledge of physical systems 2. Simulate oscillation (Recall Lecture 4)
![Page 23: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/23.jpg)
4. Cross-synthesis Choose filter for speech (vowel) Choose source to be another sound
![Page 24: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/24.jpg)
How can computers be used in making music? Synthesizing new sounds Processing and transforming sound
Demo: T-Pain
Accompanying human performers Demo: Raphael
Composing new music Demo: Copin
As new musical instruments And many other ways, too…
![Page 25: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/25.jpg)
Computer as Instrument
Demo: SMELT keyboard, motion Video: Clix Demo: Wekinator Video: CMMV, Blinky Demo: Live coding
![Page 26: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/26.jpg)
Questions: How can we….
develop new ways to synthesize sound? give a user control over synthesis parameters? make machines interactive in a musical way? augment human capabilities? design new instruments that are easy to play?
allow expert musicality? create music that is emotionally and aesthetically
compelling?
![Page 27: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/27.jpg)
Final remarks Distinctions in this presentation are superficial
Analysis, representation, and creation interact Technology draws on and contributes to our understanding of
the physics and psychophysics of sound Computer music is interdisciplinary
HCI, AI, programming languages, algorithms, systems building Also psychology, music theory, acoustics, signal processing,
engineering, physics, performance practice, library science, applied math & statistics, …
Technology is constantly complicating and changing the landscape of our musical experiences as creators, participants, listeners, and consumers.
![Page 28: Digital audio and computer music · Final remarks Distinctions in this presentation are superficial Analysis, representation, and creation interact Technology draws on and contributes](https://reader031.vdocument.in/reader031/viewer/2022041016/5ec7e1f54d6ec5103c1fa7c4/html5/thumbnails/28.jpg)
http://soundlab.cs.princeton.edu/