Digital audio & computer music

Digital audio and computer musicCOS 116, Spring 2012Guest lecture: Rebecca Fiebrink1OverviewPhysics & perception of sound & musicRepresentations of musicAnalyzing music with computersCreating music with computers21. Sound and music3

What is sound?

Discussion TimePressure wave4DEMO: SndpeekWhat do we hear?PitchLoudnessTimbreLocationMeter, rhythm, harmony, melody, structureetc...http://www.youtube.com/watch?v=EvxS_bJ0yOUhttp://www.youtube.com/watch?v=wY1EMwDeaBwhttp://www.youtube.com/watch?v=nIt9QF_5C_w

5PsychoacousticsPsychoacoustics: relationships between physical phenomenon and our perceptionFrequency: pitch (20-20,000Hz)Amplitude: loudnessTimbre: Identities and strengths of frequencies present

+=

6Show sndpeek again with sounds

Mention that they will explore these issues in the lab.

Discussion TimeWhat is music?Organized sound Psychoacoustics play an important roleAlso dependence upon history, culture, experienceEngages listeners psychological mechanisms for expectation/reward

72. Representations of sound and music8Score:

Digital waveform

Spectrogram

How do you represent music?

9Digital representation of music

10CompressionA better representation with fewer bitsWhy? Security, transmission, storageHow?Psychoacoustic principlesMP3: MaskingPhysical principles of sound production (uses models of sound source)

11Choosing a representationRepresentations make compromisesStandard representations are somewhat arbitraryAppropriate choice is task-dependent123. Using technology to analyze sound and music13Analyzing speechReal-life apps:Customer service phone routingVoice recognition software

14Auditory Scene AnalysisApplications: Archival and retrieval, forensics, AI

15Music information retrievalAnalyzing musical dataQuery, recommend, visualize, transcribe, detect plagiarism, follow along scoreSites/apps you can trymidomiThemefinder.comPandora.com (includes human-powered algorithms)Shazaam16

Machine learning for analysis

174. Using technology to create music and sound18Creating music: Synthesis

19Four approaches to synthesis1. Additive synthesisFigure out proportions of various frequenciesSynthesize waves and superimpose them

Modify amplitude using an envelope:

++=

202. FM Synthesis

Modulate the frequency of one sine oscillator using the output of another oscillator

DEMO: FM in ChucK

213. Physical ModelsStart with knowledge of physical systemsSimulate oscillation (Recall Lecture 4)

4. Cross-synthesisChoose filter for speech (vowel)Choose source to be another sound

23How can computers be used in making music?Synthesizing new soundsProcessing and transforming soundDemo: T-PainAccompanying human performersDemo: RaphaelComposing new musicDemo: CopinAs new musical instrumentsAnd many other ways, too

24Computer as InstrumentDemo: SMELT keyboard, motionVideo: ClixDemo: WekinatorVideo: CMMV, BlinkyDemo: Live coding

25Questions: How can we.develop new ways to synthesize sound?give a user control over synthesis parameters?make machines interactive in a musical way?augment human capabilities?design new instruments that are easy to play? allow expert musicality?create music that is emotionally and aesthetically compelling?26Final remarksDistinctions in this presentation are superficialAnalysis, representation, and creation interactTechnology draws on and contributes to our understanding of the physics and psychophysics of soundComputer music is interdisciplinaryHCI, AI, programming languages, algorithms, systems buildingAlso psychology, music theory, acoustics, signal processing, engineering, physics, performance practice, library science, applied math & statistics, Technology is constantly complicating and changing the landscape of our musical experiences as creators, participants, listeners, and consumers.27http://soundlab.cs.princeton.edu/

28