ai lab workshop(211206)

António Pedro António Pedro OliveiraOliveira

DEI - FCTUCDEI - FCTUC21/12/200621/12/2006

Creative Generation ofCreative Generation ofAffective MusicAffective Music

05/02/23 António Pedro Oliveira 2

OutlineOutline

IntroductionIntroduction State of the ArtState of the Art ArchitectureArchitecture Work to be doneWork to be done

Experiments ExamplesExamples ConclusionConclusion

05/02/23 António Pedro Oliveira 3

IntroductionIntroduction

ProblemProblem No automatic method to detect emotions

related to music Music induce different emotions on

different listeners Context, user profile

HypothesisHypothesis Music is the main language of emotions

05/02/23 António Pedro Oliveira 4

IntroductionIntroduction

MotivationsMotivations Music is a ubiquitous media

Cinema, TV, radio, dance, transport, shopping centres, etc.

Emotional content can be influenced by specific musical features

ObjectivesObjectives Discover the isomorphic mapping between

musical features structure and emotions structure Generate music to induce emotions

Retrieve music features Sequence, compose music

05/02/23 António Pedro Oliveira 5

IntroductionIntroduction

MethodologyMethodology Music - Select, manipulate, sequence music

samples, adapting to intended emotion Audience - Compare intended and recognized

emotions using Affective Computing technology

05/02/23 António Pedro Oliveira 6

State of the ArtState of the Art

Music PsychologyMusic Psychology Emotions and Music Music perception and cognition Music theory

Computer MusicComputer Music Affective music generation Algorithmic composition

05/02/23 António Pedro Oliveira 7

State of the ArtState of the Art

Musical Signal ProcessingMusical Signal Processing Music Features Extraction

Melody Harmony Rhythm Instruments Dynamics

05/02/23 António Pedro Oliveira 8

State of the ArtState of the Art

Affective ComputingAffective Computing Psychophisiological techniques for emotion

recognition 2 Dimensional

Space

05/02/23 António Pedro Oliveira 9

System – stage 1System – stage 1

ArchitectureArchitecture

05/02/23 António Pedro Oliveira 10

System – stage 2System – stage 2

ArchitectureArchitecture

05/02/23 António Pedro Oliveira 11

System – stage 3System – stage 3

ArchitectureArchitecture

05/02/23 António Pedro Oliveira 12

System – stage 4System – stage 4

ArchitectureArchitecture

05/02/23 António Pedro Oliveira 13

Work to be done Work to be done

Research TopicsResearch Topics Music Psychology

Computational models to test isomorphic mappings between musical features structure and emotions structure

Computer Music Algorithms for Affective Music Composition Automatic Affective DJ Bridge the semantic gap in Music Features Extraction

Affective Computing Algorithms for emotion induction

05/02/23 António Pedro Oliveira 14

ExperimentsExperiments

Knowledge Base ConstructionKnowledge Base Construction N relations between musical features and emotions

Music Base LabellingMusic Base Labelling Music Features Extraction

Music GenerationMusic Generation Music composition, selection

Emotions RecognitionEmotions Recognition Affective Computing (Psychophisiological

techniques)

05/02/23 António Pedro Oliveira 15

ExamplesExamples

RhythmRhythm Tempo – X BPM

EmotionsEmotionsBob

Sinclair

Zero 7

05/02/23 António Pedro Oliveira 16

ExamplesExamples

MelodyMelody Melodic motion - pitch variation, notes density

EmotionsEmotionsSheryl

Crow

R.E.M.

05/02/23 António Pedro Oliveira 17

ExamplesExamples

HarmonyHarmony Consonance – music tension

EmotionsEmotionsDamien

RiceFaithless

05/02/23 António Pedro Oliveira 18

ExamplesExamples

InstrumentationInstrumentation Timbre – Spectral features

EmotionsEmotions

LouisArmstrong The Corrs

05/02/23 António Pedro Oliveira 19

ExamplesExamples

DynamicsDynamics Loudness – X Energy

EmotionsEmotionsGreen DayU2

05/02/23 António Pedro Oliveira 20

ConclusionConclusion

Scientific contributionsScientific contributions Knowledge base with relations between musical features

and emotions System to be used by musicians, psychologists, health

scientists

Engineering contributionsEngineering contributions Unifying functional system Autonomous DJ application

Artistic contributionsArtistic contributions Composition of new pieces of music

05/02/23 António Pedro Oliveira 21

ConclusionConclusion

Bring the knowledge of this Bring the knowledge of this cross-disciplinary approach cross-disciplinary approach into a computer systeminto a computer system

System advantagesSystem advantages Automatic Affective Music Generation

Computer games, movies, dance, etc. Flexibility to update:

Knowledge Base – useful for Music Psychology Music Base - useful for Computer Music