epia 2007

António Pedro António Pedro OliveiraOliveira

and Amílcar Cardosoand Amílcar Cardoso06/12/200706/12/2007

A Computer System to Control Affective Content in Music Production

António Pedro Oliveira and Amílcar Cardoso 2

OutlineOutline

IntroductionIntroduction

Proposed SystemProposed System

Emotional output validationEmotional output validation

Prototype (demo)Prototype (demo)

ConclusionConclusion


OutlineOutline








Problem No automatic method to express and

induce an intended emotional description with music

Objective Design, implement and assess a computer

system that can control the affective content of pre-composed music represented at a symbolic level


Introduction – Topics of Introduction – Topics of ResearchResearch

Music and emotions psychology Relations between emotions and musical

features

Affective computingAffective computing Techniques to recognize emotions

Music computingMusic computing Approaches to generate affective music


OutlineOutline







Proposed System - Proposed System - ArchitectureArchitecture(first stage)(first stage)


Emotions representationEmotions representation

2 Dimensional and Lexical 2 Dimensional and Lexical spacespace


Proposed System - Proposed System - ArchitectureArchitecture(second stage)(second stage)


Computational Computational systematization – emotions systematization – emotions and musicand music Systematize relations between Systematize relations between

musical features in a musical features in a computational contextcomputational context


Proposed System - Proposed System - ArchitectureArchitecture(third stage)(third stage)


Music SegmentationMusic Segmentation

Segment music to allow a Segment music to allow a better control of affective better control of affective contentcontent


Music Features ExtractionMusic Features Extraction

Extract features to relate with Extract features to relate with emotionsemotions


Proposed System - Proposed System - ArchitectureArchitecture(fourth stage)(fourth stage)


Control of affective content – Control of affective content – music selection and music selection and manipulationmanipulation Selection and manipulation of pre-

composed symbolic music


Music SelectionMusic Selection

Select relations from the knowledge base by Select relations from the knowledge base by matching the emotional descriptionmatching the emotional description

Select music from the music base according to Select music from the music base according to selected relationsselected relations

Similarity metrics among music features (melody, rhythm, etc.)


Music ManipulationMusic Manipulation

Focus of the workFocus of the work


Music SequencingMusic Sequencing

Music sequencing is done with Music sequencing is done with the help of k-means clusteringthe help of k-means clustering

e.g., by note density (3 clusters)


Music transformationMusic transformation

Produce a set of musical Produce a set of musical segments that sound fluentsegments that sound fluent


Inclusion of composition and Inclusion of composition and arrangementarrangement

Possible use of third party Possible use of third party software / algorithms of software / algorithms of composition and arrangement composition and arrangement to manipulate musicto manipulate music


OutlineOutline








Assessment methodologyAssessment methodology Produced music is played in experimental

contexts (e.g., clinical or entertainment) Data is collected about the perceived and

induced emotions Facial electromyography, skin conductance,

heart rate measures and questionnaires



Recognized emotional state is compared with the intended emotional state

Comparisons will be used to refine the mappings in the knowledge base

New mappings can be added; old mappings can be adapted or discarded with Case-Based or Rule-Based techniques


OutlineOutline







Prototype - DemoPrototype - Demo


ExamplesExamples

Manipulated music samples Manipulated music samples Different affective content (e.g., timbre,

loudness, tempo and register) http://student.dei.uc.pt/~apsimoes/PhD/Mu

sic/epia2007/affective.rar

High arousal High valence Low arousal Low valence

http://student.dei.uc.pt/~apsimoes/PhD/Music/epia2007/affective.rar

http://student.dei.uc.pt/~apsimoes/PhD/Music/epia2007/affective.rar


OutlineOutline








Design a computational model to control music affective content grounded on psychological research

Validate the model with users to promote a reliable expression and induction of emotions by using music

System can be applied in areas that intend to produce music given an emotional input

Music Therapy, soundtracks for arts, movies, dance, theater, virtual environments, computer games, etc.

epia 2007

Engineering