audio culture researcher requirements, metadata design and

Audio Culture Researcher Requirements, Metadata Design and Potentials of Automated Feature Extraction

Birger Larsen & the RSLIS LARM/CoSound teamInformation Systems and Interaction DesignRoyal School of Library and Information ScienceUniversity of CopenhagenDenmark

www.iva.dk/blar - [email protected]

Marianne Lykke AAU

Mette SkovAAU

Toine BogersRSLIS

Haakon LundRSLIS

CoSound, June 21, 2013

This talk

• Audio culture researcher requirements for an audio research infrastructure

• User-driven development of a metadata scheme for radio broadcast archives

• Reflections on the potential of automated feature extraction

Slide 3


Audio culture researcher requirements for an audio research infrastructure

LARM Audio Research Archive• Research infrastructure for radio and

auditive cultural heritage, 2010-1013• Funded by the Danish Ministry of Science, Technology

and Innovation• Consortium of 13 Danish institutions, universities +

Danish Radio + State and University Library

Objective: To further radio and audio research by maturing research source material

• IT-infrastructure (main architecture and interface)• Solutions to copyright issues• National bibliography of radio• Toolbox: Tools for metadating, annotating, search and

disseminationOrganisation: technical solutions, national bibliography,

research cases

Slide 4


Two main challenges

• Large heterogeneous collection• 1 million hours of broadcast radio• 1937-present (24/7 coverage from 1989)• Extremely sparse metadata, no funding to index• Streaming access only because of copyright issues

• Heterogeneous user group• All audio culture researchers... • ...but very diverse research interests• No public access due to copyright > large scale

crowdsourcing not possible

Slide 5


Audio culture researcher requirements

Initial wish: a common philology / metadata / classification scheme to be used across researchers

How to design a metadata scheme for audio culture research?

Analysis of Humanities information needs• Questionnaire to Danish and international audio culture

researchersAnalysis of Humanities research tasks and needs

• Wiki for describing and analysing used concepts and coding wishes and practises

Slide 6


Questionnaire results

Brain storming + tagging sessions

Respondents17 Danish + 51 international

Preferred search entries (how do you/would you like to search for audio for your research?)

• On average title, genre and topic are important, media specific metadata less so

• But large individual differences

Slide 7


48%

45%48%

60%

Questionnaire results

Indexing level: Which unit of material is important??

• Several levels of indexing is necessary

Needed more detail:• Wiki for describing

and analysing used concepts and coding wishes and practises

Slide 8


Sample research cases

WP5.8 – The auditive syntax of radio• Jacob Kreutzfeldt: Urban Soundscapes

• Identifying ‘real sound’ from Copenhagen• Analysing how the ‘experience of space’ is

constructed • Investigating the correlation between technology

and the portrait of ‘the sound of the city’• Need for search and annotation

• Sound category (e.g. traffic, shop, car, bell, goat)• Free tagging• Mix (sound level (fore/background), fade in/out)• Authenticity (real/scenographic /effect)• Anchorage (space, language, rhythm, theme)• Other

• Torben Sanglid: DR Jingles and Theme Songs• Investigating changes of focus over time• Theme songs (intro, separators, outro)• Station IDs, Spots


Developing a metadata scheme

Need for • effective retrieval of radio broadcasts• adding research-specific annotations• at both the broadcast level as well as at segments of

broadcasts. Needs of humanities researchers are so diverse that it is

unlikely that a single unified subject list will suit allMain metadata requirements

• easy to work with• easily extensible• would provide for flexible data exchange

Different sources• Original archive data• User generated

• Of a general nature• Project specific

Slide 10


Conceptual metadata scheme

Slide 11


Channel, program title, start/end time, narrative, creators and roles

Program title (missing or wrong), persons (participants/subject), genre, related objects, subject, tags, annotation

[project specific]

Slide 12


Archivemetadata

Administrative metadata

LARM metadata

Project metadata



Conceptual metadata scheme

Consistency across project metadata?• project metadata scheme encoded in streaming

interface• Coding manual saved in repository

• Documentation• Reuse

Current state• 3-level schema implemented in larm.fm• In use by some research groups• Individual project schemas needs to hand coded

Slide 15


Reflections on the potential of automated feature extraction

Flexible annotation made possible• But still relatively few programs can be annotated

Automatic feature extraction has the potential to enrich the audio files automatically

• First pass to e.g. segment the files• More manual passes to refine and correct automatic

output• Data driven humanities

For search in radio programs, automatic speech recognition holds great potentials

• But not easy to do well for Danish• On-going tests on DR radio news and the Queen’s New

Year addresses in collaboration with SpeechOp.com

Slide 16


References and links

LARM – http://www.larm-archive.orgCoSound – http://cosound.dk

Unlocking radio broadcasts : User needs in sound retrieval / Lykke, Marianne; Skov, Mette. Proceedings of the 4st international conference on Information interaction in context. Association for Computing Machinery, 2012. s. 298-301.

CHAOS: User-driven Development of a Metadata Scheme for Radio Broadcast Archives / Lykke, Marianne; Bogers, Toine; Larsen, Birger; Lund, Haakon. iConference 2013 Proceedings. 2013. (Poster)

Slide 17


Search and Interaction in Media Archives

Open seminar at DR, 9:00-13:00 Thursday June 27, 2013

Designing the Search ExperienceTony Russell-Rose, Director of UXlabs, UK

larm.fm master classAndreas Røll Larsen, DR Archive, Research & Rights

Good Practices for Finding InformationKristian Norling, Enterprise Search Evangelist at Findwise, Sweden

Interaction with Sound Art InstallationsMarianne Lykke, Professor at University of Aalborg, Denmark

The Future of Music Interaction Daniel Boland, PhD student at University of Glasgow, UK


http

://b

it.ly

/DRse

min

ar

audio culture researcher requirements, metadata design and

Documents