docling 2016 david nathan anthony jukes audio theory and practice for language documentation

DocLing2016

David Nathan & Anthony Jukes

Audio theory and practicefor language documentation

An epistemology for audio in documentation an audio recording is made in order to be experienced by a

human listener a recording conveys what a human listener would

experience at a particular location in an event setting documentation goals define recording methodology a recording should capture spatial information metadata about the recording and the recording setting are

required for full interpretation ethical recording respects speakers and honours their

contribution through your effort and skill

Evaluating recordings accuracy: how well is the signal captured, as true to its

sources and without distortion? intelligibility/information accessibility: can the desired

content be identified? signal vs. noise: is the ratio acceptable? can the focal

source be separated from all sources of noise? listenability/comfort/aesthetics: is it easy on the ears?

will it be comfortable to listen to for an extended time?

Evaluating recordings localisation of sources: is enough spatial information

captured? separation of noise: can all sources of noise be

separated? representation of environment: are the acoustic

properties of the recording space appropriately represented?

Evaluating recordings content (identity, performance, uniqueness, coverage):

were the right people recorded doing the right things? editability/repurposeability: is the recording suitable for

turning to relevant purposes?

Recording audio making it is both art and science a critical and ethical responsibility strongest relationship to communities it’s not necessary to record everything, but it is necessary

to record well

SIGNAL & NOISE

Evaluating recordings signal noise signal to noise ratio listenability (eg comfort, consistency) fit for purpose

Evaluating recordings audio professionals use their human ears as evaluator of

audio quality and value, while many linguists (mistakenly?) look to formats, spectrographs, wave-forms, analyses etc

44.1 KHz, 24 bit

Signal - what you want content contextual and spatial information fidelity comfortable to listen to

Noise - what you don’t want from environment:

near: people, animals, activities far: traffic, generators, planes machines: refrigerators, fans, computers not hearable: mobile phones, electrical interference acoustic: reflections/resonance

Noise - what you don’t want generated by unwanted parts of event

shuffling papers, clothes table banging backchannel from interviewer equipment handling, especially microphones and

cables (and recorders with built-in mics)

Avoiding handling noise use stands and cradles etc

Noise - what you don’t want generated by equipment

wrong input levels circuity noise (cheap or incompatible) compression loss or distortion ALC/AGC effects (pumping) video camera motors

External noise sources

example possibilities for dealing with it

traffic investigate, record in quiet timeface awayuse damping materials

children get them involvedshow something to satisfy curiosity

animals choose time of day

weather (wind, thunder, rain etc)

use dead cat; wait; reschedule

Dead cat

Close-up noise sources machines

example possibilities for dealing with itrefrigerator pre-survey what comes on intermittently

turn offrelocate

motors, switching monitor

fans monitor, dead cat (windshield)

Dealing with noise sources be prepared and aware seek collaboration monitor use or modify room acoustics

location direction surfaces reflection absorption isolation

Utilising room acoustics location

away from doors, windows, traffic areas direction

face away from noise sources reflection

avoid parallel surfaces surfaces

avoid hard smooth surfaces choose or create soft or rough surfaces

isolation find an ‘’airtight’’ place

When is a noise not a noise? When it is part of the content, for some interpretation of

the eventPerformance of John Cage 4”33’Available on iTunes (150 yen)

PERCEPTION & PSYCHOACOUSTICS

Audio perception/psychoacoustics a human listener has:

location, orientation in a physical setting two ears - incredibly sensitive a brain/mind

the mind selects from various sources of sound and other sensory information, using long- and short-term memory

listening is actually a “hallucination”

Psychoacoustics and recording microphones don’t have a mind: they can't

distinguish wanted from unwanted sound microphones don’t have “edges” like camera

lenses

Psychoacoustics and recording the recording process loses acoustic

information if you only care about transcription, then you

are going to throw away over 99% of the acoustic information anyway!

real worldrecord acoustic phenomenarepresent (some) linguistic componentsderive data

Implications for recording typical recording methods are unscientific! … so what should we do?

Implications for recording plan and manage recording

goals equipment preparation and settings other preparation environment and setup sources changes, actions, settings

Implications for recording why is it important to record spatial information? what other information (acoustic or non-acoustic) do we

need?

“Sound stage” spatial information is an essential part of

audio we are amazingly attuned to it we should record in stereo

“Sound stage” ... or ORTF (binaural)

MICROPHONES

Microphones and audio quality microphones are the greatest factor in audio recording

quality selection of appropriate microphone(s) for the task placement and handling

Microphone types principle: dynamic vs condenser directionality: omni, cardoid, and shotgun spatiality: mono, stereo, ORTF, binaural

Microphone physical principles dynamic

generate signal from sound pressure more robust, less accurate used for musical and live performance

condenser more fragile, sensitive and accurate need power source - battery or phantom power

in general, use condenser microphones for language documentation

Omni lavalier or tie-clip microphones are typically omni-

directional

Microphone directionality - omni

omni-directional

Cardioid many “standard” handheld microphones are cardioid

units

Microphone directionality - cardioid

cardioid

Shotgun (= directional, hypercardioid) shotguns are good for

quiet sources in some noisy environments video work

Microphone directionality - shotgun

shotgun/directional/hypercardioid

Head-mounted microphones head-mounted microphones are excellent for very noisy

environments or mobile activties, and may be omni-directional or cardioid

Stereo microphones spatial information is an important part of audio

Full “sound stage”: ORTF

Superlux S502

Full binaural on dummy head

ORTF & Binaural

ORTF is now the “best practice” for field recordings” (Austrian Academy of Science, Vienna Phonogrammarchiv)

Simulating ORTF with 2 cardioids

17cm

110°

Microphones - quality generally, you get what you pay for each model has its own subjective “colour” good microphones for language documentation cost

from US$180 to US$500

Reputable makers - examples AKG Audio Technica Beyerdynamic Røde Sennheiser Shure Sony

Microphone placement

Microphone usage principles where should the microphone be? in general, about 20cm from the speaker’s mouth

the inverse square law is your friend ...

The inverse square law

Using the inverse square law if you have noise sources, increase the signal to noise

ratio by: placing the microphone as close as possible to the

signal source placing the microphone as far as possible from the

noise source

Microphone connections plugs cable types cables for stereo/mono, multiple wireless power sources for condenser microphones - battery or

phantom power

see http://www.hrelp.org/archive/advice/microphones.html

Microphone connections

1/8 inch “mini-jack”

RCA/ “phono plug”

1/4 inch “jack plug”

XLR “Canon”

XLR the physical connection is independent of the

electrical connection transmits phantom power low-noise over long cable runs you can use XML to mini-jack cables or

converters for recorders with mini-jack inputs

Choice of Recorders

Connectors - XLR or mini-jack or both?

Ruggedness and build Accuracy Media type Battery life Cost

AUDIO WORKFLOW

Audio workflow

who/what/where /why/how do you want to record?

contact people

audio training

budget, research, and buy equipment

assemble, test, practise

Before you go

Audio workflow

transport safely

check environment, situations, permissions

make test recordings

local training & collaboration

On site, before recording

Audio workflow

record!monitor!

collect metadata

check quality

monitor

Sessions

select equipment(microphones)

Audio workflow

(label) check quality

backup add information (metadata, metadocumentation, transcriptions, annotations, etc)

After sessions

Audio workflow

add information (metadata, metadocumentation, transcriptions, annotations, etc)

package and send to archive

Later

other audio outcomes and resources

End !