the soundness of documentation: towards an epistemology for
TRANSCRIPT
![Page 1: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/1.jpg)
1
The 1st International Conference on Language
Documentation and Conservation
University of Hawai’i Manoa
Sat March 14, 2pm
David Nathan
Endangered Languages Archive SOAS, London
The soundness of documentation: towards an epistemology for audio in documentary linguistics
![Page 2: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/2.jpg)
2
Aims
to share some ideas about audio within language documentation
to seek feedback/correction etc
to promote critical thought about practices
to stimulate experimentation and change
with apologies to the acoustically and philosophically better informed
![Page 3: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/3.jpg)
3
Past influences
1990s and multimedia: linguistic audio as evidence, not performance
documentary linguistics
Dietrich Schüller
training documenters
![Page 4: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/4.jpg)
4
From evidence to performance
![Page 5: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/5.jpg)
5
Linguists and audio
little or no training
poor recording techniques
little actual usage of audioas dataas performance
ie
audio is just an inconvenience on the way to transcriptions, description, analysis
there was no methodology or epistemology for audio
![Page 6: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/6.jpg)
6
Language documentation?
an offshoot within linguistics, starting 10 years ago as a response to global language endangerment
a multipurpose and comprehensive record of the linguistic practices characteristic of a speech community .. the emphasis is on the collection and representation of primary datarather than theory and analysis
(Himmelmann 1998)
![Page 7: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/7.jpg)
7
Audio is not data
real events
recordings
representations
data, abstractions
![Page 8: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/8.jpg)
8
Audio is an event or a resource
making it is both art and science
a critical and ethical responsibility
strongest relationship to communities
it’s not necessary to record everything
![Page 9: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/9.jpg)
9
Linguists and audio information
information theory ‐ it’s necessary to lose some data to get information/knowledge
for reduction to writing a lot of data is lost
recording can involve a massive and uncontrolled loss of data
but loss of audio information is not consequential!
![Page 10: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/10.jpg)
10
How are we losing audio information?
non‐optimal equipment and techniques
lack of training
noise
spatial information (how can we arbitrarily ignore this?)
we have worried about resolution not signal to noise ratio (SNR)compression not spatial information
![Page 11: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/11.jpg)
11
What we’ve been doing about it
![Page 12: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/12.jpg)
12
What we’ve been doing about it
training
field trials
challenging assumptionsrecording needs to be done unpreparedequipment is intrusive
![Page 13: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/13.jpg)
13
Training themes
signal and noiseneed to define and control signal and noise
metadata: information that provides for the discovery, usage and understanding of data
![Page 14: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/14.jpg)
14
Training themes
monitoring
evaluating
psychoacoustics
44.1 KHz, 24 bit
![Page 15: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/15.jpg)
15
Training themes
room acousticsshotgun vs omni with pub noiseORTF
equipment: frame with sleeping bag headphones, speakers, amplifiers/distributionmics and cablesrecording of pub noise, portable player“performers”
![Page 16: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/16.jpg)
16
Stereo
interview in noisy environment
![Page 17: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/17.jpg)
17
Stereo
interview in noisy environment
![Page 18: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/18.jpg)
18
Binaural/ORTF
![Page 19: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/19.jpg)
19
ORTF
17cm
110°
![Page 20: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/20.jpg)
20
Preliminary results from ORTF
listeners agree about localisation
participants can be “separated”
more knowledge about environment
can be distracting (> “information masking”)
some environments result in very disturbing recordings
preliminary experiment: evaluation and information extraction, comparing degraded ORTF with uncompressed mono recordings
![Page 21: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/21.jpg)
21
How do we perceive spatial information?
two ears – generate a difference
spatial information from:phase/delayfrequency fallofffrequency colourationintensity variation(head orientation)
and combinations
of all of these!
![Page 22: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/22.jpg)
22
What happens in 10ms? (1/100 sec)
a few hundred distinguishable amplitude readings, just enough to identify most speech‐relevant frequencies
sound has travelled about 3 metres
so our potential discrimination for environmental/spatial acoustic information is similar in scale to our ability to detect frequencies
(compare to vision; even in 50ms you won’t notice any change at all)
![Page 23: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/23.jpg)
23
Arguments from video enthusiasts
arguments for use of video ‐many apply to audio!disambiguating participantsparalinguistic expressionsemotionscapturing locations, environments/settingsattraction of multimedia products
are those pro‐video points mainly about spatiality?
has video been used/advocated to make up for the inadequacies in audio practice?
and more questions about non‐spatial information ...
![Page 24: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/24.jpg)
24
Some implications
hearing impaired people have trouble with “cocktail party/cafeteria effect”
if we said that recording is “for a human listener”:
a recording that does not appropriately distinguish the focal speaker from background talk is making a recording “as heard by a hearing impaired person”
![Page 25: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/25.jpg)
25
“Energetic vs Information masking”
What are the implications for:
listeners? “native speakers”? linguists?
![Page 26: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/26.jpg)
26
Glimpse effect
for “native speakers”, the gaps together with the redundancy in language may make this intelligible or acceptable
what about for linguists?
![Page 27: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/27.jpg)
27
Implications
who/what is audio for?if “native speakers” then glimpse effect can make audio acceptable, but for linguist unacceptableif audio is to be listened to for long periods, comfort will be important
audio data collection should also include metadata about sources, environment, locations, orientations
who notes the content of stereo channels?
![Page 28: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/28.jpg)
28
Implications
signaldeciding what is the signalquality criteria (human arbiter, lip smacking?)understanding its other properties
![Page 29: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/29.jpg)
29
Implications
noise:deciding what is noisewhat is noise may changeunderstanding its propertieshow to record, vis‐a‐vis noise(all of these will have vastly greater
influence than selection of recorder or resolution etc)
![Page 30: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/30.jpg)
30
Implications
we should record audio relative to goals and usages
teaching materials (low SNR will reduce effectiveness)songs/stories/performancestalking dictionariestranscription!
![Page 31: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/31.jpg)
31
Implications
don’t view audio only through “linguistic tools”:
songs interactive player – linguistic tools don’t acknowledge verses!
different equipment and setups for different languages, or events that have different acoustic properties
listening is important!listenability (comfort etc) ‐ actual usageslistening environment/method
![Page 32: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/32.jpg)
32
Intelligibility vs listenability
SNR Signal to noise ratio
Perform
ance / Evaluation
intelligibility
quality
![Page 33: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/33.jpg)
33
Implications
as participants in events that are recorded, we have a huge range of opportunities to choose equipment, physical layouts, manipulation of equipment and environment, and influence on performances – all of which overwhelmingly influence audio recording quality
if we don’t collect good audio we may as well do fieldwork by phone or Skype!
![Page 34: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/34.jpg)
34
Audio processing?
Not treating audio recording as data‐gathering, eg “remove the noise later”:
when noise reduction algorithms are applied, intelligibility goes down
![Page 35: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/35.jpg)
35
Possible desiderata
accuracyintelligibility /information accessibilitylistenability /comfort/aestheticslocalisation of performersrepresentation of environmentseparation of environmentseparation of noise sourceseditability /repurposeabilitycontent: performance, identity, uniqueness, coverage
![Page 36: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/36.jpg)
36
The future
who can we turn to?speech processingphoneticiansmusicradiofilmvarious users of audiodocumenters who take up the challenge
![Page 37: The soundness of documentation: towards an epistemology for](https://reader031.vdocument.in/reader031/viewer/2022021008/6203a574da24ad121e4bd2aa/html5/thumbnails/37.jpg)
37
Epistemology for audio in language documentation
an audio recording represents what a human listener would experience at a particular location and orientation in a settingan audio recording is to convey an audio experience to a human listenerthe context, goal and methodology define the audio information to be capturedrelevant spatial and configuration metadata should be recordeda multipurpose record should capture the maximum spatial information