Theatre captions through the looking glasses!

Louise FryerZoe MooresPablo Romero Fresco

Background!

UK: 11m people with hearing loss; 15.6m by 2031 (AOHL)

Before May 2000: BSL, loop and infrared, script and torch

Stagetext: 18 years of captions (LED screen)

Semi-live captioning

Trained captioner prepares script, attends rehearsal and outputs on the day

Limited number of accessible performances(mostly one-offs and London-based)

> Can captioning be automatised?

> How would this be received by the audience?

Background!

•  I-Caption for Broadway shows: based on lighting cues, captions on handheld devices, erratic timing, mixed feedback

•  Airscript for foreign language captions:needs dedicated operator, too expensive

Background!

Too much head movement Too small a screen

Annoying for other viewers

Lack of comfort Potential for immersion

Need to keep focusing

•  to develop existing speech following software and triggers from the lighting and sound desk to automatically output captions and translation subtitles

•  to test different types of display devices in order to assess which one give the best user experience.

CaptionCue: aims!

•  Four measures:

Views on the quality of the automatic captions

Distribution of attention between stage and captions

Pre- and post-test preferences per device

Sense of presence and engagement

CaptionCue: reception study!

CaptionCue: material

Shakespeare’s Henry V Enda Walsh’s Chat Room Moira Buffini’s Dinner Arnold Wesker’s Roots Stephen Sondheim’s Old Friends

Participants

Gender

Total Male Female

BSL-user 0 3 3

Deaf 12 19 31

Deafened 5 7 12

Hard of Hearing 23 34 57

Hearing 21 42 63

Total 61 105 166

Conclusions

Overall satisfaction: - as good as normal captions regarding readability, accuracy, format, speed and display mode. - synchrony could be improved.

Effective distribution of attention: - better than in TV captions - better than recalled - better for side LEDs, then integrated LEDs and then tablets - better for musicals, worse for fast-paced plays - tablets good for musicals

Month!!

Show!

October / November! Beginning!January! Pinocchio!February! Pinocchio!

March! Macbeth/Pinocchio!April! Macbeth/Pinocchio!May! Macbeth/Nine Nights!June! Macbeth/Translations/Octoroon!

July! Translations/Exit the King/Octoroon/Lehman!

August! Translations/Exit the King/ Lehman!September! Exit the King /Anthony and

Cleopatra!

Month!!

Show!

October / November! Beginning!January! Pinocchio!February! Pinocchio!

March! Macbeth/Pinocchio!April! Macbeth/Pinocchio!

Month!!

Show! Participants!

October / November!

Beginning! 16!

January! Pinocchio! 7!February! Pinocchio! 7!

March! Macbeth/Pinocchio! 21!April! Macbeth/Pinocchio! 21!

Constant adjustments!

Rifle microphones at the front of the stageSpeech follower engine acting alone

No help from lighting and sound cues or times from human captioner

Radio microphones on the actorsAdded lighting and sound cue triggers Added recorded times from human captionerAdded user options for left, centre and right aligning textChanged the nose-piece on the glasses for a different model

Songs that were pre-timed by a captioner Added auto-reconnecting for glasses that disconnect from WifiLocked out all unused buttons on the handset controller

Questionnaires!

Questionnaires!

Questionnaires!

Initial findings!

: Comfort

- Most participants found the glasses comfortable on their ears (50% vs 36%), but this depends on the play. Better results for Macbeth.

- Almost even distribution between participants who thought the glasses were comfortable on the bridge of their nose and those who didn’t. Better results for Pinocchio (2hr55 mins) than Macbeth (2 hr 30 mins) and Beginning (1 hour 50 mins).

- There was a negative  correlation between finding the glasses too heavy and them being “comfortable on the bridge of my nose”.

Initial findings: Comfort!

- Most participants did not experience sensations like a headache, dizziness or tiredness (50% vs 28%). The results improved as adjustments were introduced in the glasses.

- The glasses stayed in a good position throughout the play(55% vs 21%).

- Most participants (39.5%) were undecided as to whether the glasses are less tiring than traditional LED captions.34% consider them less tiring.20% consider them more tiring.

Initial findings: Self-consciousness!

For most participants (76% vs 8%), wearing the glasses did not make them feel self-conscious.

Initial findings: Immersion!

- The more immersed the participants were in the play,the more likely they were not to find them tiring, and vice versa.

- Causality?Immersed because there is no strainorNo strain because you are immersed

Initial findings: Immersion!

- People watching Macbeth found them more comfortable than people watching Beginning.

- Immersion depended on the play: more in Macbeth and Pinnochio than in Beginning.

Initial findings: Colours!

- Most participants (66%) tried all coloursand the majority chose white (82%).

Initial findings: Opinion!

- An overwhelming majority of participants would be happyto use the glasses again (89% vs 5%).

- This opinion is related to the statement “I could read the captions easily without strain”, but not to the accuracyor timing of the subtitles.

Initial findings: Opinion!

- Most participants (65% vs 21%) felt that the glasses were not as good compared to traditional LED screens.

- This is not show-dependent.

Initial findings: Opinion!

People who found them most comfortable were most likelyto use them again, whether or not they found them worse than existing LED screens.

“Was able to watch the show more closely. Better than just reading a black box”.

“As the captions appear where you look, you can easily watch the performance without feeling you're watching a game of tennis”.

Conclusions!

On-going research for prototype = interim resultsDesign almost readyReasonably good results for full plays

Correlation between comfort and immersion

Conclusions!

Not self-conscious

No distraction for actors or hearing audience

Happy to use them again

Replacing LED screens may meet resistance

Has the decision been taken already?

Theatre captions through the looking glasses!

Louise FryerZoe MooresPablo Romero Fresco