human improvised theatre augmented with artificial intelligence · pdf file augmented with...

Click here to load reader

Post on 15-Jul-2020




0 download

Embed Size (px)


  • Human Improvised Theatre Augmented with Artificial Intelligence

    Piotr Mirowski HumanMachine London, UK [email protected]

    Kory Wallace Mathewson University of Alberta Edmonton, AB, Canada [email protected]

    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

    Copyright held by the owner/author(s). C&C ’19, June 23–26, 2019, San Diego, CA, USA ACM 978-1-4503-5917-7/19/06.

    Abstract Improvisational theatre (improv) has been proposed as a grand challenge for general artificial intelligence (AI) [7]. Current state-of-the-art conversational intelligence models lack proper grounding, language understanding, and gener- ate meaningless meandering responses [8]. Utilizing them as improvised comedy partners (improvisors) is doomed to fail - curiously, this limitation makes their use particularly appealing. Improv theatre celebrates risk taking and fail- ure by inviting performers to express themselves without hesitation or fear of being judged [11]. Our installation is an interactive improv workshop for a group of interested partic- ipants, culminating in a live public performance. Attendees are invited to observe and interact with AI-based improvi- sational theatre technology. The workshop is facilitated by two improv theatre professionals with a combined 30 years of experience in teaching, training, and touring. The perfor- mance features various AI tools for augmented creativity.

    Author Keywords Improvisation; Theatre; Storytelling; Performance; Actor Training; Chatbot; Language Models; Conversational AI

    CCS Concepts •Human-centered computing → Interaction techniques; •Applied computing → Performing arts;

    Session: Poster & Demo Reception CC ’19, June 23–26, 2019, San Diego, CA, USA


  • Motivation

    Figure 1: Performers receiving lines from AI chatbot via headset.

    Figure 2: Example of improv performance (downstage) with one improviser receiving lines from an AI chatbot (upstage).

    Figure 3: Example AI improv performance technical setup.

    Improvisational Theatre Improvisation (impro or improv ) is a complex theatrical art form modelled on natural human interaction and demand- ing constant adaptation to an evolving context; it has been qualified as “real-time dynamical problem solving” [4, 10] in the settings of both jazz music and theatre. Improv re- quires performers to exhibit acute listening to both verbal and non-verbal suggestions coming from the other improvi- sors, split-second reaction, rapid empathy towards the other performers and the audience, short- and long-term memory of narrative elements, and practiced storytelling skills [11]. From an audience point of view, improvisors must express convincing raw emotions and act physically to reproduce the experience of a scripted play.

    The Three Minds of an Improvisor Individual improvisors, or improv troupes, typically com- bine several behavioural channels (or “minds”) over the course of a performance. Upright Citizens Brigade teacher Billy Merritt introduced three concepts of pirate, robot and ninja [15] to qualify players who fearlessly initiate new scenes with “no idea of what will happen next” (pirates), players who use their sense of logic and acting skills to ground the scene into reality (robots), and players who support the improvised storytelling by introducing characters and situa- tions necessary to move the story forward or by reincorpo- rating narrative elements to bring the story towards a con- clusion (ninjas). An alternative subdivision into “realms of the body” [16] could be seen in the practice of French actor, dancer and trainer Francois Delsarte (1811-1871), namely into head, heart and gut. In the context of improv, the head focuses on storytelling, narrative elements, interest, hu- mor, analogy, metaphor, and reincorporation; the heart con- cerns itself with reacting truthfully in the moment [14]; and the gut is the wild-card channel through which spontane-

    ity emerges [1]. Good improvisers or improv troupes can balance these three channels, effortlessly switching as the performance progresses.

    Using AI as an Actor Training Tool? AI models have been used for generating narrative struc- tures by playing a role similar to the head [3]: an AI sto- ryteller can be used in improv exercises where the actors focus on being in the scene (i.e. similar to a director calling edits). AI models have also been used for putting perform- ers with challenging and novel situations (gut) [12, 13]. The latter study [13] collected feedback of a large number of hu- man performers who qualified the AI conversational partner as an “X factor”. The pirate-like conversational AI forced them to take care of the narrative, and enabled them to fo- cus on the emotional aspects of the performance. These AI models can therefore serve challenging educational and inspirational tools for performance development.

    Engaging in Human-Machine Conversations To learn how to perform alongside AI-based improvisors, we have developed several exercises which channel the golden rule of improvisational theatre, saying: “Yes, and. . . ” The first exercises focus on conversational dialogue. We use several conversational AI-based chatbot systems:

    • rule-based systems (e.g., ELIZA [19]);

    • retrieval based models such as Jann (Just Approxi- mate Nearest Neighbour) 1, combining the Universal Sentence Encoder [5] embeddings of Cornell Movie- Dialogs Corpus [2] with approximate nearest neigh- bor search;


    Session: Poster & Demo Reception CC ’19, June 23–26, 2019, San Diego, CA, USA


  • • generative language such as A.L.Ex (Artificial Lan- guage Experiment) [12], based on sequence-to- sequence recurrent neural networks [6, 18] trained on the OpenSubtitles corpus [17].

    Figure 4: Example of improv performance with two humans and one robot. On the right, an audience volunteer curates the suggestions generated by the AI chatbot before sending them to the PA system of the theatre (audio) and to the robot (embodiment).

    Figure 5: Example of improv performance between a human and a robot (EZ-Robot).

    This progression illustrates the limitations and benefits of the different models. It invites participants to explore how each AI-based system might contribute to performance the- atre in different ways. In these interactions, we focus on the dynamics necessary for successful improvisation alongside AI. We discuss timing, status, physical dynamics, and the undesirable—albeit quite common—dismissal of nonsen- sical lines from conversational AI models. Teachings from our AI-centered workshop are transferable, for instance for developing performance and public speaking skills [20].

    Exploring AI Stage Partner Embodiment Over the course of the workshop, we explore how the em- bodiment of the conversational dialogue system can affect theatricality and communicative dynamics. We illustrate and demonstrate the differences by comparing a robotic platform [9] to a video projection-based system. We also explore how the lines generated from the conversational models can be supplied (via headphones) to the humans in the performance (Fig. 1). In this way, a subset of the hu- man performers can deliver nonsensical AI-generated lines using uniquely human emotion, instinct, and timing, while the other human performers justify and ground the scene. By modifying the embodiment we are able to compensate for timing limitations of computational systems through non- verbal acting and emotional subtext.

    Introducing New Modes of Interaction in Improv We will demonstrate several novel interactive systems for human-machine collaboration in improvisational theatre. AutomaTED invites performers to deliver an improvised

    TED style talk based on slides generated by an AI-based system [20]. dAIrector generates a sequence of plot points for the human(s) to improvise with [3]. We also present explorations in multi-modal generation, specifically, we in- troduce “emotion driven music generation”, where a sys- tem provides underscoring for improvised scenes based on emotional content of the dialogue.

    Discussion We aim to augment and enhance human performance by building and deploying challenging AI-based improv sys- tems. The interaction between the humans and the ma- chines has been explicitly explored in previous research. We discuss these findings and open the discussion to con- ference participants to provide their own reflections on in- teracting with the systems. We particularly look forward to conference participants’ engagement in discussions on giv- ing up control to AI-based collaborators [14].

    We also wish to explore audience reactions to these tech- nologies, namely, the balance between public excitement surrounding AI and the fear and misinformation about the capabilities of such systems. One of the major goals of our work is to invite the public into the conversation: we give in- tuitive explanations about AI during our shows and value lis- tening to the audience’s perspectives on AI. Our past work and performances have illustrated the limitations of the cur- rent state-of-the-art models, to great comedic success.

    Acknowledgements We would like to thank Adam

View more