Feeling FireworksDorothea Reusser1,2, Espen Knoop1, Roland Siegwart2, Paul Beardsley1

1 Disney Research2 ETH Zurich

dorothea.reusser@gmail.com

ABSTRACTWe present Feeling Fireworks, a tactile firework show. Feel-ing Fireworks is aimed at making fireworks more inclusivefor blind and visually impaired users in a novel experiencethat is open to all. Tactile effects are created using directablewater jets that spray onto the rear of a flexible screen, withdifferent nozzles for different firework effects. Our approachis low-cost and scales well, and allows for dynamic tactile ef-fects to be rendered with high spatial resolution. A user studydemonstrates that the tactile effects are meaningful analogsto the visual fireworks that they represent. Beyond the spe-cific application, the technology represents a novel and cost-effective approach for making large scalable tactile displays.

Author KeywordsLarge Interactive Screen; Haptic Device; Accessibility

ACM Classification KeywordsH.5.2 User Interfaces: Haptic I/O

INTRODUCTIONFirework shows are enjoyed globally as an essentially visualexperience. To create an experience that is more inclusive forpeople with sight loss, we have made a large-scale tactile dis-play to generate tactile fireworks that are directly analogousto physical fireworks in the sky. Inclusive and assistive tech-nology typically has a functional goal, but our work differs inhaving aesthetic intent. We envisage it as an installation thataccompanies a physical firework show to attract all crowdmembers, sighted and unsighted, to share the experience offeeling fireworks.

Rendering a tactile firework show requires a tactile displaythat is fast with high spatial resolution, and support for touchinteraction is also desirable. Large-scale tactile displays suchas inFORM [2] and TableHop [7] are not suitable becausethey do not meet these requirements at reasonable cost. TheTactile Brush [3] and ultrasound haptics [5] do not support theinteraction modality of a screen. Tesla Touch [1] is unlikelyto provide the tactile gamut and spatial resolution required forcompelling firework effects.

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).

UIST ’17 Adjunct October 22–25, 2017, Quebec City, QC, Canada

c© 2017 Copyright held by the owner/author(s).

ACM ISBN 978-1-4503-5419-6/17/10.

DOI: https://doi.org/10.1145/3131785.3131811

Figure 1. Feeling Fireworks is a large-scale tactile display which gener-ates a tactile firework show, providing an extra dimension for blind andvisually impaired users, in the context of an enjoyable experience sharedby people with sight loss and sighted users.

We propose a novel solution to the problem which is to createtactile effects by spraying water jets onto the back of a flexi-ble screen, while a user senses the impact on the front surface,as shown in Fig. 1. The water nozzles are mounted on con-trollable pan-tilt units for moving tactile sensations, and dif-ferent nozzles create different effects. This results in highlydynamic tactile effects that can be moved smoothly aroundthe screen. Feeling Fireworks further incorporates projectedvisual fireworks for visually impaired users and sighted users,and is interactive to user presses on the screen.

TECHNOLOGYFig. 2 shows a system schematic. The size of the device is106×96×215 cm. The screen is latex and has dimensions∼1×1 m. Water moves through the system in a closed cy-cle, using a controllable pump that provides variable wa-ter pressure. Different nozzles create different firework ef-fects. Three jet nozzles create rockets and explosions, oneshower nozzle is for crackle effects, and one nozzle with anovel design is for a blooming flower effect. Effects canbe sequenced, for example a rising point that explodes intoa blooming flower. A rear-mounted projector provides vi-

Demonstration UIST’17 Adjunct, Oct. 22–25, 2017, Québec City, Canada

65

Projector

Kinect

Pan-tiltunits

Valves

Pump

Control box

Flexible screen

Reservoir

Nozzles

Figure 2. Schematic drawing of the Feeling Fireworks tactile display.

(a) (b) (c) (d)

Figure 3. Custom nozzle developed for the flower effect. Increasing thewater pressure causes the membrane (blue) to bulge outwards and thebeams of water to diverge.

sual effects, and a rear-mounted Kinect detects a user’s fingerpress on the flexible screen to provide interaction similar tothe eTable [4]. An Arduino controls the pump, the pan-tilt ofthe nozzles, and the opening and closing of the nozzle valves,while a laptop controls the projector and Kinect.

Flower NozzleThe flower is a popular firework effect — an explosion thatstarts at a single point and forms an expanding circle of lightas in Fig. 1. To mimic the effect, we developed the customnozzle design in Fig. 3. The nozzle has a silicone mem-brane with small holes through which the water flows. Themembrane is flat when no water pressure is being applied(Fig. 3a), and produces a single jet of water at the lowestoperating water pressure (Fig. 3b). Further increasing thepressure causes the membrane to bulge and hence create acontinuously-expanding cone of water (Fig. 3c-d).

DeploymentThe system is free standing, on caster wheels for easy move-ment, and runs off a single power cord. It is modular fortransport and setup. The pump is a near-silent medical prod-uct, while the water jets make a light drumming sound onthe plastic screen which is unobtrusive. The back-projectedvisual fireworks are visible outdoors at night or in typical in-door light. The device sits in a plastic tray to prevent acciden-tal water leakage.

FOCUS GROUP AND USER STUDYThe development process included work with a focus groupfrom the blind community to discuss the experience, and a

user study with sighted users to evaluate the correspondencebetween the tactile firework effects and the physical fireworksthat they represent. For the user study, 18 sighted partici-pants were asked to sample individual tactile fireworks andassign them to video clips. The study was based on ten tac-tile fireworks and a corresponding ten video clips. The resultsshowed strong correspondence between the tactile fireworksand the visual fireworks they represent. For sighted users,the results indicate that experiencing a tactile firework alongwith seeing the physical firework in the sky has meaningfulcorrespondence. And for people with sight loss, the tactilefirework show is a meaningful way to experience the physicalshow.

CONCLUSIONThis work has presented a novel design of tactile screen thatis scalable and economic, and which has been used to makea tactile firework show. It provides an extra dimension forpeople with sight loss at a traditional firework show, in thecontext of an inclusive experience shared by all. A user studyshowed that the tactile fireworks are meaningful analogs ofvisual fireworks. Beyond this work, there are further possi-bilities — for example, the use of balloons by deaf people tofeel music [6] suggests that a tactile-visual screen could alsoprovide a platform for an inclusive musical experience.

ACKNOWLEDGMENTSThanks to the Schweizerischer Blinden- und Sehbehinderten-verband (SBV) for invaluable perspective from the blind andvisually impaired community, and to Maurizio Nitti for theproject banner.

REFERENCES1. Bau, O., Poupyrev, I., Israr, A., and Harrison, C. Tesla

Touch: Electrovibration for touch surfaces. Proc. 23rdAnnual ACM Symp. User Interface Software andTechnology (2010), 283–292.

2. Follmer, F., Leithinger, D., Olwal, A., Hogge, A., andIshii, H. inFORM: Dynamic physical affordances andconstraints through shape and object actuation. Proc. 26thAnnual ACM Symp. User Interface Software andTechnology (2013), 417–426.

3. Israr, A., and Poupyrev, I. Tactile Brush: Drawing on skinwith a tactile grid display. Proc. CHI Conf. on HumanFactors in Computing Systems (2011).

4. Kingsley, P. eTable: A haptic elastic table for 3Dmulti-touch interactions. Master’s thesis, University ofBristol, 2012.

5. Long, B., Seah, S., Carter, T., and Subramanian, S.Rendering volumetric haptic shapes in mid-air usingultrasound. ACM Trans. Graphics 33, 6 (2014).

6. Nature News. Deaf people use ‘mind’s ear’ to processvibrations. http://www.nature.com/news/2001/011127/full/news011129-10.html, 2001.

7. Sahoo, D. R., Hornbæk, K., and Subramanian, S.TableHop: An actuated fabric display using transparentelectrodes. Proc. CHI Conf. Human Factors inComputing Systems (2016), 3767–3780.

Demonstration UIST’17 Adjunct, Oct. 22–25, 2017, Québec City, Canada

66