Did You See Me Do That: Inconspicuous Interaction Techniques for Social Contexts

Abstract Human beings are information omnivores who crave for new information every minute. The urge to stay connected and updated led us to a world infested with wearables. However, these devices and the interactions used to attend notifications emanating from them are still frowned upon owing to their attention-grabbing nature. Our study attempts to enhance the experience of receiving and attending to notifications by encouraging face-up interactions over their heads-down counterparts. We implement indirect light based feedback using a wearable prototype at different physio-locations on the arm to notify users of incoming notifications and compare its acceptability in social contexts with the tactile vibration-based feedback. We also discuss a future implementation of our prototype using conductive fibers that potentially blends this novel feedback modality into our daily fashion. In an attempt to do so, we introduce a new gesture called the “brush-off technique” that makes the user interactions natural and inconspicuous.

Author Keywords Wearable; feedback; vibration; distraction; notification, mobile; physio-location.

Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of publication and its date appear, and notice is given that copying is by permission of ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee.

Aakash Maddi Rochester Institute of technology Rochester, NY 14623, USA am5602@rit.edu Archit Jha Rochester Institute of technology Rochester, NY 14623, USA aj6806@rit.edu Manu Suresh Rochester Institute of technology Rochester, NY 14623, USA ms5529@rit.edu

ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous

Introduction The main objective of wearables is to not only convey the arrival or update of information, but also to change our interactions from heads down to face up. However, they all demand significant user attention and disrupt the user activity when notifying the user of an incoming alert. Przybylski and Weinstein [8] demonstrated that the visible presence of mobile phones during in-person conversations attenuated the interest of participants in the conversation and negatively impacted their rapport. The surge of interest in wearables and on-body technology could safely be attributed to this growing impact of mobile phones in disrupting social interactions. Hand-held devices such as smartphones limit the movements that the human body is capable of and therefore steer away from the natural (more social) modes of interaction. On the contrary, using specific movements, that complement the social context and physical environment, for communication or interaction with technology bears the potential of crafting more natural user experiences [4]. While wearables pave the way for making the interactions with technology more natural, moving from hand-held to hands-free, they face challenges of social acceptability. Perception of a wearable technology amongst peers and its ability to fit in cultural norms play a decisive role in its popularity [7]. Many consider wearables as disruptions for individual activities such as exercising at gym [6]. Although, not just the visual appearance of a wearable but the interactions used to communicate with it also fall under the lens of social judgement. For example, twisting one’s legs to control a device may draw more

flak than turning the wrists, as the former is physically inconvenient [10]. Gestures need to be designed responsibly, especially for wearables since their position on the body may transition from one user pose to another, resulting in change in needs of the user [5].

We present the concept of an inconspicuous wearable device that blends into our daily fashion and operates by notifying the users without disrupting their interaction with people around them. Also, we consider the idea of new gesture based interaction techniques and suitable locations on the body to make them look and feel natural in a social context. As we explore the possible, most viable, and least socially awkward physio-locations to use this device, we developed a study to capture the effectiveness of light-based feedback modalities versus vibration-based.

In our venture to develop a natural interaction that stays invisible in a social context, we propose a new interaction technique called “Brush off technique”. The motivation behind this technique is to eliminate the attention and incorporate a gesture that is common in our daily lives. Brushing off dust from our clothes is one of the most intuitive actions to eliminate dirt and dust from the body. We propose to bring this gesture to wearables to ‘brush off’ notifications in order to discard or snooze them.

Related Work Our study aims to eliminate users’ disconnect from the physical world by conceptualizing a new feedback modality. The idea is to inform the users of notifications by providing a visual stimulus through glowing light under their clothes. Harrison et al. found out that users react the fastest and more consistently to visual stimuli

from the wrist and arms [1]. We therefore propose a mechanism that produces a visual feedback under the sleeves to notify the user of notifications and aim to compare the results with tactile feedback at the same location to derive its social effectiveness. Although works of Harrison [1] argue that the response to a visual stimulus on body may depend on user activity and posture, we seek to more closely study the social impact of such a feedback. Moreover, we introduce a novel, more natural ‘brush-off’ gesture for interaction that people use to dust their clothes off. GestureSleeve by Schneegass and Voit [9] was a similar attempt using conductive fibers to take input via clothes but it focused on expanding the input area of smartwatches and implemented gestures recognized by the smartwatch. Our study on the other hand is focused on using one gesture to interact with multiple devices. The motivation for this project comes from two studies that explore the indirect light-based feedback and the social awkwardness when interacting with wearables.

Pohl et al.[3] investigated a novel feedback modality for smartwatches in the form of indirect light output for notifications. Unlike most of the existing feedback mechanisms that are attention starved and disrupt user activity, indirect light feedback is a subtle way of indicating notifications. The LEDs are placed at the base and the light coming off of them is absorbed by the skin. The reflected light is the output which is indirect and scattered around the space where the watch is worn. The idea is interesting since micro-controller based LEDs can be embedded with clothing while being centrally controlled by a smartphone, scattering light-based feedback notifications, both direct and indirect, over the body space and not just around the contours of the smartwatch. The

comfort level of the user and noticeability of these feedback systems by the people around the user are the points of major interest of this project.

Profita et al[2]’s goal was to understand the perception of a third-party when a user is interacting with a wearable e-textile interface. Their main point of the study was to examine the social acceptance of interacting with an interface placed at different location on the body of the user and to gain cultural insight to the same. They implemented an e-textile interface of their own development called the “Jog Wheel”, which was used in the evaluations that supported touch and slide interactions. The participants were asked to view a series of videos of other people using the Jog wheel at different on-body locations. This gave them an understanding of the attitude towards the usage behavior and placement of the system. The study was carried out as a survey in the US and South Korea to incorporate the difference in culture and how this affected the attitude towards placement and usage behavior [2].

Methodology Prototype The project began with the exploration of suitable SoC components and circuits. This idea required a system that enabled a connection through Bluetooth or Wi-Fi. One of the best possible out-of-box options available that suited the requirements was Particle.io’s Photon, a lightweight Wi-Fi bundle. We had previously considered using Arduino MCU but substituted it with Photon due to its ease of use. In order to embrace a modular design, we chose a 150mAh Li-ion battery to power the entire setup. A 5v/100mA vibration motor and a bright 3V LED strip were used for vibration and light-based

modalities respectively. A web app was built using JavaScript to control and transmit instructions to the prototype device while the I/O for photon was coded in C. The instructions included 3 intensities (high, medium and low) of light and vibration for the LED and the motor respectively. The schematics of the circuit are shown in Fig 1.

In an attempt to create a more portable and usable prototype, we explored various designs to house the components. The final version of the housing was chosen to maintain the modularity of the concept while making it more portable to be worn on a user’s shirt using a slide-in clip-based design (Fig 2). While the other components were sewn into a faux sleeve that could be fastened on the user’s arm, the battery and the photon were encased in a 3d printed plastic material clipped-on to the same sleeve (Fig 3).

The faux sleeve prototype was designed to be worn inside the user’s layers of clothing. Both the vibration motor and the LEDs were on the inside of the sleeve and in contact with the user’s skin. LEDs were placed on the edge of the sleeve and a push button switch was provided on the outside to turn off any vibration or light feedback from the prototype. The idea was to simulate the control of circuit through conductive fibers woven into the user’s clothing that would act as a switch.

User Study The user study was designed to capture the participant’s likeness and perceived effectiveness of the feedback modalities in both a social setting and a personal setting. It consisted of two phases and was carried out in public spaces such as the library and the cafe at the Rochester Institute of Technology. All the

participants were handed out a background questionnaire at the beginning of the study.

In the first phase, the moderator put on the prototype under a full-sleeved t-shirt and struck a general conversation with the participant as the observers cycled through the 6 inputs by sending signals over Wi-Fi to the prototype (3 vibration and 3 light-based) while observing the participant responses from a distance. The participants weren’t informed about the prototype on the moderator’s hand, but were debriefed about the prototype after completion of the first phase. Whereas in the second phase, the participants were asked to put on the prototype on two physio-locations (the wrist and the upper arm) on their dominant hand. With the prototype on wrist, the users were asked to draw basic shapes and concentric circles on a sheet of paper and with the prototype on their upper arm, they were asked to read a book to summarize the contents for the moderator. The participants were asked to press the switch to turn-off feedback from the prototype as soon as they noticed it but were requested not to consciously keep an eye out for it. Both the phases were followed up with a brief task questionnaire about the perceptions of the users on the two feedback types. After the completion of both the phases, there was another questionnaire at the end of the study to collect comparative data on vibration versus light feedback.

Demographics A total of 16 participants (9F, 7M) were recruited for the study. 15 participants were graduate students while 1 was a Bachelor’s student at Rochester Institute of Technology who belonged to the age range 20-27. 15 participants were right-handed while 1 was left handed. All the participants had a technical background except

Figure 1: Circuit Diagram for the prototype

Figure 2: 3D prototype concept for housing photon with crevices for wiring that can be clipped to the user’s sleeve.

one who was a psychology major. All of them were regular users of smartphone but only 3 had experience with smartwatches while 1 used fitness trackers.

Findings Prior to the study, 50%(8 out of 16) of our participants agreed to the notion that vibration based feedback was disturbing during work while 15%(2) of them disagreed and 25%(4) were neutral. The participants pointed out that emails and missed calls take a higher priority for them over social media and IM app notifications.

In the first phase of the study, 14 participants (87.5%) did not notice the illumination of light under the moderator’s sleeve. Also, none of them was able to identify any vibration during their conversation with the moderator. Therefore, none of them found the feedback from the prototype disruptive to the conversation in particular when acting as observers. When asked if wearing such a device was acceptable in social/public settings, the participant’s response was favorable and 80%(13) of them agreed and 20%(3) were neutral.

After trying the prototype for themselves, 80%(13) of the participants noted that the illumination of light was noticeable at the wrist, while all of them indicated that it wasn’t noticeable at all at the upper arm. Also 60% (10) of the participants indicated that the light-feedback wasn’t attention grabbing for people nearby while the remaining 40% (6) were undecided on their opinion. 60% (10) of the participants were unable to tell the difference in intensities of light-based feedback and 40%(6) could tell the difference in intensities of vibration. 60%(10) participants believed that vibration was better in public places than light-based feedback. However, 80%(13)

agreed that it was distracting and all of them preferred it for high-priority notifications.

On the subject of embedding light feedback in clothes, 60% of the participants were against it and found it awkward in public while 40%(6) embraced the idea. On the other hand, 80%(13) chose not to use light feedback when speaking to someone and 60%(10) chose not to use it in classrooms.

Limitations The participants commented that the bulkiness of the prototype was a discouraging factor for such a concept. While the concept was aimed to clip onto the sleeve, the faux sleeve prototype was wrapped around the arm or wrist and was held in place rather than letting it hang freely. The studies were conducted in a well-lit place which might have undermined the effectiveness of the light-based feedback.

Future Work We propose to expand this study by involving a wider population sample that constitutes people with both technical and non-technical background from a broader age group. On the prototype front, we seek to implement a ‘brush-off gesture’ that uses pressure-sensitive conductive fibers embedded in user’s sleeve to disengage the circuit in the prototype. This is supposed to be a more natural interaction than pressing a push-button to turn-off feedback from the device. Also, various vibrational and light patterns can be incorporated into the study to test their effectiveness and determine if the users can identify the corresponding notification type. Moreover, an extended study that hooks up the prototype to user’s devices such as smartphones and smartwatches to relay device notifications would produce more detailed results.

Figure 3: Faux Sleeve Prototype with all the components (Top: Upper Arm, Mid: Wrist, Down: Un-fastened)

References 1. Chris Harrison, Brian Y. Lim, Aubrey Shick, and

Scott E. Hudson. 2009. Where to locate wearable displays?: reaction time performance of visual alerts from tip to toe. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 941-944. DOI: http://dx.doi.org/10.1145/1518701.1518845

2. Halley P. Profita, James Clawson, Scott Gilliland, Clint Zeagler, Thad Starner, Jim Budd, and Ellen Yi-Luen Do. 2013. Don't mind me touching my wrist: a case study of interacting with on-body technology in public. In Proceedings of the 2013 International Symposium on Wearable Computers (ISWC '13). ACM, New York, NY, USA, 89-96. DOI: http://dx.doi.org/10.1145/2493988.2494331

3. Henning Pohl, Justyna Medrek, and Michael Rohs. 2016. ScatterWatch: subtle notifications via indirect illumination scattered in the skin. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 7-16. DOI: http://dx.doi.org/10.1145/2935334.2935351

4. Jin Moen. 2007. From hand-held to body-worn: embodied experiences of the design and use of a wearable movement-based interaction concept. In Proceedings of the 1st international conference on Tangible and embedded interaction (TEI '07). ACM, New York, NY, USA, 251-258. DOI: http://dx.doi.org/10.1145/1226969.1227021

5. Lyons, K. and Profita, H. 2014. The multiple dispositions of on-body and wearable devices. In IEEE Pervasive Computing, 13(4), 24-31. DOI: http://dx.doi.org/10.1109/MPRV.2014.79

6. Misha Patel and Aisling Ann O'Kane. 2015. Contextual Influences on the Use and Non-Use of Digital Technology While Exercising at the Gym. In Proceedings of the 33rd Annual ACM Conference on

Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2923-2932. DOI: https://doi.org/10.1145/2702123.2702384

7. Norene Kelly and Stephen Gilbert. 2016. The WEAR Scale: Developing a Measure of the Social Acceptability of a Wearable Device. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 2864-2871. DOI: https://doi.org/10.1145/2851581.2892331

8. Przybylski, A. K. and Weinstein, N. (2013). Can you connect with me now? How the presence of mobile communication technology influences face-to-face conversation quality. In Journal of Social and Personal Relationships, 30(3), 237-246. DOI: http://dx.doi.org/10.1177/0265407512453827

9. Stefan Schneegass and Alexandra Voit. 2016. GestureSleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA, 108-115. DOI: https://doi.org/10.1145/2971763.2971797

10. Tetsuya Yamamoto, Tsutomu Terada, and Masahiko Tsukamoto. 2011. Designing gestures for hands and feet in daily life. In Proceedings of the 9th International Conference on Advances in Mobile Computing and Multimedia (MoMM '11). ACM, New York, NY, USA, 285-288. DOI: http://dx.doi.org/10.1145/2095697.2095757

11. Particle Photo LED control circuit and documentation https://docs.particle.io/guide/getting-started/examples/photon/#control-leds-over-the-39-net