quantifying user experiences for integration into a home ... · a case study of canister and...

Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015

1

Quantifying user experiences for integration into a home appliance design process: a case study of canister and robotic vacuum cleaner user experiences

Ai MIYAHARAa, Kumiko SAWADAb, Yuka YAMAZAKIb, Miwa NAKANISHIa

aFaculty of Science & Technology, Department of Administration Engineering ,Keio University,

Yokohama, Kanagawa 223-8522, JAPAN; bIndustrial Design Center, Mitsubishi Electric Corporation, Kamakura, Kanagawa 247-8501, JAPAN

In this study, we established an evaluation method for the home appliance user experience (UX), an added-value feature in product development, to introduce UX into the product design process. A mobile application was built to efficiently collect “good experiences” big data from users. The forms of human and home appliance interactions were used to classify the collected data and a probabilistic model, showing the relationship between contextual conditions and the experienced UX, was developed. A quantifiable evaluation value was calculated for the comparison of UX. Keywords: user experience, product design, vacuum cleaner, home appliance, big data

1. Introduction

Home appliances, such as vacuum cleaners and refrigerators, have already achieved high standards of functionality, efficiency, and usability, and have become so-called “mature products.” Such products are no longer easily distinguished by the abovementioned qualities, and each manufacturer is now obligated to add new value to their products. The added-value approach to product development focuses on user experience (UX), aiming to provide “good experiences” to their users. Currently, product design follows this “good experiences” concept. However, UX is the result of user–product interactions (ISO9241-210:2010) and is considerably affected by the context of such interactions (Hassenzahl & Tractinsky, 2006). Therefore, designers need to predict features that will deliver “a good experience” under different contextual conditions.

This study is part of a project to develop a UX-based design method for home appliances. Our developed process will derive which UXs should be improved, in other words, what constitutes “the good experiences,” from big data analysis.

2. Method

This study proceeded as follows: 1) We established a data collection method for “experiences” that were accompanied by positive feelings from the user. 2) We defined the forms of interaction between the user and the home appliance from the collected data and developed a probabilistic model, which shows the relationship between such conditions and the kind of UX.

2.1 Data collection method

2.1.1 The aim

User experiences can be extracted from ethnographic research, which intensively observes user actions alongside the user. Despite the strength of this approach, it is unsuitable for rapid product development, which is required for home appliances, because it necessitates the long-term observation of only a few targets. In response to this time-factor dilemma, Hama (2014) built a mobile terminal application for the purpose of collecting utility-time UX for automobiles. We built our application using Hama’s two application requirements as a reference, which are: 1) To obtain detailed information (actions, feelings, and context) of the UX, as though observing alongside the user and 2) To efficiently collect a large amount of information on various UXs.


2

2.1.2 Input items

Hassenzahl and Tractinsky (2006) stated that “UX is a consequence of a user’s internal state (predispositions, expectations, needs, motivation, mood, etc.), the characteristics of the designed system (e.g. complexity, purpose, usability, functionality, etc.) and the context (or the environment) within which the interaction occurs (e.g. organisational / social setting, meaningfulness of the activity, voluntariness of use, etc.)”. Based on this idea, we considered the input items to extract the “good experiences” of users from the following viewpoints: 1) from the user’s characteristic, 2) from the characteristics of the home appliance, and 3) from the form of interaction. These categorized input items are listed in Table 1.

Table 1. Input items of the mobile application.

Intention of the input items Input items Perceiving the user’s characteristic ! age

! sex ! frequency of use

Perceiving home appliance’s characteristic ! action ! emotion ! context of use (4W1H)

Perceiving the interaction form ! explanation of the home appliance by the user ! context of use (4W1H)

Burke (1969) stated “Any complete statement about motives will offer some kind of answers to these five

questions: what was done (act), when or where it was done (scene), who did it (agent), how he did it (agency), and why (purpose)”. Thus, we asked respondents to resolve the context of use into 4W1H (Who, Where, When, What, How), with the aim of perceiving the characteristic of the home appliance and the interaction form.

Tanaka (2012) divided UX extracted by his questionnaire into groups using the KJ method and proposed “24 types of UX in housework and cooking (UX24).” In this study, we re-worked Tanaka’s UX24, as shown in Table 2 (new UX24), and asked respondents to evaluate their experiences using our application’s 24 alternatives.

Table 2. New UX24 evaluation criteria for evaluating user’s interactions with home appliances.

No. Contents 1 Having heartwarming feelings 2 Noticing a new self 3 Remembering past good experiences 4 Praised by someone 5 Able to please someone 6 Able to do easily / quickly 7 Arouses aspirations 8 Feeling an emotional connection 9 Able to trust something to someone 10 Experiencing visual beauty 11 Experiencing auditory beauty 12 Feeling fulfilled 13 Able to concentrate

14 Able to do well 15 Able to do at timing 16 Feeling comfortable / relaxed 17 Feeling refreshed 18 Feeling fulfilled 19 Feeling tidy 20 Feeling unusually luxurious 21 Able to imagine good experiences in the

future 22 Feeling moderate 23 Understanding what had not previously been

understood 24 Feelings of progress

2.1.3 Structure

To acquire detailed information (actions, feelings, and context) on the UX, as though we were observing alongside the user, we employed a conversational interface throughout the application. Quesenbery and Brooks (2011) provided some examples of active listening skills, including repetition and feedback, which we used to shape the reaction and character of the application’s responses to the user’s answers.


3

2.1.4 Implementation methods

Our mobile application was released on October 23, 2014 for iPhone users, although only the Japanese version is available at present. Examples of typical application screens are shown in Figure 1.

Figure 1. Examples of user screens for our mobile application.

2.2 The evaluation of UX

2.2.1 Conceptual model

When analyzing the human user-system interactions and the kinds of reported UXs, we used a conceptual model shown in Figure 2 and described below. 1) The interaction forms were defined through the combination of meanings of the information, the presence or absence of the operation, and the senses. 2) The generative probability of a specific kind of UX occurring was increased for specific combination situations. 3) The balance among each kind of UX that product users anticipated having depended upon the kind of product. Therefore, 4) a minimum gap was defined as the difference between what users wanted and which product generates significant value to users. The categories were classified according to UX24.

Figure 2. Conceptual model.


4

2.2.2 The systemization of the interaction form

We set three items to define the form of the human user-system interaction, as shown in Table 3.

Table 3. Forms of human user-system interaction.

Senses Meanings of the information Presence or absence of the operation

Sight Inform state Control

Hearing Inform result Non-control

The sense of smell Inform the user of what should be done

Gustation Have user make a projection of the result

Somatesthesia Meaningless

I. Senses

Among the senses, we examined the sense of sight, hearing, smell, gustation, and somatesthesia.

II. Meanings of the information

The contents were classified and the meaning of the information that occurred at the time of interaction was determined. Inoue (2013) defined “the system image” and “the operation image” of the operational object as a mental model. He stated that it is necessary for designers to understand a users’ mental model and reflect it through design, otherwise, users cannot make good use of the products. Figure 3 shows the relationship between a mental model and a design model. Figure 3. Relationship between a mental model and a design model (Inoue, 2013) In addition, Yamaoka (2012) proposed five items to estimate the degree of the construction of a mental model. Table 4 lists these items.

Table 4. Items to estimate the degree of construct in a mental model (Yamaoka, 2012)

Item Contents Understanding of indications Be able to understand terms and contents on instruments or their

screen display Understanding of conditions Be able to understand conditions of instruments during operation Planning Be able to understand what to do to attain operational objectives Prediction of system behavior Be able to predict how to use functions and what to do during

instrument operation Interaction between system elements Be able to understand the relationship between parts of instruments

and between the layered structure of displays


5

Because our aim was not to eliminate difficulty in product use, but rather to consider the influence of positive interactions in human-products experiences, we adapted this idea to create the following classification:

a) Inform the state Information that informs the state of the home appliance, e.g., operating status or end status. b) Inform the result Information that informs the result from the use of the home appliance. c) Inform the user of what should be done Information that informs the user of what actions to take. d) Have the user make a projection of the result Information that has the user make a projection of the result. e) Meaningless Information that does not have meaning, such as the above options, or a meaningless user experience.

III. Presence or absence of the operation

We classified the collected answers as to whether the user operated the home appliance (e.g., pushed a button, lifted the lid) or not. This classification clarifies the starting point of human-product interaction.

2.2.3 Determination of the relationship between interaction forms and the kind of UX

We used 280 survey answers, collected till January 16, 2015, to associate interaction forms with each kind of UX. We covered cases where the same combination of the interaction elements occurred for no more than 10 of the 280 conditions of interaction. We then calculated the conditional probability of each kind of UX under each conditions of interaction (Ai) by the following formula: 𝑃𝑈𝑋𝑗𝐴𝑖=𝑡ℎ𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑎𝑛𝑠𝑤𝑒𝑟𝑠 𝑖𝑛𝑐𝑙𝑢𝑑𝑖𝑛𝑔 𝑈𝑋𝑗 𝑎𝑛𝑑 𝐴𝑖𝑡ℎ𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑎𝑛𝑠𝑤𝑒𝑟𝑠 𝑖𝑛𝑐𝑙𝑢𝑑𝑖𝑛𝑔 𝐴𝑖.

(1)

2.2.4 Calculation of the generative probability of each kind of UX

We hypothesized that each kind of UX generated through interaction with the entire product could be explained by summing all the human-product interactions. Therefore, we calculated the generative probability using the following formula: 𝑃𝑈𝑋𝑗=𝑃(𝑈𝑋𝑗|𝐴𝑖)𝑃(𝐴𝑖). (2) 𝑃(𝐴𝑖): Appearance ratio of conditions of interaction (i); 𝑃(𝑈𝑋𝑗|𝐴𝑖): Conditional probability of generating UXj under a condition of interaction (Ai); i=1〜 {the number of conditions of interaction}; j=1〜24 {the number of UX categories}

2.2.5 Case study: canister versus robotic vacuum cleaners We evaluated UX by examining the concordance rate between the calculated generative probability of each kind of UX and the balance of the user-anticipated UX for the product. In this study, we calculated both variables for two types of vacuum cleaners: a canister vacuum and a robotic vacuum First, a questionnaire survey was completed to clarify the users anticipated product experience. Users were presented with a list of the 24 UX categories, defined by UX24, and were asked to make multiple response selections regarding their desired experiences. The degree of anticipation for each kind of UX was calculated using the following formula. 𝑈𝑖= {𝑡ℎ𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑟𝑒𝑠𝑝𝑜𝑛𝑑𝑒𝑛𝑡𝑠 𝑡ℎ𝑎𝑡 𝑐ℎ𝑜𝑠𝑒 𝑈𝑋𝑖}𝑡ℎ𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑙𝑙 𝑟𝑒𝑠𝑝𝑜𝑛𝑑𝑒𝑛𝑡𝑠.

(3)


6

Next, the generative probability was calculated for each kind of UX using formula (1) and (2). The users’ desired UX responses were used to define a “good experience.” Finally, a quantitative UX evaluation value was produced using the following formula: 𝐸=24−𝑖=124𝑑𝑒𝑔𝑟𝑒𝑒 𝑜𝑓 𝑎𝑛𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑖𝑜𝑛 𝑓𝑜𝑟 𝑈𝑋𝑖−𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑖𝑣𝑒 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑈𝑋𝑖}. (4)

3. Results

3.1 Collected big data analysis

From the anticipated UX responses, 319 “good experience” data points were collected from October 23, 2014 to January 16, 2015. This data was manipulated after anonymization. Figure 4 shows the breakdown of these responses.

(a) (b)

(c) (d) (e) Figure 4. Breakdown of user responses: species structure of (a) home appliances, (b) UX24, (c) senses, (d) meanings of the information, and (e) control.

3.2 Relationship between conditions of interaction and the kind of UX

Table 5 shows the conditions of interaction and the conditional probability of generating UX.

Table 5. Conditions of interaction and conditional probability of generating UX.


7

3.3 Vacuum cleaner case study results

3.3.1 User-anticipated UX

Figure 5 shows that users anticipated the experiences of being recognized for their work and having a fulfilling experience when using a canister vacuum cleaner. On the other hand, users anticipated comfort and ease of use when using a robotic cleaner.

Figure 5. User-anticipated UX for canister and robotic vacuum cleaners.

3.3.2 Product-generated UX responses

After product interaction, 23 responses for canister vacuum cleaner use and 10 responses for robotic cleaner use were collected using the computer survey application, creating a baseline for a “good experience.” Figure 6 illustrates that canister vacuum cleaners generate an experience in which the user feels that they can accomplish their task briskly, while the robotic cleaner generates feelings of emotional connection, fulfillment, and comfort.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Feelin

g hear

twar

min

g

Noticin

g a

new

self

Rem

em

bering

past

good

exp

eriences

Pra

ised

by s

om

eone

Be a

ble t

o p

leas

e s

om

eone

Be a

ble t

o d

o e

asily

/ q

uic

kly

Aro

usi

ng

aspi

rations

Feelin

g em

otional

connections

Be a

ble t

o leav

e s

om

eth

ing

to

som

eone w

ith t

rust

Feelin

g vi

sual

beau

ty

Feelin

g au

dito

ry b

eau

ty

Feelin

g fu

lfill

ing

Be a

ble t

o c

oncentr

ate

Be a

ble t

o d

o w

ell

Be a

ble t

o d

o a

t tim

ing

Feelin

g com

fort

/ r

ela

xing

Feelin

g re

fresh

ingn

ess

Feelin

g fu

lfill

ment

Feelin

g be

ing

tidy

Feelin

g unusu

al luxu

ry

Be a

ble t

o im

agin

e g

ood

exp

eriences

in t

he futu

re

Bein

g m

ode

rate

am

ounts

or

degr

ee

Unde

rsta

ndi

ng

what

hav

e n

ot

been u

nde

rsto

od

Feelin

g pr

ogr

ess

es

degr

ee o

f ant

icip

atio

n

UX24

canister vacuum cleaner robotic cleaner


8

Figure 6. Product interaction-generated UX for canister and robotic vacuum cleaners.

3.3.3 The evaluation value for each vacuum cleaners

According to our calculations, using equation (4), the evaluation value is 17 for the canister vacuum cleaner and 16 for the robotic vacuum cleaner. The higher evaluation value indicates that canister vacuum cleaners fulfill more of the users’ anticipated experiences than the robotic cleaners.

4. Conclusion

In this study, we gathered and analyzed “good experiences” of vacuum cleaner use to clarify the positive effects that interactions between human and home appliances have on UX. We developed a mobile application to efficiently collect 319 responses, which were used to define what users considered as “good experiences”. Next, we succeeded in calculating the generative probability of each kind of UX for each condition of interaction. Finally, we proposed a UX evaluation method. Using this method, we calculated both the generative probability of each kind of UX and the balance of the users’ anticipated interactions, which were then used to calculate the UX evaluation value. This method has the potential to become instrumental in future product design because it converts big data into quantitative measurement.

References

K. Burke. 1969. “A Grammar of Motives.” University of California Press, 15. Hassenzahl, M. and Tractinsky, N. 2006. “User Experience: A Research Agenda.” Behavior & Information Technology

25(2): 91–97. K. Inoue. 2013. “INTERFACEDESIGNNOKYOUKASYO (The textbook of the interface design).” Maruzen Publishing ISO9241-210. 2010. Ergonomics of Human-System Interaction–Part210: Human-Centered Design for Interactive

Systems. Geneva: International Standard Organization. W. Quesenbery and K. Brooks. 2011. “Storytelling for User Experience: Crafting Stories for Better Design.” Rosenfeld,

1–298 T. Yamaoka and T. Doi. 2012. “MENTALMODELNIMOTODUKU.DESIGNSAKUSEIGUIDELINEAN (the guideline for

design based on the mental model).” Japan Society of Kansei Engineering, 7: 109–111

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Feelin

g hear

twar

min

g

Noticin

g a

new

self

Rem

em

bering

past

good

exp

eriences

Pra

ised

by s

om

eone

Be a

ble t

o p

leas

e s

om

eone

Be a

ble t

o d

o e

asily

/ q

uic

kly

Aro

usi

ng

aspi

rations

Feelin

g em

otional

connections

Be a

ble t

o leav

e s

om

eth

ing

to

som

eone w

ith t

rust

Feelin

g vi

sual

beau

ty

Feelin

g au

dito

ry b

eau

ty

Feelin

g fu

lfill

ing

Be a

ble t

o c

oncentr

ate

Be a

ble t

o d

o w

ell

Be a

ble t

o d

o a

t tim

ing

Feelin

g com

fort

/ r

ela

xing

Feelin

g re

fresh

ingn

ess

Feelin

g fu

lfill

ment

Feelin

g be

ing

tidy

Feelin

g unusu

al luxu

ry

Be a

ble t

o im

agin

e g

ood

exp

eriences

in t

he futu

re

Bein

g m

ode

rate

am

ounts

or

degr

ee

Unde

rsta

ndi

ng

what

hav

e n

ot

been

unde

rsto

od

Feelin

g pr

ogr

ess

es ge

nera

tive

prob

abilit

y

UX24

canister vacuum cleaner robotic cleaner

quantifying user experiences for integration into a home ... · a case study of canister and...

Documents