IN DEGREE PROJECT TECHNOLOGY,FIRST CYCLE, 15 CREDITS

, STOCKHOLM SWEDEN 2019

E-wallet - designed for usability

BERCIS ARSLAN

BLENDA FRÖJDH

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

E-wallet - designed for usability

Bercis Arslan and Blenda Fröjdh

2019-06-07

Bachelor’s Thesis

Examiner

Gerald Q. Maguire Jr.

Academic adviser

Anders Västberg

KTH Royal Institute of Technology

School of Electrical Engineering and Computer Science (EECS)

Department of Communications

SE-100 44 Stockholm, Sweden

Abstract

As the use ofmobile payment applications (apps) and electronicwallets (e-wallets)

increases, so does the demand for a improved user experience when interacting

with these apps. The field of Human-Computer interaction (HCI) focuses on

the design, evaluation, and implementation of interactive computing systems for

human use. One aspect of HCI is usability, i.e., the quality of the interactions with

a product or system.

This thesis investigates how an e-wallet can be designed to provide a high level of

usability by conforming to best HCI practices and by formative evaluation using a

set of usability evaluation methods.

The research process consisted of an initial literature study and development

of a prototype, which was evaluated iteratively through Thinking-aloud-protocol

(TAP) and a combination of performance measurements and questionnaire by a

chosen test group.

By each iteration, the results of the performance measurements, as well as the

verbal data improved. The most complex or difficult task, for the test subjects

to perform, was, according to the results, Pay via manual input. All goals were

achieved for all tasks except for the performance goal of a percentage of errors

below 5%.

To conclude, it was clear that the test subjects had more trouble understanding

the concept of the e-wallet rather than navigating and completing tasks. The

difficulties lay in understanding how currencies were stored and how transactions

happened. When developing this e-wallet we noticed that the most important

issuewas tomake new functions and concepts familiar to the user through relating

it to recognizable ideas.

Keywords

Usability, usability testing, e-wallet, mobile payments, Think-aloud protocol,

performance measurements

i

Abstract

I ochmed att användning av mobila betalningslösningar (appar) och elektroniska

plånböcker (e-plånböcker) ökar, ökar även efterfrågan på en förbättrad

användarupplevelse vid interaktion med dessa appar. Området människa-

datorinteraktion (MDI) fokuserar på design, utvärdering och implementering

av interaktiva datorsystem för mänsklig användning. En aspekt av MDI är

användbarhet, dvs kvaliteten på interaktionerna med en produkt eller ett system.

Detta kandidatexamensarbete undersöker hur en e-plånbok kan utformas för

att ge en hög användbarhet genom att anpassas till MDI praxis och formativ

utvärdering av designen med hjälp av en uppsättning utvärderingsmetoder för

användbarhet.

Forskningsprocessen bestod av en litteraturstudie och utveckling av en prototoyp,

som utvärderades iterativt genom Thinking-aloud Protocol (TAP) samt en

kombination av prestationsmätningar och frågeformulär av en vald testgrupp.

Efter varje iteration förbättrades resultaten av prestationsmätningarna, såväl

som för den verbala datan. Den mest komplexa eller svåra uppgiften,

för testpersonerna att utföra, var, enligt resultaten, Betalning via manuel

inmatning. Alla mål uppnåddes för alla uppgifter förutom prestationsmålet för

en procentandel av fel under 5 %.

Avslutningsvis var det tydligt att testpersonerna fann det svårare att förstå

konceptet av e-plånboken än att navigera och slutföra uppgifterna. Svårigheterna

låg i att förstå hur valutor lagrades och hur transaktioner gick till. När vi

utvecklade den här e-plånbokenmärkte vi att den viktigaste uppgiften var att göra

nya funktioner och koncept förståerliga för användaren genom att koppla dem till

igenkännliga idéer.

Nyckelord

Användarbarhet, användarbarhetstestning, e-plånbok, mobilbetalningar, tänk-

högt protokoll, prestationsmätningar

ii

Acknowledgements

We would like to thank our examiner Gerald Q. Maguire Jr. at KTH Royal

Institute of Technology, for his support and much needed guidance during our

project. For his detailed and useful feedback, given whenever needed.

Secondly we would like to thank Henrik Gradin at Centiglobe for giving us the

opportunity to do this project.

Stockholm, June 2019

Blenda Fröjdh and Bercis Arslan

iii

Authors

Bercis Arslan, bercis@kth.se and Blenda Fröjdh, blendaf@kth.seKTH Royal Institute of Technology

Place for Project

CentiglobeStockholm, Sweden

Examiner

Gerald Q. Maguire Jr.KTH Royal Institute of Technology

Supervisor

Anders VästbergKTH Royal Institute of Technology

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.5 Benefits, Ethics, and Sustainability . . . . . . . . . . . . . . . . . . . 3

1.6 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.7 Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.8 Requirements from Centiglobe . . . . . . . . . . . . . . . . . . . . . 7

1.9 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.10 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Background 92.1 Human computer interaction . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.2 Usability evaluation . . . . . . . . . . . . . . . . . . . . . . . 10

2.1.3 Analytical modeling . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.4 Inspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.5 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.6 Inquiry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.7 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.8 Measurement of usability . . . . . . . . . . . . . . . . . . . . 13

2.1.9 Prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1.10 Design principles, guidelines, and theories . . . . . . . . . . 14

2.1.11 Task flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 E-wallet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.1 E-wallets on the market . . . . . . . . . . . . . . . . . . . . . 18

2.2.2 Transaction and payment methods . . . . . . . . . . . . . . . 18

2.2.3 Task flow of existing E-wallets . . . . . . . . . . . . . . . . . 19

2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.1 User experience of Bitcoin wallets of usability and security . 20

2.3.2 Designing mobile wallets . . . . . . . . . . . . . . . . . . . . 21

v

2.3.3 User experience of a mobile app . . . . . . . . . . . . . . . . 21

2.3.4 Acceptance of mobile wallets in Oman . . . . . . . . . . . . . 21

2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3 Method 253.1 Research process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2 Research paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.1 Test group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.2 Consent form . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.3.3 Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3.4 Sample size . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3.5 Target population . . . . . . . . . . . . . . . . . . . . . . . . 30

3.4 Experimental design/planned measurements . . . . . . . . . . . . . 31

3.4.1 Performance measurement . . . . . . . . . . . . . . . . . . . 31

3.4.2 Thinking-aloud protocol . . . . . . . . . . . . . . . . . . . . . 33

3.4.3 Test environment . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4.4 Software and Hardware to be used . . . . . . . . . . . . . . . 33

3.5 Assessing reliability and validity of the method and data collected . 34

3.5.1 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5.2 Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6 Planned Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.7 Evaluation framework . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.7.1 Collection of data . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.7.2 Evaluation of data . . . . . . . . . . . . . . . . . . . . . . . . 37

4 The Work 414.1 Tasks to be analyzed . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4.2 Test group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.3 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.4 Data collection and analysis . . . . . . . . . . . . . . . . . . . . . . . 43

4.5 The prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5 Result and Analysis 45

vi

5.1 Results iteration 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.1.1 Changes after iteration 1 . . . . . . . . . . . . . . . . . . . . . 46

5.2 Results iteration 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.2.1 Changes after iteration 2 . . . . . . . . . . . . . . . . . . . . 48

5.3 Results iteration 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.3.1 Changes after iterations 3 . . . . . . . . . . . . . . . . . . . . 50

5.4 Results iteration 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.5 Reliability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.6 Validity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6 Conclusions and Future work 576.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

6.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

6.4 Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

References 62

A First Appendix 69

B Second Appendix 71

C Third Appendix 72

D Fourth Appendix 78

vii

List of figures

2.1 Example of the task flow of buying milk . . . . . . . . . . . . . . . . 17

2.2 Task flow of Paypal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.1 SEQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.1 Example of changes made to the prototype between iteration 1 and

iteration 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.2 Example of changesmade on the prototype between iteration 2 and

iteration 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.3 Example of changesmade on the prototype between iteration 3 and

iteration 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

A.1 Task flow of Swish . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

A.2 Task flow of Paypal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

C.1 Task instructions for iteration 1 . . . . . . . . . . . . . . . . . . . . . 72

C.2 Task instructions for iteration 2 . . . . . . . . . . . . . . . . . . . . 74

C.3 Task instructions for iteration 3 4 . . . . . . . . . . . . . . . . . . . 76

viii

List of tables

3.1 The performance measurements. . . . . . . . . . . . . . . . . . . . . 31

3.2 The performance goals. . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.1 The test subjects and their perceived experience using mobile

payment apps and experience of software development. . . . . . . . 42

5.1 Median value of performance measurements iteration 1 . . . . . . . 46

5.2 Median value of performance measurements iteration 2 . . . . . . . 48

5.3 Median value of performance measurements iteration 3 . . . . . . . 50

5.4 Median value of performance measurements iteration 4 . . . . . . . 52

D.1 Exerpt of concept coded verbal data . . . . . . . . . . . . . . . . . . 78

ix

List of acronyms and abbreviations

App Application

E-wallet Electronic Wallet

HCI Human-Computer Interaction

High-fi High Fidelity

ICF Informed consent form

ICT Information and Communication Technology

Low-fi Low fidelity

Nav Navigation

NFC Near Field Communication

QR Quick Response

RPA Referring Phrase analysis

SDGs Sustainable Development Goals

SEQ Single Ease Question

TAP Think-aloud protocol

UE Usability evaluation

UEM Usability evaluation method

UI User interaction

UN United Nations

x

1 Introduction

The demand for improved user experience when interacting with computers has

grown and developed into an important multidisciplinary field named Human-

Computer Interaction (HCI). This field consists of material from computer

science, psychology, ergonomics, and many more subject areas [6]. Today

electronic payment systems deployed on electronic wallets (e-wallets) are

becoming more common [46]. In this thesis, we investigate how an e-wallet can

be designed to ensure that the user’s experience will be as pleasing as possible and

will do so by applying HCI best practices and evaluation methods.

1.1 Background

The shift from analogue technology to digital technology, better known as ”The

Digital Revolution” has had an impact on our lives and enabled new possibilities

in different activities of society. One of these activities is electronic purchasing and

the digitization of money 1. This digitization introduces the need for e-wallets as

the next step in the digital revolution, as part of the transformation fromaphysical

wallet or plastic card payments to an all-electronic payment system [43].

An e-wallet is defined as a digital system that enables a user to perform electronic

transactions, including but not limited to, purchasing from (for example a store),

transferring money, receiving money, etc. Not only can monetary value be stored

but there is also the possibility to store ID documents, driver licences, and other

information that would normally be stored as cards in a wallet [26].

There are two important components of an e-wallet: the software and the

information. The information is stored in a database containing names, credit

cards, and payment methods. The software component handles the personal

information of the user and provides security through encryption of data.

Furthermore, it is possible to transfermoney via several techniques, such asQuick

Response (QR codes), Near Field Communication (NFC), Bluetooth, etc [39] [14].

1Throughout this thesis we will use the term money in its broadest sense, i.e., as a record thathas some value that can be used in an exchange

1

At the time of writing this thesis, there are some existing e-wallets in the market,

each of which are used (some more frequently than others). This raises the

importance of examining how users use these applications (apps) and whether

these apps are designed in a way that makes it easy for people to interact with

them or not. This is where the field of HCI comes into play.

One aspect of HCI is the usability of a system. Usability refers to the ease of

access or ease of use of a product, and it is the features and context of the use

that determine an app’s level of usability [62].

Centiglobe is a trading company [9] that wants to develop a mobile payment app

(specifically an e-wallet), which enables payment with cryptocurrency as well as

international payments and has tasked us with researching how amobile payment

app can be designed that conforms to HCI practices, specifically usability.

1.2 Problem

As e-wallets are a fairly new invention, there is a lot of research still needed on

the usability (user experience) of these apps. This thesis will investigate how

an e-wallet can be designed to provide a high level of usability by conforming

to best HCI practices (from a usability perspective) as well as try to answer user

experience questions that can occur when designing new and innovative apps that

use unconventional technology. The question we want to answer is:

• How should a mobile payment app be designed in order to improve its

usability, with regards to a set of pre-defined usability criteria?

1.3 Purpose

The purpose of this bachelor’s thesis project is to improve the usability of e-wallets

and ultimately contribute to an improved and optimized user experience when

usingE-wallets. The overall aimof this thesis is to present and discuss the findings

from user testing of a series of prototypes of a mobile payment app.

The aimof the project is to create a prototype of amobile payment app and through

2

iterative user testing, improve the prototype from a usability standpoint. The

resulting prototype will be the basis for a finished mobile payment app, to be

developed by Centiglobe.

1.4 Goal

The end goal of the project is to produce a prototype of a mobile payment app

with a design based on HCI principles. This prototype will be evaluated through

usability evaluation. This goal has been divided into the following six sub-goals:

• Gather information about previous usability research related to mobile

payment apps, existingmobile apps, and usability evaluation practices. This

will be done by conducting a literature study.

• Choose an evaluation method for user testing.

• Define measures through which to evaluate the usability of the design.

• Create a series of prototypes of a mobile payment app.

• Conduct iterative user-based usability testing on these prototypes.

• Iteratively improve the prototypes based on the evaluation of the previous

prototypes.

1.5 Benefits, Ethics, and Sustainability

Sustainable development implies the development that meets the needs of the

present without compromising the ability of future generations to meet their

own needs [8]. Three aspects are included: ecological, economic, and social

sustainability. There are a number of Sustainable Development Goals (SDGs),

defined by the United Nations (UN), aiming to increase sustainable development,

globally [38].

The result of this thesis might contribute to the design of mobile payment apps,

from a usability perspective. Hopefully, this improved design will increase the

use of such apps. An increased use of mobile payment apps, has some advantages

3

froman ecological standpoint, as its increased usemay lessen the need for physical

cards and wallets (the production of which has an impact on the environment).

The use of mobile payment apps may instead increase the use of electricity,

smartphones, etc. The benefit of this thesis from an ecological standpoint is

dependant on whether or not the reduced use of cards (and paper) make up for

the increased use of electricity and the manufacturing of the e-wallet.

Should the result of this thesis contribute to the increased use of mobile payment

apps, companies developing apps in general, but more specifically companies

developing mobile payment apps could benefit economically. As they could

improve the usability of their product, based on the results of this thesis. An

improvement in the app’s usability could lead to greater customer satisfaction and

result in a larger customer base. This, in turn, can lead to a higher rate of return

and more loyal customers. Providing economic sustainability to the companies.

E-wallets may contribute to both social sustainability and economic sustainability

for society. Because of the high mobile penetration, especially in rural and poorer

areas, a large number of people have the possibility to gain access to mobile

financial services [24, pp. 12-14]which theymaynot have had access to previously,

because of geographical location. Access to financial services contributes to the

reduction of vulnerability to economic, social, and environmental shocks [51].

Thus increasing the resilience of the poor and contributing to the SDG number

1, ”No Poverty”.

Furthermore, increased access to financial services, through mobile payments,

contribute to the development of new business. Targeting SDGs number

eight, ”Decent work and Economic growth”, nine ”Industry, Innovation and

Infrastructure”, and ten ”Reduce inequality within and among countries” [51].

However, there exist some possible downsides with mobile payment. Firstly,

many of themobile apps available rely on an internet connection to function. Only

some types of transactions can be made without an internet connection. Limiting

the use of these apps to people with funds for and access to internet connection

or mobile data. The need for internet also disables the use of these apps during

internet blackouts. However, some developing and existing mobile apps are not

reliant on internet connectivity, as they offer an offline-mode; for example, by

4

using tokens that can be used offline, but downloaded online [47, 57]. There

are also a number of security risk with e-wallets. Since phones are vulnerable to

malware and hacking, there exists a risk of sensitive financial information ending

up in the wrong hands, when using mobile payment apps [60].

1.6 Methodology

When conducting a degree project, the choice and use of methods and

methodologies are important. Methods and methodologies are tools through

which to assure the quality of the research. They also help guide the work and

help ensure proper and well founded results. There are a variety of methods and

methodologies from which to choose. The choice of methods and methodology

must match the research actually conducted in order to have any effect on the

work [23]. There are two categories of methods: quantitative and qualitative. A

quantitativemethod aims to prove a phenomenon by objectivemeasurements and

the statistical, mathematical, or numerical analysis of large sets of data [33]. In

contrast, a qualitative method is concerned with gathering non-numerical data

while studying a phenomenon or artefact in order to create theories or products

by examining the environment [23].

Within these categories, there are different research methods from which to

choose. These methods determine how the research process is conducted. A

few examples of methods are empirical research methods and analytical research

methods. Using an analytical research method, pre-planned hypotheses are

tested based on existing knowledge and findings. Using an empirical research

method hypothesis are instead tested based upon experiments, observations, and

experiences [23].

The method used in this degree project will be both quantitative and qualitative.

Some of the data will be quantitative, 5 seconds to do a given task, 4 errors when

doing the task, 3 negative comments etc. Additionally, some of the data will be

qualitative, content of comments from the user. However, as the test group is

small data is not expected to have statistical significance. To test the usability and

gather knowledge regarding the design, tests will be conducted. Since it is through

5

testing and observations that data will be gathered, the research method used is

an empirical research method.

To ensure sufficient background knowledge in the area, a literature study will

be conducted. The literature study will explore existing usability research about

mobile payment apps, the design of current mobile payment apps, and HCI

practices. Based on this literature study and grounded theory in usability we will

choose an evaluation method and a set of usability measurements. A prototype of

the app’s design will be created. The design will be evaluated through iterative

testing where tests subjects will comment on the usability of the design and

performance measurements will be measured.

1.7 Stakeholders

The stakeholders for this degree project are first and foremost Centiglobe (the

company that will market, sell, and profit from the end result, a potential design

of an e-wallet mobile app). They will be directly affected by this thesis since the

design of the app candirectly or indirectly potentially affect their profits and future

development.

Furthermore, the finished app could be sold to other companies as a white label

app. Enabling these other companies to adopt the app and adapt it to realize

their own product. The finished app is intended to be used together with other

existing products and companies, such as other mobile payment apps and banks.

These potential customers will be indirectly affected by the design of the app and

therefore the outcome of this project.

Further stakeholders also include potential users of the app. The app’s design is

directly affected by the outcome of this thesis project. The target users for the app

are expected to have all sorts of different backgrounds, ages, and nationalities.

However, the early adopters will most likely be younger, technically skilled

persons.

6

1.8 Requirements from Centiglobe

The requirements on the app from Centiglobe is that it has to be suited for

international use, have the ability to perform transactions to other users, (not

limited to users of the same app) and it should support several currencies

(cryptocurrencies as well).

1.9 Delimitations

We will only use a small test group as we do not have access to a large number of

people. Therefore, our results are not statistically reliable and do not necessarily

apply to our demographic of target users outside that of our test group. The

test group used is not an accurate representation of the target users of the final

product. Our results will only be applicable for the test group used. Furthermore,

due to limited resources, the same test group will be used for all iterations.

Because of time constraints we will only test some specific functions and not

present a fully developed app. This means that a high-fidelity (high-fi) prototype

will not be presented. We will only draw conclusions regarding the tested

prototypes.

The usability of the prototypes will only be tested based on a set of pre-defined

measurements hence there could be some aspects of usability that we could not

take into consideration since we had limitations in time and other resources.

Furthermore, the focus of the usability testing is discoveringmajor problems with

the design of the prototype. Therefore, no major effort will be put into finding

minor usability problems with the design.

A widely discussed topic in conjunction with e-wallets is the security level

of the apps. Studies show that users’ experience a lack of privacy and

confidentiality in transaction information and are therefore reluctant to perform

online transactions [54]. However, this is something that will not be discussed

further in our thesis since it is outside of our scope. On the other hand, if the

design or functionality of an e-wallet app can affect the user’s sense of security

this will be highly relevant to discuss.

7

1.10 Outline

The following chapter in this thesis contains an in-depth theoretical background

to explain what HCI is and what methods can be used when evaluating apps.

Different e-wallets on the market and what technology and task flows exist will

also be presented. The method and methodology used in our study are presented

in Chapter 3. This chapter covers how the research process and data collection

was performed. The validity and reliability of the method are also discussed. This

is followed by a description of the implementation and results in Chapters 4 and 5

(respectively). Finally, a discussion of our work will be given and conclusions will

be presented in Chapter 6.

8

2 Background

In this chapter, we present a detailed description of the background areas relevant

to answering our thesis questions involving HCI theories and practices as well as

describe how e-wallets work and inspect some of the apps currently on themarket.

2.1 Human computer interaction

HCI is the interdisciplinary study of interactions between humans, that is users,

and information technology design [58]. HCI is about designing interfaces in a

human-centredway, taking account of humanabilities andpreferences. It ensures

that systems are accessible, usable, and acceptable [6]. It encompasses several

areas of research, such as computer science, cognitive science, and human factors

engineering [58].

2.1.1 Usability

There are several ways to test if a design is ”good”, from an HCI standpoint. One

such way is testing the usability of the system design. There is not a set definition

of usability. However, the formal definition of usability from the ISO 9241-11

standards is ”The extent to which a product can be used by specified users to

achieve specified goalswith efficiency, effectiveness, and satisfaction in a specified

context of use” [28, p. 6]. A shorter definition is that usability is the quality of

the interactions with a product or system [6, p. 77]. As the definition of usability

can differ so can the parameters that define the usability. Some models include

parameters such as how safe the system is to operate in the context it will be used

[6, p. 81]. Others define the parameters as accessibility, clarity, learnability, and

feedback [12]. In the ISO model, however, the parameters are those ones stated

in the definition of usability, i.e., efficiency, effectiveness, and satisfaction.

In keeping with the definitions of the three parameters as given by ISO 9241-11

[28, pp. 9-12], efficiency is the resources used in relation to the results achieved,

effectiveness is the accuracy and completeness with which users achieve specified

9

goals, and satisfaction is the extent to which the user’s physical, cognitive, and

emotional responses that result from the use of a system, product, or service meet

the user’s needs and expectations.

2.1.2 Usability evaluation

When evaluating the usability of a system different usability evaluations methods

(UEM) are used. These can be divided into different categories. Ivory and Herst

(2001) propose the following five classes and definitions [29]:

Analytical modelling

an evaluator employs user and interface models to generate usability

predictions.

Inspection

an evaluator uses a set of criteria of heuristics to identify potential usability

problems in an interface.

Inquiry

users provide feedback on an interface via interviews, surveys, and the like.

Simulation

an evaluator employs user and interface models to mimic a user interacting

with an interface and reports the results of this interaction.

Testing

an evaluator observes users interacting with an interface to determine

usability problems

Depending onwhether user participation is required or whether expert evaluators

are employed, a UEM is either user-based or expert-based. Expert-based UEMs

all include the involvement of an expert in the field of usability or the design of

interactive systems. In contrast, user-based UEMs instead employ a group of

people, preferably representative of the target users, to evaluate usability [34,

p. 256]. UEMs from the analytical, inspection, and simulation class can all be

expert-based. However, testing and inquiryUEMs are dependant on users and are

therefore user-basedmethods [29]. Furthermore, aUEMcanbe either summative

10

or formative in nature. Each of these two types of testing is more appropriate

for different stages in a product’s development and purpose of evaluation [34,

p. 260]. Formative evaluation is best suited for the early stages of development.

In this case, testing is a part of the iterative design process, to explore designs that

are or are not usable. Formative testing is exploratory in nature and the focus

is on qualitative feedback and moderator observation. In contrast, summative

evaluation involves measuring the usability of a specific design choice. The focus

is onmetrics and quantitativemeasurements [48]. In short formative testing is for

discoveringwhat needs to be improved, while summative testing exploreswhether

the improvementswere successful. For example, according toMelodyY. Ivory and

Marti A. Hearst are UEMs in the testing, inspection and inquiry are best used for

evaluations that are formative in nature, while analytical and simulation methods

are summative [29].

2.1.3 Analytical modeling

Based on some representation or model of the UI and/or the user, analytical

modelling methods enable an evaluator to predict the usability of the UI, i.e., the

user’s performance when interacting with the UI [30].

One method, classified as analytical modelling, is GOMS analysis. This method

developed by David Kieras is based upon evaluating Goals, Operators, Methods,

and Selection rules [32]. AGOMSmodel specifies a set ofmethods that are used to

accomplish specific goals. Themethods are composed of operators. Operators are

steps that a user performs. Steps that are assigned an execution time. If several

methods can be used to achieve a goal, then selection rules are used to decide on

the correct method. Based on this model predictions of how users will use the

modelled system can be made.

2.1.4 Inspection

Inspection includes evaluation methods where experts examine the usability of

an interface based on a set of guidelines that range from very detailed to broad

descriptions of the guidelines [30]. However, studies show that the inspection

11

methods are not always useful due to the fact that designers can be biased

towards aesthetically pleasing interfaces instead ofmeasuring the efficiency of the

design [52]. Examples of inspectionmethods are cognitive walkthrough, heuristic

evaluation, and guideline review.

2.1.5 Simulation

With simulation, it is possible to mimic a user interacting with an interface

with the help of computer programs. The simulations are done in a controlled

environment where the values of parameters can be chosen for each simulation.

This provides the designer with quantitative data which can be easy to interpret.

2.1.6 Inquiry

Methods that are categorized as inquiries include field observations, user

feedback, interviews, and questionnaires. When conducting evaluations using

these methods, the goal is not to test performance but rather collect data on

opinions. These methods are especially relevant in early stages of product

development but also after a product has been released in order to collect feedback

as can be done when conducting for example field observations [30].

2.1.7 Testing

Testing methods are the fundamental way of knowing how humans are going

to interact with a certain interface since participants are those who will test the

product. The participants will use a prototype or a system where the goal is for

them to complete a task given by the tester who will record the results and act on

them.

Testing methods include a think-aloud protocol (TAP) where the participant talks

during testing, performancemeasurement where the tester records the usage data

during the test, and coaching where the participant can ask the tester questions

about the interface.

12

When using TAP, for usability evaluation, the comments can be audiotaped and

then transcribed. The transcription can be analyzed in multiple different ways.

One way is analyzing through three progressive steps: referring phrase analysis

(RPA), assertional analysis, and script analysis. In RPA each phrase is coded,

with names of concepts, based on the category of words contained in the phrase.

Concepts are categories based on the meaning of a word. For example, one

concept can be ”value” and refers to all words that are a rating or scaling of

usefulness, importance, or worth [21].

2.1.8 Measurement of usability

When measuring usability through usability evaluation, both performance and

subjective measures can be used. Performance measures are quantitative

measures that are observed. Such measurements can be the time it takes for a

user to learn how to perform a specific function, the rate of errors that occur

during use, the speed of task performance, or the number of observations of

frustration. Subjective measures are instead based on the subjective opinions of

the test subjects. They can be both qualitative and quantitative. This data can

be collected in the form of user comments or a user’s rating on a scale. The data

can be gathered a number of different ways. Some examples are surveys or a test

moderator observing the test subject as the subject performs certain tasks [17,

pp. 184-188].

When evaluating the three parameters for usability, as defined by ISO 9241-

11:2018 [28], some metrics of effectiveness are percentage of goals achieved,

functions learned, and number of errors. Some metrics for efficiency are the time

to complete a task, learning time, and time spent correcting errors. Lastly, some

metrics for the measurement of satisfaction include ratings for satisfaction, ease

of learning, and error handling [30, p. 7].

Measuring satisfaction using the rating of satisfaction can be done through

different types of questionnaires. These questionnaires can be distributed at

the end of a test, measuring the satisfaction of the entire system or they can be

distributed at the end of each task, measuring the satisfaction of that specific task.

13

One such post-task questionnaire is Single Ease Question (SEQ). It consists of one

single question that refers to how easy or difficult participants think a task is to

perform. The respondent can rate the difficulty on a scale ranging from one to

seven. Where one is ”Very difficult” and seven corresponds to ”Very easy” [50].

The time it takes before a user gets frustrated is about 1 minute to perform one

task. If a system requires more than one minute to perform a task, it is likely that

the site will be abandoned [44].

According toMeasuringU [35], the average result when testing the difficulty of a

task is between 4.8-5.1 on a 7 level Likert scale.

2.1.9 Prototypes

A prototype of a product is an early sample or model of a product. Prototypes

are created to test a concept or a process [7]. When designing and evaluating

interactive products, prototyping is heavily relied on. Producing a prototype, of a

future product, provides the opportunity to experiment with alternative designs,

fix any problems that might occur, and provide a conceptual idea of the product

that can be used during testing. Since a prototype is relatively easy to change the

design can quickly be adjusted according to the results of testing. Depending on

the level of detail, a prototype has different degrees of fidelity in relation to the

final product [16]. Ranging from a low fidelity (low-fi) prototypes to high fidelity

(high-fi) prototypes.

2.1.10 Design principles, guidelines, and theories

There are several different guidelines andprinciples for the development of “good”

interactive products from a human interaction design standpoint. Some of these

principles are given in Ben Shneiderman’s eight golden rules of interface design.

If a design aligns with these principles, then these are the strengths, while those

that violate it will be the weaknesses. These rules [17] are:

• Strive for consistency,

• Enable frequent users to use shortcuts,

14

• Offer informative feedback,

• Design dialogues to yield closure,

• Offer simple error handling,

• Permit easy reversal of actions,

• Support internal locus of control, and

• Reduce short term memory load.

Additionally, there exist several other well known rules of thumb, such as Don

Norman’s six design principles and JakobNielsen’s 10 usability heuristics for user

interface design. For example, Nielsen’s 10 heuristics are defined as [40]:

• Visibility of system status

Give the user appropriate feedback, in reasonable time.

• Match between system and the real world

Use language that user understands instead of system-oriented terms.

• User control and freedom

Support undo and redo, a user should not have to go through several steps

when a mistake has occurred.

• Consistency and standards

Follow conventions, thus do not use ”bye” and ”exit” interchangeably, rather

stick to one of them.

• Error prevention

Try not to put the user in situations prone to error. Present users with a

confirmation option before committing an action.

• Recognition rather than recall

Minimize users’memory load. Make actions visible so the user does not have

to remember them.

15

• Flexibility and efficiency of use

Customize the interface for both the novice and the experienced user, for

example with shortcuts or actions that speed up the interaction for expert

users.

• Aesthetic and minimalist design

Do not present irrelevant information since it competes with relevant

information.

• Help user recognize, diagnose, and recover from errors

Use error messages that are easy to understand, and suggests solutions to

the problem.

• Help and documentation

The system should preferably be manageable without documentation but if

this documentation exists, it should be easy to find, be concrete, and not too

large.

Guidelines similar to the ones named above can in broad terms be summarized

as striving for consistency, give the user control, and be aware of a user’s limited

memory.

One way to conform to the principles stated above is to create local rules for each

design. As there exists a lot of guidelines which sometimes can be contradictory

it is very important to specify these rules beforehand.

An example of a local rule would be that all ”back” buttons in an interface have to

be red and with a width of at least 5 percent of the window width.

2.1.11 Task flow

Before developing an app it is important to have a clear mind map of where each

button press, swipe, etc. will take you to ensure that each interaction serves

an important purpose for the user. This can be done by creating task flows. A

task flow is the sequences of steps required to perform a certain task. It is often

represented by a flow chart showing the relations between each step [5]. See the

example in Figure 2.1.

16

Figure 2.1: Example of the task flow of buying milk

2.2 E-wallet

E-wallets are a fairly new invention that in recent years have become increasingly

popular as we enter the digitization era, where a transition from physical money

and payments to electronic money and cryptocurrencies is currently taking place.

The markets for these electronic payment methods has a promising future, but

their successes are uncertain due to potential new technological inventions [15].

An e-wallet or digital wallet transforms the way people purchase and pay for

things, by changing the means of payment to be done via apps on mobile phones

[26]. All information that is stored in a wallet is encrypted through the use of

public and private key-pairs to ensure that payments and other data are handled

securely.

There are wallets for conventional currencies as well as for cryptocurrencies (such

as) Bitcoin, which require the same functionality, i.e., the ability to perform

transactions, check balance, etc. As there exist various e-wallets and the goal of

this thesis is to investigate how one can be designed from a usability standpoint,

it is desirable to present and compare e-wallets currently on the market. This

comparison is done in the following subsections.

17

2.2.1 E-wallets on the market

It is important to be aware of existing apps, what functions are present, and to

understand their design when developing an e-wallet yourself. The agreements

between the e-wallet app and banks or other involved entities can limit the

user group for the app. Some e-wallets are location specific and cannot be

used overseas due to transactions only being enabled with country specific

requirements and some are only available to users who are customers of a defined

set of banks. Nonetheless, a desire for global use is not unusual.

Some examples of e-wallets are: the Swedishmobile app Swish [55],multinational

Apple Pay [4], EcoCash app for the Zimbabwean market [36], and the Chinese

apps WeChat Pay [61] and Alipay [2].

2.2.2 Transaction and payment methods

There are a few communication styles to choose fromwhen developing an e-wallet

app. The selected communication style refers to the technical functionality that

is chosen when a user of an e-wallet wants to transfer money to another account.

Different wallets have adopted different technical functionality and some of these

will be presented in this subsection.

Swish is a smartphone app consisting of minimal functions that enable receiving

and transferring money through a phone number connected to the bank. Both

users have to have their phone numbers known to Swish through their bank,

then subsequently the bank performs a real-time money transfer between the

associated accounts of these users [56]. When the party sending money has

confirmed the transaction, the money has been transferred. The user also needs

to have an electronic identification app that is only available to Swedish citizens.

Swish was developed for companies. It has the possibility to provide payment

information through aQR code [55] (a two-dimensional black andwhite code that

can be read by machines). A camera app can be used to decode and evaluate the

information encoded in the QR-code as cleartext [45].

18

Apple Pay is a bit different from Swish because its main feature is electronic

purchases using any of the Apple products, but it does not supportmoney transfer.

It is currently only possible to transfermoney in the USA through a feature named

Apple Pay Cash. Apple Pay is using wireless payment via NFC.

NFC is an umbrella term for techniques using wireless data transfer for distances

shorter than 10 cm. Smartphones can through peer-to-peer communication

communicate with anything that has an NFC interface, thus they can receive

information from the payment terminal and this information enables purchases.

With the EcoCash app, a user can transfer money, or make deposits and

withdrawals of money [1]. The wallet links a user’s bank account and phone

number. To transfermoney the user needs the receiver’s phone number. EcoCash

is in partnership with Cassava Remit, which together enables money transfers

from the UK to Zimbabwe.

WeChat Pay is a Chinese messaging and social media app with the added

functionality of payment services. The app supports multiple payment methods

including QR code scanning for purchasing and smoother transactions. In-app

payments are also available [37]. In-app payments are payments made from

within the app. With in-app payments, the user can choose the payment method

directly inside the app instead of being redirected to another application or web

page.

Alipay is another popular e-wallet app that serves the Chinese market where

transactions can be made internationally by Chinese customers. Similar to

WeChat Pay, this app supports QR code scanning for local in-store payments [11]

and other services such as bank account management and peer-to-peer transfer

(money transfer without the need of a bank, i.e. money transfer directly between

two users).

2.2.3 Task flow of existing E-wallets

Many of the E-wallets in the market have similar functions. However, the task

flow within these apps differ. A functional requirement for the app to be designed

is, amongst other things, to facilitate international transactions as well as the

19

exchange of currency. For this project, the most relevant mobile payment apps to

consider are therefore apps actively used in different countries, apps that facilitate

international payments, and apps that facilitate payment with cryptocurrency.

Appswith different communicationmethods, for example, NFC, QR code, are also

relevant. As noted earlier, the task flow of an app can be presented in the form of

a flow chart. The apps, whose task flow and/or overall design is considered in

this report, are Swish, Paypal, Alipay, WeChat, and Cassava Remit. The flowchart

for Swish and Paypal are presented in Appendix A. The flowchart for Paypal is

also presented in Figure 2.2. These five apps represent a small percentage of

the mobile payment apps available, namely less than 2 percent. At least 100 e-

wallet existed 2016 according to the list made by [59], and [19] listed the 70 best

cryptocurrency wallets of 2019 together being at least 200 e-wallets. However,

there is a possibility that even more exists. In Figure 2.2 is the task flow of Paypal

shown.

2.3 Related Work

This section present previous research done on the subject of e-wallets designed

for usability.

2.3.1 User experience of Bitcoin wallets of usability and security

In Abdulla Alshamisi and Peter Andras paper ”User perception of Bitcoin

usability and security across novice users” [3], they examine how digital

payment systems with cryptocurrencies (such as Bitcoin) influence new users in

comparison to credit card payments. They used surveys to collect data about

users’ perception and concluded that the users responded more positively to

conventional credit card payments which influenced their negative perception of

e-wallet security. Ultimately they say that a deeper understanding and education

about digital payment systems is needed together with improved user-centred

designs in order to elevate each user’s experience and thus increase acceptance.

Alshamisi and Andras tested and compared the usability of payment systems with

cryptocurrencies and credit card payment using subjective measures. Their test

20

subjects completed a survey, responding to statements regarding usability. The

respondents could grade what level they agreed with each statement, through a

Likert scale.

2.3.2 Designing mobile wallets

Mia Olsen, Jonas Hedman, and Ravi Vatrapu have in their paper ”Designing

digital payment artefacts” [42] describe their scientific inquiry of a mobile wallet.

Four different user groups were identified, and then they developed and evaluated

mobile wallets from the perspective of its design. They presented sketches

and low-fi prototypes to user groups who were interviewed in order to collect

data. They concluded that there are two types of properties that are relevant

when designing a mobile wallet: (1) the functional properties and (2) the design

properties and that evaluation criteria needed to be expanded in order to take

everyday life contexts into consideration.

2.3.3 User experience of a mobile app

Fanny Chan and Sofia Johansson have in their bachelor’s thesis ”Evaluation

of user experience on a mobile application” [10] investigated if there could be

any design improvements in Shownight’s mobile app in order to increase the

quality of the user’s experience. The evaluation method they used consisted of a

combination of interviewswith the user group andmaking performancemeasures

in order to collect data. They were able to suggest improvements in the design but

left it as future work to redesign the app.

2.3.4 Acceptance of mobile wallets in Oman

Sujeet Kumar Sharma, Sachin Kumar Mangla, Sunil Luthra, and Zahran Al-Salti

in their article ”Mobile wallet inhibitors: Developing a comprehensive theory

using an integrated model” [53] reflect on what is hindering the acceptance

of mobile wallets in Oman as mobile wallets are increasingly accepted in both

developing countries and developed ones. They developed a hierarchical model

21

and concluded from it that anxiety and lack of understanding of new technology

are some of the key reasons why the promotion of the use of mobile wallets in

Oman is difficult.

2.4 Summary

This chapter has presented several HCI practices, guidelines, and models.

Usability is one of theHCI practices that ismost significant in our thesis. The three

parameters defining usability and different types of usability testing methods

(user-based, expert-based, and automated testing) were introduced. An overview

of various e-wallet apps on themarket and their task flow and transactionmethods

were also presented.

Today it is common to investigate how usability and security of the mobile wallets

affect the acceptance of apps and how amobile wallet can be designed tomaximize

its acceptance by users. The next chapter will present details of the method used

in this thesis.

22

Figure 2.2: Task flow of Paypal

23

24

3 Method

The purpose of this chapter is to provide an overview of the researchmethod used

in this thesis. Section 3.1 describes the research process. Section 3.2 details the

research paradigm. Section 3.3 focuses on the data collection techniques used

for this research. Section 3.4 describes the experimental design. Section 3.5

explains the techniques used to evaluate the reliability and validity of the data

collected. Section 3.6 describes themethod used for data analysis. Finally, Section

3.7 describes the evaluation framework used.

3.1 Research process

This subsection lists the steps conducted in order to carry out this research.

1. Literature study

At first, a literature study was made in order to gather information about

HCI practices, usability, evaluation methods and to get an overview of the

functionality of e-wallets. This was done in order to accurately determine

suitable evaluation metrics and methods. The choice of sources were

mainly books on the subject as well as relevant papers in databases such as

ScienceDirect and Scopus. The search words among others were: mobile

wallets + usability, e-wallets + usability, HCI + mobile wallets, HCI +

mobile payment. A lot of information was also found on the internet where

HCI communities and interaction design foundations share their knowledge

and experiences. The credibility of these sources was checked through a

comparison of other material and inspected to see if there were any biased

or outdated information on the sites. The primary results of the literature

study are presented in Chapter 2.

2. Determine functionality

Secondly, a decision on the required functionality of the appwasmade. A list

of functions was produced together with Centiglobe. These functions were

the foundation of the prototype.

25

3. Select usability evaluation parameters

In order to evaluate the usability of these functions, evaluation parameters

were selected. The selected parameters were the ISO 9241-11:2018 standard

parameters for usability: Efficiency, Effectiveness, and Satisfaction.

4. Create a prototype

A prototype had to be created based on Schneiderman’s eight golden rules

[17], Nielsen’s 10 heuristics [40], and from the app comparisons in the

literature study. The prototypes are presented in Appendix E.

5. Define a test group

Thereafter, a test group was constructed. The test group needed to be as

diverse as possible. The selection of this group is described in Section 3.3.1.

6. Choose evaluation methods

After the test group was constructed, the usability of the prototype was

evaluated through user-based evaluation. The UEMs that were used are

classified as a testing and inquiry UEMs (according to the classification by

Ivory and Hearst [30]). The methods we chose were TAP (testing method)

and a combination of performance measurement (testing method) and

questionnaire (inquiry). The questionnaire was used to collect performance

data regarding user satisfaction. We observed the test subjects performing

each task, having them comment during testing and measured a set of

measurements. For example, the time to perform a task. At the end

of each test, the subjects rated their satisfaction of the interaction with

the prototype. The test subjects were provided with a question regarding

the ease of performing the task and rated the ease of performance (on a

7 level Likert scale). Therefore, we gathered both subjective (TAP and

questionnaire) and objective (performance measurement) data. Since the

purpose of the usability evaluation was to develop a design of a mobile

payment app, the evaluation method was formative.

26

7. Design experiments

Based on the methods and test groups chosen we designed the experiments.

These experiments were conducted four times during individual meetings

with the test subjects in a quiet setting. Details of the experiment are given

in Section 3.4.

8. Capture data

To capture data, participants expressions and comments were audiotaped

with mobile phones as well as noted on paper. Performance data such

as time was timed and noted on paper. The user interactions with the

prototype as well as errors made were recorded using screen recording. The

questionnaire was filled out online by the participant using Google forms.

9. Analyze and interpret data

The data from testing were both quantitative and qualitative. Qualitative

data (such as the content of comments and expressions from the TAP and

the types of errorsmade) weremore difficult to analyse than the quantitative

data. The data from the TAP was analyzed through a simplified version of

RPA. The number of negative, neutral, and positive commentswere counted.

The median value of the performance measurements was calculated. The

quantitative data was only used to measure the level of usability of the task

while the qualitative data was used to find areas of improvement.

10. Critique UI to suggest improvements

After the data was analyzed, suggested improvements were based on the

qualitative data as well as the quantitative data.

11. Iterate process

The process of designing and evaluating the prototype was iterated in order

to improve its usability. For each iteration, data was captured, analyzed

and new suggested improvements were decided and implemented in the

prototype. This iterative process made it possible to find new faults with

the design and remove them.

27

3.2 Research paradigm

We are collecting data to describe the experience of using the prototype for a

certain number of people. We are also basing our conclusion on observing what

is happening in the world. We are therefore assuming that there exists some

experience that can be observed or tested empirically. Hence we are taking a

positive stance towards this phenomenon. The research paradigm that this work

adheres to is subsequently positivism.

3.3 Data collection

Data was collected during tests by several different means. The TAP data, i.e.

the verbal data, was collected through audiotape that was later transcribed.

Performance measurements data was collected using a stopwatch on a mobile

phone, to time the activity, and notes on paper. Performance data was also

recorded by screen recording. The responses to the SEQ were collected online

using Google forms.

No personal information was recorded or collected during the tests. The only

information that can be regarded as sensitive, that was collected, were the

comments expressed during the TAP. However, these cannot be tied to any one

person, since no names were recorded and therefore can’t be linked to them. The

testing process presented no risks to the participants.

3.3.1 Test group

The test group used during the user-based usability evaluation was limited to

7 people. This number of people was considered adequate since the research

performedwasmainly qualitative, to provide insight into the design, togetherwith

the fact that most usability problems can be discovered with a test group of five

people [41]. Furthermore, because of limitations in resources, the same test group

was used for all iterations. Since the target users are not limited to any specific

group it was important to construct a diverse group. However, we had limited

28

resources when choosing test subjects. Because of this, the difference in the test

subjects’ characteristics was limited to:

• Experience using mobile payment apps.

• Experience of software development.

We chose ”experience using mobile payment apps” as one characteristic because

it could affect how the user interacts and experience the prototype. A user with

experience using mobile payment apps is used to performing similar actions that

were performed during testing. This could affect the speed of performance as

well as the number of errors they make. Furthermore, can they be more critical

as they can compare the prototype to other mobile payment apps. A user’s

experience of software development can possibly also affect how they experience

and interact with the prototype. A software developer may view the prototype

from a different perspective and be more technically skilled. The test subjects

rated their experience usingmobile payment apps and their experience of software

development on a 5 level Likert scale, respectively. Level 1 being ”no experience”

and level 5 being ”extensive experience”.

The language that was used for the instructions and the ICF for the participants

were in English, but any verbal interaction was done in Swedish. The language

used in the app prototype is in English.

3.3.2 Consent form

The ICF was designed by us, but it was partly based on a sample ICF provided

by the World Health Organisation [63]. Our goal was to obtain informed consent

from our test subjects. Therefore we designed a comprehensive consent form,

providing the test subjects with information about what the tests entailed. As

advised by Joseph S. Dumas and Janice C. Redish [17], in our ICF we explained

the procedure that we would follow, the purpose of the test, any risks to the

participants, the opportunity to ask questions, and the opportunity to withdraw

at any time. Furthermore, we made sure that the participants comprehended the

information by using clear language. The test subjects were not offered anything

in exchange for participating. All test subjects had reached the age of majority,

29

i.e., 18 years old, in Sweden and were clear of mind.

The ICF is presented in Appendix B.

3.3.3 Sampling

As the test group was limited in size, all data collected from the TAP testing as

well as data from the performance measurements was needed when iterating and

improving the prototype. Therefore, no sampling was done.

3.3.4 Sample size

The size of the test group, responses during TAP, responses to the questionnaire

and the performance measurements taken determined the sample size. The

number of performance measurements and responses to the questionnaire were

static in size. However, as we used TAP as one testing method, the total sample

size varied with each iteration of testing and for each participant. Because the

number of comments and thoughts expressed varied for each iteration and for

each participant.

3.3.5 Target population

Since themain functionality of the app is the transaction ofmoney, a functionality

not limited to any specific group, and the app is intended for international use,

the target population was not limited. However, to what extent the app will be

adopted is expected to vary across groups. For example, early adopters will most

likely be younger, tech-savvy users. Furthermore, the app is limited to owners of

smartphones. The app will also require access to a bank account of some sort,

this will affect the minimum age of the users, as many banks have a minimum age

limit. There is no age limit to owning cryptocurrency. However, there is often an

age limit of 18 when selling and buying on trade sites [13].

30

3.4 Experimental design/planned measurements

The subjects were scheduled for individual meetings in a quiet setting. To

facilitate using TAP. The sessions were audiotaped and then transcribed. After

subjects signed an informed consent form (ICF), described in Section 3.3.2, they

were randomly given an instruction and scenario card (with limited instruction

regarding what task to perform and the goal to achieve). Furthermore, the test

subjects were informed that performance measurements were to be performed

and that they were encouraged to constantly talk and think aloud as they

performed the task. If the test subjects were quiet for more than a few seconds

we gently reminded them to ”keep thinking out loud”. The test subjects then tried

to perform the specific task on the interactive prototype that was displayed on a

smartphone. At the end of each task, each test subject answered the questionnaire

concerning the specific task. During the test, performance measurements were

also measured and notes were taken. The interactions between us, as observers,

were limited during the performance of tasks. Only if a reminder, to keep thinking

aloud, was needed did we intervene.

We evaluated the usability of the prototype according to the ISO 9241-11:2018

definition of usability. Each parameter (efficiency, effectiveness, and satisfaction)

was evaluated using both TAP and performance measurements in combination

with the SEQ.

3.4.1 Performance measurement

The performance measurements we chose for each parameter can be found in

Table 3.1.

Table 3.1: The performance measurements.

Performance measurements

Efficiency Effectiveness Satisfaction

Time to complete a task Percentage of errors Questionnaire

31

The efficiency was measured through the time to complete a task which is defined

as:

Tasktime = Endtime− Starttime (1)

The time was measured with a stopwatch on a mobile phone. When we asked

the participant to start, the stopwatch was started, and when the participant was

finished with the task and stopped interacting with the prototype the watch was

stopped.

The percentage of errors the participant makes when performing a task is the

chosen effectivenessmeasure. Errors are defined as slips, mistakes or unintended

actions. The screen and the user’s interaction, during the interaction with the

prototype, was recorded. Based on this recording the number of errors and

the total amount of interaction the user had with the prototype were noted.

Interaction is in this case defined as any interaction with the touch screen such as

press, swipe, etc. To calculate the percentage of interactions that were mistakes,

for each task, the following formula was used:

Number of errors

Total number of interactions(2)

The satisfaction of each task was measured through an SEQ. After each task, the

subjects were given the questionnaire. In the questionnaire, the test subjects rated

how difficult a task was to perform on a 7 level Likert scale. The single question

questionnaire, the SEQ, is presented in Figure 3.1 below.

Figure 3.1: SEQ

32

3.4.2 Thinking-aloud protocol

The comments made during the usability evaluation were, as stated above,

audiotaped and then transcribed. Then sections of the transcript that were

identified as not reflecting verbal thoughts, such as when subjects were given or

reading task instructions, and filler words, such as ”um”, ”ah” etc, were eliminated

from the transcript. The transcript was then analyzed using a simpler version

of RPA. A set of concepts (categories of ideas) were defined based on the verbal

data. The phrases were concept coded based on the category of the idea a phrase

expressed. The concept codedphraseswere then categorized into these categories:

negative, positive, and neutral, based on whether the phrase expressed a positive,

negative, or neutral idea.

3.4.3 Test environment

Each participant decided the place to conduct the testing, meaning that the testing

environment was not the same for each participant. The only demand on our side

was that the environment had to be quiet so it would be easier to focus and not

being disturbed by otherswhen doing the testing. We chose a group roomat either

the Royal Institute of Technology in Kista or Campus if a participant wanted us to

decide where the testing should take place.

The rooms had to have at least one chair for the participant and a table where a

laptop for the questionnaire was placed along with the prototype. The prototype

was developed with the software ”Figma” displayed on a Huawei P20 (further

explained in Section 3.4.4). Other materials that were needed were pens, a

notebook, a laptop for taking notes and a mobile phone with a voice recorder and

a stopwatch. To record the screen during testing software available on theHuawei

P20 was used.

3.4.4 Software and Hardware to be used

The software used for prototyping was ”Figma”. Figma is a freemium, browser-

based interface design application. Among other things, a feature of Figma is

33

making designs interactive. Meaning that the design can mimic the functions of

a real application and respond to the user’s interactions. For example, pushing a

button, in the design, can prompt a new view to appear or one can scroll on a page.

In presentation mode, only the interface is shown.

Furthermore, will the Figma mirror app be used to present the app on the phone.

With presentation mode the prototype can be viewed on a smartphone taking up

the whole screen of the phone, mimicking a real application [20].

The smartphonemodel used to display the prototype wasHuawei P20. The phone

had a screen size of 5,8 inches (measured diagonally) [27]. Making it a good

representation of the average smartphone. Themost popular smartphone’s screen

size varying between 4,7 to 6,5 inches [31].

The software used for recording the screen and users interaction was a built-

in screen recorder found on the Huawei P20. The software used for the

questionnaire was Google forms. The software used to calculate the median value

of the performance measurements was excel.

3.5 Assessing reliability and validity of the method and datacollected

The reliability (Section 3.5.1) and validity (Section 3.5.2) of our results will be

discussed down below as well as the reliability of the chosen method.

3.5.1 Reliability

In order to get reliable data from the TAP, we chose to audiotape the conversation

tominimize the disturbance to be able to transcribe when the testing was finished,

which otherwise can affect the results.

Furthermore, to get reliable results it is important that the analysis of the

transcription is done consistently. Same expressions should have the same value

for each participant and iteration. When analyzing the transcription it is necessary

34

to be unbiased and that we do not draw conclusions in order to accept a certain

hypothesis [22].

The methods we have chosen, such as for example the TAP, are well known

and scientifically used in the field of HCI and in usability testing. According to

Hertzum and Jacobsen [25] some UEM’s as TAP can suffer from ”The Evaluator

Effect” which means that novice and expert evaluators that evaluate the same

system with the same marked problems come up with different issues with the

design. They concluded in their report that it is not recommended to use only one

evaluator as a reliable conclusion cannot be made due to this effect. As we used

several test subjects or evaluators of our prototype we say that it is reliable.

3.5.2 Validity

In order to get valid results from the TAP, it is important to not disturb or affect the

participants when talking and to not behave in a way that makes them say things

that they do not agree with. The only time the observer should intervene is when

the participants stay quiet and that is through saying ”keep talking” which does

not disrupt their thought process. Disrupting the thought process is something

that affects the validity of the results [18, 22].

Furthermore, the only opinions and expressions we can record are the ones

the participant verbally expressed through the TAP. Meaning that all views and

opinions of the participant that have not been shared will be unknown to us.

The sample size (number of participants) can affect the validity of the results. We

chose 7 which is argued for in Section 3.3.1 to why it is enough in our case.

3.6 Planned Data Analysis

As mentioned earlier we chose to analyze the data received from the TAP through

a simplified version of RPA.

The verbal data collected during TAPwas foremost used to find usability problems

and areas of improvement of the design for each task. Furthermore, the number

35

of negative, neutral and positive comments was used to evaluate the prototype’s

usability.

Based on the screen recording the number of errors during the interaction could

be calculated and described. Furthermore, could the total number of interactions

be calculated. The percentage of errors could then be derived from the number of

errors and the total number of interactions. Themedianwas used to get an average

value for each performance measurement (time for performing a task, percentage

of errors, and rating of satisfaction) for each task. We chose the median value,

because of the small size of the test group. The arithmetic mean tends to be less

accurate for small sample sizes because of a few outliers and the geometric mean

cannot handle negative or zero values. [49].

The mean values of the performance measurements were used to check if an

improvement, of the usability, had beenmade between each iteration and if the set

performance goal had been reached. More specifically if the design had improved

in regards to and fulfilled the set performance goals for efficiency, effectiveness

and satisfaction of the interaction with the prototype.

3.7 Evaluation framework

The following subsections describe how we chose to collect and evaluate the data

from our testing.

3.7.1 Collection of data

As presented in Section 3.4, during the tests, we timed how long it took for each

participant to perform each task as well as record the screen to later count the

number of errors a user made and the total number of interactions for each

task. The percentage of interactions that were errors was then calculated. The

participants also rated their satisfaction of the interaction with the prototype. We

then calculated themedian value of themeasurements for each task. When timing

the user performing a task, the start timewas considered themoment the user was

told, by us, to start. The phone, on which the prototype was presented, was lying

36

screen down on a surface in front of the user. When the user was told to start

performing the task, the user could pick up the phone and start performing the

task. The time it took the user to read the task instruction cards was not included,

only the time the user interacted with the prototype was included. The user was

instructed, beforehand, to put the phone down on the surface in front of them

when they considered that they had finished the task. The moment the user put

down the phone on the surface was regarded as endtime.

The errors a usermade during the performance of a taskwas counted. The number

of interactions the user performed on the device was also counted in order to

calculate the percentage of errors the user did. Interaction is defined as any press

or swipe on the touch screen. Some examples of what was considered errors were:

• Clicking on a wrong button

• Performing a wrong action

• Clicking on something that isn’t clickable

• Typing in a value in the wrong field

The verbal data collected during TAP was analysed using RPA, as stated above.

The number of negative, neutral, and positive comments expressed for each task

was counted.

3.7.2 Evaluation of data

The mean values of the performance measurements were compared to the values

from the previous iteration. Had the values improved, it was assumed that the

respective usability parameter had improved for the specific task. A decrease

in the percentage of errors, time to complete a task and an increase in rating

of satisfaction is considered an improvement of the efficiency, effectiveness, and

satisfaction of the prototype respectively.

If the mean value for every performance measurement (time to perform a task,

number of errors, and rating of satisfaction) had reached the set performance

goals the task was considered to have a satisfactory level of usability in regards

to performance measurements. If the number of positive comments, in the

37

verbal data, was above the defined goal limit the task was considered to have

a satisfactory level of usability based on user’s comments. If the level of

usability of the task was considered satisfactory based on both the performance

measurements and the verbal data the overall level of usability was considered

satisfactory and testing on that specific task was considered finished. We chose

this definition of done since there exists, in principle, no limit to how good the

usability of a design can become and since we were limited in time, we chose that

the testing of that task was then to be considered complete.

The qualitative data (the content of the comments and description of errors)

was used to find areas of improvement and was used during the design of the

next iteration of the prototype. The qualitative data was therefore not used to

evaluate the level of usability of the task. Only the quantitative data (performance

measurements and the number of comments) was used as a basis for the level of

usability of the task.

The set performance goals are seen in Table 3.2:

The set goals for comments for each usability parameter was that the number of

positive comments was above 50%. Meaning that the number of comments that

were categorised as positive for each category of usability parameter was to be

either half or more of the number of total comments in regards to that usability

parameter.

38

Table3.2:

Theperform

ance

goals.

Usability

parameter

Measurement

Goalmetric

Unit

Motivation

Efficiency

Tim

eto

complete

atask

1min

time

Userwillgetfrustratedafterthis

time

Effectiveness

Numberoferrors

5%

percentage

Userwillgetfrustratedby

alarge

amou

ntof

errors.Errorsaddon

theperform

ance

time.

Satisfaction

User’sratingof

ease

ofuse

5Likertscaleof1-7

Aratinghigher

than

5isarating

aboveaverage.

39

40

4 The Work

This chapter presents the work process of creating, developing and improving the

prototype for each test iteration.

4.1 Tasks to be analyzed

The tasks chosen for testing were the following:

Pay via QR

Transfer money through the input of payment information via QR.

Pay via manual input

Transfer money through manual input of payment information.

Charge

Increase the balance of existing currencies in the wallet.

Exchange

Exchange currency.

These tasks were chosen because they test the most vital functions of the app.

These tasks were also the most complicated and had the greatest potential for

improvement.

To describe to the user what task should be performed and what goal they tried to

achieve instruction and scenario cards were created and presented to the users on

paper. The instruction and scenario cards presented to the users for each task

are presented in Appendix C. The task was, as stated previously, given to test

subjects in random order. This to prevent that the learning effects of performing

one task would influence the result of the remaining tasks. As the design of the

prototype changed, the task instructions were slightly modified to conform to the

new design.

41

4.2 Test group

The test group and their characteristics are compiled in Table 4.1 shown down

below. Each test subject was given a number to represent them as seen in the

first column of Table 4.1. The second column shows their perceived experience of

mobile payment apps, and the third column shows their perceived experience of

software development based on a rating from a five-level Likert scale.

With regards to experience, the Likert scale ranges from 1: Strongly disagree, 2:

Disagree, 3: Neither agree or disagree, 4: Agree and 5: Strongly agree, on the level

of agreement of the two statements (1) You have experience with using mobile

payment applications and (2) You have experience in software development.

The Likert scale ranges from 1: No experience, 2: Little experience, 3: Some

experience, 4: A lot of experience and 5: Extensive experience.

Table 4.1: The test subjects and their perceived experience using mobile paymentapps and experience of software development.

Test subject Mobile payment apps Software development

1 4 5

2 2 4

3 4 3

4 3 4

5 3 4

6 5 1

7 2 5

Furthermore, three of the test subjects used smartphones with an Android

operating system and the other four test subjects used smartphones with an IOS

operating system.

4.3 Testing

The prototype was based on the given requirements from Centiglobe. All

prototypes are presented in Appendix E. Before the test subjects started

42

interacting with the prototype and after they had read the task instructions they

were asked if they needed any clarification. It was noted that the concept of the

app was quite difficult to understand for the test subjects based on only the task

instructions.

4.4 Data collection and analysis

As stated previously the concepts used for the RPA was defined after the

transcription of the verbal data collected from iteration 1. The concepts used for

concept coding the verbal data, gathered from the TAP were the following:

Instruction

Repeating of instructions given, e.g ”I am to input text”

Error

Expression of an error made, e.g ”This was wrong”

Interpretation

Interpretation of view, e.g ”If I click on the red button I will navigate to view

B”

Value

Expression of a value, e.g ”fast”, ”slow”, ”smooth”, ”effective”, ”strange” etc.

Misinterpretation

Expression ofmisinterpretation, e.g ”The red button did not navigate to view

B”

Ambiguousness

Expression of ambiguousness or uncertainty, e.g ”I think this is correct”, ”I

am not sure this is right” etc.

Observation

Expression of observation of screen, e.g ”A number popped up on the screen”

Question

Gathering of information or expression of lack of understanding, e.g ”Huh?”

43

Success

Expression of successful or correct interaction, e.g ”Yes, that was right”

Exploration

Expression of exploring, e.g ”I will try to push button A”

Action

Expression of performing an action, e.g ”I am pressing button A”

Furthermore, the comments were categorised into neutral, positive, and negative.

Phrases coded as questions, ambiguousness, errors, and misinterpretation were

categorised as negative since they expressed fault with the design. Phrases coded

as values could be categories as either negative or positive based on whether the

phrase expressed a positive or negative value. For example, a phrase containing

the value ”too fast” or ”bad” was considered negative, while a phrase containing

the value ”great” was considered a positive one. Phrases coded with success

were categorised as positive. Every phrase coded with solely instruction, action,

interpretation, exploration, and observation were categorised as neutral. Phrases

that could be categorised as both negative and positive were categorised as such.

During testing iteration 1, we discovered that the test subjects often wanted to

give further opinions about what could be improved after the testing of a task was

finished. We found that this was a very effective way of getting more information

about why certain errors weremade and what a good improvement would be. The

post-test comments were not used as a result of testing, but we did consider them

when improving the prototype. During the rest of our testing, we started asking

the test subjects if they had any post-test comments they wanted to share with us.

4.5 The prototype

The prototypes were developed in consultation with the product owner. Design

choices made during the development of all prototypes were based on personal

experience and opinions as well as HCI best practices such as Schneiderman’s

eight golden rules and Nielsen’s 10 heuristics.

44

5 Result and Analysis

In this chapter, the results of the usability evaluation are presented and discussed.

In Sections 5.1 - 5.3 the results are presented. In Section 5.5 a reliability analysis

is performed. A validity analysis is performed in Section 5.6. Lastly, in Section 5.7

the results are discussed.

5.1 Results iteration 1

7 participants participated in iteration 1. Themajor problems thatwere discovered

based on the data collected during iteration 1, were:

Icons

The meaning of the icons was unclear for some test subjects. They were

tightly placed which made it more difficult to distinguish them.

Titles

Some views were missing explanatory titles which lead to uncertainty

amongst users.

Task instructions

As the idea of the app was very difficult to grasp, the task instructions were

not detailed enough. Task instructions can be found in Appendix C.

Action order

In some tasks, the form filling had to be executed in an order which made

users confused.

Navigation

The test subjects had difficulty to find and use the navigation bar.

End of task

The test subjects were not sure if and when they had finished a task. There

was no feedback.

All of these above have to be changed before proceeding with Iteration 2.

45

The median values of the performance data gathered during iteration 1 of testing

were the following:

As shown in Table 5.1, only two performance goals were met for only one task.

The median value of the rating of satisfaction for the task exchangewas above the

set performance goal of 5. The median value of task time for the task exchange

was below the set maximum of 60 seconds. The task Pay via manual input stood

out as the task with the worst results with regards to performance measurements.

The largest problem test subjects had with performing the task was navigating to

the correct view. However, the percentage of negative comments for all tasks met

the goal of being below 50%.

Table 5.1: Median value of performance measurements iteration 1

Task Time Error Satisfaction Negative comments

Pay via QR 109,70s 72% 4 31%

Pay via manual input 167,54s 55% 2 30%

Charge 84,79s 50% 4 17%

Exchange 55,79s 54% 5 32%

An excerpt of the concept coded verbal data is presented in Appendix D.

5.1.1 Changes after iteration 1

Icons were changed, based on post-test comments from the test subjects. For

example, was the icon for Charge changed from an inbox icon to a plus icon,

this change can be seen in Figure 5.1. Titles were added to all views. The task

instructions were changed to give test users a better understanding of the function

of the app. Instead of having a bottom navigation (nav) bar we changed the

prototype to have a top nav bar containing a hamburger menu, QR icon, and back

button. In the menu, all alternatives were described with names instead of icons,

see Figure 5.1. Lastly, we added a confirmation message that popped up when

users were finished with a task.

46

(a) Iteration 1 (b) Iteration 2 (c) Iteration 2 Menu

Figure 5.1: Example of changes made to the prototype between iteration 1 anditeration 2

5.2 Results iteration 2

7 participants participated in iteration 2. The major problems that were

discovered based on the data collected during iteration 2, were:

Cluttered views

Some views were very cluttered, containing a lot of text and information.

Input fields were ambiguous

Test subjects found it difficult to discern titles and descriptions from input

fields. Particularly pointing out underlined titles as confusing.

QR icon

Test subjects had problems finding the icon for scanning a QR code.

Pay icon

The meaning of the manual pay icon was unclear for some test subjects.

Understated titles

Many test subjects did not read the title of the view, i.e ”Charge”, ”Exchange”,

47

etc.

Wording

Some wording was confusing and unclear for test subjects

Menu button

Many test subjects ignored or did not see the menu button.

All of these above have to be changed before proceeding with Iteration 3.

As seen in Table 5.2 all tasks had a median satisfaction rating above the set goal

of 5. Furthermore was the median value of task time for tasks exchange and

charge below the maximum of 60 seconds. Pay via manual input and Pay via

QR code had the worst results, with regards to all performance measurements

and the verbal data. This result is not surprising as these functions are the most

complex. However, all results were improved.

Table 5.2: Median value of performance measurements iteration 2

Task Time Error Satisfaction Negative comments

Pay via QR 81,29s 44% 5 14%

Pay via manual input 84,6s 38% 6 0%

Charge 37,05s 25% 6 8%

Exchange 54,9s 27% 6 0%

5.2.1 Changes after iteration 2

When developing prototype 3, i.e., the prototype used for testing iteration 3, the

product owner wanted major changes to the functions of the app. Meaning that

some functions were altered. The major changes that were made were that an

additional option of payout method was added to the pay view and an additional

option of transfer method was added to the charge view. Furthermore, some

wording was adjusted. Because of the changes made to the prototype the task

instructions given to the test subjects had to be altered.

Based on the results of iteration 2 the QR icon was made bigger and was put in

the centre of the top nav bar for better visibility. We also added a coloured box for

48

encapsulation around the input fields in order to make the user understand what

input fields that were grouped together. This change is found in Figure 5.2.

The product owner wanted to merge some functions to a single view which was

done per request.

(a) Iteration 2 (b) Iteration 3

Figure 5.2: Example of changes made on the prototype between iteration 2 anditeration 3

5.3 Results iteration 3

7 participants participated in iteration 3. The major problems that were

discovered based on the data collected during iteration 3, were:

49

Menu button

Test subjects still ignored or did not see the menu button.

Overload of information/ cluttered view

Test subjects found some views to be cluttered and that there was a lot of

information displayed.

All of these above have to be changed, if possible, before proceeding with Iteration

3.

As seen in Table 5.3 the results for task Charge worsened. Some results for

the task Pay via manual input also worsened. This is most likely due to the

functional changes implemented on the request of the product owner. These

changes made the functions more complex. The results for the tasks that were

not changed, Exchange and Pay via QR, improved, however. Task Pay via QR

met all performance goals.

Table 5.3: Median value of performance measurements iteration 3

Task Time Error Satisfaction Negative comments

Pay via QR 27,72s 0% 7 0%

Pay via manual input 77,13s 29% 6 10%

Charge 47,62s 40% 5 15%

Exchange 38,62s 27% 7 0%

5.3.1 Changes after iterations 3

We decided to change the visibility of the menu button since it was obvious from

the testing that some had trouble noticing the button. To make it more visible

contrasting colours were added instead of having a black and white icon and

background. This change can be seen in Figure 5.3.

Furthermore, tomake it easier for the eye to read information andmake the design

more minimalist we decided to change the input fields from having a bubble form

to an underlined text instead. This change can be seen in Figure 5.3. Furthermore,

was the design of the menu changed.

50

(a) Iteration 3 (b) Iteration 4

Figure 5.3: Example of changes made on the prototype between iteration 3 anditeration 4

5.4 Results iteration 4

7 participants participated in iteration 4. The major problems that were

discovered based on the data collected during iteration 3, were:

Wording

Some test subjects found Transfer to be a better name for the function Pay

via manual input instead of Pay

As seen in Table 5.4 the performance goals regarding time, negative comments

and ratings of satisfaction were all met. However, no task reached a percentage

51

of error below 5%. Based on the results, the most complex tasks for this iteration

are Pay via manual input and Exchange. There is a noticeable improvement of

the percentage of errors for the task Charge which dropped from 40% to 9% in

comparison with iteration 3, however, it is still not fulfilling the performance goal.

Table 5.4: Median value of performance measurements iteration 4

Task Time Error Satisfaction Negative comments

Pay via QR 21,90s 17% 7 0%

Pay via manual input 49,80s 33% 6 8%

Charge 48,42s 9% 7 0%

Exchange 30,32s 33% 7 7%

After the last iteration, for all tasks, all goals were met bar one. The results for all

tasks met the goals of task time, rating of satisfaction and percentage of negative

comments. However, no task had a result of a percentage of errors below the goal

of 5%. Furthermore, was the result of task time for all task quite a lot below the

goal of 60 seconds.

5.5 Reliability Analysis

Since we used the same test subjects for each iteration of testing the test subjects

became familiar with the prototype. This can affect the results since task time and

percentage of errors is expected to decrease as the test subjects are familiarised

with the design. However, the prototypes were majorly altered between each

iteration, lessening the learning effects.

After the first round of testing, we realised that the questionnaire used for

measuring satisfaction wasmisinterpreted by the test subjects. Some interpreting

the rating of 7 meaning that the prototype was difficult to use, while it was, in fact,

the other way around. After realising this, we instead asked the test subjects about

their rating, and ourselves wrote in the rating into the questionnaire.

52

5.6 Validity Analysis

Since we interpreted the data from the TAP, there is a possibility that the users’

phrases and expressions are incorrectly categorized. For example, a positive

comment could be interpreted as negative. As we do not have a large sample size

this could negatively affect the validity of the results.

5.7 Discussion

As expected, the prototypes that were developed did not fully imitate the

interaction with the actual app. This was in keeping with our expectations as a

full emulation of the prototype was not the intention or goal. For example, the

emulation did not allow text input or full interactivity. This meant that all buttons

could not be pressed, and so on. The response time of the prototype varied, with

some actions having quite a long response time in comparison to others. This

presented a problem during testing since it increased the task time, resulting in

the task time results being affected. However, the biggest problem was that the

test subjects interpreted the long response time as an incorrect push and then

proceeded to push again or push somewhere else. Furthermore, the prototype

did not allow to input information in just any order. This was because we did not

have time to design the prototype to function for all possible outcomes/cases. The

result was that the test subject performed an interaction that generated an error in

the prototype, but that would have been correct in the final application. Therefore

our results are not fully equivalent to how a userwould navigate through the actual

app rather than our prototype. A consequence of this was that the percentage of

errors when interacting with the prototype are higher than they would be in the

final product which can be seen from the results of testing. Because of this source

of error, testing was concluded when all other performance goals were met.

The results were also affected by the fact that test subjects thought out loud while

performing the task, and that the task they were supposed to do was given to them

rather than known to them. It can be assumed that test subjects would perform

tasks slightly fast, would they not be required to think out loud during the process.

Furthermore, would test subjects probably have performed tasks faster, would

53

they themselves have been inspired to do the tasks. During testing all information

theywere supposed to put in and choose, was given to themmeaning that they had

to refer back to the instructions at some points. For some test subjects, this added

quite a lot to the time it took to perform a task.

It was quite difficult to concept code the verbal data. Very often the test subjects

did not speak in full sentences and it was difficult to discern what they were

referring to at all times. When analysing the verbal data the data was divided into

phrases, and then each phrase was concept coded. However, since the test subject

did not speak in full sentences it was difficult to divide the data into phrases.

Because the percentage of negative comments were simply counted, the result

can be heavily influenced by how many phrases the verbal data was split into.

Although the design of the prototype was not strictly evaluated by the percentage

of negative comments, but also included the performance measurements - this

should compensate for some faults in the analysis of the verbal data. Furthermore,

some comments referred to the software used to realize the prototype, rather than

the design of the app. For example, some commented on the slow responsiveness

of the prototype.

It is worth to consider that the first prototype was designed with a bottom nav

bar, something more commonly used in Apple IOS apps. However, the second

and third prototype was designed with a hamburger menu, more commonly

used in Android apps, but also used in a number of IOS apps. Depending on

what operating system the test subjects were used to, this could influence how

recognisable the navigation design was and how quickly they could adapt to it. In

fact, about half of the test subjects were used to the IOS operating system and half

were used to the Android operating system.

It is important to note that we chose to change the task instructions to correspond

to the changes we made in the prototype. The altered instructions are presented

in Appendix C. Furthermore, it was understood that the instructions were not

detailed enough for the test subjects to fully understand what the goal of their

task was and the concept of the app. Not having a clear understanding of the

task and the concept of the app resulted in several errors that could have been

avoided otherwise. An onboarding process or beginners guide should probably

54

be implemented in the final product to ensure that the concept of the app and

the different functions available are fully understood by users. In our opinion,

a ”good” design of the app is insufficient to convey how the app works and its

concept.

The results of iteration 1 showed that almost all performance measurements

were below the set goals. However, all the median values of the percentage

of negative comments were below the goal of a maximum of 50%. The results

for the percentage of negative comments were inconsistent with the results of

the performance measurements. From this, we can conclude that the goal

we set for the percentage of negative comments, was too generous. However,

as stated previously the design of the prototype was not evaluated only based

on the percentage of negative comments, but also considered the performance

measurements. This should compensate for small faults in the analysis of the

verbal data.

As the functions of the app were changed between iterations 2 and 3 the testing

results were affected. Not only was the design changed, but also the task flow.

This affected the result in that the results worsened, for some tasks, in between

iterations 2 and 3. The changes made task Charge more complex, and therefore

the usability of it declined. Since the product owner was consulted during the

design of a newprototype the product owner alsomade an evaluation of the design

of the prototype.

As the language used, both in the instructions and the prototype, was not the

native language of the test subjects this could have contributed to some problems

users had interacting with the prototype.

55

56

6 Conclusions and Future work

The following subsections will discuss the positive effects and drawbacks of the

work as well as an evaluation of the results. Section 6.2 will discuss the limitations

of the results and Section 6.3 suggests future work and summarizes what is left

undone.

6.1 Conclusion

As discussed in Chapter 1, there is yet research to be done in the usability of e-

wallets and mobile payment apps. Since mobile payment apps usually feature

more complicated functions than the average smartphone app, but also because

of the nature of the functions. As the functions concern money, the demand for

usability and a ”good” design that instils a sense of security is higher. The concept

of this particular e-wallet and its functions was shown to be confusing to a lot of

users we tested it with. Particularly users had difficulty understanding how the

currencies were stored and how the transactions happened. What was important

to do and also the biggest challenge was to introduce new concepts, terminology,

and functions in a way that the users were familiar with and to relate them to

recognizable ideas. This involved displaying a large amount of information in

a clear and minimalist way. This was the most important issue that we had to

deal with in order for the app to work in an environment with people who have

no earlier experience of e-wallets. When testing we noticed that our experience

and the product owner’s technical experience and knowledge of the app made

us blind to some major issues. Some people had more difficulty grasping the

idea and purpose of the app as opposed to having difficulties with navigating

and completing the tasks. This showed when participants had trouble finishing

some tasks in the first iteration, but later on, were able to finish the same tasks

without any issues - despite the fact that they could still not grasp the meaning

of some information. To work around this issue, creating an onboarding process

for new users of the app was suggested to the product owner, but no testing of

this performed. The onboarding process does not necessarily have to include

information about for example the location of buttons, but rather could consist

57

of animations or pictures combined with text to describe the bigger picture and

general idea of the app and its purpose. We believe that such a process would

increase understanding, especially for new users.

Our results showed that usability testing resulted in a huge improvement in

the design of the app. Even at the second iteration, huge improvements were

made as discussed in Chapter 5. It was clear that without any testing and

feedback from real users, more time and effort would have to been required

to redesigning a ”finished” app. In the worst case, the app would not have

performedwell with users, without improvements in its design. Reflecting on how

users experience apps in early stages broadens the perspective of developers and

motivates developers to come up with smart ideas to solve problems early on.

We met our set performance goals regarding time, meaning all tasks were

performed under 1 minute. We also met our goal of a satisfaction rate 5 or higher

on a Liker scale of 1 to 7. The goals for verbal data, a percentage of negative

comments under 50 % were also met. However, as for the percentage of errors,

we could not reach the goal of a maximum of 5%. The reason for this depended on

several factors such as: what is considered as an error in the prototype versus in

the final app, as well as the maximum number of interactions to complete a task

being a small number. Even though a tester only performed one error during the

whole task, we would never, even in that case, reach our goal. The only time we

would be able to reach such a goal would be if the user performed zero errors when

performing a task.

The drawbacks that affected the results and our efforts weremainly the limitations

in possibilities and flexibility of ”Figma” prototyping. The problem we had was

that it was too time-consuming tomake the whole prototype interactive so we had

to prioritize the most important functions and only make those tasks available

for the tester. There was no quick way to make the whole prototype interactive

and at the same time be easy to change after the feedback was received from the

testing. Because ”Figma” did not support text input we were not able to test the

design featuring a keyboard. Furthermore, could the prototype not enable the use

of an actual QR scanner or a camera. Instead, a QR code was simply displayed

on the view to mimic this feature. We noticed that other software for prototyping

58

were all similar to ”Figma” so it would not help if we switched to another. There

existed no ”perfect” software for creating prototypes for usability testing that was

free of charge and also supported mirroring on the phone which we prioritized as

a feature. The learning effects of using the same test group for each iteration also

affected the results. The results were affected by the fact that people thought out

loud, but foremost by the fact that the task they were supposed to do were given

to them rather than known to them.

Would we have done the project again we would have not used the TAP. Instead,

we would have collected verbal data and comments about the prototype through

post-test interviews. We realised that post-test interviews would have been the

preferredmethodwhen test subjects offered spontaneous comments, after testing,

that were very useful for altering the prototype. Furthermore, did the verbal data

collected through TAP not offer much information and was time-consuming to

analyse.

6.2 Limitations

During this project, we limited ourselves to focusing on finding themajor usability

problems with the design of the prototype. Therefore a small, ad hoc test group

was sufficient. To discover minor usability problems a larger test group would

need to be used with new test subjects at each iteration. The fidelity of the

prototypes developed was the same throughout testing. Initially testing on a

low-fi prototype and later testing on a high-fi prototype would have made the

testing more time efficient since less time would have been spent developing the

prototypes while there were still major changes to bemade. As the app is intended

for use internationally and by all kinds of users, we should have selected a more

diverse test groupwith regards to nationality, technical experience, and disability.

Such a broader test group would have more accurately represented the target

users.

59

6.3 Future Work

There is a lot that is left to be done in the future. The work that we did is only

the initial design and testing of a prototype which later on must be developed

into a real application for the company. Before releasing such an app it would

be appropriate to do further testing to address the limitations introduced by the

”Figma” prototyping when these would not occur in an actual app. This testing

should be done to find detailed usability problems that are too small to find when

testing on prototypes. Furthermore, a fully functional app would enable testing

with full interactivity and better response time. It would also enable testing

features like QR scanner and text input. If better software for prototyping for

usability testing is found, then further testing could be made to compare with our

results.

6.4 Reflection

Performing usability testing on an app design early and continuously during

production can save a large amount of time, effort, and money. This has a direct

effect on the economic costs of the company and potentially reduce the time to

market for the final product - thus enabling the company to begin to generate

revenue earlier and also provide a larger group of users who can generate feedback

to further improve the design. Discovering major problems early in the design

process can vastly reduce the amount of redesign of the app. Furthermore, it can

possibly ensure a better design of the final product. The app will also be designed

by keeping the users inmind. This should result in the appperformingbetterwhen

used by actual users. Having an easy to use e-wallet could have a major impact,

especially in a society thatwants to shift to electronic payments rather than the use

of coins and paper currency. The shift to electronic payments is expected to have

environmental benefits by reducing the needs to the currency and its replacement

due to wear and reducing the need for armoured cars to transport the currency.

Additionally, this transition is expected to reduce the costs and risks associated

with dealing with currency.

As mobile payment is in its infancy, the privacy and security requirements are

60

still quite unclear. Therefore there are some privacy concerns with regards to e-

wallets. Furthermore, it is also unclear if e-wallets will be designed to track all

of the transfers in an auditable way or if they will hide these transactions from

access by other parties. There is also an ethical problem concerning whether the

interface and device ensure that only a legitimate user canmake use of the e-wallet

and that such a user is not under duress.

61

References

[1] About Us - EcoCash. URL: https://www.ecocash.co.zw/about (visited on

03/22/2019).

[2] Alipay. URL: https://intl.alipay.com/ (visited on 03/27/2019).

[3] Alshamsi et al. “User perception of Bitcoin usability and security across

novice users”. In: International Journal of Human-Computer Studies 126

(June 2019), pp. 94–110. ISSN: 10715819. DOI: 10.1016/j.ijhcs.2019.

02 . 004. URL: https : / / linkinghub . elsevier . com / retrieve / pii /

S1071581918301459 (visited on 03/22/2019).

[4] Apple Pay. Apple (Sverige). URL: https://www.apple.com/se/apple-

pay/ (visited on 03/21/2019).

[5] Baskanderi, Naema. UX Glossary: Task Flows, User Flows, Flowcharts

and some new-ish stuff. Apr. 2017. URL: https : / / uxplanet . org / ux -

glossary-task-flows-user-flows-flowcharts-and-some-new-ish-

stuff-2321044d837d (visited on 03/29/2019).

[6] Benyon, David. Designing interactive systems: a comprehensive guide to

HCI and interaction design. Third edition. Boston: Pearson, 2013. ISBN:

978-1-4479-2011-3.

[7] Blackwell, Amy and Manar, E. UXL Encyclopedia of Science. 3rd ed. 2015.

ISBN: 978-1-4144-3086-7.

[8] Brundtland, Gru et al. Report of the World Commision on Environment

and Development: Our Common Future. Tech. rep. United Nations, May

1987, p. 5. URL: https : / / www . are . admin . ch / are / en / home /

sustainable- development/international- cooperation/2030agenda/

un-_-milestones-in-sustainable-development/1987--brundtland-

report.html (visited on 03/21/2019).

[9] Centiglobe. Centiglobe. URL: https://www.centiglobe.com (visited on

03/26/2019).

62

[10] Chan, Fanny and Johansson, Sofia. Evaluation of user experience on a

mobile application : A study conducted on themobile application provided

by Shownight. 2016. URL: http://urn.kb.se/resolve?urn=urn:nbn:se:

kth:diva-202389 (visited on 03/28/2019).

[11] Cheng, Evelyn. China is living the future of mobile pay right now. Oct. 8,

2017. URL: https://www.cnbc.com/2017/10/08/china-is-living-the-

future-of-mobile-pay-right-now.html (visited on 03/27/2019).

[12] Claromentis.Theprinciples of usability. Aug. 2015.URL: https://medium.

com / @Claromentis / the - principles - of - usability - 1c12c6b4fe16

(visited on 03/21/2019).

[13] Coinbase - Buy/Sell Digital Currency. URL: https://www.coinbase.com/

legal/user_agreement (visited on 04/03/2019).

[14] Covert, Adrian. Bluetooth 4.0: the wireless technology that could make

sense of the digitalwallet. CNNMoney. Dec. 9, 2013. URL: https://money.

cnn.com/2013/12/09/technology/mobile/bluetooth-digital-wallet/

index.html (visited on 06/06/2019).

[15] Dahlberg, Tomi et al. “Past, present and future of mobile payments

research: A literature review”. In: Electronic Commerce Research and

Applications. Special Section: Research Advances for theMobile Payments

Arena 7.2 (June 1, 2008), pp. 165–181. ISSN: 1567-4223. DOI: 10.1016/j.

elerap.2007.02.001. URL: http://www.sciencedirect.com/science/

article/pii/S1567422307000075 (visited on 03/21/2019).

[16] Dam, Rikke and Siang, Teo. What Kind of Prototype Should You Create?

Mar. 2019. URL: https://www.interaction-design.org/literature/

article / what - kind - of - prototype - should - you - create (visited on

03/22/2019).

[17] Dumas, Joseph S. A practical guide to usability testing. Rev. ed..

Human/computer interaction. Exeter: Intellect, 1999. ISBN: 1-84150-020-

8cased.

[18] Ericsson, K. Anders and Simon, Herbert A. Protocol analysis: Verbal

reports as data. Protocol analysis: Verbal reports as data. Cambridge, MA,

US: The MIT Press, 1984. 426 pp. ISBN: 978-0-262-55012-3.

63

[19] Falk, Tim. 2019’s

best cryptocurrency wallets | 70+ compared. finder.com.au. May 3, 2018.

URL: https://www.finder.com.au/cryptocurrency/wallets (visited on

04/08/2019).

[20] Figma. Viewing Prototypes with Presentation View - Figma. URL: https:

/ / help . figma . com / article / 202 - viewing - prototypes - with -

presentation-view (visited on 04/03/2019).

[21] Fonteyn, Marsha E., Kuipers, Benjamin, and Grobe, Susan J. “A

Description of Think AloudMethod and Protocol Analysis”. In:Qualitative

Health Research 3.4 (Nov. 1, 1993), pp. 430–441. ISSN: 1049-7323. DOI:

10 . 1177 / 104973239300300403. URL: https : / / doi . org / 10 . 1177 /

104973239300300403 (visited on 04/01/2019).

[22] Güss, C. Dominik. “What Is Going Through Your Mind? Thinking Aloud

as a Method in Cross-Cultural Psychology”. In: Frontiers in Psychology 9

(Aug. 13, 2018). ISSN: 1664-1078. DOI: 10.3389/fpsyg.2018.01292. URL:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6099082/ (visited on

04/03/2019).

[23] Håkansson, Anne. “Portal of Research Methods and Methodologies for

Research Projects and Degree Projects”. In: WORLDCOMP’13 - The

2013 World Congress in Computer Science, Computer Engineering, and

Applied Computing (July 2013), pp. 22–25.

[24] Hellström, Johan and Tröften, Per-Einar. The innovative use of mobile

applications in east Africa. OCLC: 644254073. Stockholm: Swedish

international development cooperation agency (Sida), 2010. ISBN: 978-91-

586-4129-7.

[25] Hertzum, Morten and Jacobsen, Niels Ebbe. “The Evaluator Effect: A

Chilling Fact About Usability Evaluation Methods”. In: International

Journal of Human–Computer Interaction 15.1 (Feb. 1, 2003), pp. 183–

204. ISSN: 1044-7318. DOI: 10 . 1207 / S15327590IJHC1501 _ 14. URL:

https : / / doi . org / 10 . 1207 / S15327590IJHC1501 _ 14 (visited on

06/06/2019).

64

[26] How Digital Wallets Work. HowStuffWorks. Sept. 19, 2012. URL: https:

/ / electronics . howstuffworks . com / gadgets / high - tech - gadgets /

digital-wallet.htm (visited on 03/20/2019).

[27] Huawei Technologies Co., Ltd. HUAWEI P20 Smartphone, Dual Leica

Camera, AI Master | HUAWEI Global. URL: https://consumer.huawei.

com/en/phones/p20/ (visited on 04/03/2019).

[28] ISO. ISO 9241-11:2018 Ergonomics of human-system interaction – Part

11: Usability: Definitions and concepts. Tech. rep. Swedish Standards

Institute, 2018.

[29] Ivory, Melody Y. and Hearst, Marti A. “The state of the art in automating

usability evaluation of user interfaces”. In: ACM Computing Surveys 33.4

(Dec. 2001), pp. 470–516. ISSN: 03600300. DOI: 10 . 1145 / 503112 .

503114. URL: http://portal.acm.org/citation.cfm?doid=503112.

503114 (visited on 03/27/2019).

[30] Ivory, Melody Yvette and Hearst, Marti. An empirical foundation for

automated web interface evaluation. Vol. 1. University of California,

Berkeley, 2001. URL: http://webtango.berkeley.edu/papers/thesis/

thesis.pdf.

[31] Jkielty. Viewport, resolution, diagonal screen size and DPI for the most

popular smartphones. Apr. 2019. URL: https://deviceatlas.com/blog/

viewport-resolution-diagonal-screen-size-and-dpi-most-popular-

smartphones (visited on 04/03/2019).

[32] Kieras, David. “GOMS Models for Task Analysis”. In: (Mar. 2019). URL:

https://web.eecs.umich.edu/~kieras/docs/TA_Modeling/GOMSforTA.

pdf.

[33] Labaree, Robert V. Research Guides: Organizing Your Social Sciences

Research Paper: Quantitative Methods. Research Guide. 2019. URL:

http : / / libguides . usc . edu / writingguide / quantitative (visited on

03/20/2019).

[34] Lazar, Jonathan. Research methods in human-computer interaction.

Chichester: John Wiley, 2010. ISBN: 0-470-72337-8.

65

[35] MeasuringU: 10 Things To Know About The Single Ease Question (SEQ).

URL: https://measuringu.com/seq10/ (visited on 04/11/2019).

[36] Money Transfer Online - Cassava Remit. URL: https : / / www .

cassavaremit.com/ (visited on 03/21/2019).

[37] Muitipal Payment Methods - WeChat Pay. URL: https://pay.weixin.

qq.com/wechatpay_guide/intro_method.shtml (visited on 03/22/2019).

[38] Nations, United. Sustainable Development Goals

.:. Sustainable Development Knowledge Platform. 2015. URL: https://

sustainabledevelopment.un.org/?menu=1300 (visited on 03/25/2019).

[39] NFC Mobile Wallets. Setomatic Systems. URL: https : / / www .

setomaticsystems.com/nfc-mobile-wallets/ (visited on 06/06/2019).

[40] Nielsen, Jakob. 10 Heuristics for User Interface Design: Article by Jakob

Nielsen. Nielsen Norman Group. 1994. URL: https://www.nngroup.com/

articles/ten-usability-heuristics/ (visited on 03/25/2019).

[41] Nielsen, Jakob.Why You Only Need to Test with 5 Users. Nielsen Norman

Group. Mar. 19, 2000. URL: https://www.nngroup.com/articles/why-

you-only-need-to-test-with-5-users/ (visited on 04/03/2019).

[42] Olsen, M., Hedman, J., and Vatrapu, R. “Designing digital payment

artifacts”. In: ACM International Conference Proceeding Series. 2012,

pp. 161–168. DOI: 10.1145/2346536.2346568.

[43] Olsen, Mia, Hedman, Jonas, and Vatrapu, Ravi. “HCI issues in mobile

wallet design”. In: The Tenth Annual Pre-ICISWorkshop onHCI Research

in MIS. Shanghai, China, 2011, p. 2.

[44] Powers of 10: Time Scales in User Experience. Nielsen Norman Group.

URL: https://www.nngroup.com/articles/powers-of-10-time-scales-

in-ux/ (visited on 04/09/2019).

[45] QR - Uppslagsverk - NE.se. URL: https://www-ne-se.focus.lib.kth.

se/uppslagsverk/encyklopedi/l%C3%A5ng/qr (visited on 03/21/2019).

[46] Rolfe, Alex. The rise of digital and mobile wallet: Global usage statistics

from 2018. Nov. 2018. URL: https://www.paymentscardsandmobile.com/

mobile-wallet-global-usage-statistic/ (visited on 06/04/2019).

66

[47] SamsungDevelopers. Token Handling by Samsung Pay | SAMSUNG

Developers. URL: https://developer.samsung.com/tech- insights/

pay/token-handling-by-samsung-pay (visited on 03/26/2019).

[48] Sauro, Jeff. Are The Terms Formative And Summative Helpful Or

Harmful? Feb. 2010. URL: https : / / measuringu . com / formative -

summative/ (visited on 03/22/2019).

[49] Sauro, Jeff.MeasuringU: Average Task Times in Usability Tests: What to

Report? en. Apr. 2010. URL: https://measuringu.com/average-times/

(visited on 04/09/2019).

[50] Sauro, Jeff. MeasuringU: Using Task Ease (SEQ) to Predict Completion

Rates and Times. en. Oct. 2018. URL: https://measuringu.com/seq-

prediction/ (visited on 04/02/2019).

[51] Scharwatt, Claire. The impact of mobile money on the UN Sustainable

Development Goals. Oct. 2016. URL: https : / / www . gsma . com /

mobilefordevelopment/uncategorized/the-impact-of-mobile-money-

on-the-un-sustainable-development-goals/ (visited on 03/25/2019).

[52] Sears, AndrewL. “AIDE: a step towardmetric-based interface development

tools”. In:UIST (User Interface Software andTechnology): Proceedings of

the ACM Symposium. Proceedings of the 1995 8th Annual Symposium on

User Interface Software and Technology, UIST’95. ACM, 1995, pp. 101–110.

URL: https://pennstate.pure.elsevier.com/en/publications/aide-

a-step-toward-metric-based-interface-development-tools (visited

on 03/29/2019).

[53] Sharma, Sujeet Kumar et al. “Mobile wallet inhibitors: Developing a

comprehensive theory using an integratedmodel”. In: Journal of Retailing

and Consumer Services 45 (Nov. 1, 2018), pp. 52–63. ISSN: 0969-6989.

DOI: 10 . 1016 / j . jretconser . 2018 . 08 . 008. URL: http : / / www .

sciencedirect.com/science/article/pii/S0969698918305058 (visited

on 03/22/2019).

[54] Shin, Dong-Hee. “Towards an understanding of the consumer acceptance

of mobile wallet”. In: Computers in Human Behavior 25.6 (Nov. 1, 2009),

pp. 1343–1354. ISSN: 0747-5632. DOI: 10 . 1016 / j . chb . 2009 . 06 .

67

001. URL: http : / / www . sciencedirect . com / science / article / pii /

S0747563209000958 (visited on 03/21/2019).

[55] Swish - Betala enklare. Swish. URL: https://www.getswish.se/ (visited

on 03/21/2019).

[56] Swish - Uppslagsverk - NE.se. URL: https://www- ne- se.focus.lib.

kth . se / uppslagsverk / encyklopedi / l % C3 % A5ng / swish (visited on

03/21/2019).

[57] Tebbe,Harald. “Offlinemobile payment process”.US20150006386A1. Jan.

2015. URL: https://patents.google.com/patent/US20150006386/en

(visited on 03/26/2019).

[58] The Interaction

Design Foundation. What is Human-Computer Interaction (HCI)? URL:

https://www.interaction- design.org/literature/topics/human-

computer-interaction (visited on 03/21/2019).

[59] Top 101 Digital Wallet Companies. Due. Oct. 3, 2016. URL: https : / /

due . com / blog / top - 101 - digital - wallet - companies/ (visited on

04/08/2019).

[60] Wang, Y., Hahn, C., and Sutrave, K. “Mobile payment security, threats,

and challenges”. In: 2016 Second International Conference on Mobile

and Secure Services (MobiSecServ). Feb. 2016, pp. 1–5. DOI: 10.1109/

MOBISECSERV.2016.7440226.

[61] WeChat Pay. URL: https://pay.weixin.qq.com/index.php/public/

wechatpay (visited on 03/21/2019).

[62] What is Usability? The Interaction Design Foundation. URL: https : / /

www.interaction- design.org/literature/topics/usability (visited

on 03/20/2019).

[63] WHO. WHO | Informed Consent Form Templates. 2019. URL: https://

www.who.int/rpc/research_ethics/informed_consent/en/ (visited on

04/01/2019).

68

A First Appendix

Figure A.1: Task flow of Swish

69

Figure A.2: Task flow of Paypal

70

Informed Consent Form (ICF) Background We are two students Bercis Arslan bercis@kth.se and Blenda Fröjdh blendaf@kth.se doing research as part of a 1st cycle degree project at KTH, together with Centiglobe. We are doing research on usability on the design of a mobile payment application. We want to investigate how an e-wallet can be designed to ensure that users’ experience is as pleasing as possible. Type of research This research will involve you being given a task to perform on a prototype of a mobile payment application, talking out loud as you perform it, and us observing you. You will also be asked to fill out a questionnaire regarding your experience using the prototype. This will take place in a quiet setting and will take about 30-60 minutes. Voluntary participation Your participation in this research is entirely voluntary. It is your choice whether to participate or not. You can change your mind at any time and immediately stop participating. Procedures You will be asked to perform a certain task and presented with a card containing information about the task. You can continue to refer to the instructions on the card as you go through the test. As you perform the task you will be asked to think aloud as you perform it, expressing whatever comes to your mind regarding the prototype and your experience using it. Furthermore, the meeting will be audiotaped. The audiotape will be transcribed and presented in our report but the audiotapes are confidential and no one else except us, Blenda Fröjdh and Bercis Arslan, will have access to the tapes. The tapes will be destroyed after 20 weeks. At the end of the meeting, you will be asked to fill out a survey using Google forms. The questions included in the survey will be limited to your experience using the prototype. This consent form will be retained by us for three years and will be destroyed after that time has passed. Confidentiality The information that we collect from this research will be presented in our report. Our report will be published and available in full-text in DiVA which is a database open to anyone. Any information about you in the report will have a number on it instead of your name. Certificate of consent I have read the foregoing information and I consent voluntarily to be a participant in this study Signature of participants _____________________ Date ___________________________

Signature of Researcher_______________Signature of Researcher_______________ Date ___________________________

1

B Second Appendix

71

Pay via QR

Task Scan a QR code and choose the payment method Swish.

Scenario You are using an e-wallet application to transfer money and store different currencies. You now want to transfer money. To input the payment information you want to scan a QR code. Then you want to choose the payment method Swish.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Pay via manual input

Task Transfer 100 CRW to Nisse via SEK.NORDEA.VOLVO using the payment method Swish.

Scenario You are using an e-wallet application to transfer money and store different currencies. You now want to transfer money. You want to input the payment information manually (). You want to transfer 100 CRW to Nisse via SEK.NORDEA.VOLVO. You don’t want to input a message. You want to use the payment method Swish.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the prototype ● Don’t ask any questions related to the use of the prototype when performing the task

C Third Appendix

Figure C.1: Task instructions for iteration 1

72

Charge your account

Task Increase the balance of CRW with 100 CRW using the payment method Swish.

Scenario You are using an e-wallet application to transfer money and store different currencies. Currently, you have 1234.95 CRWs stored in your app. You now want to increase the amount of CRW stored in the wallet with a 100 CRW. The payment method you want to use to increase the balance is Swish. CRW is a type of currency.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Exchange

Task Exchange from one currency to another

Scenario You are using an e-wallet application to transfer money and store different currencies. You want to exchange an amount of money in the currency SEK.Nordea.Nordea to the currency CRW. The amount you want to change is 100 SEK.Nordea.Nordea.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Charge

Task Increase the balance of CRW through charging 100 SEK from your Swish account.

Scenario You are using an e-wallet application to transfer money and store different currencies. Currently, you have 100 CRW stored in your wallet. You now want to increase the amount of CRW stored in the wallet with 100 SEK (100 SEK is 50 CRW). You want to charge through your Swish account. In the end you have added 50 CRW to your wallet. Instructions

● Speak your thoughts or whatever you do as much as possible when using the prototype

● Don’t ask any questions related to the use of the prototype when performing the task.

Exchange

Task Exchange from one currency to another

Scenario You are using an e-wallet application to transfer money and store different currencies. You want to exchange 100 SEK to the currency CRW. You want to exchange from your Nordea account to your Centiglobe account. Start by putting in the amount and currency you want to sell, then the currency you want to buy.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the

prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Figure C.2: Task instructions for iteration 2

74

Pay via QR

Task Scan a QR code and send 150 SEK to ICA Bank from your Nordea account.

Scenario You are using an e-wallet application to transfer money and store different currencies. You are at the ICA store and now want to buy something for 150 SEK. To input the payment information you want to scan a QR code. The currency you want to pay with is SEK. The account you want to send from is Nordea. You don’t want to add a message. The transfer method is Swish.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the

prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Pay

Task Transfer 100 SEK from your Swish account to 150 KRC to John’s Mpesa account.

Scenario You are using an e-wallet application to transfer money and store different currencies. You now want to transfer money. You want to send money to John. You want to send 100 SEK from your Swish account. You want John to receive the money as 150 KRC to his Mpesa account. You don’t want to input a message. The transfer method is Swish.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the

prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Charge

Task Increase the balance of CRW through charging 100 SEK from your Swish account.

Scenario You are using an e-wallet application to transfer money and store different currencies. Currently, you have 100 CRW stored in your wallet. You now want to increase the amount of CRW stored in the wallet with 100 SEK (100 SEK is 50 CRW). You want to charge through your Swish account. In the end you have added 50 CRW to your wallet. The transfer method is also Swish. The CRW account is Centiglobe. Instructions

● Speak your thoughts or whatever you do as much as possible when using the prototype

● Don’t ask any questions related to the use of the prototype when performing the task.

Exchange

Task Exchange from one currency to another

Scenario You are using an e-wallet application to transfer money and store different currencies. You want to exchange 100 SEK to the currency CRW (100 SEK is 50 CRW). You want to exchange from your Nordea account to your Centiglobe account. Start by putting in the amount and currency you want to sell, then the currency you want to buy.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the

prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Pay via QR

Task Scan a QR code and send 150 SEK to ICA Bank via provider Nordea.

Figure C.3: Task instructions for iteration 3 4

76

Scenario You are using an e-wallet application to transfer money and store different currencies. You are at the ICA store and now want to buy something for 150 SEK. To input the payment information you want to scan a QR code. The currency you want to pay with is SEK. The provider you want to send money via is Nordea. You don’t want to add a message. The transfer method is Swish.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the

prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

Pay

Task Transfer 100 SEK via provider Swish to 150 KRC to John via provider Mpesa.

Scenario You are using an e-wallet application to transfer money and store different currencies. You now want to transfer money. You want to send money to John. You want to send 100 SEK using the provider Swish. You want John to receive the money as 150 KRC via provider Mpesa. You don’t want to input a message. The transfer method is Swish.

Instructions ● Speak your thoughts or whatever you do as much as possible when using the

prototype ● Don’t ask any questions related to the use of the prototype when performing the task.

D Fourth Appendix

Table D.1: Exerpt of concept coded verbal data

Phrase Coding Category

okej, scanna in qr kod är väl där. Interpretation Neutral

Den gjorde det automatiskt Observation Neutral

Jag antar att det ska vara någon kamera Interpretation Neutral

Från sek. Action Neutral

Aa den är automatisk, nice. Observation, Value Positive

78

TRITA TRITA-EECS-EX-2019:176

www.kth.se