understanding and supporting mobile application usage
DESCRIPTION
Abstract (Dissertation defense talk) In recent years mobile phones have evolved significantly. While the very first cellular phones only provided functionality for conducting phone calls, smartphones nowadays provide a rich variety of functionalities. Additional hardware capabilities like new sensors (e.g. for location) and touch screens as new input devices gave rise to new use cases for mobile phones, such as navigation support, taking pictures or making payments. Mobile phones not only evolved with regard to technology, they also became ubiquitous and pervasive in people’s daily lives by becoming capable of supporting them in various tasks. Eventually, the advent of mobile application stores for the distribution of mobile software enabled the end-users themselves to functionally customize their mobile phones for their personal purposes and needs. So far, little is known about how people make use of the large variety of applications that are available. Thus, little support exists for end-users to make effective and efficient use of their smartphones given the huge numbers of applications that are available. This dissertation is motivated by the evolution of mobile phones from mere communication devices to multi-functional tool sets, and the challenges that have arisen as a result. The goal of this thesis is to contribute systems that support the use of mobile applications and to ground these systems’ designs in an understanding of user behavior gained through empirical observations. The contribution of this dissertation is twofold: First, this work aims to understand how people make use of, organize, discover and multitask between the various functionalities that are available for their smartphones. Findings are based on observations of user behavior by conducting studies in the wild. Second, this work aims to assist people in leveraging their smartphones and the functionality that is available in a more effective and efficient way. This results in tools and improved user interfaces for end-users. Given that the number of available applications for smartphones is rapidly increasing, it is crucial to understand how people make use of such applications to support smartphone use in everyday life with better designs for smartphone user interfaces.TRANSCRIPT
Understanding and Supporting Mobile Application Usage
Matthias Böhmer
Dissertation Defense TalkSeptember 6, 2013
Saarland UniversityFaculty of Natural Sciences and Technology ISaarbrücken Graduate School of Computer Science
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
1983
Evolution
Today - Hardware changed
- Connectivity improved
- Apps arose
Growth of Mobile Ecosystem1.1.3 The Age of Application Stores 7
0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000 900,000
1,000,000
Jun-
08
Nov-
08
Apr-0
9
Sep-
09
Feb-
10
Jul-1
0
Dec-
10
May
-11
Oct
-11
Mar
-12
Aug-
12
Jan-
13
Apple AppStore Google Play Market
Figure 1.3: Number of applications available per application stores between June 2008 toJune 2013.14
providers to develop, market and distribute their applications [7], and for end-userssuch platforms provide a convenient way to access applications since the end-usersdo not have to handle any technical details [124]. While the customization of aphone’s look and feel and audio profiles was a very important feature of first mo-bile phones [109], being able to also customize phone’s functionality in terms ofapplications also became increasingly important [17]. As such, the most importantaspect of application stores that we will focus on in this work is that the end-userherself is able to customize the functionality of her own device. Due to the varietyof services available on application stores, e.g. recreational applications and spir-itual applications [53], mobile phones were integrated even deeper into people’slives [17].
Resulting from the popularity of mobile application stores, the number of availableapplications is steadily increasing. At time of writing this thesis there were morethan 775,000 applications available for Apple’s iPhone and more than 900,000 ap-plications for the Android platform.15 One can expect these numbers to be outdatedsoon, and therefore Figure 1.3 shows the recent growth trend of mobile applica-tions stores, based on which a further increase can be anticipated. The number ofapplication downloads, i.e., the number of times people installed applications on14Data source: Wikipedia, http://en.wikipedia.org/wiki/App_Store_(iOS) and http://en.wikipedia.
org/wiki/Google_Play, data interpolated, last accessed on 28.06.2013.15Wikipedia: List of mobile software distribution platforms. http://is.gd/pzjWb6, last accessed on
07.06.2013.
Available Applications
- Number of available mobile apps is increasing- Number of app downloads is growing rapidly- Daily time spent with apps also increases
7
Research Question
How do people use apps on their smartphones, and how can we design systems to support people
in making effective and efficient use of apps?
8
EngineeringTheory Methods
Launching Housekeeping Discovering Multitasking
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
Launching Housekeeping Discovering Multitasking
How do people utilize the apps they have installed?
Related WorkNu
mbe
r of u
sers
Number of apps
Verkasalo, 2009
Froehlich et al., 2007Demumieux and Losquin, 2005
this work
Stud
y Du
ratio
n
11
Do et al., 2011 Falaki et al., 2010
00
resea
rch in
the l
arge
4,000
22,000
App Lifecycle
being used
not being used
closeopen
install uninstall
update
12
Logging app usage AppSensor: Tracing App Usage
who wherewhen how longwhich app
A
Data from Deployment
- 4,125 users from various countries- 22,626 apps from 20 categories- 4.92 million data points- 127 days
14
During Course of a Day
- App usage correlates with circadian circle- Type of apps used changes during the day
25,000
50,000
75,000
100,000
125,000
150,000
175,000
200,000
12 a
m
2 am
4 am
6 am
8 am
10 a
m
12 p
m
2 pm
4 pm
6 pm
8 pm
10 p
m
Appl
icat
ion
laun
ches
15
Application Chains
!"#$
%&"
'#()*%
'#((+,
)*-.)#,
/,.&".-),(
&,.
0),-,*&
1-(&%
2&-3.4
5)6"-")&%78
79&(
#
5):&%.;3&
<+3.)(
&=)-
>&$
%
?"#=
+*.)@
).;
A&:&"&,*&
B&..),C%
B4#D
D),C
B#*)-3
BD#".%
E4&(
&%
E##3%
E"-@&3
F,G,#
$,
B-(D3&% F%&"% HDD%!"#$%&" IJKL MJNL MMJOL PJPL PJML MJQL PJIL PJIL PJKL RJQL RRJOL MJOL PJNL RJSL MJNL RQJNL PJQL PJML OJRL IJIL NJRL KOTMSU ITRUM U'#()*% NJQL UJKL MNJRL PJPL PJIL KJOL PJNL PJIL PJNL QJIL IJSL KJRL PJNL IJIL QJIL KJML PJNL PJKL OJKL IJSL QJPL MRTIQO RTSQK RTIIP
'#((+,)*-.)#, QJSL IJSL NQJQL PJPL PJIL RJQL PJRL PJRL PJIL RJML IJRL IJQL PJML RJPL RJSL KJOL PJKL PJRL QJPL RJKL MJIL KMKTUSK ITOMU KKU/,.&".-),(&,. NJSL NJRL INJRL PJPL PJPL MJML PJNL PJPL PJNL QJNL PJNL IJOL PJPL MJML SJIL MJML MJML PJPL OJML QJNL RNJSL ROP NQ IO
0),-,*& RPJML MJSL MSJML PJPL RJOL IJUL PJIL PJML PJIL RJQL OJNL MJQL PJRL RJQL QJQL NJRL PJSL PJRL RPJNL RJUL MJRL RTKUN MKS RRS1-(&% RRJOL QJUL MPJKL PJPL PJML RQJRL PJML PJKL PJSL RJPL IJRL KJIL PJSL RJQL NJQL KJPL PJOL PJRL OJML RJSL KJIL OTNIP RTPSS UUQ2&-3.4 MJOL KJOL MKJML PJPL PJML IJQL NJRL PJNL RJIL NJRL IJUL MJRL RJNL IJML NJPL KJUL PJOL PJPL RIJKL IJML MJUL RTKNN MIO RMP
5)6"-")&%7879&(# NJPL MJSL IMJML PJPL PJIL IJML PJML IJNL PJOL RJML RJSL MJIL PJML RNJIL RRJUL MJSL PJML PJRL RMJKL MJIL QJQL MTUMN RTPOI UP5):&%.;3& OJIL QJML RSJML PJPL PJRL KJPL PJQL PJNL MJPL PJUL IJML KJML PJSL IJML IOJSL MJRL PJIL PJKL RPJIL IJIL QJQL KTNSM RTMOM MPM
<+3.)(&=)- NJIL RPJQL MOJIL PJPL PJIL RJKL PJNL PJIL PJKL IJQL IJQL NJIL PJML IJPL RJOL KJKL PJML PJKL UJQL MJIL UJRL RITKQR RTMSN QM>&$% MMJNL MJML MMJML PJPL PJQL RJNL PJIL PJRL PJIL RJKL MJUL IJUL PJKL RJKL MJPL MJSL PJKL PJPL NJQL RJPL IJKL IQTRMR RTKKP MRI
?"#=+*.)@).; SJKL QJPL MOJQL PJPL PJKL IJNL PJKL PJIL PJNL IJOL IJOL SJIL RJRL MJOL KJOL QJRL PJNL PJML UJSL IJKL KJKL MRTRRM RTUQK KUOA&:&"&,*& RMJRL KJQL MKJML PJPL PJIL SJQL PJNL PJML RJPL RJPL IJQL KJNL IJUL RJSL QJIL KJRL PJKL PJIL UJOL RJSL KJKL ITNRR QQI RUUB&..),C% OJUL QJNL INJML PJRL PJIL RJOL PJKL QJIL PJSL IJPL IJNL NJUL PJQL PJPL QJNL KJSL PJNL PJQL RRJNL KJOL RRJRL RMTQSN RTONM RB4#DD),C OJQL SJOL IMJIL PJPL PJKL KJOL PJKL PJUL UJNL PJUL IJOL QJIL PJSL MJPL KJSL KJML PJQL PJQL RNJNL RJNL MJOL IRTSOO ITIPS RMI
B#*)-3 IKJRL MJPL MQJML PJPL PJML IJML PJIL PJIL PJML RJIL IJUL IJOL PJML RJQL IJSL RIJKL PJSL PJRL QJML RJIL MJML MQTPON RTQUM IMUBD#".% SJKL KJML KMJML PJRL PJKL IJQL PJKL PJIL PJML RJML MJPL KJOL PJQL IJKL MJOL QJKL SJNL PJPL SJPL RJQL MJUL ITSUM MOS RMQ
E4&(&% OJQL RPJIL MSJIL PJPL PJIL IJKL PJRL PJIL RJKL MJIL PJKL KJSL PJKL MJML NJQL MJNL PJRL RJIL OJNL MJML KJNL RTUIU RSQ RSQE##3% RRJPL QJRL MNJRL PJPL PJIL IJSL PJML PJKL PJNL IJRL IJKL KJIL PJNL IJRL QJQL KJRL PJKL PJIL RQJSL IJOL MJQL OOTURR ITMOK RTMRPE"-@&3 NJSL UJRL MNJIL PJRL PJIL IJML PJML PJQL PJSL RJUL RJNL NJSL PJKL QJPL IJUL KJKL PJML PJIL RPJIL NJNL MJNL RITQQN RTKPM IOR
F,G,#$, RPJSL KJKL KPJOL PJRL PJIL IJRL PJIL PJML PJNL MJUL RJOL MJIL PJML MJUL IJUL KJSL PJML PJIL NJKL RJQL RRJNL KOTMSU RTUSI RTISS
16probability of transitions
Support for App Launching
17
- Adaptive launcher menu- Support visual search for apps- Presenting 5 icons for next app
- Implements different models- Sequentially used apps- Prediction model- Locally most used apps- Most recently used apps- Most frequently used apps
- Application AppKicker- Extension as a widget - Deployed on app store- 53,000 installations
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
Launching Housekeeping Discovering Multitasking
How do people organize applications on their phones?
- Barreau and Nardi, 1995; Ravasio et al., 2004- Studies on file organization on stationary computers- People dedicate screen areas to different purposes - People cluster documents by their types
- Shipman et al., 1995- People create implicit structures
when manipulating layouts
- Ziefle and Bay, 2004- People built mental models of their phone menus
- Häkkilä and Chatfield, 2006- Customization is a way of personalizing devices- Stages: (1st) alter look-and-feel, (2nd) customize
functionality, (3rd) change complicated settings
Related Work
Ravasio et al. 2004
20
2.3.1 General Mobile Phone Use 35
participants were active users of mobile phones, and over 90% had had two or more phones in active use during the last year. The respondents consisted of 42 males (70%) and 18 females (30%), and were predominately in their 20’s (30%) and 30’s (55%). The time participants had used their current (new) mobile phone was mostly between two weeks and a month (43%), or from one to two months (45%). The participants were predominately Finnish.
The study consisted of an online survey, which the participants filled out anonymously. The survey consisted both multiple-choice and free text questions. The following mobile phone customisation items were investigated:
- Background image (wall paper) - Ringing tone - Message alert tone - Screen saver - UI Theme (UI skin) - Audio profiles - Specified a ringing tone for certain contacts - Alarm clock tone - Speech commands - Adding photo to a phonebook contact - Defining fast dial numbers - Reorganising menu items - Soft key shortcuts - Active idle shortcuts - Screen brightness - Screen backlight off timer - Automatic keylock
In addition, we also investigated the editing of access point and email settings, although these are typically not considered as personalisation items. Figure 1 illustrates some of these personalisable elements of the Nokia Series 60 mobile phone.
Figure 1. Personalisation elements on the idle mode screen.
RESULTS
Intensity of Customisation A primary motivation of this survey was to examine how and when users personalise their mobile phones. We asked
each respondent to indicate when, if ever, they personalised each of the seventeen different personalisable features of the Nokia Series 60 mobile phone. The study results illustrated active customisation of the phone, with most personalisation occurring shortly after using the new phone for the first time. Overall we found that 66% of all features were personalised, see table 1. Note the table describes the percentage against all of the answers.
Number of personalised items (n = 983) First Use
First Day
First Week
Later Never
133 (13.5%)
189 (19.2%)
191 (19.4%)
133 (13.5%)
337 (34.3%)
Table 1: Personalisation time period.
Users described the act of personalisation to be both enjoyable and frustrating. Users reported that the personalisation was designed to make the phone feel and appear as ‘your own’, or to make it look and feel closer to a previous phone the users had used. The motivation ‘to make the phone feel like the one I had before’ appeared in several comments, and was linked with comments where the participant wanted to be able to find device functions and navigate the phone menus as they had done with an older phone.
The most common features that were personalised were the ringing tone (customised by 95% of the respondents), audio profiles (93%) and background image (90%). Other features that had been personalised by over 75% of the participants were the UI theme (86%), message alert tone (83%), soft key shortcuts (82%), and menu item reorganisation (76%). The least customised features were automatic keylock (45%), fast dial numbers (42%) and speech commands (38%).
What Was Customised When The results show that half (50%) of all personalisation occurred during the first day, and almost four-fifths (79%) within the first week. The study reveals that personalisation does not happen arbitrarily, but patterns can be seen of what kind of features are customised when.
Audio settings were typically customised very shortly after getting a new phone, and they seem to be the first features to be personalised. During the first time of the use, almost half (43%) of the participants responded that they had changed the ringing tone. The message alert tone and audio profiles were changed almost as commonly (32% and 30% respectively). All other settings were customised much less frequently when using the phone for the first time.
In general, the features affecting on the outer appearance of the phone were personalised most predominately at the beginning. Whereas audio settings were typically personalised in the very beginning, either during the first
410
(a) By Häkkilä and Chatfield [109]
3.1.1. Content personalisation
Users can personalise their iPhones by downloadingapplications from the Apple AppStore. In this way,users add capabilities and content to their devices.Some applications have a one-time monetary cost tothe user, but many are free, allowing all iPhone usersto personalise the content of their iPhones.
In scoring content personalisation, more applica-tions added to the iPhone represented a higherdegree of personalisation. A common sigmoid func-tion was used that grows the most near 100applications, but slowest at the extreme ends of thescale. Default applications are not included in theequation, since their presence does not indicate anycustomisation.
Table 1. Items measured to determine personalisation scores.
Personalisation item Measurement and (weight) Scoring
ContentInstalled apps Count of new apps (.10) No new apps – 0
Each new app increasesScore on sigmoid function
InterfaceMoved apps on bottom bar (BB) Count of apps moved from bottom bar
(.15)1 – % of original apps in bar
Assess order of apps on bottom bar (.15) .25 for each app that movedFrom original location on the bottom
bar
Moved apps on 1st screen Count apps moved from first page (.15) 1 – % of original apps on page
Assess order of apps on first page (.15) 1 pt for each nearest neighbour in samecategory and divide by number ofapps on page
1st & subsequent screens Count of holes on each page (.15) Count number of holes on each page
Ringtones View voice call settings (.05) No change – 0Change, no download – 50Change with download – 100
Physical/appearanceiPhone case View exterior of phone (.05) No case – 0
Case – 100
Lockscreen image View lockscreen (.05) No change – 0Change to library image – 50Change to personal image – 100
Figure 1. iPhone lockscreen and springboard pages.
4 C.C. Tossell et al.
(b) By Tossel et al. [246]
Figure 2.1: Personalization elements of mobile phones as taken into account in re-lated work.
phone calls. Comparing genders, the paper reports that men are more likely touse applications like games, video and office applications.
Häkkilä and Chatfield [109] investigate the different ways people customize theirmobile phones and study different properties that people modify, such as audiosettings and look and feel like those shown in Figure 2.1. Their study of 60 partic-ipants reveals that customization is a highly relevant way of personalizing devices.Häkkilä and Chatfield were able to distinguish temporal patterns: While modifi-cations of device look and feel are early personalizations, functional settings (e.g.short cuts and quick dial keys) are changed over the long term, and even morecomplicated features (e.g. access point settings) are often left unchanged. AlsoTossel et al. [246] investigate personalization of smartphones, and look for genderdifferences, and relations between personalization, device use and perceived us-ability. They develop a score to assess a user’s personalization of her device (foriPhone only). This score for instance takes into account the number of applicationsinstalled, moves of icons from the quick launch menu and on the first page of themenu, adding an phone case, and setting lock screen image. They collect data fromiPhone users and find that people who personalize their phones perceive it as moreusable. They also report that women customize their phones differently than men;e.g., females are more concerned about appearance. However, they do not qualita-tively investigate the concepts that people use for arranging icons; this is what wedo in Chapter 4.
In [196] Rahmati and Zhong present a four-month study on smartphone usage by14 teenage novice users with little or not experience with mobile phones. The studytook place in 2007 and authors traced mobile phone usage (power, display status,
Häkkilä & Chatfield, 2006
Study method
Quantitative data, e.g.- number of apps- number of folders- number of icons on page- x/y position of icons
Qualitative data- participants‘ experience levels- concepts of icon arrangement- participants labeled with
concepts„most used apps first page, groups of apps 2nd space, then games“
„most-used items should be on the first page, otherwise I try to group items (e.g., news outlets together)“
...
1
2
Screenshot Study
grounded theory
majority rule
template matching
- 132 participants- 1,486 screenshots
Usage-based icon arrangement
Relatedness-based icon arrangement
Usability-based icon arrangement
Aesthetics-based icon arrangement
External concepts for icon arrangement
llll lllllllll
l ll l
ABC123...
=?
22
5 Concepts for Arranging Icons4.3.2 Results of Screenshot Study 117
(a) Usage-based (b) Relatedness-based (c) Usability-based (d) Aesthetic-based
Figure 4.5: Example screenshots of participants who used certain concepts for arrangingtheir icons: (a) one participant who reports to “put the most frequently used applicationson the first screen”; (b) a user with five folders on his first page who tries “to group[applications] by what they do or what I use them for”; (c) a participant who says hewould “keep third row available for easy swiping to the next page”; (d) a participant whohas created a checkerboard pattern: “most icons are blue, so on my first page of icons italternates between blue and brown and I try to keep that consistency throughout”.
Some people also explicitly stated that they have no concept for arrangingtheir icons. Yet, since every icon arrangement has an inherent order, it isunclear how this order emerged. It is most likely that people who do nothave any explicit concept also follow an external concept, e.g. just leave thearrangement as it was preinstalled or add the icons of new installed applica-tions to the first free spot in the menu.
Hybrid Concepts and Co-Occurrence. It is worth mentioning that these fiveconcepts are not mutually exclusive, i.e. a user may apply two or more conceptsin parallel. For further analysis, all participants have been categorized based onthe five concepts we found. To reduce the subjectiveness of the categorization, thelabeling has been done by three different analysts whose results have been mergedby the principle of majority rule. For this reason we can take their merged clas-sification as ground truth. We have been able to partially cross-validate people’stextual description given in their self-reports with the screenshots they provided:For people who said that they group by similarity, we found folders of applications,and those who claimed to exploit icons’ colors have also been proven to be right;for instance the participant who said that “most icons are blue, so on my first pageof icons it alternates between blue and brown and I try to keep that consistencythroughout” is shown in Figure 4.5(d) — while the color blue is recognizable, thecolor brown is rather fuzzy. We had to trust participants’ self-reporting feedback
4.3.2 Results of Screenshot Study 117
(a) Usage-based (b) Relatedness-based (c) Usability-based (d) Aesthetic-based
Figure 4.5: Example screenshots of participants who used certain concepts for arrangingtheir icons: (a) one participant who reports to “put the most frequently used applicationson the first screen”; (b) a user with five folders on his first page who tries “to group[applications] by what they do or what I use them for”; (c) a participant who says hewould “keep third row available for easy swiping to the next page”; (d) a participant whohas created a checkerboard pattern: “most icons are blue, so on my first page of icons italternates between blue and brown and I try to keep that consistency throughout”.
Some people also explicitly stated that they have no concept for arrangingtheir icons. Yet, since every icon arrangement has an inherent order, it isunclear how this order emerged. It is most likely that people who do nothave any explicit concept also follow an external concept, e.g. just leave thearrangement as it was preinstalled or add the icons of new installed applica-tions to the first free spot in the menu.
Hybrid Concepts and Co-Occurrence. It is worth mentioning that these fiveconcepts are not mutually exclusive, i.e. a user may apply two or more conceptsin parallel. For further analysis, all participants have been categorized based onthe five concepts we found. To reduce the subjectiveness of the categorization, thelabeling has been done by three different analysts whose results have been mergedby the principle of majority rule. For this reason we can take their merged clas-sification as ground truth. We have been able to partially cross-validate people’stextual description given in their self-reports with the screenshots they provided:For people who said that they group by similarity, we found folders of applications,and those who claimed to exploit icons’ colors have also been proven to be right;for instance the participant who said that “most icons are blue, so on my first pageof icons it alternates between blue and brown and I try to keep that consistencythroughout” is shown in Figure 4.5(d) — while the color blue is recognizable, thecolor brown is rather fuzzy. We had to trust participants’ self-reporting feedback
4.3.2 Results of Screenshot Study 117
(a) Usage-based (b) Relatedness-based (c) Usability-based (d) Aesthetic-based
Figure 4.5: Example screenshots of participants who used certain concepts for arrangingtheir icons: (a) one participant who reports to “put the most frequently used applicationson the first screen”; (b) a user with five folders on his first page who tries “to group[applications] by what they do or what I use them for”; (c) a participant who says hewould “keep third row available for easy swiping to the next page”; (d) a participant whohas created a checkerboard pattern: “most icons are blue, so on my first page of icons italternates between blue and brown and I try to keep that consistency throughout”.
Some people also explicitly stated that they have no concept for arrangingtheir icons. Yet, since every icon arrangement has an inherent order, it isunclear how this order emerged. It is most likely that people who do nothave any explicit concept also follow an external concept, e.g. just leave thearrangement as it was preinstalled or add the icons of new installed applica-tions to the first free spot in the menu.
Hybrid Concepts and Co-Occurrence. It is worth mentioning that these fiveconcepts are not mutually exclusive, i.e. a user may apply two or more conceptsin parallel. For further analysis, all participants have been categorized based onthe five concepts we found. To reduce the subjectiveness of the categorization, thelabeling has been done by three different analysts whose results have been mergedby the principle of majority rule. For this reason we can take their merged clas-sification as ground truth. We have been able to partially cross-validate people’stextual description given in their self-reports with the screenshots they provided:For people who said that they group by similarity, we found folders of applications,and those who claimed to exploit icons’ colors have also been proven to be right;for instance the participant who said that “most icons are blue, so on my first pageof icons it alternates between blue and brown and I try to keep that consistencythroughout” is shown in Figure 4.5(d) — while the color blue is recognizable, thecolor brown is rather fuzzy. We had to trust participants’ self-reporting feedback
4.3.2 Results of Screenshot Study 117
(a) Usage-based (b) Relatedness-based (c) Usability-based (d) Aesthetic-based
Figure 4.5: Example screenshots of participants who used certain concepts for arrangingtheir icons: (a) one participant who reports to “put the most frequently used applicationson the first screen”; (b) a user with five folders on his first page who tries “to group[applications] by what they do or what I use them for”; (c) a participant who says hewould “keep third row available for easy swiping to the next page”; (d) a participant whohas created a checkerboard pattern: “most icons are blue, so on my first page of icons italternates between blue and brown and I try to keep that consistency throughout”.
Some people also explicitly stated that they have no concept for arrangingtheir icons. Yet, since every icon arrangement has an inherent order, it isunclear how this order emerged. It is most likely that people who do nothave any explicit concept also follow an external concept, e.g. just leave thearrangement as it was preinstalled or add the icons of new installed applica-tions to the first free spot in the menu.
Hybrid Concepts and Co-Occurrence. It is worth mentioning that these fiveconcepts are not mutually exclusive, i.e. a user may apply two or more conceptsin parallel. For further analysis, all participants have been categorized based onthe five concepts we found. To reduce the subjectiveness of the categorization, thelabeling has been done by three different analysts whose results have been mergedby the principle of majority rule. For this reason we can take their merged clas-sification as ground truth. We have been able to partially cross-validate people’stextual description given in their self-reports with the screenshots they provided:For people who said that they group by similarity, we found folders of applications,and those who claimed to exploit icons’ colors have also been proven to be right;for instance the participant who said that “most icons are blue, so on my first pageof icons it alternates between blue and brown and I try to keep that consistencythroughout” is shown in Figure 4.5(d) — while the color blue is recognizable, thecolor brown is rather fuzzy. We had to trust participants’ self-reporting feedback
Occurrences of Concepts(1) (2) (3) (4) (5)
(1) usage-based
(2) relatedness-based
(3) usability based
(4) aesthetic-based
(5) external concepts
62 % 28 % 6 % 2 % 4 %
28 % 60 % 6 % 3 % 3 %
6 % 6 % 9 % 2 % 0 %
2 % 3 % 2 % 5 % 0 %
4 % 3 % 0 % 0 % 9 %
- Usage-based and relatedness-based most used- Participants applied hybrid concepts- Concept impacts icon layout
- More apps on first page if usage-based- More folders on first page if relatedness-based 23
% o
f par
ticip
ants
usin
g co
ncep
ts
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
Launching Housekeeping Discovering Multitasking
How can we support people discovering new applications?
- Finding good apps can become a difficult task- Recommender systems for discovery- Mobile apps are a special type of items- Context is important
- The location of the user- The time of the day- The previously used application
26
App Recommender
Related Work
27
- Woerndl et al., 2007- Location-aware recommender system for apps- Only based on installations as feedback
- Jannach and Hegelich, 2009 - Recommendation of mobile games- Personalization increased number of views and sales
- AppAware by Girardello and Michahelles, 2010- Tracking installations, updates, and removals of apps- Recommending popular apps by aggregating events
- AppJoy by Yan and Chen, 2011- Based on recency, frequency and usage times of apps
- No usage-centric evaluation method available
52 2.3 Related Work
Figure 3 shows the user interface to configure the triggers. The administrator can select types of Point-of-Interests and specify within what circumference of an actual POI an application is recommended. The types in Fig. 3 are some of the return POI type values of the ArcWeb service. The POI types represent the context attributes that are used for this recommender.
The advantage of this recommender is that administrators can specify exactly when an application is suitable (rule-based recommender). On the negative side, the registration of applications requires additional effort.
4.3. User interface
In this chapter we describe the user interface of our application. The screens are indented to be as simple as possible. The user interface is in German only at this time.
A user can start the client program and log in. She then gets a form where she can select one of our four recommenders (Fig. 4). In this screen the user gets a list with a non-technical explanation of the available recommenders. For example, the CFAppRecommender(2.row) reads: “Users with a comparable taste chose …”.
Figure 4. Step 1: choosing a recommender
As result of selecting one recommender process, the user receives a list of recommended mobile applications on the next screen (Fig. 5). The list is ranked, i.e. the “best” or most suitable application – according to the used recommendation algorithm – is on top. Then, she can browse the list and choose an application she is interested in.
Figure 5. Step 2: list of recommended applications
By selecting one application, the detailed description is shown (Fig. 6) and the user can either install or use the application, or select “Nicht wieder empfehlen” (do not recommend anymore) to express that she does not want to use this application. The user can always click “Zurück” (back) to go back to the previous screen.
Figure 6. Step 3: application details
The actions of the user are recorded to be used in subsequent recommendations. If the user starts an application, this information is stored as a positive rating for the CFAppRecommender. The ratings include available context information. At this time, we record the GPS coordinates when the application is started. Other than the option to express dislike (Fig. 6) in an application, we do not let users explicitly rate applications, because this would presumably be too
876Authorized licensed use limited to: FH Munster. Downloaded on March 06,2010 at 14:31:35 EST from IEEE Xplore. Restrictions apply.
(a) Woerndl et al. [262]Figure 1: Catalog navigation and categories
Method Description
CF-Item Item-to-item collaborative filtering [7].Content-based A content-based method based on item
descriptions and the cosine similarity ofTF-IDF vectors.
Hybrid A switching hybrid of the first twowhich used the content-based methodin cases where less than 8 item ratingswere available.
SlopeOne An item-based filtering technique de-scribed in [4].
TopSeller Ordering based on sales rank.TopRating Ordering based on average customer
rating.
Control Manually-edited lists (mostly based onrelease date) as before the experiment;no “My Recommendations” section.
Customers remained in their groups to which they wererandomly assigned throughout the experiment. Each groupcomprised around 22,300 customers. Note that from all por-tal visitors, only such customers were selected for the exper-iment, for which all algorithms were able to provide rec-ommendations. Thus, it was also ensured that only similarcustomer groups (frequent users) were compared. The itemcatalog consisted of about 1,000 games. Since the numberof explicit item ratings was particularly low (less than 2% ofthe customers issued at least one rating), also implicit itemratings were taken into account. On a rating scale from�2 to +2, an item view action was interpreted as a ratingof 0 (medium); an actual purchase corresponds to a ratingof 1 (good). Explicit customer ratings, finally, override theimplicit ones.
2. RESULTS ANALYSISIn order to determine the “business value” of a recom-
mender system (as opposed to the system’s accuracy), di�er-ent measurements can be made. Obviously, one can measureand compare the total number of sold items per recommen-
dation technique. In addition, one can determine whetheror not personalized recommendations can raise additionalinterest in certain items by measuring the number of itemviews. This number might be particularly important in pay-per-click advertisement scenarios. Finally, also “conversionrates” that, e.g., measure how many site visitors are turnedinto buyers, can serve as an indicator for the business value.
In this paper, we focus on the question, whether usersthat receive personalized recommendations (a) view moreitems, and (b) buy more items than those users that re-ceive non-personalized or no recommendations. In our ex-periments (see [3] for additional measurements and results)also hypotheses regarding two di�erent conversion rates weretested. In short, these measurements show that in certainnavigational situations a slight increase with respect to con-version rates can be observed (e.g., more visitors also ac-tually purchased at least one item). Overall, however, rec-ommender systems did not measurably help to turn morevisitors into buyers, most probably because the conversionrates are already relatively high as we only consider frequentusers. Long-term e�ects and“indirect revenues”as describedin [1] were also not in the focus of the current study, becausein our application domain items are only bought once.
In the following, we summarize a selected subset of im-portant observations from our study in the context of thefollowing navigational situations; see [3] for more details.
My Recommendations:Figure 2 shows on how many items a user clicked when
viewing the personalized “My Recommendations” list.1 Wecan observe that all personalized methods (except Slope-One) successfully stimulated users to view the details of theproposed items, i.e., they have raised more interest in theo�ered items than a simple list of top-selling or top-rateditems did. The di�erences between the following groups oftechniques are statistically significant (p < 0.01) when con-sidering the absolute numbers: CF-Item/Hybrid – Content-based – SlopeOne/TopRating/Topseller.
Figure 2: Average number of item detail views per“My Recommendations” visits
With respect to downloads (Figure 3) things look dif-ferent. Although the content-based method raised addi-tional interest with respect to item views, this did not leadto significantly more downloads compared with the non-personalized methods. This indicates that users are gen-erally interested in things they have bought before (e.g.,sports games) as they view many of them. Once they haveseen di�erent ones, they however download only one game
1As mentioned above, the control group had no access tothe “My Recommendations” section.
206
(b) Jannach and Hegelich [132]
section 4 we follow up discussing its implications for then summarizing the AppAware idea in section 5.
2. RELATED WORK
In this section, we briefly review the state of the art and related work that have informed our design and indicate how AppAware differs from these.
2.1 Mobile Applications Websites
At present, the Android application portal can be accessed just from the Market mobile application and, in a limited way, from the related website. To overcome these design decisions by Google, many third-party developers are launching new services to access applications’ details from a personal computer. These services enable users to search for and download Android applications on the web instead of doing it directly from a mobile device. Good examples are AndroLib2 and AppBrain3. The major difference between the two is that AppBrain provides a user with an applications shopping cart that can be synced with the device through an Android client application. However, the idea is not innovative since it is trying to port the concept of Apple's iTunes to the Android world. In fact, iTunes already allows its users to browse and sync applications from their computer to an iPhone. AppAware does not aim at replacing the Android Market or providing a proxy, it is rather a companion to plan users' serendipity [5] in applications finding.
2.2 Appazaar and aTrackDog
Appazaar [2] is a recommender system for mobile applications, and is a project of the Lab for Software Engineering at Münster University of Applied Sciences. Based on a user current and historical locations and applications usage, Appazaar recommends applications that might be of interest for her. Therefore, Appazaar applies different algorithms from the research field of context awareness to analyze all the input data and create profiles of different situations. Another tool related to AppAware is called aTrackDog4. It is a program for Android devices that makes sure a user has the latest version of every installed application by checking the release information from either the Android Market, other users’ devices or the vendors' web site. Doing this manually takes time, thus aTrackDog supports the user in this activity. Even more, data from users’ devices is used to generate a most popular apps list that can be sorted by category, time, and price. Despite AppAware generates similar stats and providing apps recommendations is an appealing feature, we focused towards new ways to explore mobile apps on an application portal (i.e. real time stream, proximity based) and we further underlined the users presence into these activities.
3. CONCEPT AND DESIGN
In this section, we describe the system design, AppAware's most relevant features and their implementation in the user interface.
2 http://www.androlib.com 3 http://www.appbrain.com 4 http://atrackdog.a0soft.com
AppAware shares online users' installations, updates and removals of Android applications. In this way a user becomes conscious of what is hot on the Android application portal. To meet these conditions, the AppAware system consists in a client-server architecture.
3.1 General Concept
The client component in this system is the Android mobile application, which represents AppAware's graphical user interface (GUI) and allows following installations, updates and removals of applications shared by other users. Most of the core functionalities are supported by the main screen (see Figure 2) and are accessible from the application's menu or by touching a list item. Each list item represents a single event with its details, namely: the name of the application with its description, the type of event (installed, updated or removed), the user involved and the Android version together with the phone model. Moreover, to distinguish the type of event at a glance the application’s name is colored in red in case of a removal, green for an installation and blue for an update.
Figure 2. Real-time stream of installed, updated and removed
applications (a) and analogous events in user’s proximity (b).
The server component is a web application accessible though a standard browser and at the same time integrated in the mobile client through an Android WebView element that displays web pages. The client connects to the server through a RESTful interface that accepts and then stores events from users. Among the required parameters we have: the user ID, the application package name (used by the Android Market as unique identifier for an app), event type (installed, updated or removed) and the location (latitude and longitude, whenever allowed by the user). The server offers its data through an RSS feed that the AppAware client uses as data source for its core functionalities as described in Sections 3.2 and 3.3. This design decision allows at the same time any standard feed reader to keep track of installations, updates and removals of Android apps. Along with this architecture, AppAware integrates with Twitter too. This allows a user to share applications' installations, removals and updates on the Twitter account, thus letting her followers to see what applications are being installed by that user. An example of a Twitter status update is: “Just updated Google Translate http://appaware.org/1z on my #Nexus One - via #AppAware”. This tweet tells that the application “Google Translate” has been just updated on the Android phone model
432
(c) AppAware [103]
Figure 2.5: Screenshots of systems recommending mobile software for download.
applications also became a recent research topic, especially in the field of context-aware recommendation, since mobile application usage is context-dependent (aswe will discuss in Chapter 3). However, although the ecosystem of mobile appli-cations is rapidly growing as discussed in Chapter 1, so far there is little researchon recommender systems for mobile applications.
Pioneering work before the application store era was done by Woerndl et al. [262]and Jannach and Hegelich [132]. Woerndl et al. [262] present a recommendersystem for mobile applications that exploits context information and is based oncapturing the installations of applications in relation to this context (basically lo-cation), though installation times are irrelevant compared to measuring the actualusage.19 Their recommender engine is based on a hybrid engine following a multi-dimensional approach. Jannach and Hegelich [132] evaluate recommender enginessuggesting game applications for downloads on a mobile internet platform. Theyfound that the personalization of recommendations results in an increased numberof views and sales. They did not investigate contextual factors at that time.
Girardello et al. [103, 104] present AppAware: a recommender system that is basedon people’s overall application installations, uninstallations and updates. The sys-tem recommends applications based on their popularity, i.e. how many times anapplication was installed but not removed. The system’s assumption is that goodapplications are typically not removed once installed. The AppAware system also19Jakob Nielsen’s Alertbox: iPhone Apps Need Low Starting Hurdles. http://tiny.cc/eyie8, last ac-
cessed on 12.06.2013.
52 2.3 Related Work
Figure 3 shows the user interface to configure the triggers. The administrator can select types of Point-of-Interests and specify within what circumference of an actual POI an application is recommended. The types in Fig. 3 are some of the return POI type values of the ArcWeb service. The POI types represent the context attributes that are used for this recommender.
The advantage of this recommender is that administrators can specify exactly when an application is suitable (rule-based recommender). On the negative side, the registration of applications requires additional effort.
4.3. User interface
In this chapter we describe the user interface of our application. The screens are indented to be as simple as possible. The user interface is in German only at this time.
A user can start the client program and log in. She then gets a form where she can select one of our four recommenders (Fig. 4). In this screen the user gets a list with a non-technical explanation of the available recommenders. For example, the CFAppRecommender(2.row) reads: “Users with a comparable taste chose …”.
Figure 4. Step 1: choosing a recommender
As result of selecting one recommender process, the user receives a list of recommended mobile applications on the next screen (Fig. 5). The list is ranked, i.e. the “best” or most suitable application – according to the used recommendation algorithm – is on top. Then, she can browse the list and choose an application she is interested in.
Figure 5. Step 2: list of recommended applications
By selecting one application, the detailed description is shown (Fig. 6) and the user can either install or use the application, or select “Nicht wieder empfehlen” (do not recommend anymore) to express that she does not want to use this application. The user can always click “Zurück” (back) to go back to the previous screen.
Figure 6. Step 3: application details
The actions of the user are recorded to be used in subsequent recommendations. If the user starts an application, this information is stored as a positive rating for the CFAppRecommender. The ratings include available context information. At this time, we record the GPS coordinates when the application is started. Other than the option to express dislike (Fig. 6) in an application, we do not let users explicitly rate applications, because this would presumably be too
876Authorized licensed use limited to: FH Munster. Downloaded on March 06,2010 at 14:31:35 EST from IEEE Xplore. Restrictions apply.
(a) Woerndl et al. [262]Figure 1: Catalog navigation and categories
Method Description
CF-Item Item-to-item collaborative filtering [7].Content-based A content-based method based on item
descriptions and the cosine similarity ofTF-IDF vectors.
Hybrid A switching hybrid of the first twowhich used the content-based methodin cases where less than 8 item ratingswere available.
SlopeOne An item-based filtering technique de-scribed in [4].
TopSeller Ordering based on sales rank.TopRating Ordering based on average customer
rating.
Control Manually-edited lists (mostly based onrelease date) as before the experiment;no “My Recommendations” section.
Customers remained in their groups to which they wererandomly assigned throughout the experiment. Each groupcomprised around 22,300 customers. Note that from all por-tal visitors, only such customers were selected for the exper-iment, for which all algorithms were able to provide rec-ommendations. Thus, it was also ensured that only similarcustomer groups (frequent users) were compared. The itemcatalog consisted of about 1,000 games. Since the numberof explicit item ratings was particularly low (less than 2% ofthe customers issued at least one rating), also implicit itemratings were taken into account. On a rating scale from�2 to +2, an item view action was interpreted as a ratingof 0 (medium); an actual purchase corresponds to a ratingof 1 (good). Explicit customer ratings, finally, override theimplicit ones.
2. RESULTS ANALYSISIn order to determine the “business value” of a recom-
mender system (as opposed to the system’s accuracy), di�er-ent measurements can be made. Obviously, one can measureand compare the total number of sold items per recommen-
dation technique. In addition, one can determine whetheror not personalized recommendations can raise additionalinterest in certain items by measuring the number of itemviews. This number might be particularly important in pay-per-click advertisement scenarios. Finally, also “conversionrates” that, e.g., measure how many site visitors are turnedinto buyers, can serve as an indicator for the business value.
In this paper, we focus on the question, whether usersthat receive personalized recommendations (a) view moreitems, and (b) buy more items than those users that re-ceive non-personalized or no recommendations. In our ex-periments (see [3] for additional measurements and results)also hypotheses regarding two di�erent conversion rates weretested. In short, these measurements show that in certainnavigational situations a slight increase with respect to con-version rates can be observed (e.g., more visitors also ac-tually purchased at least one item). Overall, however, rec-ommender systems did not measurably help to turn morevisitors into buyers, most probably because the conversionrates are already relatively high as we only consider frequentusers. Long-term e�ects and“indirect revenues”as describedin [1] were also not in the focus of the current study, becausein our application domain items are only bought once.
In the following, we summarize a selected subset of im-portant observations from our study in the context of thefollowing navigational situations; see [3] for more details.
My Recommendations:Figure 2 shows on how many items a user clicked when
viewing the personalized “My Recommendations” list.1 Wecan observe that all personalized methods (except Slope-One) successfully stimulated users to view the details of theproposed items, i.e., they have raised more interest in theo�ered items than a simple list of top-selling or top-rateditems did. The di�erences between the following groups oftechniques are statistically significant (p < 0.01) when con-sidering the absolute numbers: CF-Item/Hybrid – Content-based – SlopeOne/TopRating/Topseller.
Figure 2: Average number of item detail views per“My Recommendations” visits
With respect to downloads (Figure 3) things look dif-ferent. Although the content-based method raised addi-tional interest with respect to item views, this did not leadto significantly more downloads compared with the non-personalized methods. This indicates that users are gen-erally interested in things they have bought before (e.g.,sports games) as they view many of them. Once they haveseen di�erent ones, they however download only one game
1As mentioned above, the control group had no access tothe “My Recommendations” section.
206
(b) Jannach and Hegelich [132]
section 4 we follow up discussing its implications for then summarizing the AppAware idea in section 5.
2. RELATED WORK
In this section, we briefly review the state of the art and related work that have informed our design and indicate how AppAware differs from these.
2.1 Mobile Applications Websites
At present, the Android application portal can be accessed just from the Market mobile application and, in a limited way, from the related website. To overcome these design decisions by Google, many third-party developers are launching new services to access applications’ details from a personal computer. These services enable users to search for and download Android applications on the web instead of doing it directly from a mobile device. Good examples are AndroLib2 and AppBrain3. The major difference between the two is that AppBrain provides a user with an applications shopping cart that can be synced with the device through an Android client application. However, the idea is not innovative since it is trying to port the concept of Apple's iTunes to the Android world. In fact, iTunes already allows its users to browse and sync applications from their computer to an iPhone. AppAware does not aim at replacing the Android Market or providing a proxy, it is rather a companion to plan users' serendipity [5] in applications finding.
2.2 Appazaar and aTrackDog
Appazaar [2] is a recommender system for mobile applications, and is a project of the Lab for Software Engineering at Münster University of Applied Sciences. Based on a user current and historical locations and applications usage, Appazaar recommends applications that might be of interest for her. Therefore, Appazaar applies different algorithms from the research field of context awareness to analyze all the input data and create profiles of different situations. Another tool related to AppAware is called aTrackDog4. It is a program for Android devices that makes sure a user has the latest version of every installed application by checking the release information from either the Android Market, other users’ devices or the vendors' web site. Doing this manually takes time, thus aTrackDog supports the user in this activity. Even more, data from users’ devices is used to generate a most popular apps list that can be sorted by category, time, and price. Despite AppAware generates similar stats and providing apps recommendations is an appealing feature, we focused towards new ways to explore mobile apps on an application portal (i.e. real time stream, proximity based) and we further underlined the users presence into these activities.
3. CONCEPT AND DESIGN
In this section, we describe the system design, AppAware's most relevant features and their implementation in the user interface.
2 http://www.androlib.com 3 http://www.appbrain.com 4 http://atrackdog.a0soft.com
AppAware shares online users' installations, updates and removals of Android applications. In this way a user becomes conscious of what is hot on the Android application portal. To meet these conditions, the AppAware system consists in a client-server architecture.
3.1 General Concept
The client component in this system is the Android mobile application, which represents AppAware's graphical user interface (GUI) and allows following installations, updates and removals of applications shared by other users. Most of the core functionalities are supported by the main screen (see Figure 2) and are accessible from the application's menu or by touching a list item. Each list item represents a single event with its details, namely: the name of the application with its description, the type of event (installed, updated or removed), the user involved and the Android version together with the phone model. Moreover, to distinguish the type of event at a glance the application’s name is colored in red in case of a removal, green for an installation and blue for an update.
Figure 2. Real-time stream of installed, updated and removed
applications (a) and analogous events in user’s proximity (b).
The server component is a web application accessible though a standard browser and at the same time integrated in the mobile client through an Android WebView element that displays web pages. The client connects to the server through a RESTful interface that accepts and then stores events from users. Among the required parameters we have: the user ID, the application package name (used by the Android Market as unique identifier for an app), event type (installed, updated or removed) and the location (latitude and longitude, whenever allowed by the user). The server offers its data through an RSS feed that the AppAware client uses as data source for its core functionalities as described in Sections 3.2 and 3.3. This design decision allows at the same time any standard feed reader to keep track of installations, updates and removals of Android apps. Along with this architecture, AppAware integrates with Twitter too. This allows a user to share applications' installations, removals and updates on the Twitter account, thus letting her followers to see what applications are being installed by that user. An example of a Twitter status update is: “Just updated Google Translate http://appaware.org/1z on my #Nexus One - via #AppAware”. This tweet tells that the application “Google Translate” has been just updated on the Android phone model
432
(c) AppAware [103]
Figure 2.5: Screenshots of systems recommending mobile software for download.
applications also became a recent research topic, especially in the field of context-aware recommendation, since mobile application usage is context-dependent (aswe will discuss in Chapter 3). However, although the ecosystem of mobile appli-cations is rapidly growing as discussed in Chapter 1, so far there is little researchon recommender systems for mobile applications.
Pioneering work before the application store era was done by Woerndl et al. [262]and Jannach and Hegelich [132]. Woerndl et al. [262] present a recommendersystem for mobile applications that exploits context information and is based oncapturing the installations of applications in relation to this context (basically lo-cation), though installation times are irrelevant compared to measuring the actualusage.19 Their recommender engine is based on a hybrid engine following a multi-dimensional approach. Jannach and Hegelich [132] evaluate recommender enginessuggesting game applications for downloads on a mobile internet platform. Theyfound that the personalization of recommendations results in an increased numberof views and sales. They did not investigate contextual factors at that time.
Girardello et al. [103, 104] present AppAware: a recommender system that is basedon people’s overall application installations, uninstallations and updates. The sys-tem recommends applications based on their popularity, i.e. how many times anapplication was installed but not removed. The system’s assumption is that goodapplications are typically not removed once installed. The AppAware system also19Jakob Nielsen’s Alertbox: iPhone Apps Need Low Starting Hurdles. http://tiny.cc/eyie8, last ac-
cessed on 12.06.2013.
52 2.3 Related Work
Figure 3 shows the user interface to configure the triggers. The administrator can select types of Point-of-Interests and specify within what circumference of an actual POI an application is recommended. The types in Fig. 3 are some of the return POI type values of the ArcWeb service. The POI types represent the context attributes that are used for this recommender.
The advantage of this recommender is that administrators can specify exactly when an application is suitable (rule-based recommender). On the negative side, the registration of applications requires additional effort.
4.3. User interface
In this chapter we describe the user interface of our application. The screens are indented to be as simple as possible. The user interface is in German only at this time.
A user can start the client program and log in. She then gets a form where she can select one of our four recommenders (Fig. 4). In this screen the user gets a list with a non-technical explanation of the available recommenders. For example, the CFAppRecommender(2.row) reads: “Users with a comparable taste chose …”.
Figure 4. Step 1: choosing a recommender
As result of selecting one recommender process, the user receives a list of recommended mobile applications on the next screen (Fig. 5). The list is ranked, i.e. the “best” or most suitable application – according to the used recommendation algorithm – is on top. Then, she can browse the list and choose an application she is interested in.
Figure 5. Step 2: list of recommended applications
By selecting one application, the detailed description is shown (Fig. 6) and the user can either install or use the application, or select “Nicht wieder empfehlen” (do not recommend anymore) to express that she does not want to use this application. The user can always click “Zurück” (back) to go back to the previous screen.
Figure 6. Step 3: application details
The actions of the user are recorded to be used in subsequent recommendations. If the user starts an application, this information is stored as a positive rating for the CFAppRecommender. The ratings include available context information. At this time, we record the GPS coordinates when the application is started. Other than the option to express dislike (Fig. 6) in an application, we do not let users explicitly rate applications, because this would presumably be too
876Authorized licensed use limited to: FH Munster. Downloaded on March 06,2010 at 14:31:35 EST from IEEE Xplore. Restrictions apply.
(a) Woerndl et al. [262]Figure 1: Catalog navigation and categories
Method Description
CF-Item Item-to-item collaborative filtering [7].Content-based A content-based method based on item
descriptions and the cosine similarity ofTF-IDF vectors.
Hybrid A switching hybrid of the first twowhich used the content-based methodin cases where less than 8 item ratingswere available.
SlopeOne An item-based filtering technique de-scribed in [4].
TopSeller Ordering based on sales rank.TopRating Ordering based on average customer
rating.
Control Manually-edited lists (mostly based onrelease date) as before the experiment;no “My Recommendations” section.
Customers remained in their groups to which they wererandomly assigned throughout the experiment. Each groupcomprised around 22,300 customers. Note that from all por-tal visitors, only such customers were selected for the exper-iment, for which all algorithms were able to provide rec-ommendations. Thus, it was also ensured that only similarcustomer groups (frequent users) were compared. The itemcatalog consisted of about 1,000 games. Since the numberof explicit item ratings was particularly low (less than 2% ofthe customers issued at least one rating), also implicit itemratings were taken into account. On a rating scale from�2 to +2, an item view action was interpreted as a ratingof 0 (medium); an actual purchase corresponds to a ratingof 1 (good). Explicit customer ratings, finally, override theimplicit ones.
2. RESULTS ANALYSISIn order to determine the “business value” of a recom-
mender system (as opposed to the system’s accuracy), di�er-ent measurements can be made. Obviously, one can measureand compare the total number of sold items per recommen-
dation technique. In addition, one can determine whetheror not personalized recommendations can raise additionalinterest in certain items by measuring the number of itemviews. This number might be particularly important in pay-per-click advertisement scenarios. Finally, also “conversionrates” that, e.g., measure how many site visitors are turnedinto buyers, can serve as an indicator for the business value.
In this paper, we focus on the question, whether usersthat receive personalized recommendations (a) view moreitems, and (b) buy more items than those users that re-ceive non-personalized or no recommendations. In our ex-periments (see [3] for additional measurements and results)also hypotheses regarding two di�erent conversion rates weretested. In short, these measurements show that in certainnavigational situations a slight increase with respect to con-version rates can be observed (e.g., more visitors also ac-tually purchased at least one item). Overall, however, rec-ommender systems did not measurably help to turn morevisitors into buyers, most probably because the conversionrates are already relatively high as we only consider frequentusers. Long-term e�ects and“indirect revenues”as describedin [1] were also not in the focus of the current study, becausein our application domain items are only bought once.
In the following, we summarize a selected subset of im-portant observations from our study in the context of thefollowing navigational situations; see [3] for more details.
My Recommendations:Figure 2 shows on how many items a user clicked when
viewing the personalized “My Recommendations” list.1 Wecan observe that all personalized methods (except Slope-One) successfully stimulated users to view the details of theproposed items, i.e., they have raised more interest in theo�ered items than a simple list of top-selling or top-rateditems did. The di�erences between the following groups oftechniques are statistically significant (p < 0.01) when con-sidering the absolute numbers: CF-Item/Hybrid – Content-based – SlopeOne/TopRating/Topseller.
Figure 2: Average number of item detail views per“My Recommendations” visits
With respect to downloads (Figure 3) things look dif-ferent. Although the content-based method raised addi-tional interest with respect to item views, this did not leadto significantly more downloads compared with the non-personalized methods. This indicates that users are gen-erally interested in things they have bought before (e.g.,sports games) as they view many of them. Once they haveseen di�erent ones, they however download only one game
1As mentioned above, the control group had no access tothe “My Recommendations” section.
206
(b) Jannach and Hegelich [132]
section 4 we follow up discussing its implications for then summarizing the AppAware idea in section 5.
2. RELATED WORK
In this section, we briefly review the state of the art and related work that have informed our design and indicate how AppAware differs from these.
2.1 Mobile Applications Websites
At present, the Android application portal can be accessed just from the Market mobile application and, in a limited way, from the related website. To overcome these design decisions by Google, many third-party developers are launching new services to access applications’ details from a personal computer. These services enable users to search for and download Android applications on the web instead of doing it directly from a mobile device. Good examples are AndroLib2 and AppBrain3. The major difference between the two is that AppBrain provides a user with an applications shopping cart that can be synced with the device through an Android client application. However, the idea is not innovative since it is trying to port the concept of Apple's iTunes to the Android world. In fact, iTunes already allows its users to browse and sync applications from their computer to an iPhone. AppAware does not aim at replacing the Android Market or providing a proxy, it is rather a companion to plan users' serendipity [5] in applications finding.
2.2 Appazaar and aTrackDog
Appazaar [2] is a recommender system for mobile applications, and is a project of the Lab for Software Engineering at Münster University of Applied Sciences. Based on a user current and historical locations and applications usage, Appazaar recommends applications that might be of interest for her. Therefore, Appazaar applies different algorithms from the research field of context awareness to analyze all the input data and create profiles of different situations. Another tool related to AppAware is called aTrackDog4. It is a program for Android devices that makes sure a user has the latest version of every installed application by checking the release information from either the Android Market, other users’ devices or the vendors' web site. Doing this manually takes time, thus aTrackDog supports the user in this activity. Even more, data from users’ devices is used to generate a most popular apps list that can be sorted by category, time, and price. Despite AppAware generates similar stats and providing apps recommendations is an appealing feature, we focused towards new ways to explore mobile apps on an application portal (i.e. real time stream, proximity based) and we further underlined the users presence into these activities.
3. CONCEPT AND DESIGN
In this section, we describe the system design, AppAware's most relevant features and their implementation in the user interface.
2 http://www.androlib.com 3 http://www.appbrain.com 4 http://atrackdog.a0soft.com
AppAware shares online users' installations, updates and removals of Android applications. In this way a user becomes conscious of what is hot on the Android application portal. To meet these conditions, the AppAware system consists in a client-server architecture.
3.1 General Concept
The client component in this system is the Android mobile application, which represents AppAware's graphical user interface (GUI) and allows following installations, updates and removals of applications shared by other users. Most of the core functionalities are supported by the main screen (see Figure 2) and are accessible from the application's menu or by touching a list item. Each list item represents a single event with its details, namely: the name of the application with its description, the type of event (installed, updated or removed), the user involved and the Android version together with the phone model. Moreover, to distinguish the type of event at a glance the application’s name is colored in red in case of a removal, green for an installation and blue for an update.
Figure 2. Real-time stream of installed, updated and removed
applications (a) and analogous events in user’s proximity (b).
The server component is a web application accessible though a standard browser and at the same time integrated in the mobile client through an Android WebView element that displays web pages. The client connects to the server through a RESTful interface that accepts and then stores events from users. Among the required parameters we have: the user ID, the application package name (used by the Android Market as unique identifier for an app), event type (installed, updated or removed) and the location (latitude and longitude, whenever allowed by the user). The server offers its data through an RSS feed that the AppAware client uses as data source for its core functionalities as described in Sections 3.2 and 3.3. This design decision allows at the same time any standard feed reader to keep track of installations, updates and removals of Android apps. Along with this architecture, AppAware integrates with Twitter too. This allows a user to share applications' installations, removals and updates on the Twitter account, thus letting her followers to see what applications are being installed by that user. An example of a Twitter status update is: “Just updated Google Translate http://appaware.org/1z on my #Nexus One - via #AppAware”. This tweet tells that the application “Google Translate” has been just updated on the Android phone model
432
(c) AppAware [103]
Figure 2.5: Screenshots of systems recommending mobile software for download.
applications also became a recent research topic, especially in the field of context-aware recommendation, since mobile application usage is context-dependent (aswe will discuss in Chapter 3). However, although the ecosystem of mobile appli-cations is rapidly growing as discussed in Chapter 1, so far there is little researchon recommender systems for mobile applications.
Pioneering work before the application store era was done by Woerndl et al. [262]and Jannach and Hegelich [132]. Woerndl et al. [262] present a recommendersystem for mobile applications that exploits context information and is based oncapturing the installations of applications in relation to this context (basically lo-cation), though installation times are irrelevant compared to measuring the actualusage.19 Their recommender engine is based on a hybrid engine following a multi-dimensional approach. Jannach and Hegelich [132] evaluate recommender enginessuggesting game applications for downloads on a mobile internet platform. Theyfound that the personalization of recommendations results in an increased numberof views and sales. They did not investigate contextual factors at that time.
Girardello et al. [103, 104] present AppAware: a recommender system that is basedon people’s overall application installations, uninstallations and updates. The sys-tem recommends applications based on their popularity, i.e. how many times anapplication was installed but not removed. The system’s assumption is that goodapplications are typically not removed once installed. The AppAware system also19Jakob Nielsen’s Alertbox: iPhone Apps Need Low Starting Hurdles. http://tiny.cc/eyie8, last ac-
cessed on 12.06.2013.
Woe
rndl
et
al.,
2007
Jann
ach
and
Heg
elic
h, 2
009
App
Aw
are,
201
0
Design Space
- Covers dimensions of recommender engines- Users, items, context, relevance
- Implicit or explicit parameter capturing- Propose possible signals for capturing
28
App Recommender System
29
- Implementation of a recommender system- Context-aware (location, time and previous app)- Based on traces of application usage (AppSensor)
- Application appazaar- Deployed on app store- 7,200 installations
- Testbed for different recommender engines
Figure 4. Examples of screens a user sees when traversing the stages of our testbed’s conversion funnel. This usage-
centric action sequence can be captured for evaluation of the di↵erent recommender engines.
a certain stage, e.g. the long-term usage metric is thenumber of apps that have been used in the long-term.Conversion rates are the quotients of two counters ofstage events, e.g. the ratio of the number of people whoviewed a recommended app to the number of people whoused it in the long term.
Recommender Engines under TestThe focus of this paper is on the AppFunnel as an eval-uation framework and not on investigating new enginesthemselves. Therefore we have used the recommenderengines that already have been developed and deployedwithin appazaar for testing our framework. As shown inTable 1, the engines under test can be classified accord-ing to the two dimensions of personalization and contextawareness. The personalized engines take informationabout specific users into account, i.e. their app usagehistories. The context-aware recommender engines gen-erate recommendations based on users’ current context,e.g. time, location, and previously used apps.
Non-personalized Personalized
Context-
less
– App popularity – Usage-based CF
Context-
aware
– App-aware filtering – Location-aware CF
– Time-aware CF
Table 1. Design space of tested recommender engines.
Within appazaar, users’ requests for app recommen-dations have been scheduled to the di↵erent recom-mender engines randomly. This allows for counter-balanced within-subject testing of the engines. All en-gines that are utilizing collaborative filtering are imple-mented based on the Apache Mahout framework.5
It is worth mentioning that not every recommender en-gine was able to produce results for every request. Thesystem su↵ers from the cold-start problem [19], in par-ticular for the context-aware engines. When no recom-mendation could be presented to the user respectively,no user interaction with the recommendation list waspossible, and as a result no AppFunnel -related data wascollected. However, to present apps to the user regard-less, the fallback strategy for all engines is to return arandom set of applications. Since the interaction with
5http://mahout.apache.org/
such random data does not contribute to the evaluationof the engines themselves, we do not take into accountthis data for the presented evaluation framework.
App Popularity
The first recommender engine implemented in appazaaris based on apps’ popularities measured by their usage.Since installations alone do not reveal much about thequality of an application6, this engine ranks applicationsaccording to their popularity based on their global usagein terms of total number of launches. This engine isneither personalized nor context-aware. Therefore, thisengine was able to return results for all issued requestsfor recommendations, since it did not require any specificadditional data.
Usage-based collaborative filtering
The second engine implements collaborative filtering. Itutilizes users’ app usage data as implicit feedback en-coded as binary values for user/item pairs: 1 meaning auser has used an app and 0 meaning a user has never usedthe app. This engine is personalized, but not context-aware. The core of this engine was implemented using auser-based collaborative filtering algorithm. Therefore,it su↵ers from the cold-start problem and cannot rec-ommend any applications to users that are new to thesystem, or to those who did not upload app usage data.7
App-aware filtering
This recommender engine is based on the idea that peo-ple’s usage sessions between screen-on and screen-o↵have a specific contextual scope. For instance, thereis an increased likelihood that people stay with gamesonce they have used a game, or with social apps oncethey have used a social app [6]. Also, for app usageprediction the preceding app is a strong predictor [22].Therefore, this app-aware engine recommends applica-tions that are similar to those that have been used re-cently within the same session. The app-aware filteringis a context-aware but non-personalized approach. Sincethis engine is based on the apps recently used by theuser, this engine has failed whenever the user asked forrecommendations without having used an app other thanappazaar previously in the same session.6Jakob Nielsen, http://goo.gl/KR09G7Users could opt out from uploading data for privacy reasons.
- Novel method for evaluation of recommendations- So far methods were based on installation counts
- Evaluate recommendation based on engagement of user with applications
- Apps can reach different stages after recommendation (AppFunnel)
Usage-centric Evaluation
VIEW INSTALLATION DIRECT USAGE LONG-TERM USAGE
RECOMMENDATION
30
Results of Case Study
- Difference in performance of engines- Context-less engines: more views- Context-aware engines: better from installation to
direct use31
A
vera
ge n
umbe
r of o
ccur
renc
es
pe
r rec
omm
enda
tion
list
0.0
0.2
0.4
0.6
0.8
1.0
Location−aware Collaborative
Filtering
App−popularity Filtering
App−aware Filtering
Usage−based Collaborative
Filtering
Time−aware Collaborative
Filtering
AppFunnel stage
view view market installation direct usage long−term usage
App-popularity
Usage-based CF
App-aware
Time-aware CF
Location-aware CF
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
Launching Housekeeping Discovering Multitasking
What are the costs of multitasking between apps?
Related Work- Jin and Dabbish, 2009
- Study of self-interruption on stationary computers- Waiting for primary task is one reason to self-interrupt
- Salvucci, 2010- Resumption is a reconstruction process rather than
memory-based, e.g. re-reading previously written text
- Oulasvirta et al., 2005- Attention to mobile device is fragmented- Mostly due to environmental distractions
- Fischer et al., 2010- Study of opportune moments for interruptions on phones- After episodes of device use reaction to notifications was
fastest34
No text input was required. Altogether 25 tasks were cre-ated, all in Finnish.
In a study like this, the route itself is an inherent part of both stimulus materials and procedure. The route consisted of several places in the Helsinki city center. Locations, situations, transportation, and times are given in Figure 1.
Training and Procedure Before the trials, the experimenter greeted the participant, committed to paper background information about her/him, and read aloud an overall description of the study (without revealing its purpose). Next, participants were trained on using the phone and browser. Training was incremental, starting from simple tasks (e.g., opening the application menu) and ending at two full tasks (e.g., looking from whatis.com at what “ITV” means).
After the training, the trial started. The search task was read aloud to the participant, with the associated bookmark num-ber (e.g., “Choose bookmark number 4”). Some situations involved doing “mobility tasks” related to that location si-multaneously with the Web search task (see Figure 1, right column); these were instructed when not implied by the situation. Some tasks were done while moving (route was provided if the participant did not know it) and others while standing or sitting. When moving, the participant led the way and the experimenter shadowed few steps behind with-out disturbing or helping the participant. After accomplish-ing the task, the participant’s answer was recorded by the experimenter. New instructions were then provided.
Each task was performed in one of the Instructed Time Pressure (ITP) conditions: (1) in the hurry condition, the instruction was to “Do as many tasks as quickly as possi-ble.” (2) In the baseline condition, it was to be done within a given (4 minutes) or implicit timeframe (e.g., “You can do the task until we arrive to the Sörnäinen metro stop”). The timeframe was sufficient to perform the task, but if exceeded, the experimenter stopped the task and instructed the next task. (3) In the waiting condition, the participants waited for something, and were told they had plenty of time to carry out the one and only task: “We’ll be waiting for a call from my colleague, you have plenty of time.”
Altogether, one trial lasted about 1½ hours.
Apparatus The Web search tasks were performed on a Nokia 6600 mobile phone running a mobile Web browser (Opera).
We aimed at making the equipment setup as unobtrusive (for the user and other people) as possible. 30 g (Watek WAT 230A) minicams were used for video recording. One minicam was attached to the test phone, capturing the phone display and keyboard. The device was also equipped with a second camera head that was focused up towards the user’s eyes. A third camera was attached to the backpack shoulder strap, facing forward, in order to record the field of vision ahead. Finally, the experimenter’s minicam, hid-
den in a phone shell, captured the overall environment. This video stream was sent wirelessly to a receiver in the partici-pant’s backpack. Since we knew that wireless video is sus-ceptible to distractions, we backed up this view onto a tape carried by the experimenter. (See Figure 2.)
The participant carried most of the equipment in a back-pack. It contained a microphone, a video camcorder, batter-ies, a wireless link receiver, and a 4-channel quad processor for building up one video from the four video streams. (See Figure 3, Video Figure, and, for a system diagram, [29].)
Coding The coders held a calibration meeting where they coded a part of one tape together to agree on and elaborate the cod-ing scheme. Several items were dropped and others simpli-fied to reach a high level of consensus. The revised, final version included measurements that could be done by rec-ognizing or counting specified circumstances (given in the scheme) from a paused video image. Each of the five cod-ers independently coded the tape by watching and pausing the playback after each event and coding it to a data sheet. The final scheme was as follows: - Time stamp: Time for the entry (accuracy of one second) - Task number: 1–25 - Location: Café / Metro platform / … (See Figure 1) - Instruction on Time Pressure (ITP): Hurry / Wait / Normal - User’s movement: Walks / Decelerates walk / Stands / Sits - Focus of user’s attention: Phone / Environment - Interaction: (User) Starts operating the phone / Stops it - Status of loading: Loading / Page scrollable with only text / All content loaded
- Crowdedness: (1) No people around / (2) Some people around (not moving) / (3) Some people around (moving) / (4) Many people moving close, crowded.
Figure 2. Configuration of recording equipment.
Figure 3. Output video data integrated on-the-fly.
CHI 2005 ʜ PAPERS: Interruptions and Attention 2: Attending to Interruptions April 2–7 ʜ Portland, Oregon, USA
923
Oulasvirta et al., 2005
app use ...... app use cont‘dinterruptiontime
...app use...time
- Study based on data set of mobile app usage- Mining for interruptions within data set
- Another application (self interruption)- Incoming phone call (external interruption)
- Duration of overhead
Detecting Interruptions
3541
overhead
app use app use cont‘d
app use
time
Findings
- Interruptions do not happen as often as expected- 8% of app use is interrupted by app switching - 3% of app use is interrupted by phone calls
- If interruptions happen, overhead may be exceedingly high
phone call app switch
Daily interruptions (% usage) 3.2 (2.2) 8.3 (5.3) per user
Regular app runtime (s) 24.8 (31.8) 18.9 (24.4)per app
Overhead duration (s) 43.2 (65.9) 34.4 (40.7)per app
mean (SD)
36
No Evolution of Phone UIs
37
- When phones became computers the design of phone UI did not change accordingly
- Still only accept and decline button- Call application has superior status
Re-Design of Phone UIs
Prototype ImplementationAndroid-based implementation of approaches b) to e)Available for study in the wild and testing:
Problem and IdeaMobile phones evolved form single-purpose devices to multi-purpose devicesThe design of phone call applications did not evolve accordinglyIncoming phone calls can interrupt concurrent application useWe revise the design of call applications to allow for higher degree of multitasking
Extending Phone Call Applicationsa) Current design: Full-screen modal dialogs providing only options to accept or decline callb) Postponing calls: Additional third option besides accept/decline to allow user to return to previous applicationc) Multiplexing: Allows user to keep attention in previous application while call is pendingd) Background noti!cations: Puts incoming call into background for user to pickup call at wille) Scheduling on app completion: Wait until task is done and display call when user leaves previous app
Revisiting Phone Call Applicationsfor Multipurpose Mobile Phones
CALLER NAME CALLER NAME
CALLER NAME
Discussion, Challenges and Future WorkA model for predicting overhead would allow to determine which option (b to e) to choose for handling callsWhen user is multitasking the caller needs to be kept in line, e.g. by signalling “user is currently writing a mail”Other modalities need to be taken into account (esp. vibration and ringtone) and aligned with visual noti!cation
a) Current design b) Postponing calls c) Multiplexing d) Background noti!cation
Interruptions do not happen as often as expected3% of app use is interrupted by phone calls (external)
8% of app use is interrupted by app switching (internal)
- Extending the design space for phone call UIs- New interaction design for phone call handling- Support for better multitasking with call notifications- Application CallHeads deployed on app store (30,000 users)
38
Housekeeping AppsLaunching AppsIntroduction
Multitasking between AppsDiscovery of Apps
Conclusion
40
Summary of ContributionsSummary of Contributions- AppSensor (tracing of mobile application usage)
- Understanding mobile application launching
- AppKicker (adaptive menu for supporting app launching)
- AppFunnel (method for usage-centric evaluation)
- Design space for mobile app recommender systems
- Reference architecture and implementation of appazaar
Launching
Housekeeping
Discovering
Multitasking
- Understanding people‘s concepts of icon arranging
- System to support icon arrangement
- Understanding the costs of mobile app multitasking
- Phone call UI for lowering the impact of call interruptions
41
Discussion and OutlookOutlook
Launching Housekeeping Discovering Multitasking
Future Work
- Understand usage of AppKicker and CallHeads- Build formal models to describe user behavior
42
Engineering
Theory
Methods
- Enhance quantitative large-scale research with qualitative methods
- Build supportive systems for new generations of devices
Thank you!Publications-Böhmer, Krüger: A study on icon arrangement by smartphone users. In Proc. of CHI 2013-Böhmer, Hecht, Schöning, Krüger, Bauer: Falling asleep with Angry Birds, Facebook and
Kindle: a large scale study on mobile application usage. In Proc. of MobileHCI 2011-Böhmer, Ganev, Krüger: AppFunnel: A framework for usage-centric evaluation of
recommender systems that suggest mobile applications. In Proc. of IUI 2013-Parate, Böhmer, Chu, Ganesan, Marlin: Practical prediction, prefetch, and prelaunch
for faster access to applications on mobile phones. In Proc. of UbiComp 2013-Böhmer, Bauer: Exploiting the icon arrangement on mobile devices as
information source for context-awareness. In Proc. of MobileHCI 2010-Leiva, Böhmer, Gehring, Krüger: Back to the app: the costs of
mobile application interruptions. In Proc. of MobileHCI 2012-Böhmer, Bauer: Improving the recommendation of mobile services by
interpreting the user’s icon arrangement. In Proc. of MobileHCI 2009-Böhmer, Prinz, Bauer: Contextualizing Mobile Applications for Context-
aware Recommendation. In Adjunct Proceedings of Pervasive 2010-Böhmer, Gehring, Hempel, Krüger: Revisiting Phone Call UIs for
Multipurpose Mobile Phones. In Proc. of MobileHCI 2013-Karatzoglou, Baltrunas, Church, Böhmer: Climbing the app wall: Enabling mobile app
discovery through context-aware recommendations. In Proc. of CIKM 2012-Böhmer, Bauer: The case for Context-Collaborative filtering in
pervasive services environments. In Proc. of CAM3SN 2009-Böhmer, Bauer, Krüger. Exploring the design space of recommender systems
that suggest mobile apps. In Proc. of CARS 2010-Böhmer, Lander, Krüger. What’s in the apps for context? Extending a sensor
for studying app usage to informing context-awareness. In Proc. of UbiMI 2013
Thank you!