8/17/2012

1

Jialiu Linjialiul@cs.cmu.edu

School of Computer ScienceCarnegie Mellon University

SCHOOL OF COMPUTER SCIENCESCHOOL OF COMPUTER SCIENCECARNEGIE MELLON UNIVERSITYCARNEGIE MELLON UNIVERSITY

CommitteeJason I. Hong (co-chair)

Norman Sadeh (co-chair)Mahadev Satyanarayanan

Sunny Consolvo

UNDERSTANDING AND CAPTURING UNDERSTANDING AND CAPTURING PEOPLE’S MOBILE APP PRIVACY PEOPLE’S MOBILE APP PRIVACY PREFERENCESPREFERENCES

Thesis Proposal, Aug 17, 2012

http://blog.comscore.com/2012/05/comscore_introduces_mobile_metrix_20_the_next_gene.html

• 614 millions units shipped in 2012 globally• >50% cell users in US own smartphones

2/50

8/17/2012

2

•Google Play Store offered 600,000 apps with more than 20 billion downloads (Google I/O 2012)

•By June 2012, Apple offered 650,000 apps with 30 billion downloads (WWDC,2012)

3/50

• Lots of mobile apps use private information in surprising ways

• A significant portion of private information goes to 3rd

parties through APIs

Pandora gathers location, gender, year of birth, etc.

Path uploads entire contact list without user full consent.

Brightest Flashlight requires full Internet access, location, etc.

Bible accesses location.

…4/50

8/17/2012

3

• Need to review the permissions before installing the app.• Cannot tell where private info goes.

Can only control limited types of resources.

People do not read permissions or cannot understand them. (Felt’12, Kelley’12),

5/50

Context Sharing Services

Mobile Apps in General

Technological Solution

User Research

App analysis

Security extensions

Permission usability studySurvey of mobile privacy concerns

My proposed workMy past work

Computational privacy

Trusted Middleware….

Identify influencing factors

Essential privacy controls

Location privacy policy learning Place Naming

Purpose- vs. social-driven

US-China Study

How to understand app behaviors from users’ perspectives

How to model users’ mobile app privacy preferences

Expectation & Purpose

6/50

8/17/2012

4

Thesis Statement:

By using crowdsourcing and user-oriented machine learning techniques, we can build accurate and understandable models of mobile apps and users’ privacy preferences to inform the design of mobile privacy interfaces and settings, and to help developers build more privacy preserving apps.

7/50

� Bridge the gap between technological solutions and user research

� Experiment new ways, i.e., crowdsourcing, to capture users’ mobile app privacy preferences

� Model users’ preferences through user-oriented ML techniques to generalize our findings

8/50

8/17/2012

5

� A new way of looking at mobile privacy, i.e. “Privacy as expectation”

� A valuable dataset containing both app behavioral attributes and users privacy ratings

� Clusters of mobile apps elicit distinct privacy concerns

� A set of privacy personas that can simplify privacy settings � i.e., common privacy policies shared by a group of users

9/50

� Provide mobile app markets models to evaluate apps from a privacy perspective

� Inform the design of usable and efficient privacy interfaces and settings of existing mobile OS

� Help developers to understand the privacy implications and user acceptance of their apps

10/50

8/17/2012

6

� Background� Related Work in Location Sharing� Related Work in Mobile App Privacy� Proposed Work: Investigating Users’ Mobile App

Privacy Preferences� Schedule and Summary

11/50

Location-sharing Apps Mobile Apps in General

Entity

Accessing

Sensitive

Resources

Other end-users Primary: Service providers,

Secondary: advertising

agent, etc.

Examples

Typical

Research

Scale

10-50 participants, a

couple of conditions

Hundreds of

participants

12/50

8/17/2012

7

� Influencing Factors� e.g., utility, relationship, purpose…

� Location Presentation� e.g., map, place names, abstractions…

� Controls and Feedback� e.g., black/white list, time-based rule, plausible deniability,

request history…

� Location Privacy Preferences Learning� e.g., place naming method prediction, default policy

generation…

13/50

� Objective: � Understand how users modulate their location

information in sharing▪ E.g., sharing “at work” vs. 5000 Forbes Ave, Starbucks vs.

downtown, etc.

� In study:� Day Reconstruction Method

▪ Captured 26 participants’ location traces for two weeks▪ Asked for their sharing preferences at each place they

visited during that day

Modeling People’s Place Naming Preferences

J. Lin, et al., "Modeling people's place naming preferences in location sharing," In Proc. UbiComp, 2010. 14/50

8/17/2012

8

� Proposed a taxonomy of place naming methods� Semantic vs. Geographic, e.g. “at work” vs. “5000 Forbes ave”

� Granularity, e.g., “Forbes & Craig”, vs. “Pittsburgh”

� Four (context-dependent) factors impact users’ choices� Relationship, familiarity, privacy concerns, place entropy

� Users’ preferences are predictable� Top category accuracy 93%

� Granularity accuracy 89%

15/50

� K. P. Tang, J. Lin, et al., "Rethinking location sharing: exploring the implications of social-driven vs. purpose-driven location sharing," In Proc. UbiComp, 2010.

� J. Lin, et al., "A Comparative Study of Location-sharing Privacy Preferences in the U.S. and China," PUC, vol. under review, 2011

16/50

8/17/2012

9

� A typical Ubicomp way of collecting user preferences.� Require a more scalable method.

� Complex tradeoffs users make between utility and privacy� Study similar tradeoffs in the broader context.

� Users’ privacy preferences are dynamic and complex, yet predictable� Extend to a more complex problem space.

17/50

� Background� Related Work in Location Sharing� Related Work in Mobile App Privacy� Proposed Work: Investigating Users’ Mobile App

Privacy Preferences� Schedule and Summary

18/50

8/17/2012

10

Permission Analysis Static Analysis Dynamic Analysis

Example [Enck, 09][Barrera, 10][Felt &Greenwood, 11][Felt&Chin, 11][Vidas,11]

[Egele,11][Chin,11][Felt&Wang,11][Enck,11]App Profiles

[Thurm, 11] : WSJ[Enck, 10]: TaintDroid[Beresford,11][Zhou,11][Hornyack,11]

Features Identify vulnerabilities and anomalies by analyzing the permission usage pattern

Profile apps by scanning the source codes or binary files

Capture the data flow while interacting with the apps

App analysis cannot (directly) tell:• The intention of certain behavior• How users feel about certain behavior

19/50

Rule-Based Approach Faking Sensitive Info

Example [Bugiel,11] TrustDroid[Felt&Wang,11] Propose IPC inspection[Nauman,10] Apex[Jeon,12] Dr Droid & Mr. Hide

[Beresford,11] MockDroid[Zhou,11] TISSA[Hornyack,11] AppFence

Features Users can define rule based on context

Substitute fake information in the data flow

• Can users fully understand what is necessary?• Can users correctly configure these settings? • Will these details just overwhelm the users?

?

20/50

8/17/2012

11

Preliminary Results

Purposes & Expectations: Understanding Users’ Mental Models of Mobile App Privacy (Ubicomp2012)

21

� Problem� Existing permission screens do not help users make

good trust decisions [Felt’12], [Kelley’12]▪ Few people read it � not the main task▪ People don’t understand the implications

� Can we ask “others” to “digest” for users?

� Solution� Other human intelligence � crowdsourcing� But what to crowdsource here?

▪ Our idea: expectations and misconceptions

22/50

8/17/2012

12

� Apply the idea of mental models for privacy� Compare what people expect an app to do vs

what an app actually does� Emphasize the biggest gaps,

the misconceptions that most people had

App Behavior(What an app actually does)

App Behavior(What an app actually does)

User Expectation(What people think

the app does)

User Expectation(What people think

the app does)

23/50

24/50

8/17/2012

13

� Showed crowd workers (on Mturk) screenshots and description of app (from market)

� Showed permissions one at a time� Only those related to privacy

� Expectation Condition� Whether they expect the app uses permission� Why they think the app uses permission� How comfortable they were with it

� Purpose Condition� We gave an explanation (based on analysis)� Asked how comfortable they were with it

25/50

� Participants� Recruited from Mturk, US Android users only� Between-subjects (one condition only)

� Method� The top 100 popular apps in Android Market captured

on Sept 12, 2011� Targeted types of resource

▪ Location: GPS (24) and Network location (29) , Unique ID(56), Contact List (25) --- 134 app-resource pairs

� 20 participants per pair per condition

26/50

8/17/2012

14

Strong correlation between expectation and perceived level of comfort, r=0.91

Apps Use GPS Comfort Level (-2 – 2)

Expectation(0-100%)

Google Maps 1.52 100%

GasBuddy 1.47 95%

Weather Channel 1.45 95%

TuneIn Radio 0.60 65%

Evernote 0.15 55%

Angry Birds -0.70 20%

Brightest Flashlight Free -0.95 10%

Toss It -0.95 5%

N=20 per appGPS location data as example

27/50

• Clarifying purposes lower concerns• All differences statistically significant• Big increases for dictionary, Shazam, and others (> 1.0)

28/50

-0.2

0

0.2

0.4

0.6

0.8

1

Device ID Contact List Network Location GPS Location

Average Comfort Rating in Both Conditions

Avg comfort rating w/ purpose

Avg comfort rating w/o purpose

8/17/2012

15

• Key features• Common misconceptions (expectations)• Purpose(s) (explanations)

• Other design principles• Simplified terms • Only show permissions that affect privacy• Prioritized list• Highlight suspicious items

• Compared with existing Android permission screens

• Higher privacy awareness• Better comprehensibility• Required slightly less time to read

29/50

� Demonstrated the feasibility of crowdsourcing� However, still not scalable enough…

� Identified two key factors --- expectation and purpose� How about other (context-free) factors?

� Proposed a preliminary design of new privacy summary interface� Making decision for every app, hmm…What about

privacy settings?

30/50

UserUser--oriented Machine Learningoriented Machine Learning

8/17/2012

16

Data Collection

Preliminary Analysis

App Clustering

Personas Generation

• Static analysis• Crowdsourcing

• Identify influencing attributes

• Select features

• Cluster apps

• Extend to a predictive model

• Identify groups of users• Generate persona

31/50

1. App selection -- meta data2. App analysis -- Resource usage data3. Crowdsource -- user feedback

Objective : Compile a dataset that include both app behaviors, and howusers feel about these apps.

32/50

8/17/2012

17

� Representative and unbiased� Strategy: apps from all 30 categories, proportional to the ratio in the Market

� Feasibility of analysis� Strategy: filter out Non-English apps, limit the total number to ~400

33/50

0

10

20

30

40

# of

app

s

Category

Free paid

0

2

4

6

8

10

# of

per

mis

sion

s

Category

free paid

34/50

8/17/2012

18

� App analysis� Tools: apktool, app analyzer, ded� Resource usage data: permissions, purpose, destination

URLs, 3rd party APIs, etc.

� Crowdsourcing user feedback� Show app screenshot, descriptions and other meta data� Show resource usage one at a time with purpose (10

privacy-related resources) � Collect participants’ expectations and levels of comfort

� Collect info about participants� e.g., phone version, years of use, demographic

information, etc.

35/50

� Objective: To understand how different factors impact users’ decisions

� Perform regression on user feedback� Determine how different types of sensitive resources

weigh in users’ mental models

� Perform feature selection based on� Correlation� Predictability

36/50

8/17/2012

19

� Objective :

� Identify collections of apps that elicit distinct privacy preferences� Extract high-level knowledge

▪ E.g., Apps frequently send location info for ads concerns users,

▪ E.g. , Apps communicate to only one server are more accepted

� By-product: predictive model to estimate user acceptance

37/50

� Utilize easy-to-interpret clustering algorithms� Choices of clustering algorithm:

▪ e.g. K-means, Bottom-up▪ Other advanced algorithms (only if the aboves not

working)

� Choices of distance measure:▪ Hamming distance or weighted hamming distance▪ Euclidean distance (require proper coding of

categorical data)▪ Other advance distance functions: regular simplex,

symbolic covariance, etc.38/50

8/17/2012

20

� Opportunities :(1) Individual differences (2) User burden

� Assuming one day� Google redesign the privacy settings, or� Privacy extensions developed by other parties

� What are the “right” settings?� Effective: capture users’ need � Usable: with low user burden

� Objective : Can we identify a set of GOOD default settings

� � Privacy Personas39/50

� Opportunities :(1) Individual differences (2) User burden

� Assuming one day� Google redesign the privacy settings, or� Privacy extensions developed by other parties

� What are the “right” settings?� Effective: capture users’ need � Usable: with low user burden

� Objective : Can we identify a set of GOOD default settings

� � Privacy Personas40/50

Privacy Persona is A set of privacy policy shared by a group of users, e.g.“I am willing to disclose my location for the functionality

purpose. I am NOT willing to disclose my location for advertising or my

call log for any purpose.”

8/17/2012

21

� How to generate default privacy personas?1. Learn the privacy policy for each participant ->

(preference vectors)2. Identify groups of users share similar preferences

(clustering) 3. Extract average policy of each group (cluster center)

� Evaluate generated personas� Repeat crowdsourcing for new apps and new

participants� Lab studies (if time allows)

41/50

� Complement existing mobile privacy research with in-depth user research

� A new way of looking at privacy, i.e. “Privacy as Expectation”

� A valuable dataset containing both app behavioral attributes and users privacy ratings

� Clusters of mobile apps that elicit distinct privacy concerns

� A set of privacy personas that can simplify privacy settings 42/50

8/17/2012

22

� Provide mobile app markets models to evaluate apps from privacy perspectives

� Inform the design usable and efficient privacy interfaces and settings of existing mobile OS

� Help developers to understand the privacy implications of their apps

43/50

Aim at finishing by Aug 2013Publication opportunities: MobiSys’13, submission deadline Dec 2012UbiComp’13, submission deadline March/ April 2013

44/50

8/17/2012

23

� This work is supported by CyLab at Carnegie Mellon under grants DAAD19-02-1-0389 and W911NF-09-1-0273 from the Army Research Office and by Google . Support was also provided by the National Science Foundation under Grants CNS-1012763 and CNS-0905562.

45/50

� App Analysis� S. Thurm and Y. I. Kane, "Your Apps are Watching You," WSJ, 2011.� E. Chin, et al., "Analyzing inter-application communication in Android," In Proc. MobiSys, 2011� M. Egele, et al., "PiOS: Detecting Privacy Leaks in iOS Applications," In Proc. NDSS, 2011.� W. Enck, et al., "TaintDroid: An Information-Flow Tracking System for Realtime Privacy Monitoring

on Smartphones," In Proc. OSDI 2010.� Y. Zhou, et al., "Taming Information-Stealing Smartphone Applications (on Android)," In Proc.

TRUST, 2011

� Security Extensions� A. Beresford, et al., "MockDroid: trading privacy for application functionality on smartphones," In

Proc. HotMobile, 2011.� P. Hornyack, et al., "These aren't the droids you're looking for: retrofitting android to protect data

from imperious applications," In Proc. CCS, 2011.� J. Jeon, et al., "Dr. Android and Mr. Hide: Fine-grained security policies on unmodified Android,"

Working paper, available at http://www.cs.ucla.edu/~jeff/docs/drandroid.pdf, 2012.

� Android Permission Studies� A. P. Felt, et al., "Android Permissions: User Attention, Comprehension, and Behavior," UCB/EECS-

2012-26, University of California, Berkeley, 2012� P. G. Kelley, et al., "A Conundrum of permissions: Installing Applications on an Android

Smartphone," In Proc. USEC, 2012.� A. P. Felt, et al., "I've Got 99 Problems, But Vibration Ain't One: A Survey of Smartphone Users'

Concerns," In Proc. SOUPS, 2012.

46/50

8/17/2012

24

� Location Sharing� B. Brown, et al., "Locating Family Values: A Field Trial of the Whereabouts Clock," In Proc. UbiComp, 2007.� G. Iachello, et al., "Control, Deception, and Communication: Evaluating the Deployment of a Location-

Enhanced," ed: unknown, 2008.� M. Benisch, et al., "Capturing location-privacy preferences: quantifying accuracy and user-burden tradeoffs,"

Personal and Ubiquitous Computing, 2010.� S. Consolvo, et al., "Location disclosure to social relations: why, when, & what people want to share," In Proc.

CHI, 2005.� J. Cranshaw, et al., "User-Controllable Learning of Location Privacy Policies with Gaussian Mixture Models,"

In Proc. AAAI, 2011.� J. Y. Tsai, et al., "Location-Sharing Technologies: Privacy Risks and Controls," In Proc. TPRC, 2009.

47/50

� Jialiu Linjialiul@cs.cmu.edu

School of Computer ScienceCarnegie Mellon University

48/50

8/17/2012

25

49

8/17/2012

26

• Maps reminds participants of the locations

they visited.

• Questions were asked for four social groups

•family member,

•close friend,

•acquaintance,

•stranger.

Place NamesPlace Names

HybridHybridSemanticSemantic

PersonalPersonal

GeographicGeographic

LandmarkLandmark

AddressAddressFunctionalFunctional

namenameBusiness Business

namename

StateState

..

..

..

..

..

CityCity

FloorFloorRoomRoomFloorFloorRoomRoom

8/17/2012

27

3. Suppose you have installed Toss it on your Android device,

would you expect it to access your precise location? (required)

Yes No

4. Could you think of any reason(s) why this app would need

to access this information? (required)

Toss it does access users’ precise location information.

precise location is necessary for this app to serve its

major functionality.

precise location is used for target advertisement or

market analysis.

precise location is used to tag photos or other data

generated by this app.

precise location is used to share among your friends or

people in your social network.

other reason(s), please specify

I cannot think of any reason.

5. Do you feel comfortable letting this app access your precise

location? (required)

Very comfortable

Somewhat comfortable

Somewhat uncomfortable

Very uncomfortable

� strong correlation observed (r=0.91) between people’s expectation and their subjective feelings

� Perceived necessity guide users to make trust decisions or prompted them to take different actions.

� W27 “Why does a flashlight need to know my location? I love this app, but now I know it access my location, I may delete it.” (Brightest Flashlight)

� W56 “I do not feel that games should ever need access to your location. I will never download this game.” (Toss it)

The comfort rating was ranging from -2.0 (very uncomfortable to +2.0 (very comfortable).

8/17/2012

28

� To prevent gaming of our study� Crowd workers’ lifetime approval rate >75%� Limit to Android users� Quality control question� Similar results comparing to lab study� Mean Square Errors are negligible

MSE Network Loc GPS loc Contact List Unique ID

expectation [0,1] 0.0354 0.0303 0.0353 0.0363

comfort level [-2,+2] 0.7081 0.8136 0.6749 0.3067

• TaintDroid was used to analyze the ground truth. • We manually categorized apps into 3 categories:

•For major functionality• for sharing and tagging (or supporting other minor functions)• for target advertising or market analysis

•Accuracy never exceeded 80% even for purely functionality purposes.• Very low accuracy when sensitive resources used for multiple purposes

8/17/2012

29

The comfort rating was ranging from -2.0 (very uncomfortable to +2.0 (very comfortable).

� http://www.appbrain.com/stats/android-market-app-categories

8/17/2012

30

App meta data (directly obtained)

ranking, rating, description, download, category, price, developer type

Resource usage data (Need app analysis)

Type of resource, purpose, destination, 3rd party API

User feedback (through crowdsourcing)

Subjective: expectation, comfort level,

Static: phone version, years of use, demographic information, technical affinity, privacy scale

Static App Analysis

Crowdsourcing users’ feedback

2 3

1 Meta Data

� 1: Learn the privacy policy for each user� Rearrange per-app preferences into a rule-based policy � Encode policy into a vector, each entry represents preference of an app

cluster

� 2: Perform clustering on these vectors� Group users who have similar policy vectors� E.g., K-means with Hamming distance

� 3: Learn a default persona for each cluster� Find the center of each cluster by averaging the policy vectors within each cluster