cscw 2013 - investigating the appropriateness of social network question asking as a resource for...

Investigating the Appropriateness of Social Network Question Asking

as a Resource for Blind Users

Questions or comments during the talk? Tweet @erinleebradyPhoto used via CC Liscense from

http://www.flickr.com/photos/loneblackrider/315302588/

Erin Brady, Yu Zhong, Meredith Ringel Morris, Jeffrey P. Bigham

University of Rochester || Microsoft Research

Usefulness of SNSs for Blind People

Have visual questions that need to be answered [Brady 2013]

Introduction | Survey | Field Experiment

Why use SNSs?

FreeHigh saturation

[Pew 2012]

Personalized and trusted [Morris 2010]

Matches existing model [Kane 2009, Burton 2012]

Low-costAlways-available workers

Anonymous

Images used from Facebook, Twitter, and http://imaginarywitness.files.wordpress.com/2010/03/mechanical-turk.jpg


Primary Questions

How do blind people use SNSs?How do blind people view SNSs for question

asking?How do blind people ask questions on SNSs in

practice?


Primary Questions

Survey of social networking site useHow do blind people use SNSs?How do blind people view SNSs for question

asking?

Field experiment with VizWiz SocialHow do blind people ask questions on SNSs in

practice?


Survey Design

Accessible online surveyAvailable for 3 weeks (Jan/Feb 2012)Advertised via email to NA organizations for

the blindAsked participants not to spread survey via

SNSRaffled gift card for incentiveSurvey focused on all types of question-asking


Demographics

203 completed survey191 self-reported as blindAges skewed older, which may reflect onset of

blindnessGenerally had significant internet experience


Networks Used

At 92%, general SNS use higher than average of 66%

[Pew

2012]

Face

book

Twitt

er

Link

edIn

Googl

e Plus

MyS

pace

Yam

mer

Inclus

ive

Plan

et

Orkut

0102030405060708090 80

52

40

154 3 3 1

* χ2(1, N=191) = 55.88, p < .001


Networks Used

85% used Facebook, Twitter, or bothTwitter use higher than general population

Facebook Twitter0

10

20

30

40

50

60

70

80

9080

5261

15

Blind UsersPew Study

* 61% Facebook is calculated from 66% of online adults using SNSs, 92% of those using Facebook


Network Size

Smaller than average network sizes Facebook median 100

vs. Pew median 111, Facebook stats 130 Twitter median 45

vs. reports of Twitter around 126


SNS for Question Asking

Logged in frequentlyGeneral behavior tended toward “lurking”Status message question asking was

infrequentAnswer rates were low


SNS for Question Asking

55% thought SNS question-asking could be effective

Few users of SNSs felt comfortable posting questions


Findings

High adoption rate of SNSs Despite accessibility challenges Asynchronous communication with physically remote

contacts Twitter in particular (text-based)

Status posting and question asking infrequent Poor response rates Smaller-than-average network sizes Concerns about social costs


Primary Questions


asking?


practice?


VizWiz Social

Mobile phone application that answers visual questions

Based on concept presented by Bigham et al.5,000+ blind users asked over 40K questions

in first year

“Which can is the corn?”


Identification

Description Reading Unanswerable

44% 26% 23% 7%

General Question TypesIntroduction | Survey | Field Experiment

Answer Sources - Anonymous/Crowdsourced

Questions can be sent to: Web workers

Recruited from Mechanical Turk Answers in 98 seconds VizWiz team pays 5¢ per answer

IQ Engines Human-backed object recognition VizWiz team pays approximately 1¢ per answer


Answer Sources - Social/Friendsourced

Questions can be sent to: Broadcast to Social Networking Sites

Facebook/Twitter Posts made by the VizWiz application Free to VizWiz team

Email to Individual Contacts Free to VizWiz Team


Use of Social Sources

In month-long period preceding study: 702 users asked 3116 questions (average 4.44/user) 15% of users had tried social sources, 10.7% excluding

email 5% of questions (156) were sent to social sources

94 to Twitter 47 to email 26 to Facebook

Only 3 received answers, with median response time of 2:55:00


Experimental Design

Test the impact of financial restrictionsMirror the existing costs of the VizWiz

serviceEach participant was given $25 balance for

a monthSplit into conditions:

Cheap: 1¢ for IQ Engines, 5¢ for web workers Expensive: 5¢ for IQ Engines, 25¢ for web workers

Remaining balance, plus $10 gratuity, paid at end


Modified ApplicationIntroduction | Survey | Field Experiment

Experimental Design

Recruited from 207 active VizWiz users30 were recruited, 23 asked 1 or more

questionsSplit evenly into cheap, expensive

conditionsAnalyzed pre-study behaviors:

Asked 217 questions total (average 9.86/user) 81% of questions sent to web workers 93% of questions sent to IQ Engines 14% to social sources (10% to FB, 3% to Twitter, 1%

to email)


In-Study Behaviors

23 participants asked 170 questions (average 7.3/user)

Web

Wor

kers

IQ E

ngin

es

Face

book

Twitt

er

Emai

l0

20

40

60

80

10081%

93%

10%3% 1%

83%

45%

1% 0% 0%

Pre-StudyIn-Study


Post-Study Questionnaire

12 participants completed the questionnaireDemographics:

7 male, 5 female Four aged 20-29, six aged 30-39, two aged 50-59 Most had used internet for 10+ years All used at least one social networking site



All chose crowdsourcing as preferred answer source

“Humans are much more reliable, in my opinion, and Web workers are entirely

anonymous. They might necessarily not even know that they're dealing with an

accessibility application if Amazon Turkit [sic] is involved.”



Crowdsourcing was preferred to friendsourcing 9/12 “much preferred” web workers 1/12 “somewhat preferred” web workers 2/12 had no preference between web workers or

friendsourcing



Reasons were both technical and personal Speed of response Accuracy Feedback

Preferred anonymity Didn’t want to broadcast Didn’t like SNSs in general

“…because there's no guarantee that a facebook or twitter post would get you an immediate answer. When I need something identified like a can or TV Dinner I am going to use it now, not whenever my friends get around to telling me what it is. :) ”

Web-workers are completely anonymous, and there is sometimes no reason to think they are actually assisting

with a disability related question.



Restricted question-asking behaviors for social sources 8/12 chose not to ask 1+ questions to social sources 4/12 chose not to ask 1+ questions to crowdsourced

sourcesSocial costs played a role in restricted

question-asking“Not my friend's job to tell me that stuff. Plus it clutters up people's timelinesand [sic] they

might not like it.”


Primary Questions


asking?


practice?


Findings

0. Friendsourcing can be a valuable resource for Q&A1. Blind people are heavy users of SNSs2. Blind people infrequently use SNSs for QA3. In practice, even practical & financial incentives can’t motivate blind users to use SNSs for QA

Investigating the Appropriateness

of Social Network Question Asking

as a Resource for Blind Users

Erin Brady, Yu Zhong, Meredith Ringel Morris, Jeffrey P.

Bigham

University of Rochester Microsoft

Research

Get in touch via email:

[email protected]

or Twitter: @erinleebrady

Interested in questions asked by VizWiz users? Come see our talk at CHI! Visual Challenges in

the Everyday Lives of Blind People

Supported by NSF Awards #IIS-1149709

and #IIS-1049080

cscw 2013 - investigating the appropriateness of social network question asking as a resource for...

Technology