defense 20121130
DESCRIPTION
Understanding "Just Enough" Computer Users: Motivation Style and ProficiencyTRANSCRIPT
1 of 45
Understanding “Just Enough” Computer Users:
Motivation Style and Proficiency
By Harriet King Masters Candidate in Computer Science
2 of 45
Why do some proficient daily computer users, stumble over the
unfamiliar and others easily adapt?
The Question
Introduction Study Design Motivation Observations Future Work Conclusions
More EXAMPLE: More information and detail in Supplementary Slides
3 of 45
What Is a Just Enough (JE) User?
• Daily computer user
• Competent
• Extrinsic Motivation
Introduction Study Design Motivation Observations Future Work Conclusions
4 of 45
The Hypothesis
We hypothesize that
extrinsically motivated
proficient daily computer users
have difficulty with unfamiliar computer tasks and skill transfer, whereas
intrinsically motivated daily users accomplish unfamiliar tasks readily.
Introduction Study Design Motivation Observations Future Work Conclusions
5 of 45
Who Cares?
• Software designers
• Human Computer Interactions (HCI)
• Software Users
• Stakeholders for computer literacy
“Lest we wish to change our field’s name to student-computer interaction we should make effort to find more representative participants” (Barkhuus and Rode 2012)
Introduction Study Design Motivation Observations Future Work Conclusions
6 of 45 Introduction Study Design Motivation Observations Future Work Conclusions
Study Design Overview
OUTPUT INVENTORY
scores & statistics group descriptors
OBSERVATIONS Coded & analyzed attitudes & actions
7 of 45
Richness of Data for Understanding
• Pre-questionnaire: daily users?
• Quantitative motivation inventory scores
• Demographic and interview questions
• Ethnographic observation methods: – Think Aloud Protocol
– Observation recordings
– Researcher questions and follow up
• Quantify transcripts with coding
• Post-questionnaire and JE Users questionnaire (Sim 1999; Rose, Shneiderman, Plaisant. 1995)
Introduction Study Design Motivation Observations Future Work Conclusions
8 of 45
MOTIVATION
9 of 45
Motivation Background
Motivation Styles, adapted from Ryan and Deci (2000) ‘Taxonomy of Human Motivation’. Low interest and enjoyment are on the left ranging to high interest and enjoyment on the right. (Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004; Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)
Introduction Study Design Motivation Observations Future Work Conclusions
More More information and detail in Supplementary Slides
10 of 45
Motivation Inventory
L to R: Richard Ryan and Edward Deci (Photo: Adam Fenster, August 2010)
Introduction Study Design Motivation Observations Future Work Conclusions
Source Factors
Guay, Vallerand, Blanchard (2000)
1. Amotivation 2. External Regulation 3. Identified Regulation
Ryan and Deci (IMI 2012) 4. Interest/Enjoyment 5. Perceived Choice 6. Perceived Competence
11 of 45
Adapting Questions
Precedent: (Shroff and Vogel 2009). Confirmed Inventory with two pilot studies.
Introduction Study Design Motivation Observations Future Work Conclusions
12 of 45
Precedents for Scoring Inventory
Likert scale IS-An ordinal measure of ranking
“We did violate some mathematical assumptions in creating an interval level of measurement index out of ordinal components, but as previously indicated, this is common practice in the social and behavioral sciences.” (Sirkin, R. M., 2006. “Statistics for the Social Sciences.” 3rd edition, Sage Publications.
Precedent for averaging motivation inventory scores
1. Pavlas, Jentsch, Salas, Fiore, and Sims, 2012
2. Shroff and Vogel, 2009
3. McAuley, Duncan, and Tammen, 1989
Introduction Study Design Motivation Observations Future Work Conclusions
13 of 45
Who Took the Inventory? Everybody!
Community Classmates Faculty Internet
• Ages 13 to 87 from FIVE continents • 9 countries: USA, China, Turkey, Australia, Sweden, U.K.,
South Africa, India, and France • 130+ total completed questionnaire • Used 66 for total respondents • 16 participants observed (7 intrinsics, 9 extrinsics)
Introduction Study Design Motivation Observations Future Work Conclusions
14 of 45
Required Correlation
Correlation of Interest/Enjoyment & Perceived Choice Factors
n = 66
All Respondents
n =16
All Observed
Correlation 0.602 0.815
Significance (2-tailed)
p < 0.01 p < 0.01
Table 8: Pearson Correlation of
Interest/Enjoyment & Perceived Choice
Introduction Study Design Motivation Observations Future Work Conclusions
15 of 45
Grouping Variables
Venn Diagram is External Regulation > 4.0
intersecting Interest/Enjoyment > 4.0
Total and percent inventory responses by group with n=66
Introduction Study Design Motivation Observations Future Work Conclusions
16 of 45
Inventory T Test Results
Factor Different Significance
Age* NOT different p=0.396
Digital Native* NOT different p=0.166
Perceived Competence* NOT different p=0.071
Amotivation* Different p=0.012
External Regulation Different p<0.001
Interest/Enjoyment Different p<0.001
Perceived Choice Different p=0.001
Significant Differences in Inventory
Scores, Age, & Digital Native * Asterisk indicates non parametric Mann-Whitney U test
All other are Independent Samples T-test
Introduction Study Design Motivation Observations Future Work Conclusions
More More information and detail in Supplementary Slides
17 of 45
Intrinsics: Digital Native or Not
Interest/ Enjoyment Perceived Choice Perceived
Competence
No
n-n
ativ
e
Dig
ital
nat
ive
No
n-n
ativ
e
Dig
ital
nat
ive
No
n-n
ativ
e
Dig
ital
nat
ive
6.43 5.86 6.33
5.00 6.71 4.71 5.14 2.67 6.67
4.14 6.14 4.14 5.29 3.00 6.83
5.57 6.57 4.29 4.86 5.17 7.00
Side by side comparison of digital non-natives (3) on left and digital natives (4) on right. Ordered from low to high competence
Introduction Study Design Motivation Observations Future Work Conclusions
More
18 of 45
Digital Natives not significantly different
Inventory RespondentsObserved Participants
IntrinsicsExtrinsics
0
10
20
30
40
50
60
70
27
64
2
66
16
79
Digital Natives All
Groups
Num
ber o
f Peo
ple
41%
37% 57% 22%
Introduction Study Design Motivation Observations Future Work Conclusions
19 of 45
Not Significantly Different Age, Perceived Competence, & Digital Native or not
Mean Perceived Competence with error bars for standard deviation
Mean Age with error bars for standard deviation
Introduction Study Design Motivation Observations Future Work Conclusions
9 Extrinsics 7 Intrinsics0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
55.6746.57
Age
9 Extrinsics 7 Intrinsics1.00
2.00
3.00
4.00
5.00
6.00
7.00
3.70
5.38
Me
an
Pe
rce
ive
d C
om
pe
ten
ce
20 of 45
Grouping Variable
with error bars showing standard deviation
Introduction Study Design Motivation Observations Future Work Conclusions
Mean Inventory Results
Grouping Variable
0
1
2
3
4
5
6
7
8
Like
rt S
cale
1 -
7
wit
h n
eutr
al a
t 4
n=66 Respondents n=16 Observed n=9 JE Users n=7 Intrinsics
More
21 of 45
Data Screening Extra High Perceived Choice
Mean Perceived Choice with standard deviation error bars Extrinsic Molly = 5.57!?
2.3 standard deviations above
5.57
More Introduction Study Design Motivation Observations Future Work Conclusions
22 of 45
OBSERVATIONS
23 of 45
Observation Phases
Introduction Study Design Motivation Observations Future Work Conclusions
24 of 45
Near Skill Transfer
Introduction Study Design Motivation Observations Future Work Conclusions
More
25 of 45
Participant Hesitation Wording “uhhhh”
“I’m looking for a way to
do...”
“maybe if I go here”
“what’s this?”
“I can’t...”
“ummm”
“let’s go back here”
[giggling]
“aaaaannnnnnnd”
“I could try like..”
“no I can’t drag that..”
“I’ll look in here, no I just
looked in there”
“I think I can just... click on
this here, and... that didn’t
work”
“ok, that didn’t work”
“I looked at the bottom but
there’s nothing there”
“I saw this click to ... but
that isn’t it”
“hmmm”
“contacts....contacts....
contacts”
“that doesn't look very
promising”
[sigh]
“no, that's not it”
“maybe this”
“so, we're not doing that”
“I wouldn't think it'd be
under that”
“I'm going to try right click
again”
“I forgot what you said to
do”
“this damn mouse”
Introduction Study Design Motivation Observations Future Work Conclusions
26 of 45
For Prompting the Participant
“go ahead and tell me what you’re seeing”
“please tell me what you’re thinking”
“Are you trying to decide something, can you tell me about it?”
“did that work?”
“what seems odd about this?”
“what are you thinking?”
“you’re giggling, …you’re sighing…you sound angry, what are you feeling?”
Introduction Study Design Motivation Observations Future Work Conclusions
27 of 45
Rubric for Coding Observations
CODE RULE
Stumble [action] >= 20 seconds
Fall [action] >= 1 minute
Persist [action] >= 3 minutes
Quit attitude towards a task
Resist attitude towards a task
Introduction Study Design Motivation Observations Future Work Conclusions
28 of 45
Transcript Example
time OLIVIA [action] “quote” (time on video) analysis stu
mb
le
fall
qu
it
resis
t
pers
ist
b 7:58
e 9:08
[while looking for spam, stumbles across trash 7:58 and says I’ll empty the trash
instead, I say go ahead] Participant: “I have no idea how to do that. It’s already IN the
trash” me: “Look around. ...you can empty the trash.” (8:10) Participant: “It’s already IN
trash. Where do you empty trash to? I’m thinking that I never empty my trash because
there’s no way to empty trash because it’s already trash.” (8:25) me: “no, there is a way
to empty trash.” Participant: “There’s no trash emptying.”
[ask about her agitation] Participant: “I’m not agitated at all. You’re just wrong. There’s
no trash emptying.” [ask what she’s feeling] Participant: “I think it’s dumb that the trash
doesn’t have an empty.” (8:40) me: “It does actually”
Participant: “I don’t see it. If I click on something in my trash, all I can do is trash
something in my trash, which is silly because it’s already in my trash” (9:08) me: “Ok,
we’ll come back to this. Let’s look at your spam” [so resistant that I stop this task on
test. Never does trash]
1 1
1
b 9:10
e 9:45
Participant: “I don’t know if I have spam” (9:10) me: “You do have spam.” “No. Really!?
I’m looking at all my folders and I do not have one called “spam”” (9:20) me: “Did you
find “more” at the bottom?” “There’s a more. Oh look at that, there’s spam.” (9:45)
1
1
b 9:50
e 11:10
[directed to delete all spam at once, (9:50), giving her hints] me: “It’s not that tricky, it
has words and I can see them, I’m looking at it right now” (10:37) (11:10) found “delete
all messages now”
1 1
b 11:20
e 12:10
[11:20 Go to address book] Participant: “I’m not fully sure where my address book is, I
think I have to go to my calendar”, then found contacts 12:10
1
Introduction Study Design Motivation Observations Future Work Conclusions
29 of 45
Inter Rater Reliability Results
• First Rater (HK)
• 2 outside raters (SK and PM)
• Outside raters reviewed 30% of transcripts
• Stumble, fall, and persist are time related
Rater 1 Rater 2
Stumble, Fall, Persist 100% agreement 100% agreement
Quit 99.13% agreement 97.73% agreement
Resist 96.52% agreement 97.73% agreement
Introduction Study Design Motivation Observations Future Work Conclusions
30 of 45
Occurrences for Each Code
• Asterisk indicates statistically significant difference for this code
between extrinsic and intrinsic. Total occurrences with percent of
total in parentheses.
• There was no significant difference between Unfamiliar Task
compared to Near Skill Transfer for either intrinsics or extrinsics.
Introduction Study Design Motivation Observations Future Work Conclusions
Stumble* Fall* Persist* Quit* Resist
JE Users 91 (81%)
56 (84%)
15 (88%)
9 (90%)
13 (87%)
Intrinsics 21 (19%)
11 (16%)
2 (12%)
1 (10%)
2 (13%)
More
31 of 45
All Occurrences of Stumble & Fall
Introduction Study Design Motivation Observations Future Work Conclusions
More
0
5
10
15
20
stumble fall
Intrinsics on left and Extrinsics on right
Intr
insi
cs
Extr
insi
cs
32 of 45
JE User vs. Intrinsic: Marsha & Rebecca
Similar: 1. both Amotivation = 1.0 2. Both digital non-native 3. similar experience level 4. similar self rate and perceived competence 5. similar age 6. Appeared to cruise through unfamiliar tasks 7. Responsible community leaders 8. Professional women
Different: 1. Performance 2. Different motivation styles
Introduction Study Design Motivation Observations Future Work Conclusions
More
Exter: 4.5 Int/En 2.57
Exter: 4.0 Int/En 5.57
33 of 45
Resist • Only 5 out of 16 resisted • 4 extrinsic & intrinsic Mike • Olivia had 7 resists
1. Can’t empty trash 2. there is no spam 3. doesn’t “add” to group but
insists she did 4. says “check mail” button is
broken 5. won’t remove attachment, 6. says used wrong address but
was sent folder issue 7. says did not spell a word
correctly when did spell correctly
0
1
2
3
4
5
6
7
8
Lucy Mike* Miranda Marsha OliviaTo
tal O
ccu
rre
nce
s o
f R
esi
st
Introduction Study Design Motivation Observations Future Work Conclusions
More
34 of 45
Another Type of Resist
Marsha shares, “I never use the google calendar. I’m not telling them what I’m doing every day. Forget that!”
“Passionate?...I am. I’m not MAD at them [MS Word], I’m frustrated with them. … they’re leaving out the average person. And maybe that’s what open office is for. I don’t know.”
Introduction Study Design Motivation Observations Future Work Conclusions
35 of 45
Quit
Screenshot of "Contacts" button behind "Mail" in Gmail. Doesn't look like a button with no rectangle or color change.
0
1
2
3
4
5
6
7
8
Tota
l Occ
urr
en
ces
of
Qu
it
Quit Resist8 of 16 quit * Asterisk indicate intrinsic
Introduction Study Design Motivation Observations Future Work Conclusions
More
36 of 45
Persist
Walter spent about 5 minutes (7:40 to 13:10) using wrong password of “guest” and misspelled username trying to login to gmail online. He repeated the same behavior while expecting different results
0
1
2
3
4
5
6
MaryAnn
Molly Lucy Olivia Mike* Marsha Alice Walter
Tota
l Occ
urr
ence
s o
f P
ers
ist
8 of 16 Persist
Introduction Study Design Motivation Observations Future Work Conclusions
37 of 45
Lowest Interest & Choice • Extrinsics Lilly and Olivia
• Opposite attitudes (shame vs. blame)
Introduction Study Design Motivation Observations Future Work Conclusions
More
0.001.002.003.004.005.006.007.00
amo
tiva
tio
n
exte
rnal
regu
lati
on
Inte
rest
Enjo
ymen
t
Per
ceiv
edC
ho
ice
Per
ceiv
edC
om
pet
ence
Lilly Olivia
38 of 45
Just Enough User Alice (1/9)
“I don’t do ANYTHING that I’m not taught. And that is a big
drawback in my learning.”
“I know enough to get what I want, most of the time. And it
definitely is not a pleasure for me to try to figure out things on
my own. N-O-T AT A-L-L… Maybe everyone thinks they are a “Just
Enough” user.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
39 of 45
Just Enough User Lucy (3/9)
“Why would I Google it? I wouldn’t, because it’s a bunch of
teenagers who can’t spell right, who don’t use punctuation, all
lower case.”
“I am fine using the computer only for what I need. I think they
are ruining the world quite frankly, and am slightly proud I find
them somewhat repulsive machines.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
40 of 45
Just Enough User Walter (8/9)
“You are … confronting an unbelievably unfamiliar system, with all the
scariness of being surrounded by REAL fully paid, fully trained, card
carrying life member geeks … I got spooked by the surroundings. I got
intimidated by my high level of geekitude surroundings.”
“People do get on without a computer at all, so perhaps ‘No
Computer’ (or ‘The Computer They Make You Use At Work’) is the true
‘Just Enough Computer’.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
41 of 45
Future Work
• So much data!
• Bigger sample
• “Just Enough” term?
• Gender, socioeconomic status, years of experience, aversion to change?
• Separating work and play in motivation study
• Less frequent users?
• What if a “consequence” element?
• Hand held computers?
Introduction Study Design Motivation Observations Future Work Conclusions
42 of 45
CONCLUSIONS
43 of 45
Statistical Results
• Confirmed competency of JE users
• Extrinsic proficient daily users stumble, fall, persist and quit significantly more than intrinsics
• AND it is not explained by age, perceived competence, or being digital native
• JE users account for over 80% of performance difficulties in our study
• Just Enough users exist in all age groups and experience levels (18% in our sample)
Introduction Study Design Motivation Observations Future Work Conclusions
44 of 45
Observed Phenomena
• Impossible to differentiate JE user from any other competent user, until faced with the unfamiliar
• Just Enough users shed competencies as they become unnecessary
• Wide range of attitudes and experience related to exploring and performance
• Sense of “not belonging”
Introduction Study Design Motivation Observations Future Work Conclusions
45 of 45
Thank you!
A Haiku Just Enough is cool till unfamiliar and new safe routine un-do.
46 of 45
Just Enough User Lilly (2/9)
When asked during the test about her feelings, Lilly shares, “ohhh, why am I so stupid? How can I not know how to do this? I dread asking one of my kids because they have no patience.”
“I really want computers to be as unobtrusive in my daily life as can be. Just Enough term sounds a bit lazy.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
47 of 45
Just Enough User Marsha (4/9)
Marsha says, “I like to sign out, because then they, THEORETICALLY, aren’t watching me, but you know they are because advertisements for something I just looked at turn up on the *weirdest* pages.”
“My feelings are that I would like to be more than that [JE user]. I would consider a "just enough" user to be one who uses only email, or only cruises the web for news, or only uses one application.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
48 of 45
Just Enough User Mary Ann (5/9)
“When I’m at work, I’m so busy, that I don’t have time to play around... I always have to do things in the fastest way possible, which doesn’t allow exploration.”
“My feelings are that I would like to be more than that. I do not want to be a "dinosaur. I sometimes can do a little more than just enough if I get up my courage to try."
Introduction Study Design Motivation Observations Future Work Conclusions
More
49 of 45
Just Enough User Miranda (6/9)
“It seems stupid and why should I waste my time staring at the computer.”
“My feelings are, why would I spend any more time at the computer? I'd rather read a book or take a walk. Just enough is a perfect name.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
50 of 45
Just Enough User Molly (7/9)
“This all is stupid. This is ridiculous. I don’t know why anyone uses computers. … I don’t really care. I can basically do anything I need to do and I have [IT worker] and if I can’t do anything I just call [IT worker] and cry.”
“The term "Just Enough" is kind. I don't feel judged or "less than" (stupid).”
Introduction Study Design Motivation Observations Future Work Conclusions
More
51 of 45
Just Enough User Olivia (8/9)
“[it] is really annoying not to be able to find these things that you’re CLAIMING it’s on here. And it’s like, how are you supposed to know where it is.....[I’m] irritated.”
“Very proud that I can do it enuf [sic]. People should make more things easy for us.”
Introduction Study Design Motivation Observations Future Work Conclusions
More
52 of 45
“Just Enough Users”, a poem
Just enough is satisficing, works out fine till new and strange. Computer changes make life messy, then it’s struggle stumble quit. Those interest people cruise along, probably nothing ever wrong. Curse you easy flexing user. Why can’t I just find my cursor? Just Enough left me so helpless, when the web changed all my favorites. I just want to stay so lazy, stay low interest, stay low effort. OK sometimes then I stumble. Just Enough was not effective. Who to blame and who to curse? Designers! They must be the worst.
Conclusion
53 of 45
Intrinsic Motivation Characteristics
• Deeper involvement in activities; natural activity • More curiosity; exploration • Trying out more complex options • Increased persistence • Higher achievement of goals; improved
performance • Less avoidance behavior • Interest, excitement, and confidence (Martens et al. 2004; Oudeyer et al. 2007; Deci and Ryan 2000)
Motivation Inventory
Back
54 of 45
Intrinsic Motivation: Supports & By Products
• Self-esteem and general well-being
• Competence
• Autonomy
• Adaptable
• Pros/cons of praise
• Reduced by external rewards
• Supported by seeing examples; having capability
(Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004; Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)
Motivation Inventory
Back
55 of 45
Parametric or Non parametric?
Does data pass the 3 assumptions for parametric statistical analysis?
1. Independence? Yes! All different humans
2. Homogeneity? (equal variance, Levene’s test)
3. Normality? (skewness & kurtosis < |1.95|)
Does it pass for 66 respondents and 16 participants?
Motivation Inventory
Back
56 of 45
Homogeneity of Inventory Factors
Levene’s Test for Equality of Variances
n = 16 Observed. Significance
Levene’s Test for Equality of Variances n = 66 Respondents.
Significance
Amotivation 0.053 0.002
Identified Regulation 0.802 0.546
External Regulation 0.572 0.822
Interest/Enjoyment 0.989 0.842
Perceived Choice 0.492 0.218
Perceived Competence 0.152 0.010
Motivation Inventory Back
57 of 45
Normality of Inventory Factors
Respondents n=66 Observed Participants n=16
Skewness Kurtosis Skewness Kurtosis
Amotivation 1.486 1.986 1.004 -0.557
Identified Regulation -0.063 -1.048 -0.527 -1.157
External Regulation 0.038 -0.781 -0.273 -0.870
Interest/Enjoyment -.0513 -0.050 0.165 -1.358
Perceived Choice -0.213 -0.708 -0.050 -1.565
Perceived Competence -0.246 -0.609 -0.533 -0.988
Motivation Inventory
Back
58 of 45
T Test Result Detail
Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Perceived Choice (independent Samples T test) extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001.
Back
59 of 45
Intrinsics Descriptive Statistics
N Min Max Mean Std. Dev Skewness Kurtosis
Statistic
Std. Error Statistic
Std. Error
age 7 23 87 46.71 27.93 0.59 0.79 -1.96 1.59
amotivation 7 1.00 1.50 1.07 0.19 2.65 0.79 7.00 1.59
external regulation
7 1.00 4.00 2.64 1.02 -0.19 0.79 -0.06 1.59
Interest/ Enjoyment
7 4.14 6.71 5.80 0.95 -0.96 0.79 -0.11 1.59
Perceived Choice 7 4.14 5.86 4.90 0.59 0.32 0.79 -0.35 1.59
Perceived Competence
7 2.67 7.00 5.38 1.84 -0.87 0.79 -1.30 1.59
Motivation Inventory
Back Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Perceived Choice (independent Samples T test) extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001.
60 of 45
Extrinsics Descriptive Statistics
N Min Max Mean Std. Dev Skewness Kurtosis
Statistic
Std. Error Statistic
Std. Error
age 9 34 74 55.78 14.17 -0.18 0.72 -1.32 1.40
amotivation 9 1.00 2.75 1.89 0.74 0.08 0.72 -1.82 1.40
external regulation 9 4.25 6.50 5.39 0.89 -0.02 0.72 -1.92 1.40
Interest/ Enjoyment
9 1.57 4.00 2.73 0.89 -0.03 0.72 -1.35 1.40
Perceived Choice 9 1.57 5.57 2.67 1.26 1.71 0.72 3.43 1.40
Perceived Competence
9 2.50 4.33 3.70 0.72 -0.87 0.72 -0.80 1.40
Motivation Inventory
Back
61 of 45
Summary of Correlations
Relationship Correlation Significance n R^2
External Regulation with Interest/Enjoyment
- 0.821 p=0.001 16 67.40%
- 0.397 p=0.001 66 15.76%
External Regulation with Perceived Choice
- 0.879 p=0.001 16 77.26%
- 0.785 p=0.001 66 61.62%
Amotivation with Perceived Competence
- 0.602 p=0.014 16 36.24%
- 0.339 p=0.005 66 11.49%
Age with Perceived Competence - 0.710 p=0.002 16 50.41%
n=66 Inventory Respondents & n=16 Observed Participants
Motivation Inventory
Back
62 of 45
Summary of Correlations
Relationship Correlation Significance n R^2
External Regulation with Interest/Enjoyment
- 0.821 p=0.001 16 67.40%
- 0.397 p=0.001 66 15.76%
External Regulation with Perceived Choice
- 0.879 p=0.001 16 77.26%
- 0.785 p=0.001 66 61.62%
Amotivation with Perceived Competence
- 0.602 p=0.014 16 36.24%
- 0.339 p=0.005 66 11.49%
Age with Perceived Competence - 0.710 p=0.002 16 50.41%
n=66 Inventory Respondents & n=16 Observed Participants
Motivation Inventory Back
63 of 45
Digital Native Correlations
Digital Native Relationship with...
Correlation Significance n R^2
...Age 0.536 p<0.001 16 28.73%
...Interest/Enjoyment 0.561 p=0.024 16 31.47%
...Perceived Choice 0.575 p=0.020 16 33.06%
...Perceived Competence 0.647 p=0.007 16 41.86%
...External Regulation -0.534 p=0.033 16 28.52%
Digital Native Significant Correlations for Observed Participants
Motivation Inventory
Back
64 of 45
Mean Occurrences of Codes
Extrinsics Intrinsics
stumble 10.11 3.00
fall 6.11 1.57
quit 1.00 .29
resist 1.11 .29
persist 1.67 .29
Mean Number of Code Occurrences for Extrinsics and Intrinsics
Observations
Back
65 of 45
Correlations for Extrinsics
Relationship To Correlation Significance n R^2
Age Persist 0.667 0.050 9 44.49%
Digital Native -0.728 0.026 9 53.00%
Amotivation -0.713 0.031 9 50.84%
External Regulation
Perceived Choice
-0.699 0.036 9 48.86%
Extrinsic Group Significant Relationships
Observations
Back
66 of 45
Correlations for Intrinsics Relationship To Correlation Significance n R^2
Stumble Fall 0.898 .006 7 80.64%
Age 0.823 .023 7 67.73%
Digital Native -0.832 .020 7 69.22%
Interest -0.861 .013 7 74.13%
Perceived Competence -0.917 .004 7 84.09%
Digital Native Age -0.866 .012 7 75.00%
External Regulation -0.874 .010 7 76.39%
Interest/Enjoyment 0.866 .012 7 75.00%
Perceived Choice 0.866 .012 7 75.00%
Perceived Competence 0.866 .012 7 75.00%
Age External Regulation 0.757 .049 7 57.30%
Perceived Competence -0.929 .003 7 86.30%
Perceived Competence
Fall -0.768 .044 7 58.98%
Interest/Enjoyment 0.786 .036 7 61.78%
External Regulation Perceived Choice
-0.883 .008 7 77.97%
Observations
Back
67 of 45
Extrinsics Detail Asterisk denotes digital native
name
stu
mb
le
fall
qu
it
resi
st
per
sist
age
Dig
ital
nat
ive
Self
rat
e
exp
er
amo
tiva
tio
n
Exte
rnal
re
gula
tio
n
Inte
rest
En
joym
ent
Per
ceiv
ed
Ch
oic
e
Per
ceiv
ed
Co
mp
eten
ce
Alice 6 4 0 0 4 71 1 3 16to25 1.00 4.75 4.00 3.29 4.17
Lilly 10 5 1 0 0 48 1 7 6to15 1.50 6.50 1.57 1.57 4.33
Lucy* 3 3 1 1 1 34 2 6 16to25 2.75 6.00 3.00 1.57 4.33
Marsha 10 8 1 3 2 68 1 3 more25 1.00 4.50 2.57 2.43 4.33
Mary Ann 16 6 0 0 1 60 1 4 16to25 1.25 6.50 3.71 2.57 4.00
Miranda 10 9 1 2 0 58 1 4 6to15 2.00 4.50 2.00 3.14 3.33
Molly* 10 6 2 0 1 40 2 6 more25 2.75 4.25 3.43 5.57 3.67
Olivia 12 5 1 7 1 48 1 4 16to25 2.75 5.75 1.57 2.14 2.50
Walter 14 10 2 0 5 74 1 2 6to15 2.00 5.75 2.71 1.71 2.67
Observations
Back
68 of 45
Name
stu
mb
le
fall
qu
it
resi
st
per
sist
age
Dig
ital
nat
ive
Self
rat
e
exp
er
amo
tiva
tio
n
Exte
rnal
re
gula
tio
n
Inte
rest
E
njo
yme
nt
Per
ceiv
ed
Ch
oic
e P
erce
ived
C
om
pet
en
ce
Beth* 3 2 0 0 0 26 2 8 16to25 1.00 1.00 6.43 5.86 6.33
Jane* 0 0 0 0 0 27 2 9.5 16to25 1.00 2.50 6.71 5.14 6.67
Mike 8 6 1 2 2 74 1 4 16to25 1.50 2.75 4.14 4.14 3.00
Peter* 0 0 0 0 0 24 2 10 16to25 1.00 2.50 6.57 4.86 7.00
Rebecca 3 0 0 0 0 65 1 5 more25 1.00 4.00 5.57 4.29 5.17
Roger* 0 0 0 0 0 23 2 11 16to25 1.00 2.00 6.14 5.29 6.83
Wilma 7 3 0 0 0 87 1 3 16to25 1.00 3.75 5.00 4.71 2.67
Observations
Intrinsics Detail Asterisk denotes digital native Back
69 of 45
Phase 2 & 3 Extrinsic Stumbles
Total stumble occurrences for each extrinsic participant in phase 2 (blue on left) and phase 3 (orange on right)
Observations
Back
70 of 45
Experience Age A
motivation
Ext
ern
al
Reg
ula
tion
Inte
rest/
En
joym
en
t
Perc
eiv
ed
C
hoic
e
Perc
eiv
ed
C
om
pete
nce
Beth 16-25 years 26 1.00 1.00 6.43 5.86 6.33
Jane 16-25 years 27 1.00 2.50 6.71 5.14 6.67
Roger 16-25 years 23 1.00 2.00 6.14 5.29 6.83
Peter 16-25 years 24 1.00 2.50 6.57 4.86 7.00
Observations
Comparing Intrinsic Digital Native Inventory Scores Ordered from Lowest Perceived Competence to Highest
Back
71 of 45
Experience Age A
motivation
Ext
ern
al
Reg
ula
tion
Inte
rest/
En
joym
en
t
Perc
eiv
ed
C
hoic
e
Perc
eiv
ed
C
om
pete
nce
Wilma 16 - 25 years 87 1.00 3.75 5.00 4.71 2.67
Mike 16 - 25 years 74 1.50 2.75 4.14 4.14 3.00
Rebecca 25+ years 65 1.00 4.00 5.57 4.29 5.17
Observations
Comparing Intrinsic Digital Non-Native
Inventory Scores Ordered from Lowest Perceived Competence to Highest
Back
72 of 45
Using Help or Not
• Many had no experience • Or old experience from 10 years ago when help was
notoriously bad • Stumbling of intrinsic digital native Beth had different
quality because used help “Because it’s going to have 50 pages of text that I have no desire whatsoever to read about something that I use rarely, and I don’t really care to know. I don’t read instruction manuals, generally. And why would I Google it? I wouldn’t, because it’s a bunch of teenagers who can’t spell right, who don’t use punctuation, all lower case,” answers Lucy.
Introduction Study Design Motivation Observations Future Work Conclusions
Back
73 of 45
All 66 Respondents
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
0.00 2.00 4.00 6.00 8.00
perc
eiv
ed c
om
pete
nce
interest/enjoyment
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
0.00 2.00 4.00 6.00 8.00
perc
eiv
ed c
om
pete
nce
external regulation
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
0.00 2.00 4.00 6.00 8.00
perc
eiv
ed c
om
pete
nce
perceived choice
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
0.00 2.00 4.00 6.00
perc
eiv
ed c
om
pete
nce
amotivation
More
Introduction Study Design Motivation Observations Future Work Conclusions
Extrinsics
Intrinsics
Apathy
74 of 45
All 16 Observed Participants
0
1
2
3
4
5
6
7
8
0 2 4 6 8
perc
eiv
ed c
om
pete
nce
interest/enjoyment
0
1
2
3
4
5
6
7
8
0 2 4 6 8
perc
eiv
ed c
om
pete
nce
external regulation
0
1
2
3
4
5
6
7
8
0 2 4 6 8
perc
eiv
ed c
om
pete
nce
perceived choice
0
1
2
3
4
5
6
7
8
0 1 2 3
perc
eiv
ed c
om
pete
nce
amotivation
Back
75 of 45
Correlations
Introduction Study Design Motivation Observations Future Work Conclusions
Relationship To
Age Persist +
Digital Native -
Amotivation -
External Regulation
Perceived Choice
-
Relationship To
Stumble Fall +
Age +
Digital Native -
Interest -
Perceived Competence -
Digital Native Age -
External Regulation -
Interest/Enjoyment +
Perceived Choice +
Perceived Competence +
Age External Regulation +
Perceived Competence -
Perceived Competence
Fall -
Interest/Enjoyment +
External Regulation Perceived Choice
-
Intrinsics
Extrinsics
More
76 of 45
Test Effects
• Lowering of emotions
• Learning without any teaching – “do it” = “you CAN do it”
– Expect researcher to fix any problems
• Performance hindrances – Age (Mike, Wilma, Walter, Marsha)
– Eye strain (Walter, Wilma)
– Tiredness (Lucy, Miranda, Wilma, Walter)
– Distraction (Molly’s daughter, Walter)
Introduction Study Design Motivation Observations Future Work Conclusions
77 of 45
Test Effects: Stress
Intrinsics max stress JE Users max stress
Beth 1 Lilly 2
Jane 1 Mary Ann 4
Rebecca 1 Molly 5
Roger 1 Olivia 5
Mike 2 Walter 5
Peter 6 Alice 6
Wilma 10 Lucy 7
Marsha 8
Miranda 10
Max Stress Self Rating of Participant by intrinsic (left) and extrinsic (right) on
a scale of 1 to 10 with 10 being high and 1 is low. Ordered from low stress to
high for both groups.
Introduction Study Design Motivation Observations Future Work Conclusions
78 of 45
Proposed Solutions
• Limit unfamiliar tasks, software, or systems (impractical)
• Teach big picture patterns and how they relate from one situation to another
• Teach visual and vocabulary tools
• Give a sense of belonging
• Generate interest and choice
Introduction Study Design Motivation Observations Future Work Conclusions
79 of 45
JE User vs. Intrinsic: Walter & Mike
Similarities: 1. Same age 2. Both retired professors 3. Both persisting 4. Similar competence 5. Similar experience level 6. Both agitated but say they are “fine”
Differences: 1. Performance 2. Different motivation style
Introduction Study Design Motivation Observations Future Work Conclusions
More
80 of 45
Weaknesses
1. Sample was a convenient sample 2. Ordinal Likert scale results should not be
averaged 3. All participants had different tasks so they
are not easily comparable 4. Sample size was small 5. Pros and cons of qualitative ethnographic
techniques 6. Did not measure the quantity, rate, and type
of task success
Introduction Study Design Motivation Observations Future Work Conclusions
81 of 45
What could have been…
1. Randomized respondent selection for motivation inventory to get an evenly distributed sample
2. Standardized tasks assigned to measure rate and type of stumbling and success
3. Standardized unfamiliar and familiar system and software
4. Give written instructions instead of verbal
5. Keep researcher ignorant of motivation scores before observations
6. Participant alone in a room with the observer outside the room
7. Possibly observing through one way glass or video camera and screen capture
8. Eliminating researcher interaction with participants
Introduction Study Design Motivation Observations Future Work Conclusions
82 of 45
So Much Data…
Could be re-analyzed with other emphases
• Digital literacy
• Communication patterns
• Misinformation or ignorance of a novice
• Attitudes to life long learning
• Attitudes of a “refuser”
• And more…
Introduction Study Design Motivation Observations Future Work Conclusions
83 of 45
Bigger Sample
• What percent are extrinsic? Intrinsic?
• What percent are Low-Low or High-High?
• How to characterize Low-Low or High-High?
• What percent are digital natives and non-natives?
• Do age, perceived competence, or being digital native hold no difference across intrinsic and extrinsic?
Introduction Study Design Motivation Observations Future Work Conclusions
84 of 45
The Flaws Were Also Strengths
• Rich and diverse insights into identifying JE users
• Diverse population of daily proficient users
• Successfully quantified failure
• Likert scale average is standard in Social Science
• Captured individual proficiency and tested unfamiliar tasks, software, & system
• Captured motivation style
Introduction Study Design Motivation Observations Future Work Conclusions
85 of 45
Other Questions
• “Just Enough” term?
• Gender, socioeconomic status, years of experience, aversion to change?
• Separating work and play in motivation study
• Less frequent users?
• What if a “consequence” element?
• Hand held computers?
• Food and sleep deprived?
Introduction Study Design Motivation Observations Future Work Conclusions
86 of 45
Statistical Analysis of Inventory
Statistical Analysis Inventory Factors
PARAMETRIC: Passes 3 assumptions
for parametric analysis
1.external regulation 2.interest/enjoyment 3.perceived choice 4.identified regulation
NON-PARAMETRIC: Must be non-parametrically analyzed
1.amotivation 2.perceived competence 3.age 4.digital native
More information and detail in Supplementary Slides
Introduction Study Design Motivation Observations Future Work Conclusions
More
Do 3 assumptions hold for n=66 Respondents and also for n=16 Participants?
87 of 45
Significant Differences between extrinsic and intrinsic
Phase Stumble Fall Persist Quit
2 Different (p=0.003)
Different (p=0.003)
Not Significant (p=0.127)
Different (p=0.041)
3 Different (p=0.018)
Different (p=0.025)
Different (p=0.023)
Not Significant (p=0.470)
Both Phases
Different (p=0.004)
Different (p=0.005)
Different (p=0.030)
Different (p=0.014)
Extrinsics and Intrinsics Have Significant Differences in Phase 2,
Phase 3, and total occurrences.
Introduction Study Design Motivation Observations Future Work Conclusions
88 of 45
Task FAMILIAR GMAIL webmail
Difference in Work Flow
UNFAMILIAR Obscure Company webmail
UNFAMILIAR GMX webmail
compose
mail (Fig.
6)
click “compose” button, top
left, contrasting color obscure is more difficult to see
compose but all compose are in
same area of screen
click pencil/paper icon, top
left between other icons,
same color, no words
click “compose mail” button,
top left, same color
open
inbox click “inbox” word on left
top, same color NO DIFFERENCE click “inbox” word on left top,
same color click “inbox” word on left top,
same color
read mail click “[name of sender or
participant]” of mail in
center large panel, same
color, opens by replacing
same center window
Gmail replaces center panel,
others open side by side with
inbox list either below or to the
right
click “[name of sender or
participant]” of mail in center
top half panel, same color,
opens in bottom half of
center panel
click “[name of sender or
participant]” of mail in left half
of center panel, same color,
opens in right half of center
panel
reply to
mail click “arrow” icon button on
right at top of what reading,
same color, no word, OR
gray “reply” word link at
bottom, same color in
separate white box. NOTE:
if email is medium to large,
“arrow” icon button
disappears into the header
and the second choice
disappears into the footer
Gmail has two places, both
same color, one is word, one is
icon, one or both can disappear
with medium and bigger emails,
floats on top of mail view so not
always visible. Both gmx and
obscure have one step, button
with word and icon, always
visible
click “reply” button with
picture and word, top icon
bar, first of 9 buttons with
words, same color
click “reply” button with
picture and word, top icon
bar, 2nd of 7 buttons with
words, same color
forward
mail click “drop down” arrow on
“arrow” for reply to see
more options, same color,
then select “forward in drop
down menu”, all on center
right at top of email
reading, or click gray on
white words in white box at
bottom (often not visible if
reading anything other than
shortest email) NOTE:
same as for reply
Gmail has two places, one
requires two steps (select from
drop down), both ways are
same color, one is word, one is
icon, one or both can disappear
with medium and bigger emails,
floats on top of mail view so not
always visible. Both gmx and
obscure have one step, button
with word and icon, always
visible, both at top center area
click “forward” button with
picture and word, top icon
bar, 3rd of 9 buttons with
words, same color
click “forward” button with
picture and word, top icon
bar, 3rd of 7 buttons with
words, same color
89 of 45
Phase 2 & 3 No Difference
Stumble Fall Persist Quit Resist
Extrinsic (p=0.370) (p=0.147) (p=0.738) (p=0.056) (p=0.494)
Intrinsic (p=0.784) (p=0.872) (p=0.317) (p=0.317) (p=0.317)
There was no significant different between Unfamiliar
Task compared to Near Skill Transfer for either
intrinsics or extrinsics.
Introduction Study Design Motivation Observations Future Work Conclusions