The Effect of The Effect of Gender on the Gender on the Response to Response to

Technological Technological ErrorErrorCalvin Chan and Jamie JinCalvin Chan and Jamie Jin

PSY/ORF 322PSY/ORF 322

Spring 2005Spring 2005

The BeginningsThe Beginnings

Professor Cooper’s lectures inspired Professor Cooper’s lectures inspired us to think about the gender gap, us to think about the gender gap, specifically in college. specifically in college.

Research has shown that the Research has shown that the comfort level with computers is comfort level with computers is based on previous experience, which based on previous experience, which correlates with gender.correlates with gender.

We wanted to see how wide, if at all, We wanted to see how wide, if at all, the gender gap was in college. the gender gap was in college.

A Case StudyA Case Study Jane Margolis conducted a study on CS Jane Margolis conducted a study on CS

students at CMU. She gave a survey to all students at CMU. She gave a survey to all first year CS students:first year CS students:

40% of males passed the AP exam to place 40% of males passed the AP exam to place out of the intro course. out of the intro course. No females passed the exam. No females passed the exam.

53% of males indicated they were highly 53% of males indicated they were highly prepared for their classes.prepared for their classes. No females rated themselves as highly prepared. No females rated themselves as highly prepared.

50% of males indicated they had an expert 50% of males indicated they had an expert level of at least one programming language. level of at least one programming language. No females indicated they had such an expert No females indicated they had such an expert

level. level.

The Gender Gap ExistsThe Gender Gap Exists

The gender gap in college is prominent:The gender gap in college is prominent: At the top 12 research universities, females At the top 12 research universities, females

receive 18% of bachelor’s CS degrees receive 18% of bachelor’s CS degrees (Margolis 1997) (Margolis 1997)

The takers of the AB level CS AP exam were The takers of the AB level CS AP exam were 12% female in the years from 1992 to 1997. 12% female in the years from 1992 to 1997.

It seems that males are the ones It seems that males are the ones experiencing computers in high school, experiencing computers in high school, which translates into their dominance in which translates into their dominance in numbers and confidence in college.numbers and confidence in college.

The MotivationThe Motivation Everybody knows how to type an essay in Everybody knows how to type an essay in

Microsoft Word, but how do people react Microsoft Word, but how do people react when faced with an error or a crashed when faced with an error or a crashed program? program?

Does the gender gap predict how males Does the gender gap predict how males and females interact with computers? and females interact with computers?

We wanted to test if and how people We wanted to test if and how people would react to a software failure, and to would react to a software failure, and to see if gender correlated with their see if gender correlated with their exploring abilities or “trial and error” exploring abilities or “trial and error” mentality. mentality.

The Test CreationThe Test Creation

We decided to create a computer program We decided to create a computer program disguised as a survey. disguised as a survey.

However, an intentional error was However, an intentional error was programmed, so that it would activate in the programmed, so that it would activate in the middle of the survey.middle of the survey.

Information gathered included: time taken, Information gathered included: time taken, major, gender, self-rated comfort level with major, gender, self-rated comfort level with computers.computers.

We wanted to see how long the user would We wanted to see how long the user would fiddle with the program before seeking help fiddle with the program before seeking help from the surveyors.from the surveyors.

The HypothesisThe Hypothesis

The Gender Gap seems to predict The Gender Gap seems to predict that women would be quicker to that women would be quicker to seek assistance after encountering seek assistance after encountering an error, and thus would spend less an error, and thus would spend less time with the survey.time with the survey.

Also, it seemed that males would try Also, it seemed that males would try to fix the error or try to bypass it in to fix the error or try to bypass it in some way.some way.

The ExperimentThe Experiment A completely computerized approach was A completely computerized approach was

used to implement the study.used to implement the study. Users were pointed to a link, where they Users were pointed to a link, where they

would be able to download and run our would be able to download and run our “survey.” “survey.”

They were instructed to download and run the They were instructed to download and run the program, and to seek us only after they were program, and to seek us only after they were complete with the survey, wanted to stop, or complete with the survey, wanted to stop, or had a problem had a problem

A data file on the computer was created A data file on the computer was created automatically when the program was run, that automatically when the program was run, that stored all of the info we aimed to collect.stored all of the info we aimed to collect.

Get Ready for a Demo!Get Ready for a Demo!

Get Ready for a Demo!Get Ready for a Demo!

Survey.exe

Stories from the FieldStories from the Field

A majority of people would seek help A majority of people would seek help immediately after seeing an error, immediately after seeing an error, regardless of gender.regardless of gender.

Many complaints that the survey Many complaints that the survey didn’t work.didn’t work.

A few participants later mentioned A few participants later mentioned that they suspected what we were that they suspected what we were up to.up to.

Gender vs. Comfort LevelGender vs. Comfort LevelAverage Comfort Level

5.6

5.8

6

6.2

6.4

6.6

6.8

7

7.2

7.4

females males

Co

mfo

rt (

1 t

o 1

0)

Gender vs. TimeGender vs. TimeAverage Time

117.00

118.00

119.00

120.00

121.00

122.00

123.00

124.00

females males

Tim

e (s

eco

nd

s)

Gender vs. TimeGender vs. TimeMedian Time

80.00

85.00

90.00

95.00

100.00

105.00

110.00

females males

Tim

e (s

eco

nd

s)

Gender vs. RestartsGender vs. RestartsAverage Restarts

0

0.5

1

1.5

2

2.5

3

females males

Re

sta

rts

Gender vs. RestartsGender vs. RestartsMedian Restarts

0

0.5

1

1.5

2

2.5

females males

Res

tart

s

Major vs. Comfort LevelMajor vs. Comfort LevelMajor vs. Comfort Level

5.6

5.8

6

6.2

6.4

6.6

6.8

7

7.2

7.4

7.6

AB BSE

Co

mfo

rt (

1 t

o 1

0)

Major vs. TimeMajor vs. TimeAverage Time

112

114

116

118

120

122

124

126

128

130

AB BSE

Tim

e (

se

co

nd

s)

Major vs. TimeMajor vs. TimeMedian Time

95

96

97

98

99

100

101

102

AB BSE

Tim

e (

se

co

nd

s)

Major vs. restartsMajor vs. restartsAverage Restarts

1.9

2

2.1

2.2

2.3

2.4

2.5

2.6

2.7

AB BSE

Re

sta

rts

Major vs. restartsMajor vs. restartsMedian Restarts

0

0.5

1

1.5

2

2.5

BSE AB

Re

sta

rts

Comfort Level vs. TimeComfort Level vs. TimeComfort Level vs. Time

y = 6.4778x + 77.985

R2 = 0.0227

0.00

50.00

100.00

150.00

200.00

250.00

300.00

350.00

400.00

0 2 4 6 8 10

Comfort Level (from 1 to 10)

Tim

e (

se

co

nd

s)

Comfort Level vs. Comfort Level vs. RestartsRestarts

Comfort Level vs. Restarts

y = 0.0135x + 2.3535

R2 = 0.0002

0

2

4

6

8

10

12

0 2 4 6 8 10 12

Comfort Level (1 to 10)

Re

sta

rts

What does our data say?What does our data say?

It suggests a gender gap, but is It suggests a gender gap, but is inconclusive.inconclusive. There is not a large difference between the There is not a large difference between the

genders, especially when considering standard genders, especially when considering standard deviations.deviations.

Comfort level does not seem to provide a Comfort level does not seem to provide a good correlation with restarts/time spent good correlation with restarts/time spent with the program, at least for this set of with the program, at least for this set of data.data.

Outliers may be the reason the genders are Outliers may be the reason the genders are characterized the way they are.characterized the way they are.

What does our data say?What does our data say? Other important issues to note include:Other important issues to note include: Girls report lower comfort levels, but Girls report lower comfort levels, but

work past the error almost as well as work past the error almost as well as boys.boys.

Girls take longer with the survey than Girls take longer with the survey than boys, but length of time does not boys, but length of time does not correlate with error response.correlate with error response.

There is a perceived gender gap, but There is a perceived gender gap, but similar performances from the data by similar performances from the data by both groups do not support the gender both groups do not support the gender gap. gap.

For the futureFor the future

Larger sample sizes are needed to Larger sample sizes are needed to understand any sort of correlation or understand any sort of correlation or conclusion. conclusion.

The reported low confidence of girls’ The reported low confidence of girls’ comfort levels indicate a problem. comfort levels indicate a problem.

There may need to be some sort of There may need to be some sort of reform in the education system to reform in the education system to account for the gender gap.account for the gender gap.

Society would gain to have more Society would gain to have more women in the area of IT.women in the area of IT.