Analyzing Students' Behavior in a Beginner's

Programming CourseMarija Brkić, Higher Teaching Assistant

mbrkic@inf.uniri.hr

Maja Matetić, Associate Professor

majam@inf.uniri.hr

Department of Informatics, University of RijekaRadmile Matejčić 2, 51000 Rijeka, Croatia

http://www.inf.uniri.hr

Why are we doing this?

• Task 6 of the strategy of the University of Rijeka for the period 2008-2013:

• Pass rate increase for 2nd year enrollment to 75%

• The course pass rate in the academic year 2012/2013 was 61%.

• The course pass rate in the academic year 2011/2012 was 63%.

• We are facing a falling pass rate!!!

Course info

• Programming 1

• mandatory course

• 1st year of undergraduate study of Informatics

• C++

• procedural programming

• 82 students in our case study

• LMS Moodle, supplemental instruction classes

ActivityActivity ScoresScores

Online quizzesOnline quizzes 2525

Self-evaluationSelf-evaluation 44

LabsLabs 99

GlossaryGlossary 22

ExamExam 2525

ActivityActivity 66

Visualization as a pre-processing tool

2012/2013

35%

21%

35%

9%

final exam

remedial exam

fail

dropout rate2011/2012

36%

24%

36%

4%

final exam

remedial exam

fail

dropout rate

Grade distribution

2012/2013

10

7 7

5

0

2

4

6

8

10

12

A B C D

Avoiding examination

2012/2013

0

5

10

15

20

25

30

35

1st quizz 2nd quizz exam

# o

f a

bs

en

t s

tud

en

ts

regular date

extra date

Repeating the course2012/2013

79,3%

13,4%

20,7%

7,3%

novices

students repeating the coursewho failed again

students repeating the coursewho passed

2011/2012

84,2%

7,9%

7,9%

15,8%

novices

students repeating the coursewho failed again

students repeating the coursewho passed

Additional activity I

2012/2013

0

10

20

30

40

50

60

70

1st 2nd 3rd 4th 5th 6th

Labs

# o

f s

tud

en

ts

success

total

Additional activity II

Relationship with the final grade

Pre-processing

• Missing values for one part of activities have been replaced with minimum values

• Examples with the remaining missing values have been filtered out

• Additional attribute has been generated (Labs)

Data mining techniques

• Association

• Classification

• Clustering

• Outlier detection

Association rulesNo Antecedent Confidence Lift

1 Exam 1 1,761

2 Self-evaluation, Exam 1 1,761

3 Labs, Exam 1 1,761

4 Quizzes, Exam 1 1,761

5 Self-evaluation, Labs, Exam 1 1,761

6 Self-evaluation, Quizzes, Exam 1 1,761

7 Labs, Quizzes, Exam 1 1,761

8 Self-evaluation, Labs, Quizzes, Exam 1 1,761

9 Labs, Quizzes, Glossary 0,95 1,677

10 Self-evaluation, Labs, Quizzes, Glossary 0,95 1,677

11 Quizzes, Glossary 0,91 1,601

12 Self-evaluation, Quizzes, Glossary 0,91 1,601

13 Labs, Quizzes 0,88 1,55

14 Self-evaluation, Labs, Quizzes 0,88 1,55

15 Quizzes 0,83 1,467

16 Self-evaluation, Quizzes 0,83 1,467

Classification rules

Clustering

Activity6

Exam25

Self-evaluation

4

Glossary2

Quizzes25

Labs9

Total71

Items

Cluster 0 2.765 20.618 3.806 0.379 21.018 7.059 55.644 17

Cluster 1 0.765 14.412 3.361 0.296 18.021 4.412 41.266 17

Cluster 2 1.200 5.548 3.212 0.140 15.794 3.000 28.843 20

Cluster 3 0.071 1.093 2.788 0.069 9.907 1.607 15.392 14

Cluster 4 0.000 0.000 0.886 0.006 1.750 0.364 2.941 13

Outlier detection

Activity Exam Self-evaluation Glossary Quizzes Labs Total

Cluster 0 6.0 16.5 3.86 0.28 19.0 6.0 51.64

Cluster 0 1.0 25.0 4.0 1.2 23.75 6.0 60.95

Cluster 0 0.0 23.0 3.86 1.28 17.5 3.0 48.64

Cluster 0 5.0 20.0 3.86 0.0 23.75 9.0 61.61

Cluster 2 0.0 15.5 3.14 0.0 13.5 1.5 33.64

Cluster 2 0.0 19.0 3.71 0.0 14.0 6.0 42.71

Cluster 2 5.0 11.0 3.93 1.04 21.8 3.0 45.77

Cluster 2 2.0 13.0 3.68 1.12 21.0 7.5 48.3

Cluster 3 0.0 7.5 2.0 0.0 8.25 1.5 19.25

Cluster 4 4.0 1.5 3.5 0.08 18.0 0.0 27.08

Student comments on newly introduced activities

• Official evaluation

– I liked the labs because they force us to work on new materials continuously

– I liked the labs because they encourage us to exercise regularly

– professors gave us a lot of materials and organized everything perfectly – from labs to supplemental instruction

• Class evaluation

– the labs made us work continuously

– it is a good idea for getting scores, though the evaluation system should be less harsh and give partial credits

– excellent idea set to practice perfectly

– labs helped a lot for continuous engagement

Future work

• Time analysis (self-evaluation)

• Log analysis (forum, laboratory exercises, etc.)

• Classification issues

Conclusion

• We are actually not facing a falling pass rate!!!

Thank You for your attention!