gauging mooc learners’ adherence to the ......week 3 fp101x ri101x frame101x ex101x 2. behavior...
TRANSCRIPT
GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED LEARNING PATHDAN DAVIS, GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN
TU DELFT WEB INFORMATION SYSTEMS GROUP
WHAT IS A LEARNING PATH ?WHAT IS A LEARNING PATH ?
WHAT IS A LEARNING PATH ?THE SEQUENCE OF EVENTS A STUDENT TAKES TOWARDS A LEARNING OBJECTIVE
RQ to what extent do learners adhere to the designed learning path?
VIDEO QUIZ PROGRESS FORUM
WATCH START VIEW STARTSUBMITEND
SUBMITEND
VIDEO QUIZEND
QUIZSUBMIT
QUIZSTART
VIDEO
VIDEO
VIDEO
QUIZEND
QUIZEND
QUIZSUBMIT
QUIZSTART
QUIZSUBMIT
QUIZSTART
Data Analysis
Functional Programming
Framing
Responsible Innovation
SUBMIT ENDFORUMSTART
SUBMIT ENDFORUMSTART
v Q Q Q Q Q Q Q Q Q Q Q Q Q QQ
Q Q Q Q Q Q Q Q Q Q Q Q Q Q
Q Q Q Q Q Q Q Q Q Q Q Q
Q
Q Q Q Q Q Q
v v
v v v F F F F F F
F F F
F F F F F F F F F F F F F F F
v v v v v v v
v v v v v v v
v v v v v v
Q
DESIGNED LEARNING PATHS
Data Analysis
Functional Programming
Framing
Responsible Innovation
RELATED WORK RELATED WORK
Identifying latent study habits by mining learner behavior patterns in massive open online courses. In CIKM ’14MIAOMIAO WEN & CAROLYN ROSÉ
Demographic differences in how students navigate through MOOCs. In L@S ‘14PHILIP GUO & KATHARINA REINECKE
MOOC ENROLLED PASS CHAINSPASS/NON
QUIZQUESTIONSRATE
TRIES VIDEOSACCESSED
FP101x
RI101x
Frame101x
EX101x
37k
9k
34k
33k
5.3%
4.3%
2.4%
6.5%
1.06M/807K
66k/30k
95k/141k
1.02M/855k
288
75
26
136
1
1—3
2
2
67.5%
49.7%
51%
45%
*
* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%
MOOC ENROLLED PASS CHAINSPASS/NON
QUIZQUESTIONSRATE
TRIES VIDEOSACCESSED
FP101x
RI101x
Frame101x
EX101x
37k
9k
34k
33k
5.3%
4.3%
2.4%
6.5%
1.06M/807K
66k/30k
95k/141k
1.02M/855k
288
75
26
136
1
1—3
2
2
67.5%
49.7%
51%
45%
*
* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%
Frame101x
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
3
4
6
36
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
1 1
1
121
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
7 7
7
6
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
5 1
1 1
1
1
1
101
FP101x
RI101x
EX101x
Frame101x
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
3
4
6
36
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
1 1
1
121
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
7 7
7
6
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
5 1
1 1
1
1
1
101
FP101x
RI101x
EX101x
EACH COURSE IS UNIQUE
Frame101x
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
3
4
6
36
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
1 1
1
121
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
7 7
7
6
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
5 1
1 1
1
1
1
101
FP101x
RI101x
EX101x
EACH COURSE IS UNIQUE
ANALYZE ACCORDINGLY
APPROACH APPROACH 1. VIDEO INTERACTIONS2. BEHAVIOR PATTERN CHAINS3. EVENT TYPE TRANSITIONS
FINDINGSFINDINGS
1. VIDEO INTERACTIONSDO LEARNERS WATCH VIDEO LECTURES IN THE PRESCRIBED ORDER?
VIDEO INTERACTIONS
Non-Passing
Passing
Week 1 Week 2Week 3
FP101xDesigned Lecture Order
Executed Paths
Non-Passing
Passing
Week 1Week 2
Week 3
VIDEO INTERACTIONS
Non-Passing
Passing
Week 1 Week 2Week 3
Non-Passing
Passing
Week 1 Week 2Week 3
Non-Passing
Passing
Week 1Week 2
Week 3
FP101x
RI101x
Frame101x
EX101x
2. BEHAVIOR PATTERN CHAINSWHAT ARE THE MOST COMMON SEQUENCES IN THE EXECUTED LEARNING PATHS?
FORUMEND
EIGHT-STEP CHAINEXAMPLE
LECTUREWATCH QUIZSTART QUIZSUBMIT QUIZSUBMIT
QUIZEND PROGRESS LECTUREWATCH
MOOC ENROLLED PASS CHAINSPASS/NON
QUIZQUESTIONSRATE
TRIES VIDEOSACCESSED
FP101x
RI101x
Frame101x
EX101x
37k
9k
34k
33k
5.3%
4.3%
2.4%
6.5%
1.06M/807K
66k/30k
95k/141k
1.02M/855k
288
75
26
136
1
1—3
2
2
67.5%
49.7%
51%
45%
*
* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%
OPEN CARD SORTINGOPEN CARD SORTING
MOTIFA RECURRING OR REPEATED ELEMENT
MOTIF FREQTOTAL
FREQPASS
FREQFAIL
QUIZ COMPLETE 552,363(29.4%)
328,995(30.8%)
223,368(27.7%)
1.
BINGE WATCHING 149,784(8%)
59,498(5.6%)
90,286(11.2%)
2.
LECTURE -> QUIZ COMPLETE 100,179(5.3%)
50,415(4.7%)
49,764(6.2%)
3.
QUIZ COMPLETE -> FORUM 99,828(5.3%)
67,722(6.3%)
32,106(4%)
4.
FP101x
XQUIZ events only with at least 1 X = SUBMIT
WATCH events only
WATCH event(s) followed by XQUIZ events w/ >1 X = SUBMIT
XQUIZ events w/ >1 X = SUBMIT followed by XFORUM events
3. EVENT TYPE TRANSITIONSHOW DO STUDENTS NAVIGATE BETWEEN CERTAIN EVENT TYPES?
EVENT TYPE TRANSITIONS
Frame101x Non-PassingMarkov ModelState Visualization
(EXECUTED)
EVENT TYPE TRANSITIONS
Frame101x Non-Passing Frame101x Passing
(EXECUTED)
EVENT TYPE TRANSITIONS
Frame101x Non-Passing Frame101x Passing
(EXECUTED)
32%
35%
VIDEO
QUIZEND
QUIZSUBMIT
FORUMSUBMIT
FORUMEND
PROGRESS
FORUMSTART
QUIZSTART VIDEO
QUIZEND QUIZ
SUBMIT
FORUMSUBMIT
FORUMEND
PROGRESS
FORUMSTART
QUIZSTART
21%28%
44%
44%
DAVIS, D., CHEN, G., JIVET, I., HAUFF, C., & HOUBEN, G. J. (2016). ENCOURAGING METACOGNITION & SELF-REGULATION IN MOOCS THROUGH INCREASED LEARNER FEEDBACK. LAK 2016 LAL WORKSHOP
FEEDBACKEXAMPLE
Frame101x PassingFrame101x Non-Passing
RI101x Non-Passing RI101x Passing
EX101x PassingEX101x Non-Passing
FP101x Non-Passing FP101x Passing
APPLICATIONSAPPLICATIONS
1. BEYOND PASS-FAIL SEGMENTATION2. PATTERN MINING & CONFORMANCE CHECKING3. POSITIVE DEVIANCE
THANK YOUTU DELFT WEB INFORMATION SYSTEMS GROUP
DAN DAVIS, GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN
BIT.LY/WIS-LEARNING-ANALYTICS