online learning processes - what users do and why it is important
DESCRIPTION
Keynote Young Researcher's Workshop at ITK07, Finland Abstract: Online settings provide non-reactive ways to observe learners' activities. While the learning itself is not directly visible for example in a logfile or within the history of a Wiki, the growing expressiveness in online learning environments takes us a step nearer to witness and analyse learning processes. In this presentation four questions will be discussed: (1) What is the relation between possible user activities in recent E-Learning environments and actual learning processes? (2) What data is available to observe the users' activities? (3) What data analysis methods are fitting to model these activities? (4) What research designs do we need to get more insight into learning processes? As an example, some ideas, experiences and results from an ongoing research project about online user participation and motivation will be presented and discussed, focussing on a multimethod approach.TRANSCRIPT
![Page 1: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/1.jpg)
cc by Karsten D. Wolf 2007
Karsten D. WolfDidactical Design of Interactive Learning Environments
interaktiivinen tekniikka koulutuksessa 2007Wednesday, 18.04.2007
Online Learning Processes –What Users Do and Why it is Important
1
![Page 2: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/2.jpg)
Karsten D. WolfFaculty educational science
Didactical design ofinteractive learning environments
2
![Page 3: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/3.jpg)
cc by Karsten D. Wolf 2007
www.everlearn.eu
User Generated ContentDidactical PrototypingCollaborative Learning
3
![Page 4: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/4.jpg)
4
![Page 5: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/5.jpg)
© Karsten D. Wolf 2007
roadmap
1. Why educational research is hard
2. How can we produce evidence in educational research?
3. Multimethod approaches in educational research
4. Alternative methods: Intra-individual analysis of micro process timeseries
5. Educational technology as a research frontier in learning science
6. Some applied results
5
![Page 6: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/6.jpg)
cc by Karsten D. Wolf 2007
soft sciences are hard
6
![Page 7: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/7.jpg)
cc by Karsten D. Wolf 2007
7
![Page 8: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/8.jpg)
cc by Karsten D. Wolf 2007
btw: this is not a real sim.rocks do not decelerate before hitting the ground!
8
![Page 9: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/9.jpg)
cc by Karsten D. Wolf 2007
what if rocks would behave?
9
![Page 10: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/10.jpg)
cc by Karsten D. Wolf 2007
10
![Page 11: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/11.jpg)
cc by Karsten D. Wolf 2007
covariates hell
pic by Don Gato on Flickr11
![Page 12: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/12.jpg)
cc by Karsten D. Wolf 2007
3 strategies
control themlive with themr&d approach
Fischer/Waibel/Wecker (2006): Nutzenorientierte Grundlagenforschung im BildungsbereichArgumente einer internationalen Diskussion
12
![Page 13: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/13.jpg)
cc by Karsten D. Wolf 2007
experimental researchdesign based research
engineering approaches
13
![Page 14: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/14.jpg)
cc by Karsten D. Wolf 2007
14
![Page 15: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/15.jpg)
cc by Karsten D. Wolf 2007
75%15
![Page 16: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/16.jpg)
cc by Karsten D. Wolf 2007
Educational Researcher, Vol. 33, No. 2, 3–11
Educational Researcher, Vol. 32, No. 9, 3–14
16
![Page 17: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/17.jpg)
cc by Karsten D. Wolf 2007
Problems with quasi-experiments
Steyer 2007 - www.causal-effects.de
Simpson
paradox
17
![Page 18: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/18.jpg)
cc by Karsten D. Wolf 2007
Problems with quasi-experiments
Steyer 2007 - www.causal-effects.de18
![Page 19: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/19.jpg)
cc by Karsten D. Wolf 2007
Problems with quasi-experiments
Steyer 2007 - www.causal-effects.de19
![Page 20: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/20.jpg)
© Karsten D. Wolf 2007
Random Assignment & Propensity Score Matching
• Random Assignment
• is the population large enough?
• can the assignment truly random?
• doesn‘t it feel like a lottery?
• Propensity Score Matching
• need to collect covariates
• matching at an individual level between treatment and control
20
![Page 21: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/21.jpg)
© Karsten D. Wolf 2007
• self-reports (behaviors, psychological contructs, forms of behaviors)
• informant assessment
• ecological momentary assessment
• web based / computer based methods
• ability tests
• implicit methods (performance measures)
• sequential observational methods
Multimethod approaches: assessment methods
21
![Page 22: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/22.jpg)
© Karsten D. Wolf 2007
Multimethod approaches:assessment methods
• quantitative text analysis
• multilevel analysis: physiological & biochemical measures
• brain imaging
• experimental methods
• nonreactive methods
22
![Page 23: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/23.jpg)
© Karsten D. Wolf 2007
Experiments vs. Assessments
Why is Explanandum?
If (A1 and A2) then Explanandum (general law, explanans)
A1 (antecedent condition 1)
A2 (antecedent condition 2)
Therefore: E (explanandum)
Experiments aim to develop and test general lawsAssessments aim to show that the antecedent conditions are met
Erdfelder & Musch 2006
23
![Page 24: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/24.jpg)
© Karsten D. Wolf 2007
Level of nonreactivity
Low High
Type 0 Type 1 Type2 Type 3 Type 4 Type 5
Setting initially designed or selected for research yes yes yes yes yes no
Participants are aware of the research setting likely likely likely likely no no
Participants are aware of the research question likely likely likely no no no
Participants are aware of the research hypothesis likely likely no no no no
Participants are aware of the measures‘ manipulability likely no no no no no
Examples Participative expert interview
Bogus pipeline technique
Personality questionaire
Cover story experiments
Lost letter technique
Analysis of archival data
Fritsche and Linneweber 2006
24
![Page 25: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/25.jpg)
© Karsten D. Wolf 2007
Methods for analyzing multimethod data
• Multitrait-Multimethod Matrix (Campbell and Fiske 1959)
• categorical variables analysis for asessing multimethod association
• multimethod item response theory
• multilevel analysis
• structural equation models
• longitudinal methods
25
![Page 26: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/26.jpg)
cc by Karsten D. Wolf 2007
26
![Page 27: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/27.jpg)
27
![Page 28: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/28.jpg)
28
![Page 29: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/29.jpg)
29
![Page 30: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/30.jpg)
30
![Page 31: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/31.jpg)
31
![Page 32: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/32.jpg)
32
![Page 33: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/33.jpg)
I feel taken seriously (EG1 TN1 SoLe III)
33
![Page 34: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/34.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13 14
34
![Page 35: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/35.jpg)
Taken seriously Cognitive Load Participation
I feel good I understand I am interested
35
![Page 36: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/36.jpg)
I am interested for19 students in EG1 SoLe III
36
![Page 37: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/37.jpg)
I feel taken seriously (EG1 TN1 SoLe III)
37
![Page 38: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/38.jpg)
I feel taken seriously (EG1 TN1 SoLe III)Linear Trend
Trend
38
![Page 39: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/39.jpg)
I feel taken seriously (EG1 TN1 SoLe III)3rd Polynom
Trend
39
![Page 40: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/40.jpg)
I feel taken seriously (EG1 TN1 SoLe III)Spline λ=10.000
Trend
40
![Page 41: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/41.jpg)
I feel taken seriously (EG1 TN1 SoLe III)Spline λ=100.000
Trend
41
![Page 42: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/42.jpg)
I feel taken seriously (EG1 TN1 SoLe III)
Decomposition
Trend Microprocesses
42
![Page 43: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/43.jpg)
ARMA-Analysis
43
![Page 44: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/44.jpg)
AIC BIC R2 Φ
AR(1) 1201,8 1208,0 0,094 Φ1 = 0,307
AR(2) 1190,2 1200,9 0,139Φ1 = 0,238Φ2 = 0,223
AR(3) 1192,1 1206,5 0,139Φ1 = 0,237Φ2 = 0,222Φ3 = 0,007
AR(5) 1194,2 1215,8 0,145
Φ1 = 0,236Φ2 = 0,241Φ4 = -0,077Φ5 = -0,018Φ3 = 0,030
Model parameters
44
![Page 45: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/45.jpg)
AR(2)-Model
45
![Page 46: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/46.jpg)
Multivariate Analyis:Cross Correlations
FG
I
P
CL
t-3 t-2 t-1 t
I 0,1220,120
-0.143
0.146
46
![Page 47: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/47.jpg)
Multivariate Models: GMDH
FG
I
P
CL
t-2 t-1 t
0,097
-0.15
0.13
47
![Page 48: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/48.jpg)
“As the complexity of a system increases, our ability to make precise and significant statement about its behaviour diminishe until a threshold is reached … the closer one look at a real-world problem the fuzzier becomes its solution” (Kosko, 94)
Fuzzy Modelling (Zadeh)
48
![Page 49: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/49.jpg)
Fuzzification1
0,5
0Negative_X ZERO_X Positive_X
49
![Page 50: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/50.jpg)
IF
NOT_Participation (t-1) & TakenSeriously (t-2)
OR
NOT_Participation (t-1) & CognitiveLoad (t-2)
Fuzzy Rule InductionTakenSeriously (t-1) & NOT_Participation (t-2)
OR
NOT_Understand (t-1) & CognitiveLoad (t-2)
THEN FeelGood TN1 (prewhitened)
50
![Page 51: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/51.jpg)
Better communication?
Different interrelationships at different levels? (Thresholds)
non-linear models?
basis for building hypotheses?
Advantages?
51
![Page 52: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/52.jpg)
time consuming
lot of rules
no selection criteria
no „test of significance“
disadvantages
52
![Page 53: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/53.jpg)
cc by Karsten D. Wolf 2007
Online learning as a research frontier?
53
![Page 54: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/54.jpg)
cc by Karsten D. Wolf 2007
54
![Page 55: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/55.jpg)
cc by Karsten D. Wolf 2007
55
![Page 56: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/56.jpg)
cc by Karsten D. Wolf 2007
what is happening here?
how to measure this?
56
![Page 57: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/57.jpg)
cc by Karsten D. Wolf 2007
???
57
![Page 58: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/58.jpg)
cc by Karsten D. Wolf 2007
???Learning accompanying actions
BrowseSearchWrite…
58
![Page 59: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/59.jpg)
cc by Karsten D. Wolf 2007
???creation
59
![Page 60: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/60.jpg)
cc by Karsten D. Wolf 2007
???
construction
60
![Page 61: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/61.jpg)
cc by Karsten D. Wolf 2007
???
communication
61
![Page 62: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/62.jpg)
cc by Karsten D. Wolf 2007
???
cooperation
62
![Page 63: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/63.jpg)
cc by Karsten D. Wolf 2007
???
collaboration
63
![Page 64: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/64.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Some results from a prior online seminar
ZfE Cover
64
![Page 65: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/65.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Activity
# chat msg# forum entries
# created words
femalen=23
46.2(57.92)
7.0(7.44)
2301(363.5)
malen=13
24.7(23.43)
3.0(3.08)
1919(419.0)
1-tail significant test, n=36
65
![Page 66: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/66.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Activity and Post-Interest
# chat msg# forum entries
# created words
Post-interest
.11(.258)
.19(.135)
.34*(.022)
1-tail significant test, n=36
66
![Page 67: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/67.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Pre-Interest and Activity
# chat msg# forum entries
# created words
Pre-interest.30*
(.043).28
(.053).36*
(.019)
1-tail significant test, n=36
67
![Page 68: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/68.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Activity and Post-Interest
# chat msg# forum entries
# created words
Post-interest
.11(.258)
.19(.135)
.34*(.022)
1-tail significant test, n=36
factors out by pre-interest!
68
![Page 69: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/69.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Activity and Social Relatedness
# chat msg# forum entries
# created words
socialrelatedness
-.08(.319)
.30*(.036)
.31*(.035)
1-tail significant test, n=36
69
![Page 70: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/70.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Pre-Knowledge and Activity
# chat msg# forum entries
# created words
pre-problemsolving
.34*(.021)
.47**(.002)
.54***(<.001)
1-tail significant test, n=36
70
![Page 71: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/71.jpg)
© Prof. Dr. Karsten D. Wolf 2007
Activity and Knowledge
# chat msg# forum entries
# created words
post-problemsolving
.03(.425)
.28*(.048)
.43**(.005)
1-tail significant test, n=36
71
![Page 72: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/72.jpg)
© Karsten D. Wolf 2007
Which means: constructivism is right after all!
72
![Page 73: Online learning processes - what users do and why it is important](https://reader035.vdocument.in/reader035/viewer/2022062513/554bdd52b4c905706a8b5668/html5/thumbnails/73.jpg)
© P
rof.
Dr. K
arst
en D
. Wol
f 200
5
http://www.karsten-d-wolf.de/
73