![Page 1: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/1.jpg)
WCSE 2015
Faculty & Students as collaborators, co-creators and makers
[email protected]@simonpbatesbit.ly/batestalks
![Page 2: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/2.jpg)
Photo: Wikipedia
![Page 3: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/3.jpg)
Edwin Thompson Jaynes
1922-1998
Photo: Wikipedia
![Page 4: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/4.jpg)
Jaynes, E. T., 1993, `A Backward Look to the Future, ' in Physics and Probability, W. T. Grandy, Jr. and P. W. Milonni, Cambridge Univ. Press,
![Page 5: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/5.jpg)
“But it required a few years before I perceived what a science teacher's job really is.
The goal should be, not to implant in the student's mind every fact that the teacher knows now; but rather to implant a way of thinking that will enable the student, in the future, to learn in one year what the teacher learned in two years.
Only in that way can we continue to advance from one generation to the next.”
Jaynes, E. T., 1993, `A Backward Look to the Future, ' in Physics and Probability, W. T. Grandy, Jr. and P. W. Milonni, Cambridge Univ. Press,
![Page 6: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/6.jpg)
Overview
How things are changing; technology as an example driver
Implications for instructors; the anatomy for future success
The case for students as co-creators, and four examples of how to enact that
![Page 7: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/7.jpg)
CC BY-NC-SA 2.0 https://flic.kr/p/2ZdABF
![Page 8: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/8.jpg)
Technology - scale and pace
Slide credit: Eric Grimson (MIT)
![Page 9: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/9.jpg)
Technology - scale and pace
Slide credit: Eric Grimson (MIT)
![Page 10: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/10.jpg)
Technology - reach and unbundling
![Page 11: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/11.jpg)
Technology - reach and unbundling
![Page 12: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/12.jpg)
Technology - disruption
Graph extracted from http://vikparuchuri.com/blog/on-the-automated-scoring-of-essays/
![Page 13: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/13.jpg)
Technology - implications
Changing the
what, where, when, how, from whom and with whom
of many aspects of life, …and learning is included
![Page 14: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/14.jpg)
Technology - implications
So what are the
we need to embrace, develop and refine?
skills, values and habits
![Page 15: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/15.jpg)
![Page 16: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/16.jpg)
![Page 17: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/17.jpg)
![Page 18: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/18.jpg)
Students as collaborators, co-creators
Four examples (of increasing complexity to implement)
1. “Things I wish I’d known”
2. The most important course you can take
3. Undergraduate learning assistants in the classroom
4. Student-generated assessment and content
![Page 19: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/19.jpg)
“Things I wish I’d known”
![Page 20: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/20.jpg)
“Things I wish I’d known”
![Page 21: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/21.jpg)
The most important course you will take
![Page 22: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/22.jpg)
The most important course you will take
![Page 23: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/23.jpg)
The most important course you will take
![Page 24: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/24.jpg)
Undergraduate learning assistants in the classroom
![Page 25: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/25.jpg)
Undergraduate learning assistants in the classroom
S. Pollock, 2007 PERC Proc. 951, p.172
![Page 26: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/26.jpg)
Undergraduate learning assistants in the classroom
S. Pollock, 2007 PERC Proc. 951, p.172
![Page 27: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/27.jpg)
Undergraduate learning assistants in the classroom
S. Pollock, 2007 PERC Proc. 951, p.172
![Page 30: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/30.jpg)
Selected results and analysisEngagement - how do students use the system?
Benefits - what is the impact on learning?
Question quality - how good is what students produce?
Relevant publications:
Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4), 045002 (2014)
Student-Generated Content: Enhancing learning through sharing multiple-choice questions. International Journal of Science Education, 1-15 (2014).
Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014) 10, 020105
Student-generated assessment - Education in Chemistry (2013) 13 1
![Page 31: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/31.jpg)
Quality of student authored content
Bloom’s Taxonomy of levels in the cognitive domain
Score Level Description
1 Remember Factual knowledge, trivial plugging in of numbers
2 Understand Basic understanding of content
3 Apply Implement, calculate / determine. Typically one-stage problem
4 Analyze Typical multi-step problem; requires identification of strategy
Evaluate Compare &assess various option possibilities; often conceptual
Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.
Text
![Page 32: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/32.jpg)
Question quality
0%
5%
10%
15%
20%25%
30%
35%
40%
45%
50%
1 2 3 4 5 6
Taxonomic Category
Per
cent
age
of S
ubm
itte
d Q
uest
ions
First semester N = 350
Second semester N = 252
![Page 33: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/33.jpg)
Explanation quality
0 Missing No explanation provided or explanation incoherent/irrelevant
1 Inadequate Wrong reasoning and/or answer; trivial or flippant
2 MinimalCorrect answer but with insufficient explanation/justification/ Some aspects may be unclear/incorrect/confused.
3 Good Clear and detailed exposition of correct method & answer.
4 ExcellentThorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution
![Page 34: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/34.jpg)
0 1 2 3 4 0 1 2 3 40
20
40
60
Num
ber o
f que
stio
ns
Assessment 1 Assessment 2
Explanation Quality
![Page 35: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/35.jpg)
Question quality summary (UoE 2011)
2 successive years of the same course (N=150, 350)
‘High quality’ questions: 78%, 79%
Over 90% (most likely) correct, and majority of those wrong were identified by students.
69% (2010) and 55% (2011) rated 3 or 4 for explanations
Only 2% (2010) and 4% (2011) rated 1/ 6 for taxonomic level.
![Page 36: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/36.jpg)
That’s not commonBottomley & Denny Biochem and Mol Biol Educ. 39(5) 352-361 (2011)
107 Year 2 biochem students 56 / 35 / 9 % of questions in lowest 3 levels.
Momsen et al CBE-Life Sci Educ 9, 436-440 (2010)
“9,713 assessment items submitted by 50 instructors in the United States reported that 93% of the questions asked on examinations in introductory biology courses were at the lowest two levels of the revised Bloom’s taxonomy”
![Page 37: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/37.jpg)
Students as producers of assessment content for learning
Why not short answer Qs?
Why not …. anything?
![Page 38: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/38.jpg)
Students as producers of assessment content for learning
Why not short answer Qs?
Why not …. anything?
![Page 39: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/39.jpg)
Students as producers of assessment content for learning
Why not short answer Qs?
Why not …. anything?
Test Kitchen: Adaptive Comparative
Judgement
![Page 40: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/40.jpg)
Students as producers of assessment content for learning
Why not short answer Qs?
Why not …. anything? LEARNING OBJECTS
Test Kitchen: Adaptive Comparative
Judgement
![Page 41: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/41.jpg)
PHYS101: Energy and Waves
![Page 42: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/42.jpg)
PHYS101: Energy and Waves
![Page 43: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/43.jpg)
PHYS101: Energy and Waves
![Page 44: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/44.jpg)
PHYS101: Energy and Waves
![Page 45: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/45.jpg)
PHYS101: Energy and Waves
![Page 46: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/46.jpg)
Implementation logistics Cohort split into 4 groups
Each week one group tasked with creating LOs
Each submission counts for 2.5% of final grade
Repeat cycle twice per Semester
Students can submit >2 LOs & receive grade for best 2
Short survey on submission
Students encouraged to apply CC licenses
![Page 47: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/47.jpg)
Results: engagement0 100 200 300
LO 1
LO 2
LO 3
LO 4
LO 5
LO 6
LO 7
LO 8
Number of students
AssignedOptional
![Page 48: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/48.jpg)
Results: time on task
0 100 200 300 400
Less than 0.5h
0.5 to 1 h
1 to 2h
2 to 3h
3 to 4h
4 to 5h
More than 5h
Number of students
![Page 49: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/49.jpg)
Results: self-reported change in understanding
0 200 400 600 800
None
Little
Moderate
Good
Excellent
Number of students
0200400600800
Number of students
before creating it after creating it How much did you understand the topic your LO was based on
![Page 51: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/51.jpg)
Sample 2 - Standing Wave in a bowl
![Page 52: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/52.jpg)
Sample 2 - Standing Wave in a bowl
![Page 53: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/53.jpg)
Sample 3 - Colour Loss Underwater
![Page 54: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/54.jpg)
Student generated exam content
before creating it after creating it How much did you understand the topic your LO was based on
![Page 56: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/56.jpg)
Not quite the whole story• Despite these outstanding examples, many students
didn’t like the assessment
• difficulty level vs other assessed components of the course
• credit weighting
• Students dropped these assessments more than other coursework
• Strange ‘phase transition’ for LO vs exam grades
![Page 57: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/57.jpg)
![Page 58: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/58.jpg)
![Page 59: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/59.jpg)
One final thought….why this all works
![Page 60: Faculty and students; as collaborators, coproducers and makers](https://reader031.vdocument.in/reader031/viewer/2022030311/58ee88811a28ab20028b45d5/html5/thumbnails/60.jpg)