Improving lecture capture through usability testing and research
Ilkka Kukkonen
@ilkkakukkonen #mlconf13
Media & Learning 2013, Brussels 12.11.2013
Motivation
• Create the best possible learning environment for our students and clients
• Improve overall quality, commitment and effects with optimal resource allocation
• Support personal learning paths and flexibility in participation
• Improve the pedagogical models and spesifically the lecture format
• Create multiaccess courses
Project approach to support the development(”MOVIE”, 2011-2012) N~800• The development of pedagogical quality and effectiveness
of lecture capture– How the students are using the recorded lectures to
support their studies?– How can we measure of the impact in various contexts?– How the pedagogical quality of the lecture capture can
be improved?• Data from teacher training and high school students,
collected in mathematical subjects and statistics
A B C D
57%49% 50%
64%
A. Possibility for independent studies
B. Retention and repetition of classes
C. Illuminated unclear themes from lectures
D. Helped to prepare for final exams
Encouraging results (N=113) *
• 81% would like to increase the use of lecture capture in their studies• 85% would recommend lecture capture for their peers
* www.aducate.fi/movie
Findings from the study
• Students seem willing to increase the use of lecture capture in their studies
• Use of LC is seen as democratic method for participation
• It takes time to develop personal study strategies and build trust for LC
• Impact on lecture attendance seems to depend on the personal study strategies and previous experiences
• Viewing a lecture capture can feel more personal pedagogical experience than attending live lecture
* www.aducate.fi/movie
Findings from the study
• Lecture capture seems to be more efficient for lower achieving students– Supporting conceptual understanding for content,
metacognition and efficient study strategies• Viewing volumes seem to have correlation to
experienced value and grades• Challenge is to create meaningful learning contexts for
students– Include lecture capture in learning design– Summarize, contextualize and engage interaction
* www.aducate.fi/movie
Usability study (3-6/2012, N=16)
• Need to create realistic insight about LC use in genuine context
• Are there some problems related to the usability and content?
• How does the interface and features support viewing?• How students find the essential parts of the lecture?• 4 short lectures with different statistical assignments• Familiarity of the content• Pair work vs. individual performance
● Research setup records students’ eye movement, video and audio and captures also mouse clicks (Tobii Eye Tracker)
● Two similar and syncronized workstations for pair work● Downside is research setup is quite complex and
annotation and analyzing the final data is time consuming● Minor inaccuracies may occur within the people with
glasses● Supplemental interviews and survey
Eye-tracking for lecture capture
% o
f u
sers
use
d a
t le
ast
on
ceUsed features
Scenes
Timeline
Forward / ReversePersonal bookmarksTeacher’s comments
Tim
e i
n m
inu
tes
Standarddeviation
Mean
Time with assignments
Recording 1
Recording 2
Recording 3
Recording 4
* ”Flip”
Question 3
Question 1
Question 2
Recording 1 Recording 2 Recording 3 Recording 4
% a
nsw
ers
corr
ect
Performance
* ”Flip”
Results from the video annotation (N=16)
• Teacher’s comments and remarks seem to support the viewing experience (scaffolding)
• Familiarity of the content and student’s personal notes and bookmarks speeded up completing the assignments
• Writing lecture notes while playing the video in the background gave lower scores in individual level
• Learning by copying the instructor, ”vicarious learning” (Bandura) seems to lead into surface level learning, which generated lower scores individual level and pair work
• Students who spent time with making sense of the subject got better grades than expected
● Students formed pairs independently based on their social relationships
● Pairs typically shared the tasks for viewing and making notes● ”Weaker” party asks and questions the answers, so the
talented one needs to answer with thought and provide a convincing explanation about the potential answer
● In this setting stronger one seems to develop the weaker student’s grades
● Also in the final exam the weaker students outperformed the expected grades, so working in pairs with lecture capture might have developed their skills and motivation
Working in pairs with lecture capture
www.aducate.fiwww.uef.fi
Thanks!
Talk more? Collaborate?
http://aducate.fi/[email protected] @ilkkakukkonen