sang chan weber state university jane strickland idaho state university
DESCRIPTION
Examining the Multimedia Redundancy Effect among Education Majors: Assessing Pedagogical Usability of Instructional Videos. Sang Chan Weber State University Jane Strickland Idaho State University. AACE, New Orleans, LA October 29, 2014. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
Examining the Multimedia Redundancy Effect among Education Majors:
Assessing Pedagogical Usability of Instructional Videos
Sang Chan Weber State University
Jane StricklandIdaho State University
AACE,
New Orleans, LA
October 29, 20141
•Many studies (e.g., Craig, Gholson, & Driscoll, 2002; Kalyuga,
Chandler, & Sweller, 1999, 2000; Mayer, Heiser, & Lonn, 2001, etc.) compared redundant groups (graphics, text, and narration) with non-redundant groups (graphics and narration).
• Results: Non-redundant group significantly outperformed the redundant group.
2
Introduction
•Used a short treatment time
•Measured only short-term learning
• Rarely used a topic of a formal course
• Rarely discussed pedagogical usability
3
Introduction
• To further examine the redundancy effect by comparing posttest scores, delayed posttest scores, and usability perceptions of education majors enrolled in an online technology class.
• Redundant (R): Narration, graphics, & text• Non-Redundant (NR): Narration & graphics
4
Purpose
1. Is there a significant difference in posttest scores in learning Microsoft Access 2013 between R and NR?
2. Is there a significant difference in delayed posttest scores?
3. Is there a significant difference in pedagogical usability perceptions?
5
Research Questions
• Cognitive load theory
6
Theoretical Framework
• The study used convenience sampling.
• 42 students participated in the study.
• Female, traditional students (majority)
• No students with disabilities
• Native English speakers7
Participants
• T1: Narration, graphics
• T2: Narration, graphics, and text
Group Treatment Survey Posttest Delayed Posttest
T1 X1 O2 O6 O10
T2 X2 O4 O8 O12
8
Research Design
•Designed and developed the videos
•Developed a posttest and delayed posttest
•Adapted the pedagogical usability survey
• Piloted with a small group of four
9
Materials
•Obtained the informed consents•Assigned students randomly •Administered treatments
• Completed surveys (αR = .83, αNR = .88)
• Took posttest • Took delayed posttest (6 weeks later)
10
Procedure
• Little’s MCAR, χ2 (252) = 263.08, p = .303• Test scores• Violated normality & equal variances for
the independent-t test• Outliers in test scores
• Survey responses: • Met the independent-t test assumptions
11
Data Screening
• Posttest and delayed posttest • Mann-Whitney U (adjusted α= .025)• Survey• Independent t-test (α = .05)
Note: We found similar results for parametric and non-parametric tests with and without outliers.
12
Data Analysis
•No significant difference between the two
treatments on posttest scores, U = 164.50,
p = .489
• Small estimated effect size, r = Z/ = .11
13
Result: 1
•No significant difference between the two
treatments on delayed posttest scores, U
= 118, p = .538
• Small estimated effect size, r =.11
14
Result: 2
•No significant difference between the two
treatments on survey responses, t(40)
= .60, p = .549
• Small estimated effect size, d = .18
15
Result: 3
•No significant differences in posttest and delayed posttest•Not consistent with previous studies
(i.e., Craig, Gholson, & Driscoll, 2002; Mayer, Heiser, & Lonn, 2001)• Consistent with McNeill, Doolittle,
and Hicks (2009) and Wu (2011)
16
Discussions
•Useful text: Short and easy-to-read with slow narration• “The text was so helpful in understanding the
content. I have never used access [sic] before & [sic] I think it can be helpful.” • “. . . The visuals were great and i appreciate
having the subtitles to keep me on track.” • “Seeing you perform the task but also having the
words on the bottom helped reinforce the idea.” 17
Discussions
• Read text to reinforce narration.• Eight words, on average, per line• Processed the elements in different orders.• Anecdotally, text was useful.
18
Posttest Delayed Posttest
M Mnd M Mnd
Redundant 80.44 87.50 71.20 74
Non-redundant 80.33 84 60.22 67.50
Discussions
• Insignificant results on posttest and delayed posttest may result from:• Small sample size • Low power•High data variability• Low motivation• Prior preparations before tests
19
Discussions
•Useful aspects of the videos:• Review what was learned: “… Reviewing
after each topic helped reinforce the concepts.” • Clear content and logical sequence: “It is
useful that the objectives built on one another. I also thought it was helpful to see what the end product would look like.” • Multimedia elements: “I really liked the
reading at the bottom of screen so I could follow along with the speaker.”
20
Discussions
• Present the text slightly faster (1 or 2 seconds) than narration (120 words/minute) •Keep the text available on the screen with
graphics although the narration was silent•Keep the text short (8 words) and easy to
read (large font)
21
Implications
• Investigate such use of text • Explore ways to use text as a supporting
element •Use a large sample size•Use a more reliable instrument to reduce
data variability • Consider motivation: Low motivation
may result in high data variability. 22
Recommendations
Thank You!
Q & A
23
• Primary analysis: 6 subscales (75.63%)• Low reliability for some subscales• Low correlations among items• Conceptual overlapping•Unstable factor• Revised survey suggested 5 subscales.
24
Revising Survey
Factor Loading Communality1 2 3 4 5
Q20: The time allotted for completing this tutorial was adequate for me. .79 1
Q8: The difficulty level in this tutorial was appropriate for me. .75 1
Q1: Dividing this content into sections assisted me in learning the concepts. .65 1
Q2: The objectives informed me of what was expected for me to learn. .65 1
Q3: The overview of what I would be learning in this module assisted me. .56 1
Q16: The sequencing (order) of the materials helped me to learn this content.
.53 1
25
Exploratory Factor Analysis
Factor Loading Communality1 2 3 4 5
Q24: I understand how to create a Query after completing this tutorial.
.96 1
Q23: I understand how to create a Table after completing this tutorial.
.94 1
Q22: I will be able to create a database after completing this tutorial.
.88 1
Q17: I learned this material more quickly using this computer-based tutorial than I would in a traditional (face-to-face) format.
.95 1
Q12: It was more useful for me to learn the content in this format than in an instructor-led, traditional classroom.
.78 1
Q21: The tutorial was an adequate alternative to having an instructor lead me in learning this material
.74 126
Exploratory Factor Analysis
Factor Loading Communality1 2 3 4 5
Q13: I was interested in the topic of this tutorial.
.81 1
Q18: I felt motivated to learn the content from this tutorial.
.71 1
Q19: I would like to learn other technologies using this type of computer-based tutorial.
.65 1
Q7: This tutorial was designed for my needs as a future educator.
.65 1
Q9: The images in this tutorial helped me to learn the content.
.87 1
Q10: The audio narration in this tutorial helped me to learn the content.
.86 1
Q11: The animations in this tutorial helped me to learn the content.
.49 1
27
Exploratory Factor Analysis
• Factor 1: Design Considerations (α = .84)• Factor 2: Learning Confidence (α = .96)• Factor 3: Learning Format (α = .86)• Factor 4: Learning Motivation (α = .87)• Factor 5: Multimedia Elements (α = .78)•Overall reliability α = .89• Total variance = 75%.
28
Revised Survey