the effect of using mobile classroom response system on students’ engagement and performance
TRANSCRIPT
The Effects of Using Mobile
Classroom Response System
(MCRS) on Students’
Engagement and
Performance
Khaled Hamdan, PhD Instructor Kwantlen Polytechnic University
Outline
Background: Classroom Response System and Mobile
Classroom Response System
Goals & Objectives
Methods
Results and Discussion
Outline
Background: Classroom Response System and Mobile
Classroom Response System
Goals & Objectives
Methods
Results and Discussion
Classroom Response System
CRS a set of hardware
and software that
facilitates teaching
activities.
A classroom response system
(CRS)= a personal response
system, a student response system,
an audience response system, or
simpley a clicker.
Classroom Response System
Instructor poses a
multiple-choice question
to students.
Each student submits an
answer to the question
using a handheld
transmitter.
“on the fly” response bar
chart
htt
ps:
//w
ww
.um
b.e
du
Background literature
CRSs are most effective in large classes as a stimulus to
greater student involvement and interaction.
CRSs promote learning in all kinds of classes: small
and large, graduate and undergraduate, lecture and lab.
Milner-Bolotin, Antimirova, & Petrov (2010)
Sevian & Robinson (2011).
Some studies suggest that use of a classroom response
system improves learning outcomes.
Camey, Gray, & Gray 2008
Mobile Device Penetration in
Canada
The
Canadia
n R
adio
-tele
visi
on a
nd T
ele
com
munic
ations
Com
mis
sion, 2015.
Mobile Device Penetration in BC
by Age
htt
p:/
/ww
w.6
smark
eting.c
om
/blo
g/c
anadia
n-s
mart
phone-u
se-s
tatist
ics/
Mobile Classroom Response
System (MCRS)
MCRS=
student’s
mobile device
is the
classroom
response
system.
htt
p:/
/blo
g.c
engage.c
om
/tech
nolo
gy-
class
room
-dis
tract
ion-a
sset/
Outline
Background: Classroom Response System and Mobile
Classroom Response System
Goals & Objectives
Methods
Results and Discussion
The Issue
Previous research is in agreement that CRS
enhances the engagement of students.
Little research has addressed the effects of MCRS
on student engagement and performance.
Contradictory findings have been reported on the
influence of a CRS on student performance.
Goals and Objectives
to understand students’ perceptions on using
MCRS in classroom instruction.
to provide insights on the effects of MCRS usage on
students’ engagement and performance.
Explore the associations between gender/age and
perceptions on the effect of MCRS on class
engagement.
Outline
Background: Classroom Response System and Mobile
Classroom Response System
Goals & Objectives
Methods
Results and Discussion
Methods
Two class sections
Sections 1: control section, no use of a MCRS
(GEOG 1120 Earth Science Fall 2015 term). N=23.
Section 2: MCRS used throughout the entire
semester (GEOG 1120 Earth Science Winter 2016
term). N=22
Gender Distribution (%) in each of the
Study Class Sections
10
15
20
25
30
35
40
45
50
55
60
Control MCRS
Male Female
50% 50%
54%
46%
Age Distribution (%) in each of the
Study Class Sections
0
10
20
30
40
50
60
Under 20 yr-old 20-25 yr-old 25-30 yr-old
Control MCRS
5% 5%
50% 50%
45% 45%
Methods
Both sections were taught by the same instructor.
Same materials (lecture notes, assignments, and
exams) were used for both sections.
Both sections had attendance and participation
worth 5% of the course final grade.
Methods
Students (Winter 2016 term) were surveyed on
their perceptions of how using MCRS in
classroom instruction influenced their learning
experience.
Qualitative Data Analysis
Association between categorical variables:
Gender and perceptions on the effect of MCRS on class
engagement.
Age and perceptions on the effect of MCRS on class
engagement.
Conditional Percent tables and Chi-square
analysis, C.I. 95%
Quantitative Data: Control and
MCRS sections
Attendance
Midterm exam scores
Final exam scores
Paired Student t-test, C.I. 95%
Outline
Background: Classroom Response System and Mobile
Classroom Response System
Goals & Objectives
Methods
Results and Discussion
Results: Attendance
0
10
20
30
40
50
60
70
80
90
100
Attendence throughout the term Attendence in the last 2 weeks
of classes
Control MCRS
70%
(P 0.047) (P 0.371)
(a) (a)
(b) (b)75%
90% 86% The use of MCRS
improved attendance
by ~15%.
Student t-test
revealed that
attendance is
significantly
improved.
Results: Attendance
0
10
20
30
40
50
60
70
80
90
100
Attendence throughout the term Attendence in the last 2 weeks
of classes
Control MCRS
The use on MCRS Improved attendance by ~15%.
(P 0.047) (P 0.037)
(a) (a)
(b) (b)
Caused students to attend the class
more regularly?
Strongly Disagree Disagree Neither Agree Agree Strongly Agree
Nor Disagree
Most students agreed or strongly agreed
(84%) that the use of MCRS has caused
them to attend the class more regularly.
Narrative feedback: Attendance
It is a good method of increasing interest in course and
attending lectures regularly.
Results: participation (control)
Control: Despite my best effort, most classroom
activities did not result in 100% student
participation.
Results: participation (control)
Control: Despite my best effort, most classroom
activities do not result in 100% student
participation.
Typically, when I ask a question in class, only a
few students are called on to respond. Even if 25%
raise their hands, only 5% may get to share their
responses.
Results: participation (MCRS)
MCRS: ~99%
participation
With a MCRS, almost
every student responded to
every question. MCRS
clearly increased student
engagement.
htt
ps:
//w
ww
.bost
onglo
be.c
om
Made students more willing to
participate in class discussions?
Strongly Disagree Disagree Neither Agree Agree Strongly Agree
Nor Disagree
Most students agreed or strongly agreed (89%)
that the use of MCRS has them more willing
to participate in class discussions.
Maybe the two students who usually answer
all questions even with no MCRS?
Narrative feedback: participation
Students get deserving participation marks.
Makes the class participation engaging!
Consequences of improved
participation rate Strongly Disagree Disagree Neither Agree Agree Strongly Agree
Nor Disagree
74%
79%
79%
92%
Made the class more engaging?
Strongly Disagree Disagree Neither Agree Agree Strongly Agree
Nor Disagree
Most students agreed or strongly agreed (90%)
that the use of MCRS has made the class more
engaging.
Enhanced the learning experience?
Strongly Disagree Disagree Neither Agree Agree Strongly Agree
Nor Disagree
Most students agreed or strongly agreed (83%)
that the use of MCRS has enhanced their
learning experience.
Narrative feedback: engagement
I think it was a great learning experience made class
lectures really interesting…
Narrative feedback: engagement
…It helps students to learn topics in depth and improves
their own understanding.
Narrative feedback: engagement
I think it was a cool way to keep us engaged.
…. as I listen very much carefully to professor in order to
answer those questions….
Results: Performance
0
10
20
30
40
50
60
70
80
Control MCRS Control MCRS
Midterm Final Exam
61
P 0.371P 0.55
68
(a)
(a)(a)
Students’
performance
improved in the
midterm exam.
Student t-test
revealed that the
performance
improvement is not
statistically
significant.
70
56
(a)
Results: Performance
0
10
20
30
40
50
60
70
80
Control MCRS Control MCRS
Midterm Final Exam
61
P 0.371 P 0.55
68(a)
(a)(a)
There’s a trend of
improved students’
performance in
both the midterm
and final exam.
Student t-test
revealed that the
performance
improvement is
not statistically
significant.
70
56(a)
Results: Association between categorical variables-
Gender & perceptions on class engagement
68(a)
(a)(a)
Two categorical
variables are related in
the sample if at least
two rows noticeably
differ in the pattern of
row percent.
Chi-square: P=0.037
(a)
Strongly
Agree/Agree
Neither Agree
Nor Disagree Total
Male 10 2 12
Female 7 0 7
Total 19
Strongly
Agree/Agree
Neither Agree
Nor Disagree Row
%
Male 83% 17% 100%
Female 100% 0 100%
Row percent
Data
Results: Association between categorical variables-
Age & perceptions on class engagement
68(a)
(a)(a)
Age is related to the
perceptions on class
engagement because at
the rows are noticeably
differ in the pattern of
row percent.
Chi-square: P=0.021
(a)
Strongly
Agree/Agree
Neither Agree
Nor Disagree Total
<20 yrs 10 0 10
20-25 yrs 6 2 8
>25 yrs 1 0 1
Total 19
Strongly
Agree/Agree
Neither Agree
Nor Disagree Row
%
<20 yrs 100 0 100
20-25 yrs 75 25 100
>25 yrs 0 100 100
Row percent
Conclusions
Using MCRS significantly improved attendance and participation.
There’s a trend of improved performance, however, t-test
revealed it is not statistically significant.
Class is more engaging and overall a better learning experience.
Results may differ depending on class demographics as females
(52% of BC university/college students) and students < 20 year-
old (28% of BC university/college students) seem to have strong
positive opinion about how the MCRS affected class
engagement.
htt
p:/
/ww
w.a
ved.g
ov.
bc.
ca/d
ata
ware
house
/
Limitations and Opportunities
Sample size N=22
An opportunity for Collaboration