engaging students: developing and measuring guiding principles for learning
Post on 10-Jan-2016
24 Views
Preview:
DESCRIPTION
TRANSCRIPT
Engaging Students: Developing and Measuring
Guiding Principles for Learning
Chad ComptonPaul FreebairnRonald Miller
Sela Unga
Brigham Young UniversityApril 29, 2013
About BYUH
• Small campus• Diverse student population• Student focused
Challenge
• Improve Quality• Serve More Students• Decrease Costs
Half Day Faculty Retreat
• Small group discussion of principles and best practices of teaching and learning.
• Flip chart paper posted on wall (80 comments)
• 10 dots for all faculty to vote
• Top 20 best practices
Steering Committee
• Purpose: Develop framework of teaching and learning (guiding principles)
• 12 faculty, 4 students• Weekly meetings (read and discuss)
1+ years• Student/faculty interviews and focus
groups
First Things
• Teaching as a craft, art, and science
• Focused on principles of learning
Result: 6 Principles of Learning Framework
1. Motivated by desire to serve2. Active Not Passive3. Self directed and responsible4. Meaningful self reflection and
assessment5. Open to change6. Skilled in learning process
Principles to Practices
• Prepare
• Engage
• Improve
Implementation• General Education • Coordinated with student life• Increased classroom visits (lunch)• Emailed inspirational quotes regularly• Annual faculty and department reports • Presented framework to faculty and students• Implemented a 1-year new faculty orientation• Organized the Center for Teaching and Learning• Held on and off-campus seminars with guest speakers• Created brochures, website resources, and video vignettes• Conducted a student time study• Benchmarked student learning activities, opinions, and
time usage
Institutional Research TeamThe Committee
Administrators, Faculty, IR Staff, Students
IR InternSupervisor
Student Worker
Voluntee
r
Voluntee
r
Student Worker
Voluntee
r
Voluntee
r
Student Worker
Voluntee
r
Voluntee
r
Student Worker (Focus
Groups)
Voluntee
r
Voluntee
r
Framework Survey Development
Institutional Research Team (IRT) given task to
benchmark student learning activities, opinions, and time
usage
Fall 2011: IRT developed potential survey questions to provide insight about student
engagement in the learning process
Framework Survey
• About Yourself – course load, credit load, # of classes requiring preparation beforehand,
hours spent per week on various activities (school & work, exercise & entertainment, and church, service, & other)
• About the Framework– 2 open-ended questions: understanding and application of the Framework
• About Your Learning– Prepare, Engage, and Improve– 5-point scale: never (1), hardly ever (2), sometimes (3), often (4), very
often (5)
• Demographics – Gender, Age, Regional Home Area, Class, GPA, etc.
Framework Survey: About Your Learning
Prepare• Read and study materials before class• Prepare thoughtful questions before class• Understand assigned materials well enough to teach or explain it
Engage• Set personal goals • Actively participate in class • Work with other students outside of class
Improve• Take initiative to learn on your own• Increase the effectiveness of your study skills• Engage in self-assessment and meaningful reflection
Encouraging Students
Teachers giving more busy work
Training us as future leaders
Applying what we learn
Time management skills
Enter to Learn Go Forth to Serve
Spiritual Learning
Tests, Quizes, Classes, Graduating
I don't know or not familiar with it
Improve
Engage
Did not respond appropriately to the question
Prepare
Studying, Secular Learning, Education
1%
2%
2%
3%
4%
6%
6%
12%
16%
18%
21%
23%
23%
30%
In your own words, describe the BYU-Hawaii Framework for Student Learning
The majority of students
described the Framework as
“preparing”, “engaging”, or “improving”
their learning
39% of the respondents did not
know, did not respond to the
question appropriately, or were not familiar
with the Framework
Text analysis taken from Framework Survey
Assign more homework
They are merciful & patient
Review material from previous class period
Teachers are open to different views & perspectives
Require you to teach (i.e. Videos)
Expect concepts to be learned outside of class
Ask for feedback (i.e. Class flexible to student's understanding)
Office hours and tutors (i.e. Availability to students)
Group work & in-class activities
Ask students questions
Allow students to ask questions & help them understand
Encouragement to participate
Teach real life application & lead by example
Syllabi, instruction, assignments, & grades based on the framework
Spiritual activities & application (i.e. Prayer in class)
Class Discussions
Required pre-class preparation
1%
1%
1%
2%
2%
2%
3%
3%
4%
4%
5%
10%
11%
12%
14%
18%
19%
22% of the re-spondents gave
examples of how instructors help them to engage in the learning
process
Please give a few specific examples of how you see the BYU-Hawaii Framework for Student Learning being applied by
instructors in your class
37% of the respondents gave
examples of pre-class preparation or class
discussion when describing how the
Framework was being applied.
Text analysis taken from Framework Survey
Prepare: As a student, how often do you…?
1
2
3
4
5
2.84
3.68
2.713.31
3.713.25
Overall Mean
very often -
often -
sometimes -
hardly ever -
never -
Additional findingssingle married p
prepare thoughtful questions to ask in class 2.78* 3.10* 0.004
thoroughly read and study materials in advance 3.62* 3.95* 0.004
Female Male p
prepare thoughtful questions to ask in class2.74* 2.96* 0.019
Significant effects: p<.05* 1= never; 2= hardly ever; 3= sometimes; 4= often; 5= very often
Engage: As a student, how often do you…?
1
2
3
4
5
3.82
3.363.10
2.71
3.613.32Overall Mean
very often -
often -
sometimes -
hardly ever -
never -
Additional findingsMainland
USA Asia Hawaii PacificOther
International p ŋp2
discuss ideas from your readings or classes with faculty members outside of class 2.54* 2.99** 2.42* 2.98 2.85 0.000 0.046
Freshman Sophomore Junior Senior p ŋp2
discuss ideas from your readings or classes with faculty members outside of class 2.55* 2.49* 2.53* 3.07*** 0.000 0.062
Female Male p
discuss ideas from your readings or classes with faculty members outside of class 2.59* 2.88* 0.004
single married p
discuss ideas from your readings or classes with faculty members outside of class
2.62* 3.07* 0.000
actively participate in class by asking questions, expressing your point of view, and responding to others’ views
3.55* 3.85* 0.011
Significant effects: p<.05* 1= never; 2= hardly ever; 3= sometimes; 4= often; 5= very often
Improve: As a student, how often do you…?
1
2
3
4
5
3.75 3.673.95 3.83 3.75
3.79
Overall Mean
very often -
often -
sometimes -
hardly ever -
never -
Additional findingsMainland
USA Asia Hawaii PacificOther
International p ŋp2
increase the effectiveness of your study skills and habits 3.51* 3.87* 3.67 3.70 3.94* 0.001 0.039
change the way you think or act because of what you’ve learned in class 3.74* 3.84 3.91 3.89 4.21* 0.043 0.022
Freshman Sophomore Junior Senior p ŋp2
seek further opportunities to learn 3.79 3.56* 3.63 3.89* 0.011 0.024
single married p
seek further opportunities to learn 3.70* 3.96* 0.009
increase the effectiveness of your study skills and habits 3.61* 3.90* 0.004
take initiative on your own to learn things that you don’t understand 3.89* 4.16* 0.004
change the way you think or act because of what you’ve learned in class
3.74* 3.84* 0.043
Significant effects: p<.05* 1= never; 2= hardly ever; 3= sometimes; 4= often; 5= very often
About Your Learning: Prepare, Engage, and Improve
discuss ideas from readings with faculty outside of class
prepare thoughtful questions
study assigned materials with others before class
work with other students outside of class
understand materials well enough to teach
test ideas with others in class
actively participate in class
ask for guidance from the spirit
increase the effectiveness of study skills
thoroughly read and study materials before class
seek further opportunities to learn
engage in self-assessment
set personal goals and standards
change the way you think or act
take initiative on your own
0% 10% 20% 30% 40% 50% 60% 70% 80%
21%
22%
23%
35%
42%
46%
53%
56%
58%
60%
62%
63%
65%
67%
75%
Percentage of students responding with often or very often
Prepare Engage
Improve
More students...
• take initiative on their own to learn• thoroughly read and studied
materials before class• set personal goals • engage in self-assessment
About Your Learning: Prepare, Engage, and Improve
take initiative on your own
seek further opportunities to learn
engage in self-assessment
thoroughly read and study materials before class
ask for guidance from the spirit
test ideas with others in class
prepare thoughtful questions
discuss ideas from readings with faculty outside of class
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
2%
2%
6%
6%
7%
9%
11%
12%
13%
16%
18%
28%
34%
42%
44%
Percentage of students responding with hardly ever or never
Prepare Engage
Improve
Fewer students...
• discussed ideas with faculty outside of class
• studied before class with others• prepared thoughtful questions
before class• worked with others outside of class
Exploratory Factor AnalysisQuestions Factor 1 Factor 2 Factor 3
Prepare
prepare thoughtful questions to ask in class 0.085788 0.462264 0.612411
thoroughly read and study materials in advance 0.238854 0.016551 0.707059
study with others to make sure you understand assigned materials 0.154039 0.141104 0.734530
understand assigned materials well enough to teach or explain it 0.079345 0.735859 0.224951
ask for guidance from the Spirit 0.326257 0.106098 0.367008
Engage
set personal goals and standards 0.558461 0.193091 0.230917
test ideas with others in class 0.308393 0.562841 0.335045
work with other students outside of class 0.150854 0.865002 -0.061181
discuss ideas from readings and classes with faculty members 0.213787 0.620976 0.279427
actively participate in class 0.182878 0.203032 0.550451
Improve
seek further opportunities to learn 0.731545 0.173406 0.218000
increase the effectiveness of your study skills and habits 0.733764 0.208398 0.125593
take initiative to learn on your own 0.765110 -0.010432 0.170907
change the way you think or act 0.741580 0.144300 0.142496
engage in self-assessment and meaningful reflection 0.740721 0.145936 0.153575
New Structure
Questions
Prepare
prepare thoughtful questions to ask in class
thoroughly read and study materials in advance
study with others to make sure you understand assigned materials
Engage
understand assigned materials well enough to teach or explain it
work with other students outside of class
discuss ideas from readings and classes with faculty members
Improve
seek further opportunities to learn
increase the effectiveness of your study skills and habits
take initiative to learn on your own
change the way you think or act
engage in self-assessment and meaningful reflection
Validity & Reliability
New Structure
Chi-Square Statistic/p-value 395.637/ p < 0.000
Steiger-Lind RMSEA 0.135
Population Gamma Index 0.873
Joreskog GFI 0.860
Akaike Information Criterion 0.995
Cronbach’s alpha 0.836
Original Structure
Chi-Square Statistic/p-value 750.462/ p < 0.000
Steiger-Lind RMSEA 0.127
Population Gamma Index 0.838
Joreskog GFI 0.820
Akaike Information Criterion 1.838
Cronbach’s alpha 0.866
Time Usage Study
Winter 2012: The weekly schedule and
daily time usage survey were developed
Selected random groups of students to participate in the
Time Study during winter, summer, and fall 2012
Students first completed the Learning Framework
Survey before tracking their time.
1 Week at a GlanceLearning Framework Survey
(predicted from random samples of the entire student population- n=382)
Class Prepare StudyCampus
ActivitiesWork
Exercise and Entertainment
Church and
Service
Eat, Clean, Sleep
Total Hours
Mean 13.50 hrs.
6.99 hrs.
12.18 hrs.
3.08 hrs.
14.19hrs.
29.07 hrs.
7.40 hrs.
44.37 hrs.
130.78hrs.
Median 14 5 10 2 19 24 6 50
Mode 15 10 10 2 19 24 6 68
1 Week at a GlanceLearning Framework Survey
(predicted from time study participants- n=105)
Class Prepare StudyCampus Activitie
sWork
Exercise and Entertainment
Church and
Service
Eat, Clean, Sleep
Total
Mean14.04 hrs.
6.83 hrs.
11.56 hrs.
3.19 hrs.
14.05 hrs.
31.29 hrs.
8.20 hrs.
53.62 hrs.
142.78 hrs.
Median 14 4 8.5 2 19 23 7 58
Mode 14 4 15 2 19 21 6 62
Time Study(actual time recorded in student daily time logs- n=105)
Mean12.25 hrs.
5.64 hrs.
16.09 hrs.
3.16 hrs.
13.36 hrs.
42.60 hrs.
7.94 hrs.
60.47 hrs.
161.51 hrs.
Median 12 4 15.5 0.25 16 38 6 64.5
Mode 12 0 17 0 0 47 5 67
Time Study
Campus Activities
Prepare
Church and Service
Class
Work
Study
Exercise and Entertainment
Eat, Clean, Sleep
0 10 20 30 40 50 60 70
3.2
5.7
7.9
9.8
13.4
16.1
42.6
60.5
3.2
6.8
8.2
14.0
14.1
11.6
33.8
53.6
PredictedActual
Hours spent per week
Time Study Key Findings
• underestimated time spent studying outside of class
• underestimated time spent on exercise and entertainment-related activities
We're in it for the long-term
• Creating new culture of learning takes time, resources, leadership, and buy-in
• No "quick fix" available
• Patience, persistence, and vision needed
• What's next?
Please Contact Us
• Chad Compton, Associate VP for Instruction chad.compton@byuh.edu
• Paul Freebairn, Dir. Assessment & Testing freebaip@byuh.edu
• Ronald Miller, Associate Professor of Psychology ronald.miller@byuh.edu
• Sela Unga is the IR Planning Analyst Intern unga2@go.byuh.edu
top related