measuring and assessing the value of student … · 5/7/2013 · source: j. waldman, measuring the...
TRANSCRIPT
![Page 1: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/1.jpg)
STRATEGIC ENROLMENT MANAGEMENT SUMMIT 2013
CALGARY, ALBERTA
TUESDAY, MAY 7, 2013
MEASURING AND ASSESSING THE VALUE OF STUDENT ENGAGEMENT
Informing the Future of Higher Education
Richard Wiggers, Executive Director, Research & Programs & Valerie Lopes, Research Consultant
![Page 2: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/2.jpg)
WHAT IS HEQCO?
![Page 3: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/3.jpg)
Higher Education Quality Council of Ontario
• HEQCO’s creation was recommended in the Rae Review (2005)
• It is an independent agency with a mandate to conduct research and give policy advice to the Minister of Training, Colleges and Universities (MTCU) on all aspects of PSE
• The primary focus is to employ evidence to influence policy
3
Informing the Future of Higher Education
![Page 4: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/4.jpg)
HEQCO has completed or launched over 250 research projects
4
Informing the Future of Higher Education
Published Completed Underway Total
Projects
1. Accessibility/Participation 32 12 16 60
2. Accountability/System Design 18 2 2 22
3. Learning Quality 34 28 68 130
Internal Research (Various) 32 2 6 40
Manuscripts 3 0 0 3
Opportunities to Innovate Fund (OIF) 1 1 8 10
Total 120 45 100 265
![Page 5: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/5.jpg)
Many current projects focus on teaching, learning and student outcomes
Student Services Large Classes Technology-
Assisted Learning
Professional Development
Graduate Studies
NSSE Collegiate Learning
Assessment
AHELO Tuning
Work Integrated Learning
Informing the Future of Higher Education
5
![Page 6: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/6.jpg)
Course–Based Interventions
Student Services
Teaching
Student success is impacted by many different factors
6
Informing the Future of Higher Education
STUDENT SUCCESS
![Page 7: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/7.jpg)
More recently HEQCO developed and released a national “how to” Guide
Informing the Future of Higher Education
7
• Based on a similar document developed by McMaster University
• Endorsed by the Society for Teaching and Learning in Higher Education (STLHE) and the Canadian Association of College and University Student Services (CACUSS)
• September 18, 2012 release
Researching Teaching and Student Outcomes in
Postsecondary Education:
A Guide
Susan Elgie
With:
Ruth Childs
Nancy E. Fenton
Betty Ann Levy
Valerie Lopes
Karen Szala-Meneok
Richard Dominic Wiggers
![Page 8: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/8.jpg)
STUDENT ENGAGEMENT
![Page 9: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/9.jpg)
University and non-university attainment for main OECD countries, percentage of population aged 25-64, 2006
23 24
0
5
10
15
20
25
30
35
Tertiary-type B education (Non-university) Tertiary-type A education (University)
Canada already leads the OECD in PSE attainment
Informing the Future of Higher Education
9
![Page 10: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/10.jpg)
Ontario is achieving the goal of “mass education” in PSE
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
25-64 55-64 45-54 35-44 25-34
Hig
he
st e
du
cati
on
al a
ttai
nm
ent
Trades cert. & registered apprenticeship Total College or CEGEP Total University
67.5
61.4
Ontario Goal
Informing the Future of Higher Education
10
![Page 11: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/11.jpg)
The postsecondary student population is now much more varied
Dee
p
Str
ateg
ic
Su
rfac
e
En
titl
emen
t
%
sample Tentative Label
Group 1 2.62 2.67 3.49 3.15 7.4% Memorizer with expectations
Group 2 2.85 2.71 2.99 2.30 15.1% Just puttin’ in time
Group 3 2.90 3.50 2.80 2.80 14.5% Strategist
Group 4 3.22 3.28 2.41 1.88 14.5% Relaxed student
Group 5 3.43 3.03 3.05 2.86 12.4% Non-strategic worker bee
Group 6 3.49 3.52 3.31 3.65 6.9% Driven to succeed
Group 7 3.70 3.49 2.58 2.50 17.3% Inquiring student
Group 8 4.07 3.86 1.99 2.06 11.8% Student scholar
Source: Schoner, V., Andrey, J., Jayasundera, R., Joakim, E. & Hambly, D.
Students’ Learning Syles and Sense of Entitlement (2011).
Informing the Future of Higher Education
11
![Page 12: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/12.jpg)
Engaged students are curious and have a passion for learning
12
Informing the Future of Higher Education
Source: metapreneurship.net/wp-content/uploads/2012/02/CollegeProfessorsMeme.jpg
I found this course fascinating and I’m thrilled I had the opportunity to take it.
I would love to have had this class all year and I would recommend it to anyone!.
This has been one of the best classes I’ve taken in four years.
![Page 13: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/13.jpg)
Disengaged students don’t appreciate high academic standards
13
Informing the Future of Higher Education
Source: metapreneurship.net/wp-content/uploads/2012/02/CollegeProfessorsMeme.jpg
Really needs a life!
Out to fail you to compensate for whatever he’s lacking.
He is literally ruining my university career!
He is a total ASS.
![Page 14: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/14.jpg)
Millennials have a different view of the value of education
Informing the Future of Higher Education
14
Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of
Today’s College Student (2012).
1969 1976 1993 2009
The chief benefit is that it increases one’s earning power --- 44 57 67
Essential goal: to get training and skills for an occupation 59 67 70 73
1969 1976 1993 2009
Essential goal: to formulate life values and goals for my life 71 62 52 50
Essential to learn to get along with people 76 66 47 38
![Page 15: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/15.jpg)
One HEQCO study looked at the gap between expectations and reality
Source: M. Mancuso et al., Disappointment, Misunderstanding and Expectations: A Gap
Analysis of NSSE, BCSSE and FSSE (HEQCO: 2010).
-30 -20 -10 0 10 20 30 40 50
Gains in General Education
Gains in Practical Skills
Student Success Support
Diversity
Information Technology
Course Interaction
Out-of-Class Interaction
Collaborative Learning
Active Learning
Higher-Order Thinking Skills
Writing
Course Challenge
Disappointment GapBCSSE-NSSE
Misunderstanding GapNSSE-FSSE
Informing the Future of Higher Education
15
![Page 16: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/16.jpg)
16
Informing the Future of Higher Education
Helicopter parents are raising more “bubble wrap kids”
![Page 17: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/17.jpg)
Source: R. Finnie, S. Childs and T. Qiu, Patterns of Persistence in PSE (2011);
Academica, From Postsecondary Application to the Labour Market (2010);
Government of NB, High School Follow-Up Survey (2007). 17
Informing the Future of Higher Education
Reasons for Leaving PSE Ontario Rest of Canada
Didn't like it/Not for me 35.7% 30.3%
To change schools or programs 35.2% 33.7%
Health, personal or other reason 17.3% 20.5%
Not enough money 3.1% 6.7%
Wanted to work 3.1% 5.0%
Marks too low 5.6% 3.9%
More PSE students are uncertain about their direction
![Page 18: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/18.jpg)
Quite a few students arrive in PSE unprepared academically
Informing the Future of Higher Education
18
Ontario College “A”
2006 2007 2008 2009 2010
Below College English 58.7% 59.3% 59.8% 59.3% 55.0%
College English 39.4% 38.4% 38.2% 36.5% 41.0%
Total Responses (Count) 6,150 6,409 6,380 6,834 7,179
![Page 19: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/19.jpg)
PSE isn’t even the equivalent of full-time work for most students
14.0
15.2
15.9
17.0
27.3
28.4
29.3
31.7
0 10 20 30 40
below 70
70 to 79
80 to 89
90+
Hours per Week
Ove
rall
Un
ive
rsit
y G
rad
e
Total Time at School
work
Total Time Studying
Source: Lori McElroy and Alex Usher, HESA, Measuring Differences in Academic Challenges
and Grades at Canadian Universities (PowerPoint presentation for CIRPA 2010).
Informing the Future of Higher Education
19
![Page 20: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/20.jpg)
Being “networked” today means being connected/disconnected
20
Informing the Future of Higher Education
Source: W. Drexler (2008)
![Page 21: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/21.jpg)
The Paradox of Choice
21
Informing the Future of Higher Education
![Page 22: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/22.jpg)
HOW DO WE MEASURE IT?
![Page 23: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/23.jpg)
What does student “engagement” look like?
23
Informing the Future of Higher Education
• Volunteering
• Hours on campus outside of class
• Curious
• Enthusiastic
• Showing up for things
• Intentional
• Actively participating
• GPA
![Page 24: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/24.jpg)
What does student “engagement” look like?
24
Informing the Future of Higher Education
• Reflective
• Doing school work
• Empowered
• Clarity in institutional pledge to students
• Actively seeking learning
• Confident
• Active alumni
• Internally motivated
![Page 25: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/25.jpg)
What does student “engagement” look like?
25
Informing the Future of Higher Education
• Institutional pride
• Depends on the student
• Classes are full throughout
• Interest
• Asking questions
• Mental and physical health
• Time and effort students devote to their studies and related activities….
![Page 26: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/26.jpg)
What variables and factors encourage student engagement?
26
Informing the Future of Higher Education
• Passionate, engaged and well prepared faculty and staff
• Mentorship
• Activities that challenge
• Timeliness
• Student involvement
• Instruction that is relevant to where students are at
• Same level of commitment, etc. that we expect of students
![Page 27: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/27.jpg)
What variables and factors encourage student engagement?
27
Informing the Future of Higher Education
• Assuring that there is a fit when students come in
• Physical space
• Students have clear goals
• Outreach to K-12 about what it is all about
• Availability of learning resources (physcial space, etc.)
• Relevant ways to deliver services
• Mandatory voluntarism in the community
![Page 28: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/28.jpg)
What are the barriers to student engagement?
28
Informing the Future of Higher Education
• Time
• Money
• Collective agreements
• Tired and uninspired faculty and staff
• Conflicting commitments
• Tired and uninspired students
![Page 29: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/29.jpg)
What measures of student engagement are you currently using on your campus?
29
Informing the Future of Higher Education
• Attendance
• NSSE
• NCHA
• Participation in clubs and student leadership
• Required to withdraw rates
• Program satisfaction
• Retention
![Page 30: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/30.jpg)
What measures of student engagement are you currently using on your campus?
30
Informing the Future of Higher Education
• Course evaluations
• Student focus groups
• Feedback forms
• Graduation rates
• Undergraduate survey
![Page 31: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/31.jpg)
What do these measures of student engagement actually measure?
31
Informing the Future of Higher Education
• Satisfaction
• Academic performance
• Activity
• Extracurricular activities
• Impact of student supports
• Persistence rates
• Number of students engaged with peers
![Page 32: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/32.jpg)
What should we be measuring in terms of student engagement?
32
Informing the Future of Higher Education
• Why are they here and what do they want to study?
• What are their expectations?
• Satisfaction in exit surveys
• Expectations of program, faculty, etc.
• Perception of their own engagement
• Measure of student engagement and learning (transformation, desire to continue to learn?)
• How many resources are going to already engaged students
![Page 33: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/33.jpg)
What should we be measuring in terms of student engagement?
33
Informing the Future of Higher Education
![Page 34: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/34.jpg)
Students are more satisfied with their PSE experience
Informing the Future of Higher Education
34
Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of
Today’s College Student (2012).
1969 1976 1993 2009
I am satisfied overall 62 71 79 79
I would rather be going to university than anything else 69 69 75 83
I am satisfied with teaching at my institution 67 72 81 87
1969 1976 1993 2009
My grade point average is A- or higher 7 19 26 41
My grade point average is C or less 25 13 9 5
I have taken a basic skills or remedial course --- 29 32 45
I work hard at my studies --- 86 89 93
![Page 35: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/35.jpg)
Some innovations increase satisfaction but not academic performance
35
Informing the Future of Higher Education
Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to
ISSOTL, 2012)
Student Success
(A, B or C) GPA
Student
Satisfaction
Withdrawal
Rates
University of Central
Florida High None to slightly
positive effect No effect Not measured
Sheridan (FOB, FT
Faculty, PY Control) High No effect No effect
Possibly small
negative effect
Sheridan (College-
wide, many controls) Not measured
Small negative
effect No effect
Small negative
effect
![Page 36: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/36.jpg)
Course evaluations are a standard way to measure student “satisfaction”
Source: P. Gravestock and E. Gregor-Greenleaf, University of Toronto, Student
Course Evaluations: Research, Models and Trends (HEQCO: 2008).
Course evaluations typically include questions to students about the following:
Common uses for course evaluations include the
following:
Course content Teaching improvement (faculty)
Instructor’s communication skills Personnel decisions (PSE institutions)
Student-Teacher interaction Course selection (students)
Course difficulty and workload Teaching awards
Assessment practices in the course
Student self-assessment
Informing the Future of Higher Education
36
![Page 37: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/37.jpg)
NSSE has limitations as a means to measure student engagement
37
Informing the Future of Higher Education
• Too blunt for institutional comparisons, and too broad for course-based assessments
• It can be useful for program level and sub-group analyses.
• “most student subgroups are not universally engaged or disengaged, but rather show more or less engagement across items within and across benchmarks in a given program, and more or less engagement for particular items across programs”
Source: C. Conway, H. Zhao and S. Montgomery, The NSSE National Data Project
Report (HEQCO: 2011; C. Conway and H. Zhao, The NSSE National Data Project: Phase
Two Report (HEQCO: 2012).
![Page 38: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/38.jpg)
According to the CLA, students are studying less, and learning less
Source: Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College
Campuses (2011)).
Informing the Future of Higher Education
38
• Measures critical thinking, problem solving, writing, etc.
• 45% of students demonstrated no significant improvement in learning during the first 2 years of PSE
• 36% of students demonstrated no significant improvement over 4 years of PSE
• For students who show improvement, it is modest only
• Main culprit is lack of rigor in terms of reading requirements and study time (12-13 hours/week)
![Page 39: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/39.jpg)
WHAT CAN PSE INSTITUTIONS DO?
![Page 40: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/40.jpg)
Source: SSRC, 2012 from Learning to Earning, 2012, presentation by Brent
Herbert-Copley, SSHRC.
Unemployment By Student Performance (Sample Of Recent US Bachelor’s Graduates)
0
2
4
6
8
10
12
CLA scores Academic Engagement/Growth
LowMediumHigh
Informing the Future of Higher Education
40
Students must be reminded that jobs require more than a credential
![Page 41: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/41.jpg)
There are a number of different ways to introduce experiential learning
1 2 3 4
*Organize class visits to local business, government, or community workplaces
*Arrange job shadowing opportunities for students
*Invite business, government, or community guest speakers into the classroom
*Assign projects that require students to interact with local business, government or community organizations
*Invite students to share career goals with class
*Provide class time for students to reflect on their work experiences
*Design academic course content (readings, discussions) to integrate with students’ work experiences
*Use authentic assessment strategies to assess students’ ability to perform real-world tasks
*Use business/community/workplace case studies for student assignments
*Invite students to share their work experiences with class
*Talk to students individually about their work experiences
*Use business/community/workplace examples to illustrate concepts in class
*Talk to students individually about their career goals/concerns
University
College
Never 1-5 times 6-10 times 11-20 times
41 Source: Peters, J., Faculty Experiences with and Perceptions of Work-Integrated
Learning (WIL) in the Ontario Postsecondary Sector (Toronto: HEQCO, 2012).
Informing the Future of Higher Education
![Page 42: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/42.jpg)
Ontario students indicated what made a teacher effective
Factor % of Students
Selected
Clearly outlines expectations at the beginning of the semester 22.4%
Students often do well in his/her course 17.2%
A prominent researcher at the cutting edge of his/her field 13.2%
Integrates technology well into the learning environment 6.9%
Source: Ontario Undergraduate Student Alliance (OUSA), Ontario Results of the
Canadian Student Survey (PowerPoint presentation, 2010).
Informing the Future of Higher Education
42
Factor % of Students
Selected
Delivers interesting, well prepared and organized lectures 83.7%
Enthusiastic, entertaining or motivating in the classroom 74.6%
Able to communicate the subject in multiple ways 52.4%
Able to meet regularly with students 25.8%
Clearly outlines expectations at the beginning of the semester 22.4%
Students often do well in his/her course 17.2%
A prominent researcher at the cutting edge of his/her field 13.2%
Integrates technology well into the learning environment 6.9%
![Page 43: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/43.jpg)
Some ideas and advice for our faculty and programs
Informing the Future of Higher Education
43
Integrate technology where appropriate
Find ways to involve students in their own learning
Set and maintain reasonable but high standards
Articulate clear learning outcomes
Utilize transparent assessment practices (rubrics, etc.)
Encourage students to utilize student services and supports
Remind students of the value and relevance of learning
![Page 44: MEASURING AND ASSESSING THE VALUE OF STUDENT … · 5/7/2013 · Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012) Student Success](https://reader036.vdocument.in/reader036/viewer/2022081615/5fd627dca23a116f5274fadb/html5/thumbnails/44.jpg)
Thank You!
Informing the Future of Higher Education
Richard Dominic Wiggers Executive Director, Research and Programs
HEQCO
(416) 212-3881
Valerie Lopes [email protected]
Research Consultant