do student evaluations match alumni expectations?* your future in finance, october 1, 2011...
TRANSCRIPT
Do Student Evaluations Match Alumni Expectations?*
Your Future in Finance, October 1, 2011*Managerial Finance, June 2011
Carla Guevara and Scott Stewart
MSIM PROGRAM
This Research Project’s Objectives
1.Identify what alumni perceive to be of most value in coursework, once they’ve had time to reflect, and put their learned knowledge into practice
2.Explore the relationship of those perceptions with the information in student evaluations conducted at the conclusion of individual courses
Do Student Evaluation Match Alumni Expectations?
Literature ReviewStudent Evaluations
Positive correlation between instructor rankings and student perceptions of learning but not actual learning.
Beyond learning, there are other factors correlated with student ratings of teaching effectiveness.
Do Student Evaluation Match Alumni Expectations?
Literature ReviewAlumni Surveys
Alumni offer an objective perspective given their distance from involvement with the programAlumni surveys provide a basis for evaluating relationship between knowledge and skills acquired through the educational program and the ones required in the workplace
Do Student Evaluation Match Alumni Expectations?
Case Study
Program Evaluation ProcessBoston University MSIM Program
End-of-Course Evaluation
End-of-Program Survey
Alumni Survey
Assessment of Meeting AACSB Learning Goals
Do Student Evaluation Match Alumni Expectations?
Student Data
End of Course: Please rate this course…1.Course overall (course-satisfaction)2.Extent of learning (learning)3.Potential usefulness to a professional career
(relevance)4. Instructor overall (instructor)
End of Program:1.What course did you find most challenging?– What course did you find most rewarding?
Do Student Evaluation Match Alumni Expectations?
End-of-Course
Evaluation
End-of-Program Survey
Summary of Student Evaluation Measures Mean Values, By Course, First and Second Highest Values in Bold
End-of-course and End-of-Program Survey Questions, 2004-2009
Do Student Evaluation Match Alumni Expectations?
Alumni Data
Quick SurveyFocus GroupAlumni Survey
“great guest speakers”“doing as well as learning”
“something is worthy, relevant”“It helps in the real world and application”
“what I take out”“something I can apply in the real world”
“maybe a course that can’t be applied, but in which I learned something”
Do Student Evaluation Match Alumni Expectations?
Hypotheses
H1: Post‐graduation, students believe career relevance is more important in determining the level of satisfaction for a given courseH2: Post‐graduation, students believe instructor performance is less important in determining the level of satisfaction for a given course
Do Student Evaluation Match Alumni Expectations?
Results
Do Student Evaluation Match Alumni Expectations?
* Logistic coefficients / 10
Conclusions
H1: Consistent with focus group observations, career relevance clearly grows with time in importance for determining course satisfactionH2: Instructor performance becomes relatively less important
Do Student Evaluation Match Alumni Expectations?
Forecasting Alumni Satisfaction
Do Student Evaluation Match Alumni Expectations?
Key question: Can we forecast alumni course satisfaction using end-of-course student evaluation perceptions?Answer: To some extent
Regression Results
Adjusted R-Square 13.1%
F-stat 4.261 0.008
t-stat Significance
LearningS -0.121 0.357
Career RelevanceS 0.366 0.005
InstructorS -0.005 0.952
Key Takeaways for Faculty
• Instructor can improve their alumni course satisfaction if they incorporate real-world applications in the curriculum, as well as enhance their in-class presentation
• But, instructor faces a challenge: they need to teach material which will be useful in the real world, but students may not fully appreciate it, which may not help the instructor’s student evaluations
Do Student Evaluation Match Alumni Expectations?
Key Takeaways for Students
1.Analyze Career Relevance rating in addition to course and instructor ratings
2.Ask instructor how skills learned in class are applied on the job
3.Be patient in the classroom, especially with casework
4.Remember building blocks courses may not be directly applicable in your internship
Do Student Evaluation Match Alumni Expectations?
Appendix
Do Student Evaluation Match Alumni Expectations?
How address issue?
1.Measures of teaching effectiveness utilized for decision making purposes shouldn’t stress too highly traditional measures of student satisfaction
2.School’s curriculum should include material useful for the real world
3.One suggestion: form a council of alumni who can provide advice on appropriate curricula
Do Student Evaluation Match Alumni Expectations?
Literature ReviewModel of Education
Course Satisfaction
Course RewardingDo Student Evaluation Match Alumni
Expectations?
• Nature of the students• Subject being taught• Time of day• Teaching environment• Availability of resources
• Prior knowledge• Personal organization,
interest and motivation for particular subjects and tasks
• Personal Characteristics• Performance• Teaching techniques
Adapted from Ding & Sherman 2006
Forecasting Alumni Satisfaction
Correlation = 0.58
Do Student Evaluation Match Alumni Expectations?
Derivatives
Economics
Fixed Income
Equity Analysis
Port. Mngmnt.