Improving the Online Evaluation Process and Response Rates
SAIR 2011
WCU• 9,352 students• Master’s Comprehensive• Mountain location• Resident and Distance
History of Evaluations at WCU• No university-wide policy on administration,
instruments, courses covered, and use of results• Scanned, paper forms administered during class time• Long wait for results• No flexibility in types of reports• Poor security– esp. comments
Implementation Timeline2002 through 2006
• Faculty Senate establish ‘teaching evaluation committee’
• Provost mandated online evaluations (PACE initiative)• Developed SAI instruments based on course types• Developed guidelines for results usage• Researched available options for administering evals
and selected ConnectEdu’s CoursEval
Faculty “own” the evaluation process!
Timeline cont.: Course Evaluation Pilot – Spring 2007
• 5 departments• 328 courses (5689 surveys)• 45% response rate• Problems: log-in and email issues
lack of experience
Timeline cont.: Establishing Procedures (2007-2008)
• Scheduling guidelines• Low-enrolled courses• Student incentives• Refined SAI instruments• Planned validity study
• Advertising/incentives• Faculty training• Summer parts-of-term• Instrument defaults• Student email policy• Login issues
Faculty Senate Implementation Committee
Lessons Learned• Lesson 1: Involve everyone with a stake in the
evaluation procedures and results; everyone with experience in processing & administering evaluations
• Lesson 2: Consider all issues before implementing
• Lesson 3: Build a strong communication system between faculty and survey administrators
IR Responsibilities• Set-up & administer each semester’s evaluations• “Help desk” for faculty and students• Learn about new tools and related software, reporting,
and response rates• Communicate to Faculty Senate any issues arising
affecting senate policies and survey administration
Overview of Current Policies• All course sections, all terms• Open period set by course length• University-wide incentives allowed• Low-enrolled courses• Crosslisted courses• Periodic response rates & student reminders• Faculty evaluation (not course content)• Results availability• Use in faculty AFE/TPR process• IR office responsible for administration• Department heads have final say in instrument used
Response Rates over time
Spr 07 Fall 07 Spr 08 Fall 08 Spr 09 Fall 09 Spr 10 Fall 10 Spr 110%
10%
20%
30%
40%
50%
60%
70%
80% Term RateSpr 07 44.7%
Fall 07 43.5%
Spr 08 32.8%
Fall 08 46.5%
Spr 09 47.3%
Fall 09 43.4%
Spr 10 49.6%
Fall 10 73.4%
Spr 11 73.4%
MarketingStrategy Did it work?Large Banners Yes
Postcards Maybe
Emails Yes
Flyers around campus Maybe
Radio announcement Maybe
Giveaways No
“Early” grade release YES!
“Early” Grade Release• Students are encouraged to complete evaluations in
order to receive their final grades “early”• A list of all students who complete evaluations is sent
to the Registrar• Emails go out to students as faculty members submit
grades• Those who do not complete evaluations get final
grades after exam week is over
Overall, what works for us• Faculty Senate owns the process• Changes are made slowly, with much deliberation• Small, centralized survey administration• SaaS – few technical issues on our end, backed up
data, no questions about data integrity
Features of CoursEval that we like• Students and Faculty – Find all associated courses
and surveys with one click• Consolidation of people without deleting anything• Active Directory login• Automated emails, open/close, results release• Response rates – real time, easy to locate and email• Easy to use administrative tools• Customizable emails and log-in site
Features we would like• More flexibility with reports• Advance notice of outages and problems• Ability to access individual student responses (for
research purposes)• Faculty member attached to course cannot be
changed• More stream-lined set-up
Contact InformationWCU’s Course Evaluation website:
http://www.wcu.edu/8356.asp“Guidelines & Procedures for Administration & Oversight of Student Assessment of Instruction”
Institutional Planning & Effectiveness: http://www.wcu.edu/12829.asp 828-227-7239Elizabeth Snyder [email protected] 828-227-2570Alison Joseph [email protected] 828-227-3042
ConnectEdu website:http://www.connectedu.com/