increasing student interest and response rates
TRANSCRIPT
Increasing Student Interest and Response Rates
Will Miller, Ph.D.Director of Institutional
Research and Effectiveness
Flagler College• St. Augustine, FL• Approximately 2,500
students• Undergraduate only• Liberal arts tradition• Non-religious• Non-tenure system• Using IDEA since at
least 2004
Old Approach to Evaluations
• Prior to Fall 2014– Veteran faculty• Select two courses (one upper/one lower)• Paper/pencil administration• One attempt—controlled by faculty
–New and adjunct faculty• All courses evaluated• Paper/pencil administration• One attempt—controlled by faculty
Results with the Old Approach
• Received a high response rate given that students were a captured audience– However, there were significant administrative
costs– Concerns about faculty role in administration– Concerns about thoughtfulness of students
• Especially with written comments• Additionally, even with the high response
rate, we were only evaluating 50% of courses chosen by faculty
Approach to Evaluations• Beginning in Fall 2014– All faculty• All courses evaluated• Online administration through Campus Labs• Multiple approaches
Some Faculty Respond• “Students will
never take the survey online.”
• “Not every student has the capability to complete this online.”
• “They’ll all sit together and talk negatively about me.”
The Response Rate Issue• Biggest concern going online• Campus-wide conversation on how
to assure response rates are where we want them to be ( > 70%)
The Punitive Approach• Student can’t access grades
– Or at least not as quickly• Student can’t register for next term• Student can’t graduate
– The Bachelor of Arts degree is awarded to students who meet the following conditions:
• Faculty revolt
Why We Went Another Direction
• Student Autonomy– From the moment freshmen arrive, we
preach student autonomy– Thus, it is our job to demonstrate to
them WHY they should complete course evaluations
Results
Results• Faculty satisfaction (and, in some
cases, lack thereof)– Comments– Response rates– Less loss of class time– Customization options
Method• Faculty are told to set aside a day
and time to still complete the evaluations in class, if they wish, along with talking points– Loaner tablets available if they know
students will need them• Significant effort spent designing a
communication plan to reach ALL students
Communication Plan• Pre-administration social media and
campus media blasts
Communication Plan• Initial e-mail at beginning of
administration– Serious tone– Discusses importance of course
evaluations and student responsibility in completing them
– KEY: reminding them to follow faculty instruction. We leave administrations open for 2-3 weeks and allow faculty to provide guidance on timing when relevant.
Sample Language• These results are used by the institution to
improve teaching and assure students are receiving the best instruction possible. Each semester IDEA results are looked at by faculty, department chairs, and administrators. Your honest feedback helps us in our efforts to continually improve instruction. Please take these evaluations seriously as they are our only opportunity to measure your satisfaction or dissatisfaction with your courses this semester.
Communication Plan• Follow-up e-mails– Less serious in tone– Have faculty provide short case studies on
what they learn from IDEA results and how they’ve made positive changes (attempt to use faculty from their majors)
– Keep them updated on response rate
Communication Plan• Follow-up e-mails– Sample language
• I know you’re tired of hearing from my office about these, but they really are important. It will take less than 5 minutes per course to complete and will help us do a better job teaching you effectively. We’ve extended the deadline till December 7 for you to complete these and really hope you will take the time to complete them. However, the sooner you complete them, the sooner we can stop reminding you.
• Yes, it’s us again. Looks like you still haven't completed course evaluations for the following courses: ECO 331 (A): International Economics We won't notify you anymore about this. Just reminding you that your ability to evaluate these courses will close on Sunday night at midnight. Log in now (even from a mobile device) to do your part.
Communication Plan
Communication Plan
Communication Plan
Communication Plan
Tricks• Redirect to a different URL–www.flagler.edu/courseevaluations• Easier for faculty and students to remember
• Having students help in crafting messaging
• Appropriate seriousness in each message
• Subject lines of e-mails!– Pretty please! (responses and
comments)
Conclusion• It IS possible to have high response
rates without taking draconian control over elements of student lives– Faculty will be happier– Students will be happier
Questions and Discussion