Online Course Evaluations

Is there a perfect time?

Presenters:

Cassandra Jones, Ph.D., Director of Assessment

Michael Anuszkiewicz, Research Associate

Office of Institutional Effectiveness

• Founded in 1950

• Private, not-for-profit

• Coeducational

• Comprehensive university

• Four Schools• Arts and Sciences• Business Administration• Education and Human Services• Malek School of Health Professions

About Marymount

Student Body (n=3,702)

67%

33%

By Level

Undergrad

Graduate

72%

28%

By Gender

Female

Male

47%

38%

9% 6%By Race/Ethnicity

White

Minority

Nonresident

Unknown

59%15%

4%22%

By Home State

VA

MD

DC

Other/Intn'l

Course Evaluations at MU• Done in-house since Fall 2002

• Paper forms completed in class through Spring 2007

• Online student evaluations since Summer 2007

• Evaluations distributed at the end of each semester– Three terms: Fall, Spring, and Summer– Excluded: Internships; projects; courses of fewer than 2

• Timeline: 3 weeks (since Spring 2008)– Last 2 weeks of class– Exam week

Issues

• Response Rates

• Quality – Nonresponse bias

• New issue – Timing – To exam or not to exam

ISSUE 1:

RESPONSE RATES

A Closer Look at Response Rates

62%

38% 33% 36% 31% 37%

Fall Term Response Rates by Submission Date

Last Day of Class

A Closer Look at Response Rates

62%

38% 33% 36% 31% 37%

0%

54% 51% 49%41%

54%

Fall Term Response Rates by Submission Date

Last Day of Class Including Exams (+)

Issue 2: Nonresponse Bias

• Study Conducted in 2009• Examined predictor of ratings and who

participates: No common predictor

Issue # 3

Timing of Evaluations: To offer or not to offer during exams?

Timing of Responses Received

Term % Before Exams % During ExamsFall 2010 76% 24%Fall 2011 88% 12%Fall 2012 72% 28%

3-Year Average 79% 21%

Undergraduate Students

Feared Distribution of Course Evaluation Completion

F D- D+ C- C C+ C+ B- B B+ A- A0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

During Exams Linear (During Exams)

Perc

ent R

ecei

ved

Feared Distribution of Course Evaluation Completion

F D- D+ C- C C+ C+ B- B B+ A- A0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Before Exams Linear (Before Exams)

Perc

ent R

ecei

ved

How would you rate this course as a learning experience?

Submission Before Exams Submission During Exams

Very Poor

Poor Fair Good Excellent0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Overall

Perc

ent R

ating

Very Poor

Poor Fair Good Excellent0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Overall

Perc

ent R

ating

How would you rate this course as a learning experience?

F D- D D+ C- C C+ B- B B+ A- A.00

.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Before Exams During Exams

Mea

n Sc

ore

How would you rate the instructor’s teaching?

Submission Before Exams Submission During Exams

Very Poor

Poor Fair Good Excellent0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Overall

Perc

ent R

ating

Very Poor

Poor Fair Good Excellent0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Overall

Perc

ent R

ating

How would you rate the instructor’s teaching?

F D- D D+ C- C C+ B- B B+ A- A.00

.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Before Exams During Exams

Mea

n Sc

ore

Decided to Examine More Systematically

• Can one predict who responds before or during exams?

• Variables included:– Response: Before or after exams– Course evaluation rating– Course grade– Course level – Semester– Gender– Course Enrollment– Term

• Undergraduates Only

Course Evaluation Questions

• The instructor was well prepared for each class

• The instructor explained course content• The instructor was responsive to student

questions• The instructor used class time well• The instructor encouraged me to take

responsibility .. . .

Findings: Descriptives

• 3 Fall Semesters• 2784 Course Sections• 14055 student records• Composite of Five Questions – Likert scale (Strongly Disagree to Strongly Agree)– Average Course Rating: 21.61 out of 25.00

Findings: Statistical

• HGLM and Logistical Regression/ Same Results– Student Characteristics– Course Characteristics

Findings

• Predictors indicating greater chance of completing before exams– Course grade– Male– Course size– Course Level (up to 400 level course)

Findings

• Predictors indicating greater chance of completing during exams:– Gender (Female)– Higher level of course

• Important finding: Course evaluation rating: Not significant

Conclusions and Implications

• No statistically significant differences in ratings based on when a student responds

• Students who are male, and have higher grades respond before exams

• Students are more likely to complete evaluations before exams in lower level classes

Implications

• Course evaluation ratings are not impacted by timing

• The longer students are in school, the more encouragement they need to complete the evaluations earlier

• Reason to change timing?

Thank you for your time and interest!

Office of Institutional EffectivenessMarymount University2807 North Glebe RoadArlington, VA 22207