+

Measuring Teaching Quality in the Online ClassroomAnn H. TaylorDirector, Dutton e-Education InstituteCollege of Earth and Mineral Sciences

+How do we review teaching effectiveness at Penn State?

Per Policy HR-23 and the “Statement of Practices for the Evaluation of Teaching Effectiveness for Promotion and Tenure”: Student reviews via the SRTE Peer review

Procedures, including frequency, are left to academic units

“Your mileage may vary”!

+How can we review teaching effectiveness online?

Formal reviews

Student reviews

Informal reviews

+Are there differences in what we are looking for?

Seven Principles of Good Practice

Good practice in undergraduate education:

1.encourages contact between students and faculty,

2.develops reciprocity and cooperation among students,

3.encourages active learning,

4.gives prompt feedback,

5.emphasizes time on task,

6.communicates high expectations, and

7.respects diverse talents and ways of learning.

+Good teaching is good teaching.But the evidence may look quite different!

+

+What are we looking for?

Interactions with students (including response rates and availability)

Use of technology (including usability of instructor-created content)

Pedagogical strategies (including facilitation of discussions)

Overall management of the administrative aspects of the course

+

+What should we do with the results?

Celebrate and reward good teaching

Provide professional development opportunities Mentoring Learning design support Resources! (weblearning.psu.edu)

+Student Reviews

+What are students reviewing?

Course content

Interactions with instructors

Use of technology

Appropriateness of the technology

Pedagogical strategies

+What are some concerns?

What can, and cannot, students evaluate?

How can you get meaningful feedback?

Are students qualified for the task?

Will they complete online evaluations?

How do I get them to fill them out?

+What should faculty do with student feedback?

Be prepared to make a few changes

Take the small steps approach

Read results with a thick skin

Look for trends

Overlook outliers

Seek explanations, but don’t explain away

(see http://www.lehigh.edu/~infdli/FD-evaluations.htm)

+Tool 1: SEEQ

Students’ Evaluation of Educational Quality

Mid-semester feedback instrument

Developed by educational psychologist Herbert Marsh

Normed to determine the kinds of questions students really could answer!

32 standardized questions grouped into nine dimensions of teaching

+Tool 2: SALG

Student Assessment of their Learning Gains

Sample instrument that can be customized

Statements about the degree of "gain" (on a five-point scale) which students perceive they've made in specific aspects of the class

Includes open-ended questions

Delivered to students online

Sponsors provide a statistical report of the results

+Tool 3: SRTE

Student Rating of Teaching Effectiveness

177 Question Choices

4 mandatory items1. Are you taking this course as an elective? (If uncertain,

omit.)2. What grade do you expect to earn in this course?3. Rate the overall quality of this course.4. Rate the overall quality of the instructor.

2 open-ended mandatory items 1. What helped you learn in this course?2. What changes would improve your learning?

Up to 15 department items and up to 5 instructor items

+SRTE for online courses in EMSSRTE Question Bank Items:

Rate the instructor's availability for individual help and consultation.  (#42)

Rate the effectiveness of the instructor's responses to student questions.  (#48)

Rate the instructor's skill in keeping the course on schedule.  (#70)

Rate the promptness with which graded exams, reports, and other materials were returned.  (#101)

Additional questions:

Rate your agreement with the following statement:  “The instructor initiated communication with students on a regular basis.”

Rate your agreement with the following statement:  “The instructor responded to my inquiries in a timely manner.”

Rate your agreement with the following statement:  “The instructor provided meaningful feedback on my class assignments.”

Rate your agreement with the following statement:  “I would recommend this course to others.”

+Tool 4: DIY

Simple web-based form

Ask anything!

Lots of examples available

Lots of tools available Google Template SurveyMonkey Survey Gold Zoomerang

+Another alternative:Gather informal feedback

Minute papers, muddiest points

Self-check using formal instruments Peer Review of Online Teaching Faculty Competencies for Online

Teaching

Online Mentoring Program (pilot)

+How do we get started?

Start with faculty How do they define good teaching? What criteria do they think should be

measured? How do we involve part-time faculty?

Form a steering committee of key stakeholders

Take small steps and build on successes

Measure impact throughout the process

Strive to continuously improve

+What can administrators do?

Create an institutional environment where good teaching is highly valued & professional growth is expected

Provide adequate and accessible faculty development tools & resources

Tell faculty, staff, and students how this will benefit them

Measure impact!

Recognize and reward good teaching at all levels – celebrate!

Encourage trial and error

Act!

+What can faculty do?

Utilize a wide array of strategies to collect feedback

Document and share evidence of good teaching with superiors

Request faculty development programming

Work with faculty senate and other faculty bodies to strengthen support

Tell your students how this will benefit them

+So what are YOU going to do?

+Follow-up?

Ann H. Taylor

atb3@psu.edu

814-863-9070