students’ evaluation of educational quality (seeq) dr. mary benbow associate professor,department...
Post on 19-Dec-2015
219 views
TRANSCRIPT
![Page 1: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/1.jpg)
Students’ Evaluation of Educational Quality (SEEQ)
Dr. Mary BenbowAssociate Professor,Department of Environment and Geography
University of Manitoba
![Page 2: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/2.jpg)
Resources to Examine Evaluations
• Centre for Educational Advancement at Curtin University, Australia.
• “Students Rating Teaching” by Mark Lawall
• Many, many peer-reviewed research articles– A selection is
provided in the bibliography
• My web site• http://
home.cc.umanitoba.ca/~benbow/seeq.html
![Page 3: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/3.jpg)
Using the SEEQ
• Student evaluations – what we know• Student Evaluation of Educational Quality (SEEQ)• Using the SEEQ
![Page 4: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/4.jpg)
Student Evaluation: What We Know
• A lot of research (some apparently contradictory and/or inconsistent)
• Difficult to compare different instruments• Many useful reviews:
– Students Rating Teaching– Other articles (e.g. Ory, 2001)
![Page 5: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/5.jpg)
Development and Use of Student Evaluations
• Introduced in the 1960s– Students sought “a stronger voice”– Today viewed as one component of accountability
• Over 90% of institutions of higher education use student evaluations of some sort
• Comprehensive scheme for student evaluations began at the U of M in the academic year 1996-97 (Policy 425)– September 2002, Senate approved voluntary posting of
SEEQ results on U of M website
![Page 6: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/6.jpg)
Courses: Requirements, Levels and Sizes
• Electives tend to get higher ratings than required course– majors more favourable than
minors
• Higher level courses tend to get higher ratings
• There are differences between the disciplines
• Class size has little effect overall
![Page 7: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/7.jpg)
Instructors and Students
• Student ratings positively correlated to those of alumni– positive correlation between students and colleagues
• Lower ratings for less experienced instructors and teaching assistants– relationship between research productivity and ratings is
either positive or nil
• Higher ratings where there is prior interest• Slightly higher ratings where gender of instructor and
students is the same
![Page 8: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/8.jpg)
Administration and Use
• Diagnostic not prescriptive– Need to follow uniform
procedures– higher evaluations if followed by
a short speech– higher also if not anonymous– higher if the instructor stays in
the room
• Most important: not the only source of information concerning teaching
![Page 9: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/9.jpg)
Student Evaluation of Educational Quality (SEEQ)
• Widely researched– very high degree of reliability (r = 0.88 - 0.97)
• Based on psychometric analysis (teaching not content)– correlate well with a wide range of measures of learning
outcome– correlates well with instructors’ self ratings
![Page 10: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/10.jpg)
The development of the SEEQ
• Developed Herbert Marsh of the University of Western Sydney in late 1970s– Internationally recognised expert in
psychometrics
• SEEQ has been exhaustively researched– Statistical tests over 13 years, from
approximately 50,000 courses and almost 1 million students
![Page 11: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/11.jpg)
SEEQ Factors Questions developed from interviews with faculty and students
about what constitutes effective teaching
• Learning• Organization• Individual Rapport• Breadth• Assignments
• Enthusiasm• Group Interaction• Examinations• Overall Ratings
Teaching as a multi-dimensional activity
![Page 12: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/12.jpg)
SEEQ Questions
• Formative questions:– Identifies specific aspects of each factor– Organization:– Instructor’s explanations were clear
• Summative overall questions:– 30. Compared with other courses I have had at the U. of M., I would
say this course is:– 31. Compared with other instructors I have had at the U. of M., I
would say this instructor is:– 32. As an overall rating, I would say this instructor is:
![Page 13: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/13.jpg)
Student Evaluation of Educational Quality:
Using the SEEQ
• Little research on summative use• Ratings are a reliable source of data • Data must then be examined and evaluated
– Examine the data to create assessment– Followed by decisions and examination of change
![Page 14: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/14.jpg)
Using the SEEQ
• Understanding your priorities
• Undertake a self-evaluation of teaching
• Looking at your evaluations
• Identify solutions
• Making changes and smart decisions
![Page 15: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/15.jpg)
Your Priorities
• Related to your Teaching Philosophy– Describes the principles that direct your
teaching style and activities
• Identify which factors and elements are most important to you– Are your priorities consistent those of
your discipline, department or institution?– Can you establish a reason for your
priorities
• Prior to examining your evaluations, it is important to assess which matter most to you
![Page 16: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/16.jpg)
Self-Rating of Teaching
• The SEEQ correlates well with self ratings and trained observer ratings
• Useful to consider our assessment of teaching– Collect evidence of teaching and assemble in a
teaching dossier or portfolio– Consider our assessment using a self-rating form
based on the SEEQ– “Most of us await student reactions to our courses
with both eagerness and trepidation” (Delucchi and Pelowski, 2000)
![Page 17: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/17.jpg)
Strongly Disagree
Disagree Neutral Agree Strongly Agree
1. I have found the course intellectually challenging and stimulating
2. I have learned something which I consider valuable
3. My interest in the subject has increased as a consequence of this course
4. I have learned and understood the subject materials of this course
Identify the most common responses you
expect from your class
![Page 18: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/18.jpg)
• Are there any surprising responses?
• Which surprises are most relevant?
• Do the written comments reveal any more information?
• You can also use your statistical summaries of evaluation data (but differences of <0.05 are not considered significant)
Self-Analysis: Looking at your course evaluations
![Page 19: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/19.jpg)
Analysis framework
• What special features reflect the subject, the students or other contextual factors?
• Eliminate any factors that are not appropriate to your intentions in teaching this class. – What are the main strengths of your teaching? – Which are the main weaknesses in your teaching?
• How do these findings differ from your Self-Rating Survey
![Page 20: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/20.jpg)
Other Useful Data on the SEEQ
• Students Enrolled and Students Responding can indicate attendance (e.g. 62%)
• Expected Grade can be compared to the actual grade distributions
• Reason for taking course can reveal how many are required to take a course
![Page 21: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/21.jpg)
“There’s always one”
! ?
• What should we do about that one or two reponse(s) that stand
out, perhaps anomalously?
• Look at the response sheets and identify which responses are the outliers
• Look at their written comments
• Look at the Student and Course Characteristics
• Are the outliers only in one question or subject area? (e.g. use of humour)
• Are they relatable (Level of interest (37) & Learning questions)?
![Page 22: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/22.jpg)
Using the SEEQ:Making changes
• Using your priorities and self rating decide where changes would be most useful
• Need to identify sources of information – Choose information sources to suit your teaching and
learning style– Start small (quick, easy, cheap) before investing in large,
long-term changes
![Page 23: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/23.jpg)
Select key areas for improvement
• Which are the two or three areas that received the lowest ratings? – What strategies for improvement could
you use that would be appropriate to your particular situation?
• Could the Tips to Improve Academic Teaching help you in this?
• Would further personal guidance or investigation be needed to help you improve your knowledge and skills?
![Page 24: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/24.jpg)
Making Changes
• Start small before investing in long-term changes
• Why go it alone?– Could there be some changes that could be
accomplished by group action?
• It takes time to create a new habit– When you try something new, it may feel
uncomfortable at first– Give yourself reminders to try some new techniques
or ideas
![Page 25: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/25.jpg)
A good place to start are the “Tips to Improve Academic Teaching” organized in 8 sections referring to each
factor12. Instructor gave lectures that facilitated taking notesStrongly Strongly Disagree Disagree Neutral Agree Agree 9 18 3 13 7
This question comprises part of the Organization factor: choose Organization & Clarity SEEQ Factor 3
![Page 26: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/26.jpg)
Statistical Analysis of SEEQ Results
• SEEQ data cannot be analyzed or summarized statistically by administrators
• But, individuals can analyze their own data• The Statistics Advisory Group recommends the
following techniques• Can be undertaken by hand or by using a
computer spread sheet
![Page 27: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/27.jpg)
Averages, Medians and Grouped Questions
• Find the average response for each question• Find the median response for each question• Find the average for each subject grouping or
FACTOR (e.g. average response for all four questions under “Learning”)
![Page 28: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/28.jpg)
• 1. I have found the course intellectually challenging and stimulating
N/A 01 Strongly Disagree 02 Disagree 03 Neutral 34 Agree 205 Strongly Agree 2
• Average = (0*1)+(0*2)+(3*3)+(20*4)+(2*5) = 99/25 = 3.96
• Median = 4
![Page 29: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/29.jpg)
• 1. I have found the course intellectually challenging and stimulating
• 2. I have learned something which I consider valuable
• 3. My interest in the subject has increased as a consequence of this course
• 4. I have learned and understood the subject materials of this course
Av. 3.96Med. 4.0
Av. 4.16Med. 4.0
Av. 4.08Med. 4.0
Av. 4.24Med. 4.0
Learning factor: Average = 4.11, Median = 4.0
![Page 30: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/30.jpg)
You may wish to provide summaries to indicate improvements in your teaching
‘00-’01 ‘01-’02 ‘02-‘03
1. I have found the course intellectually challenging and stimulating
3.6 3.6 3.7
2. I have learned something which I consider valuable
3.8 3.9 3.8
3. My interest in the subject has increased as a consequence of this course
3.7 3.7 3.8
4. I have learned and understood the subject materials of this course
4.0 4.0 4.3
![Page 31: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/31.jpg)
You can also graph changes in averages (in a teacher dossier, include an explanation)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
2000-01 2001-02 2002-03 2003-04
Q. 30Q. 31Q.32
![Page 32: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/32.jpg)
Additional Questions in the SEEQ
• At Curtin University (Perth, Western Australia)• Staff engaged in the 1995 pilot of SEEQ were concerned
that it might not accommodate forms of teaching and learning other than conventional face-to-face lecturing
• Teaching staff across the disciplines have subsequently developed factors and questions to complement the SEEQ form.
![Page 33: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/33.jpg)
SEEQ Questions Examples: The computer laboratory
1. The laboratory was an essential part of the unit.
2. Laboratory-practical exercises reinforced the main points of the lecture.
3. By applying theoretical concepts in the laboratory, my understanding of the material improved.
4. The time and effort required for computer-based laboratory work was reasonable.
![Page 34: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/34.jpg)
Written Comments
• Type them up– Include course title,
number, year, number of students, etc.
– What additional depth do they provide for the question responses?
– e.g. if you score well in Organization the comments may bring up a clear outline, good notes, web pages, etc.
• Note: Not to be used for tenure or promotion
![Page 35: Students’ Evaluation of Educational Quality (SEEQ) Dr. Mary Benbow Associate Professor,Department of Environment and Geography University of Manitoba](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d375503460f94a1049e/html5/thumbnails/35.jpg)
• Use your priorities and self rating to decide where changes would be most useful
• Innovations to improve teaching are everywhere– Use the library, NetDoc or Google Scholar to find great
information and ideas
• The SEEQs are not the only source of information about your teaching – Develop your teaching dossier– Develop your teaching philosophy and goals
Students’ Evaluation of Educational Quality (SEEQ) :
Conclusions