welcome to teaching + learning tuesdays july 16, 2013 i 2:30pm to 4:30 pm
TRANSCRIPT
Welcome toTeaching + Learning Tuesdays
July 16, 2013 I 2:30PM to 4:30 PM
Title:Innovative Results Reporting: Student Assessment of Instruction
Presenter: Julian Hutcheson, Statistical & Research Analyst III, Aiken Technical College
3
Innovative Results Reporting:
Student Assessment of Instruction
4
Have a “SAI” in your education!”
5
In-house report development for student course evaluation results
Translating this to our vendor’s online system
Features of our online reports
Additional recent innovations
This is the Story of……
6
Fall Enrollment
around 3,000
Approx. 60 FT faculty and
100 PT faculty
Online course evaluation
began in Fall 2009
Vendor was Campus Cruiser,
Cruiser Eval product
Fall 2011 began using
IOTA Solutions
About Aiken Technical College
7
Part of FPMS, Evaluation of Instructors
Student Satisfaction
rates
Trends in courses,
departments, and programs
Collect assessment data on our QEP critical
thinking learning
outcomes
QEP Awareness
Withdrawn students feedback
Comparison of Online vs Regular
classroom
How ATC Uses the SAI Survey
8
16 core questions:
Rating Very Satisfied to
Very Dissatisfied
Converted to
numerical 4-point scale
2 QEP Awareness questions
for all
6 QEP Critical thinking
outcomes for
implementer courses
Faculty required to
keep an average of
3.0
About our SAI Survey (through Fall 2012)
9
Does your online system have standard, built-in reports that meet your needs?
Reporting Results
CourseID Section concat form Course & Section Instructor_Last_Name Instructor_First_Name UNIQUE_KEY Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14PSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 18881610 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 18970174 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 31067426 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 9723702 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 16188092 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 13940057 S S D S S S S VS S S S D S SPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 13499398 S VD D S D D VD D VD VD D VD D VDPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 22882508 VS VS VS VS VS VS VS VS VS VS VS VS VS VSSUR-102 140 SUR-102-140 SUR-102-140 Instructor_Last_Name Instructor_First_Name 16245664 VS VS VS VS VS VS VS VS VS VS VS VS VS VSSUR-102 140 SUR-102-140 SUR-102-140 Instructor_Last_Name Instructor_First_Name 8477022 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 1198689 VS D S S VS S S D S S VS S S SBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 13710771 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 18652426 VS S S VS S VS VS S S VS VS VS VS VSBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 25827011 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 14455382 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 17531362 S S S S S S S D S S S S S VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 8147534 VS VS VS VS VS VS VS S S S VS VS VS SBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 33536410 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 32656969 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 15839791 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 32074949 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 31723791 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 1838195 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 24478282 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 29149708 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 24438272 S VS S S S S S D S D D S D DBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 27803933 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 21691307 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 30463164 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 20885947 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 17212158 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 11993973 VS VS VS S VS VS VS VS S S S S VS SRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 21247300 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 17106447 VS VS VS VS VS VS VS VS VS VS VS VS VS VS
Raw
11
Diligence
16
TermCourse ID Section
Instructor Last Name
Instructor First Name
Q1: My instructor explains and clarifies course learning outcomes, grading and requirements.
2010FA1 BUS-140 011 Louisa Delay 4.00
2010FA1 ECD-135 003 Mallory Brassell 4.00
2010FA1 MTT-112 040 Fernando Desanto 4.00
2010FA1 IMT-151 041 Kurt Prettyman 4.00
2010FA1 CPT-101 151 Codi Smith 3.92
2010FA1 IMT-113 041 Ted Wagoner 3.91
2010FA1 WLD-102 018 Louisa Garabedian 3.91
2010FA1 ACR-102 040 Karina Sobel 3.91
2010FA1 MKT-240 151 Tameka Valderman 3.91
2010FA1 WLD-132 069 Eric Sturtz 3.90
2010FA1 CPT-209 029 Lakisha Turnbough 3.89
2010FA1 WLD-103 040 Jamie Quisenberry 3.89
2010FA1 CRJ-202 069 John Schieber 3.86
2010FA1 CPT-101 451 Lance Vanders 3.86
2010FA1 WLD-111 040 Clayton Shira 3.84
2010FA1 WLD-102 040 Selena Dutil 2.83
2010FA1 IMT-151 040 Hillary Filip 3.84
2010FA1 CPT-162 057 Christian Tiano 3.83
2010FA1 RPT-223 058 Louisa Delay 3.82
2010FA1 MGT-101 060 Mallory Brassell 3.82
2010FA1 CPT-101 013 Fernando Desanto 3.82
2010FA1 IST-201 102 Kurt Prettyman 2.80
2010FA1 BAF-230 063 Codi Smith 3.81
2010FA1 CPT-257 103 Ted Wagoner 3.80
2010FA1 WLD-101 040 Louisa Garabedian 3.80
2010FA1 ECD-101 064 Karina Sobel 3.73
2010FA1 EET-251 040 Tameka Valderman 3.73
17
Term CourseID Section First_Name Instructor_Last_Name Instructor Overall Average with Course Section Averages
2010FA1 CPT-162 057 Louisa Delay 3.82
2010FA1 CPT-168 059 Louisa Delay 3.58
Delay Average 3.70
2010FA1 WLD-132 069 Fernando Desanto 3.20
2010FA1 WLD-170 040 Fernando Desanto 3.20
Desanto Average 3.20
2010FA1 RPT-243 069 Codi Smith 2.98
Smith Average 2.98
2010FA1 ECD-102 002 Ted Wagoner 3.71
2010FA1 ECD-237 040 Ted Wagoner 2.95
2010FA1 ECD-131 015 Ted Wagoner 2.88
2010FA1 ECD-133 040 Ted Wagoner 2.68
Wagoner Average 3.06
2010FA1 CPT-162 015 Louisa Garabedian 3.69
2010FA1 CPT-212 003 Louisa Garabedian 3.49
2010FA1 CPT-162 013 Louisa Garabedian 3.39
Garabedian Average 3.52
2010FA1 MKT-240 151 Karina Sobel 4.00
2010FA1 MKT-240 014 Karina Sobel 3.70
2010FA1 MKT-101 058 Karina Sobel 3.62
2010FA1 MGT-120 013 Karina Sobel 3.07
Sobel Average 3.60
2010FA1 EGT-252 069 Tameka Valderman 3.00
Valderman Average 3.00
18
19
Positives
User friendly mostly
Overall averages
Sorted by course and by instructor
PDF with Navigating tabs
Fall 2010: Good but Lengthy
20
21
22
23
Compact
Seeing results for all questions on single page
12 pages versus
116 pages
Can see areas of weakness in
student satisfaction:
Referring to the survey
questions
Patterns of red areas
Online courses and QEP
Implementer courses identified
by color coding
Sorted by course and by Instructor,
5 pages each
Spring 2011 Report
24
Encouraged customization
Very different from current
vendor
Can you do this? Usually “Yes!”
Site graphics made course evaluations
look like ATC
Enter IOTA Solutions, Inc
25
Even standard reports were a
significant step up
from previous vendor
Faculty views and
Dept. Chair views
impressive
What is standard and what
is custom?
Implemented Fall
2011
Overall went well,
but not all rosy
IOTA, continued
27
28
29
• All the information of the original report, plus response rates & comments link
• Automatically generated• Numerous interactive sorting, ranking, and
filtering features• Instantly look at the data in different ways• Link to comments instantly• Printable• QEP Reports
IOTA Custom Report
30
We are in the middle of a
journey
Going to a much more customized
reporting tool has its
downsides and risks
Bugs, Failures, Beta
Customer service may be
impacted by features not
working at all times
Customization
31
• Affordable• Prompt attentive customer service to correct problems
• Automated, customized reports with features already described
Positive Side
32
• Benefits clearly outweigh the costs
• For customized, automatic reporting features, more tolerance and patience are warranted to allow the development process to occur
Final Analysis:
33
• Moved from 16 questions on instruction to:• 5 Instructor questions• 4 Course questions, and a combination of
the following, based on delivery method• Lecture, Laboratory, Clinical Site, Online
Course, Hybrid Courses, Two-way Video Courses, Flexible / Open Lab, and Field Work
• 2 to 4 questions for each• Only the 5 instructor questions feed into
the FPMS and the Custom Report
New Development Spring 2013Questions Based on Delivery Method
34
Discussion
Discussion
Join us every Third Tuesday for more Professional Development opportunities.Next Session: August 20, 2013TIME 2:30pm – 4:30pmTITLE Preparing for the Fifth-Year Report and ReaffirmationPRESENTERS Dr. Claudette Williams, SACS-COC