rubric for improving the quality of online courses t lt april27 09
DESCRIPTION
"Rubric for Improving the Quality of Online Courses" presentation given at TLt 2009 conference April 27th in Regina, SK, CA.TRANSCRIPT
RUBRIC FOR IMPROVING THE QUALITY OF ONLINE COURSES
Denise Nelson, Virtual Campus
2
OUTCOMES
Share strategies used to measure the quality of online courses.
Discuss the impact of a quality assurance system on institutional operations and stakeholders.
3
PLAN
SIAST Context & background Rubrics...rubrics...rubrics...
Adaptation Use
Discuss quality assurance processes & results
Future plans?
4
VIRTUAL CAMPUS REVIEW
Development and Delivery on Online Courses
5
VIRTUAL CAMPUS REVIEW
External review of online courses Stakeholder focus groups Email interviews Web survey mySIAST – Intranet call for feedback
6
RUBRICS...
Used existing rubrics as guides to develop online courses CNIE Criteria for
Award of Excellence in Instructional Design BlackBoard Exemplary Course Rubric Blood-Siegfried et al.
A Rubric for Improving the Quality of Online Courses
Blood-Siegfried et al. adapted to SIAST Context
7
ANALYSIS OF EXISTING RUBRICS RELEVANT FOR ONLINE COURSE
1. California State University, Chico - What does a high quality online course look like?http://www.csuchico.edu/tlp/resources/rubric/rubric.pdf (Committee for Online Instruction, 2003)
2. Towson University, Maryland - Towson Faculty Peer Review of Online Courseshttp://www.towson.edu/~mcmahon/peerreview/On-linerubric.pdf (Ashcraft, McMahon, Lesh, & Tabrizi, 2003)
3. University of West Georgia - Assessing Interactive Qualities of Online Courseshttp://www.westga.edu/~distance/roblyer32.html (M D Roblyer & Ekhaml, 2000; M. D. Roblyer & Wiencke, 2003)
4. Grant McEwan College - Criteria for Evaluating the Quality of Online Courses (Wright, 2004)http://elearning.typepad.com/thelearnedman/ID/evaluatingcourses.pdf777
5. University of Wisconsinhttp://www.uwex.edu/disted/conference/Resource_library/handouts/Forum1.pdf (Swan, Bishop, Wisher, & Trollip, 2003)
6. Robert Wood Johnson Partnerships for Traininghttp://pftweb.org/reach/reach_home_frameset.htm (Partnership for training, 2003)
7. Measuring Success: Evaluation Strategies for Distance Educationhttp://www.educause.edu/ir/library/pdf/eqm0213.pdf (Lockee, Moore, & Burton, 2002)
8
BLOOD-SIEGFRIED ET AL. RUBRIC CRITERIA
1. Organization and Design 2. Course Content3. Instruction4. Interaction5. Evaluation and Assessment6. Student Support Services - added
9
USE OF RUBRIC FOR EXTERNAL COURSE REVIEW
Adapted rubric Obtained course context information
nature of target audience design specifications student/instructor interactions pace development and delivery schedule
Faculty team available to respond to inquiries Provided assessment items and related data
not housed with course (e.g., examinations)
10
DISTANCE COLLABORATION
11
PROCESS AND DELIVERABLES OF THE REVIEW
Reviewer received documentation & access Context and rubric documentation Access to Master version of online course Access to archived online course (demonstrates
interaction, grades) Reviewer conducted review within ~2
weeks Reviewer emailed completed review
documentation Reviewer and stakeholders held debriefing
mtg
12
EXTERNAL REVIEWER COMMENTS ABOUT USE OF RUBRIC
Lori AtkinsonInstructional Designer
13
CHOICES...
14
RATE THE USEFULNESS OF THE RUBRIC FOR COURSE EVALUATION?
Highly useful The criterion creates a checklist that will
demonstrate whether the online course provides the best shot at a quality student experience.
Overall the 7-13 objectives within each criteria captured the major areas of best practice for online learning.
I would recommend that the rubric is used for the basis of a course evaluation or course development tool. (Lori Atkinson, 2009)
15
RATE THE USEFULNESS OF THE RUBRIC TO INFORM ONLINE COURSE DEVELOPMENT?
Highly useful for instructor mediated courses
This rubric should be adapted for self-paced and blended courses to reflect different student to content (and media), student to student, and student to instructor interaction strategies. (Lori Atkinson, 2009)
16
WHAT ARE THE STRENGTHS OF THE RUBRIC?
Well grounded in research, reflected in structure and content (Lori Atkinson, 2009)
17
WHAT ARE THE LIMITATIONS OF THE RUBRIC?
Requires an additional document to synthesize major themes for improvement(Lori Atkinson, 2009)
18
ARE THERE OTHER CRITERIA/RUBRICS THAT SHOULD BE INCORPORATED?
A section that provides evaluation of the tie between the outcomes, instruction and assessment. This would help to demonstrate areas of the
course that are content heavy and not really related to the course outcomes.
As a development tool this would be helpful to assess what technology intervention is really a match for the learning outcome. (Lori Atkinson, 2009)
Learning Outcome
Learning Activities
Assessments
Write resume Practice writing resume
Write resumeCritique resume
19
LORI ATKINSON, EXTERNAL REVIEWER
20
RESULTS OF USING RUBRIC
Identify professional development needs Instructional design process Alignment of outcomes, learning
activities, and assessment Evidence for resource allocation Identify important questions (e.g.,
Should this course be instructor led, self-paced, blended?)
Improve learning experience Review debrief facilitates connection
with other SIAST teams and examples of high quality SIAST courses
21
DISCUSSION AND FUTURE PLANS?
Professional development to increase awareness of benefits of using rubric to guide development and evaluation
Dialogue re improvements needed in course development processes (e.g., dealing with heavy course content; alignment)
...
22
CONTACT
Denise NelsonInstructional DesignerSIAST Virtual Campus
Phone: 306-775-7683Email: [email protected] available at
http://programs.siast.sk.ca/nelson/#presentations