nurturing insight to assessing online teaching

WCET 19th Annual Conference Nov. 7 – 10, 2007

Nurturing Insight to Assessing Online Teaching

Romana Hughes

Texas Christian University

History

Online Graduate Programs 1999 eLearning 2001 Koehler Center for Teaching Excellence 2002 Standards and Procedures Document (2003)

Standards and Procedures for Online Courses at TCU

Written fall 2002 Approved by Provost Council April 2003

Requires an online student evaluation Launched spring semester 2004

Requires assessment of instructional design First attempt launched spring semester 2005 Not adopted by Nursing faculty – too much work

Requires online faculty to attend training Requires departmental review of all assessment to

make recommendations for continued quality and improvement

Faculty Requirements for online assessment of courses

Must be a simple format Must be one page Must be online Must have a report tool Must be easy to learn and available for peer

review

Online Self Assessment Tool – OSATCreated by Clarkson College

Systematically review courses and determine what areas of instructional design could be improved

Develop clearer perspective on course design To easily identify areas of strength and

weaknesses for future development Four level rubric

Copy of TCU OSAT is found at back of handout.

Pilot Spring 07

Used Clarkson’s OSAT to assess the spring 2006 MLA online courses

Modified the layout to fit the needs of TCU policies and procedures

Faculty approved tool April 2007

Where is OSAT now?

August 2007 completely online in TCU Portal Available for peer review and personal review

of online courses Reporting tool created September 2007 October 2007 Incorporated Program

Intelligence Manager (eCollege tool)– PIM - results with OSAT results

Course Home/Syllabus SectionOSAT online

Content Levels SectionOSAT Online

Presentation Levels SectionOSAT Online

Current Report Options

• By College

• By Date

• By Term

Example of One Report

4067 NURS 20033 070 45 37 9/10/2007 3.00 2.22 2.75 2.664067 NURS 60043 479 10 10 9/10/2007 2.56 2.11 2.00 2.224073 NURS 40632 045 60 56 9/10/2007 2.44 2.00 2.00 2.154073 NURS 20033 055 60 48 9/10/2007 2.56 2.00 2.00 2.194073 NURS 40882 104 9 9 9/10/2007 2.00 2.00 2.00 2.004075 NURS 60713 479 1 1 9/10/2007 2.56 2.00 2.25 2.274077 NURS 60713 479 5 2 9/10/2007 2.56 2.56 2.25 2.464075 NURS 60443 479 15 6 9/17/2007 2.56 2.22 3.00 2.59TermCourse Number Section Max Enr Act Enr Date Comp S1 Tot S2 Tot S3 Tot Score

Flags to watch - Score below 2.0

Program Intelligence ManagerPIM

PIM enables data-driven decision-making to Assist in more successful student outcomes

and better institutional performance Allows schools to determine what factors

contribute to the success (or lack of success) of their online courses

To improve and maintain quality course offerings online

Program Intelligence Manager – PIMReport Example

Course Completion Rates: Course Completion Rate Analysis - eCourse Terms

Fall 2006 eCourse Dev101 Course Start to Census Completion Rate

Census to Course End Completion Rate

Course Start to Course End Completion Rate

Fall 2006 eCourse MALA61173 93.33% 100.00% 93.33%




Fall 2006 eCourse NURS30543 100.00% 80.95% 83.33%









Flags to watch – retention below 87%Set by department

Faculty Development

OSAT Score should be 2.0 or greater PIM retention target is 87% Deans and/or Department Chairs review

reports with Koehler staff Faculty Development plans are created with

Dean and/or Department Chair and faculty for scores that are lower than set standard

Faculty sign up for workshops based on FD plan or request one-to-one training

Contacts

Romana Hughes

[email protected]

www.cte.tcu.edu

817-257-6631

nurturing insight to assessing online teaching

Education