A behind-the-scenes look at the suggestions enhancement process at Indiana University

About the presenters…

Anastasia S. Morrone, Associate Dean, Teaching and Learning IT, Office of the Vice President for IT and CIO, Indiana University and Associate Professor, School of Education, Indiana University Indianapolis

David A. Goodrum, Director, Teaching and Learning Technologies Centers (TLTC), Indiana University Bloomington

In this session, we will share lessons learned as we refined the suggestions enhancements process at Indiana University. We will provide an overview of the new process with an emphasis on how the functional requirements committee and the faculty priorities committee work closely together to set development priorities.

About the session …

How does your institution handle user suggestions?

Any volunteers…

How about your organization …

https://oncourse.iu.edu/access/content/user/ocadmin/developmentcl.htm

What is the suggestions enhancement process at IU?

The Oncourse Priorities Committee (OPC) is primarily comprised of faculty members who are actively engaged in teaching with Oncourse. The committee considers proposals for improvements, new features and tools. These proposals are further defined in collaboration with the Functional Requirements Committee. The OPC determines the priorities for Oncourse development.

https://oncourse.iu.edu/access/content/user/ocadmin/developmentcl.htm

Oncourse Priorities Committee

The Functional Requirements Committee (FRC) is comprised of academic, library and technology service providers as well as members of the development team who categorize and review suggestions. This committee initiates an iterative process with users, the Oncourse development team and the OPC to establish specific functional requirements for proposed improvements, new features and tools. At the conclusion of the committee's process, a report including development information and functional requirements is prepared for the OPC.

https://oncourse.iu.edu/access/content/user/ocadmin/developmentcl.htm#Pcomm

Functional Requirements Committee

Suggestions analysis process at IU

Support team responds immediately to support issues and bug reports

Remaining entries compiled monthly, forwarded to Functional Requirements Committee (FRC)

START: Suggestions, questions and comments entered by users

FRC members analyze suggestions for trends & update summary reports

FRC combines suggestions data, opt-out rationales, and other sources into a proposal to the Priorities Committee

Faculty Priorities Committee rank orders priorities for upcoming development

Excerpts from the FRC report

Report and Recommendations from the Functional Requirements Committee

Table of Contents Introduction......................................................................................... 1 Development Accomplishments......................................................... 1 Increase in User Adoption ................................................................. 2 Hurdles to Adoption ........................................................................... 2 What Do User Suggestions Indicate? ..................................... 3 What Do Instructor Opt-Out Rationales Indicate? .............. 3 The Issue of Site to Site Export – Import............................... 4 The Issue of Gradebook Functionality................................... 4 The Issue of Combining Sections ............................................ 5 The Issue of Group Awareness ............................................... 5 FRC Recommendations ..................................................................... 5 Proposed Items ......................................................................... 6 Priorities for Proposed and Existing Items............................ 6

Appendixes I. Decrease in Oncourse Opt-Out Requests............................... 8 II. Suggestions Analysis II.a. Executive Summary Analysis IUPUI ..................................... 9 II.b. Executive Summary Analysis Regional Campuses ............. 10 II.c. Executive Summary Analysis IUB Suggestions................... 11 III. Opt-Out Rationales III.a. IUPUI Analysis ....................................................................... 12 III.b. Regional Campus Analysis .................................................... 13 III.c. IUB Analysis ........................................................................... 14 III.d. Combined Analysis................................................................. 15 IV. CL Site to Site Export-Import Summary............................. 16 V. FRC Recommendations ......................................................... 17 VI. Example Projected Development Hour Estimates.............. 19

Increase in User Adoption Fall

2005 Spring 2006

Summer 2006

Fall 2006

Total Courses loaded* 24,049 23,235 10,345 24,743 Original Active 7,399 5,037 1,408 3,559 CL Active 3,014 3,338 1,234 5,435 Total Active 10,413 8,375 2,642 8,994 % Using CMS 43% 36% 26% 36% % Using original 71% 60% 53% 40% % using CL 29% 40% 47% 60%

Appendix IIc: Executive Summary Analysis for Bloomington Oncourse Suggestions The following are the most frequent suggestions from the past several months that are important to adoption of Oncourse CL on the Bloomington campus:

The ability to combine/redirect sections to a single site.

Discussion Forums in Message Center should be group aware.

In Gradebook, there are many suggestion items relating to providing comments and again other items relating to grading scales, how they are calculated and displayed, differences in what faculty see (point only) and the student sees (letter grade), weighted grades, option to enter letter grades instead of points, support of extra credit assignments and bonus points, showing a cumulative grade that does not include future assignments, ability to upload from Excel, and persistent requests for multiple gradebooks. Some faculty would like the option for students to see only points and not letter grade.

The Assignments tool needs a class-hand-in assignment and/or the ability to grade an assignment that has not been turned in electronically; faculty want to put in paper assignments, reading assignments… in fact they want to list all assignments in the assignment tool. Students might simply check that they have completed it. Also want to be able to grade a group assignment which gets posted to the gradebook.

[…]

The Issue of Gradebook Functionality

Capability Exists in Original

Oncourse

New Function-

ality Comments

Ability to drop low grade (or high grade) X

Adding bonus points or extra credit X

Enter a letter grade instead of a number X

Enter a percentage grade instead of points X

Support weighted grades X

Importing from Excel file X

include the ability to create multiple gradebooks; ability to create categories of grades that can be viewed separately

X X Compound feature building on Original functionality

include the ability to export Gradebook from one course to another

X

Selective assignment release: Include the ability to figure grades on a specific number of assignments instead of the whole semester to determine a current grade; i.e., show current grades and not just running grades through the end of the semester

X

Support commenting of grades X

Running Grades and Stats X

Bulk Assignment Creation X

Appendix IIId: Combined analysis of opt-out rationales for fall 2006 through Aug 30

issue category # of times%

missing function-ality

pref-erence

bad exper-ience

usa-bility

bad info other

gradebook functionality 155 15.4% 15.4%familiarity with original Oncourse 40 4.0% 4.0%redirect sections 95 9.5% 9.5%personal preference 96 9.6% 9.6%bad experience with CL 45 4.5% 4.5%ease of use/user friendly 118 11.8% 11.8%incomplete functionality 71 7.1% 7.1%discussion functionality 51 5.1% 5.1%Tests & Surveys 105 10.5% 10.5%dept/school recommendation 23 2.3% 2.3%problems encountered or heard 29 2.9% 2.9%group functionality 22 2.2% 2.2%mail functionality 5 0.5% 0.5%NA [no answer given] 60 6.0% 6.0%navigation 10 1.0% 1.0%readiness survey recommended 12 1.2% 1.2%roster 16 1.6% 1.6%import functionality 11 1.1% 1.1%schedule functionality 19 1.9% 1.9%time 4 0.4% 0.4%training 8 0.8% 0.8%assign proxy 4 0.4% 0.4%file access 4 0.4% 0.4%logistics 1 0.1% 0.1%

1004 43.4% 15.8% 7.4% 12.7% 12.5% 8.1%

FRC Recommendations (Message Center Example)

Tool Description Status Priority ref #

Message CenterSupport importing of Forums and Topics in Discussion Forums from another CL site; in addition, draft status should be optional

0. Proposed 1. Blocker 23

Message Center Ability to organize forums and topics in Discussion Forums 2. Awaiting Priority 1. Blocker 24Message Center Message Center group aware for Discussion Forums 2. Awaiting Priority 1. Blocker 25

Message CenterFrom the user perspective, view Private Messaging as a separate tool called Site Messages and view Discussion Forums as separate tool. [see Ancillary Materials C]

0. Proposed 2. Critical 26

Message CenterProvide instructor/site owner user statistics of # of postings and other statistics

0. Proposed 2. Critical 27

Message CenterIn Private Messages, show in message list who the message was sent to

0. Proposed 2. Critical 28

Message CenterDisplay statistics for user's postings and read messages in Discussion Forums

2. Awaiting Priority 2. Critical 29

Message Center Provide print friendly view for selected messages in Message Center 2. Awaiting Priority 2. Critical 30Message Center Ability to delete message threads in Message Center 2. Awaiting Priority 31Message Center Ability to move forum messages to different topics in Message Center 2. Awaiting Priority 32

Message CenterForum activity notifications to external email addresses in Message Center

2. Awaiting Priority 33

Message Center Insert original text in forum message in Message Center 2. Awaiting Priority 34Message Center Keyword search across forums / topics in Message Center 2. Awaiting Priority 35Message Center Moderator functionality in Message Center 2. Awaiting Priority 36Message Center & My Workspace

Show indication of awaiting messages and forum postings across sites in My Workspace

0. Proposed 3. Major 37

The FRC sends their report to the OPC one week prior to the retreat.

The retreat is an all day face-to-face meeting that takes place at the beginning of the fall and spring semesters.

At the retreat, the OPC discusses the FRC’s recommendations and votes on the development priorities for the December and May releases of Oncourse.

Refining our process

1. Creating a new tool 2. Refining an existing tool 3. Collaborating with another

institution on tool development

Examples of tool development process

Today’s Announcement @ IUSpring 2007 Oncourse CL enhancements• Starting January 1, significant enhancements in

Oncourse CL:• Gradebook: commenting on grades.• Message Center: importing forums and topics;

organizing forums; & group awareness• Roster: Students allowing themselves to be viewable,

per site, to other students.• Sectioning: combining course sections into one site (i.e.,

create a combined roster). • Library Resources: now a default tool for course sites