Rich Media Library Project Enriching the Rich Media Library: An analysis of the process of populating the Rich

Media Library with class lectures and recommendations for improving this process.

December 14, 2007

Written for:

Vic Divecha, E-Learning Specialist

Phone: (734) 763-4355

Email: rdivecha@umich.edu

Written by:

Rockin’ Media Librarians

Adrienne Leo, Client Contact

Rachael Dreyer, Project Coordinator

Ryan Lankton, Project Manager

Nathan Oostendorp, Project Secretary

Saurabh Koparkar, Faculty Liaison

Key contact information:

Adrienne Leo

Phone: (571) 262-1121

Email: amleo@umich.edu

2

Table of Contents

Executive Summary...................................................................................................................................3

Introduction .................................................................................................................................................4

Rich Media Library.................................................................................................................................4

Project Scope .........................................................................................................................................4

Methodology ...............................................................................................................................................4

Data Collection .......................................................................................................................................4

Data Analysis .........................................................................................................................................5

Findings and Recommendations .............................................................................................................5

Finding 1: The Rich Media Library is difficult to browse and is not well publicized so it remains largely unknown and unused. ..............................................................................................................6

Finding 2: The workflow for recording and editing lectures is constrained by a lack of formal procedures and tight timelines.............................................................................................................8

Finding 3: There is almost no formal training for new staff so they must rely on knowledge and skills acquired elsewhere and on the expertise of the E-Learning Specialist, when he is available. .................................................................................................................................................9

Summary...................................................................................................................................................10

Appendices Introduction .........................................................................................................................12

Appendix A: Consolidated Communication Model..........................................................................13

Appendix B: Organizational Cultural Assessment ..........................................................................14

3

Executive Summary Over the past three months we executed an in-depth consultation on the Rich Media Library at the School of Public Health Library and Informatics at the University of Michigan. In that time we evaluated the existing RML repository, interviewed six candidates who were involved in different stages of the lecture production and distribution process. After each interview we performed interpretation sessions including comprehensive modeling of communication, procedures, salient facts, and physical spaces. Ultimately we consolidated models from the multiple interviews to create a comprehensive picture of the Rich Media Library and the processes involved in its production, it is from this that we present our major findings. With these findings in mind we used design ideas and research to come up with several recommendations to improve the library and its development processes. Based on our interviews and models, we made the following major conclusions: Finding 1: The Rich Media Library is not actively maintained, not easy to browse, and is not being used by students. Finding 2: The workflow surrounding lecture recording and production is informally organized and highly constrained by tight timelines. Finding 3: Training of student Recording/Editing Assistants is done entirely with the E-learning Specialists expertise and requires prior experience in similar tasks. Based on these findings, our research, and our own design ideas we make the following recommendations: Recommendation 1: Post more information associated with lectures in the RML, such as transcripts and lecture slides. Make this data searchable, and consider using wiki software to encourage user participation. Recommendation 2: Create more formal procedures for recording and editing by organizing the lab space into an assembly line for lecture production. Enhance any existing documentation by posting checklists of each procedure on the lab walls. Recommendation 3: Encourage more training among student assistants by compiling questions into a FAQ-style manual and having students shadow each other. If possible, recruit another full time position to augment the E-learning specialist in his duties. The full report contains details for these findings and recommendations, and an appendix with some supplemental materials. We hope that you find this report and the recommendations useful for improving the Rich Media Library. Sincerely, The RML Team

4

Introduction

Rich Media Library The Rich Media Library is an online repository of class lectures recorded by the School of Public Health. The Rich Media Library is managed by the Public Health Library & Informatics, which provides library and informatics services to the school and the greater campus community. This repository supports the On-Job, On-Campus (OJOC) Executive Education Program at the School of Public Health, as well as the School's on-campus component.

Project Scope The goal of the Rich Media Library Project is to understand the process of lecture recording and editing, storing lectures on the RML, and making them available to students of the School of Public Health. It is also to recommend ways of streamlining the process of producing and releasing lectures.

Methodology

Data Collection We collected data from the following sources:

• An initial client meeting. • Six in-depth interviews, along with follow up questions to those interviews. • Browsing the RML and viewing lectures.

Initial client meeting: Our initial client meeting with the E-Learning Specialist provided an excellent overview of the structure of the RML and the process of populating it with media. At this meeting, we also received a list of key staff and faculty to interview for data collection, as well as the client's desired objectives for the project. In-depth staff interviews: We conducted six interviews in total. These interviews included four staff members and two faculty members. We obtained a bulk of our data for analysis from these interviews. Interviews took approximately one hour to complete and were performed with a primary interviewer and a note taker who recorded the content of the interview. After each interview, our group met for several hours to debrief from the interviews. During this debriefing session, we generated Affinity Notes for the Affinity Diagram that we would later use to organize our information into themes for further analysis. In addition to Affinity Notes, we also created models of the organization's workflow, communication patterns, workspace organization, and culture. In addition, the note-taker and interviewer for each interview shared digital photos that would later be collated into artifact models.

5

Informal investigation of the RML: Prior to conducting our interviews, we spent some time investigating the media stored in the RML. This allowed us to experience the resource from the perspective of its primary target audience: non-expert, student users.

Data Analysis From the data we collected, we generated almost 200 notes for our Affinity Diagram and created six consolidated work models. We compiled our notes into the Affinity Diagram over two 4 hour sessions. Using the Affinity Diagram and the work models, we engaged in a protracted process of analyzing the process by which lectures are recorded, edited, archived on the RML, and distributed to students. At the end of our Affinity Diagramming sessions, we identified three areas that could be targeted for modification:

1. The functionality of the RML in its current state 2. Media production workflow 3. RML staff training

Findings and Recommendations The Rich Media Library is a valuable and visionary resource for the School of Public Health. It serves as a repository of lectures from the School of Public Heath that can be used by students, on and off campus, to enhance their educational experience. It is a major goal of the RML to be well archived and easy for students to search, as well as easy to populate and maintain. However, it currently suffers from a number of technical and practical breakdowns that keep the project from fulfilling its true potential. Through our analysis, we have identified three major areas open for improvement. We have organized our findings and recommendations around these key opportunities for improvement, supplying evidence for each our findings. In addition, we have presented the benefits that we expect will result from implementing our recommendations. Finally, we have selected our recommendations with an eye to keeping the cost and effort of their implementation low. We have categorized these recommendations as long-term or short-term based on the time and effort we think would be required to implement.

6

Finding 1: The Rich Media Library is difficult to browse and is not well publicized so it remains largely unknown and unused. Finding 1-1: The RML taxonomy is insufficiently robust or developed to support its planned use. RML media is inadequately identified, making it difficult for students browse or search for lectures. Evidence: The E-Learning Specialist indicated during our initial interview that there is no formal taxonomy in place for organizing, identifying, or archiving the media resident in the RML. Correcting this problem has long been one of his goals. As one student staffer observed, the RML’s identification of existing materials is “too ambiguous to be beneficial” to student users. Finding 1-2: Although the RML has 165 lectures available for viewing, it is not well publicized because of concerns about the potential for copyright infringement. There is no policy in place governing the lectures available to students through the RML. Evidence: We were told by the founder of the RML that it is not widely publicized because there was some concern that it infringes on rights of some professors, especially those who have not clearly given their consent to having their lectures made available through the RML. The concern expressed to us was that this exposes the School of Public Health to risk of lawsuit. We were also asked not to publicize the existence of the RML. Finding 1-3: Lectures are being made available to students via other channels instead of through the RML. Evidence: During our interviews, a faculty member reported making lectures available to students on DVDs rather than using online copies. One of the student staff identified Ctools and iTunesU as the primary online distribution channels. Recommendations for findings 1-1 and 1-3: (A) Post more information with lectures in the RML. • Ideally, provide transcripts for each lecture. (Long term) • At least, post the original PowerPoint slides from the faculty for the lecture. (Short term) Providing a transcript can be done for $2 - $6 per page using an online transcription service such as Escriptionist.com. Posting the original PowerPoint slides for a lecture is free. Both would require a minimal investment of time, but would improve the user experience by allowing searching and printing of lecture materials. (B) Make use of a MediaWiki and its robust set of features to give greater functionality to the current and future users of the RML. (Long term) Making use of a MediaWiki will require a greater investment of time and effort than merely posting more information with lectures, but it will also provide a great deal more functionality. Extensions such as

7

MetaVid allow integrated streaming video, and online discussions of lectures could use the software’s forum capabilities. Benefits: PowerPoint slides allow users to access the contents of a lecture without having to view the lecture in real time. This makes it possible for students to locate relevant lectures quickly. The text from PowerPoint slides could also be extracted and used in searching. Transcripts offer this benefit as well, but they provide more content and allow full-text searching of lecture content. A MediaWiki provides the following functionality: • Users can tag and add comments to lectures. • Users can utilize full-text searching to locate relevant materials. • Staff can make video available through Metavid extensions. • Staff or faculty can easily revise material imbedded in the wiki. Full-text searching, especially when coupled with lecture transcripts, makes locating material easier. It also reduces, if not eliminates, the need for developing and maintaining a complex taxonomy. Tagging and user comments allow students to collaborate and discuss lectures. This potentially enhances the educational benefits of the lectures. Because the software tools employed by a MediaWiki are very likely to be familiar to users and staff from the popular Wikipedia website, it will likely be easy for students and staff to use and maintain the wiki. MediaWiki also has revision and tombstoning capability which was indicated by staff to be a desirable future function of the RML. Recommendations for findings 1-2 and 1-3: (A) The Office of General Counsel at University of Michigan should be consulted immediately about the copyright infringement concerns. (Short term) (B) The RML staff should take stock of the modules residing on the RML and in accordance with advice from the Office of General Counsel determine what permissions must be sought before it is made available to students through the RML. (Short term) (C) Proper policies and procedures should be implemented to bring the RML in compliance with the laws governing copyright. Benefits: Resolving the copyright infringement concerns will allow the RML to be safely publicized. When coupled with our recommendations above, this will make it possible to utilize this valuable resource as envisioned.

8

Finding 2: The workflow for recording and editing lectures is constrained by a lack of formal procedures and tight timelines. Finding 2-1: The existing processes for recording and editing lectures are not fully proceduralized. Evidence: Despite the problems and constraints discussed in the findings below, the lectures are successfully recorded, edited and even distributed for the School of Public Health. In fact, there are over 160 lecture modules in the RML. None of this could be accomplished without processes in place for recording and editing lectures. Finding 2-2: Lack of formal procedures for the use, storage, and maintenance of equipment gives rise to various problems associated with equipment. Evidence: It was reported to us that, when the recording equipment is not returned to the primary storage room, the student in charge of recording the next lecture has to track down the equipment. It was also noted that when batteries need to be replaced before a recording session, the students have to return to the lab where the replacements are kept. Another employee indicated that it is not uncommon for the equipment to fail. Such equipment failures tend to affect the recording resulting in issues like improper audio levels of the lecture (see Appendix A). Finding 2-3: Faculty occasionally provide insufficient notice of their desire to have a series of lectures recorded. Evidence: Although faculty typically notify their desire to record lectures a semester before they need a lecture series recorded, there are times when they make their requests at the “last minute.” All the interviewees reported such circumstances (see Appendix A). Finding 2-4: It is not uncommon for RML staff to receive a very short window of time to record, process, and distribute a lecture to students. Evidence: The E-Learning Specialist reported that some lectures have to be recorded, edited and distributed quickly to accommodate the needs of the faculty and their students. Finding 2-5: The E-Learning Specialist is central to the entire lecture production process and is solely responsible for distributing work to RML staff. Evidence: All the student employees report directly to the E-Learning Specialist. They said that when they complete a project they ask the E-Learning Specialist for a new project because he is responsible for

9

setting their schedule and he knows which projects have priority. In the E-learning Specialist’s absence, the students devote their time to the completion of the extensive backlog of old lectures. Recommendations for findings 2-1, 2-2, 2-3, 2-4, and 2-5: (A) Make the lab an “assembly line” by arranging the physical space to represent the lecture recording and editing workflow. (B) Post checklists in appropriate places within the newly arranged lab that allow RML staff to track their progress for the entire workflow process whenever required. (C) Store lectures that need to be edited in a queue which are sorted and grouped by completion deadline. (D) Standardize lab storage space for lecture tapes, and create a method for quickly and clearly indicating to RML staff whether a given batch of tapes is being actively edited. (E) The student lab assistants should be guided effectively so that they do not have to depend on the E-learning specialist for directions from time to time. Benefits: Standardization and proceduralization of the lecture production workflow will make it more efficient. Including best practices described above from the recommendations will take care of lecture production scheduling issues and equipment management issues.

Finding 3: There is almost no formal training for new staff so they must rely on knowledge and skills acquired elsewhere and on the expertise of the E-Learning Specialist, when he is available. Finding 3-1: There is no formal training, so student employees learn primarily on-the-job. Students that are hired generally have prior expertise in video editing and/or recording. Evidence: During our interviews, student employees indicated that they did not receive any training beyond a brief introduction to Camtasia. One said that he felt like he was “thrown in” to the work. Another believes he was hired because he already possessed the skills he would need for the job. Finding 3-2: The turnover rate for student employees is high. Evidence: The E-Learning Specialist reported a rapid rate of turnover for student staff. As students, they generally only stay in their position about a year or two. Finding 3-2: When student employees leave, the valuable skills and knowledge they developed on-the-job are lost.

10

Evidence: The students indicated that, while they discuss best practices and share insights about their work informally, there is no regular or formal means of doing so (see Appendix B). Finding 3-4: The E-Learning Specialist is personally responsible for training all student employees. Evidence: During our interviews, the E-Learning Specialist confirmed that he was responsible for training all the students working in the lab. Recommendations for findings 3-1, 3-2, 3-3 and 3-4: (A) Have newly hired student employees “shadow” the more experienced student employees as part of their training. (Short term) (B) Post more instructions, or procedural checklists, on the lab walls. (Short term) (C) Develop a wiki or some form of repository to record questions posed by student employees about their work and the answers to those questions. Make this FAQ available to all staff and update it as necessary. (Short term) Benefits: By externalizing and formalizing the training process, the E-Learning Specialist will save time on training students, and have more time for his other important responsibilities. The FAQ and the process of shadowing allow current student employees to pass on their valuable skills and knowledge to future employees. Recommendations for findings 3-2 and 3-3: (A) Hire an Audio-Video specialist (long term) Benefits: Since the E-Learning Specialist is a vital element in all steps of the RML process, it would be worthwhile to have another person to share his responsibilities. A full-time Audio-Visual specialist would have the skills to be able to assist in editing and recording, and they can also be responsible for administrative things, such as reminding faculty to submit recording requests, etc. In other words, a person in this position would be devoted to all steps in the process, which the student staff and the E-Learning Specialist do not have the time to be.

Summary It is clear that the RML staff work hard to meet the faculty and students’ needs for making lectures available online. We have several suggestions that could help make this process easier, and the end results more effective.

• To make the RML in shape for publicity:

o Post more information with the lectures on the RML (transcripts and/or lecture slides); consider incorporating a MediaWiki to extend greater functionality to the RML.

11

o Confer with the UM Office of General Counsel regarding copyright infringement terms and how they apply to the existing and future RML modules.

• To encourage efficiency in the workflow process:

o Arrange the physical space of the editing lab as necessary so work may be conducted in an “assembly line” procedure.

o Post checklists around the editing lab so staff can track workflow.

o Organize lectures to be editing according to deadline, and standardize storage of lecture tapes so they may be retrieved quickly when necessary.

• To relieve the training responsibilities of the E-Learning specialist:

o Have new student staff “shadow” the experienced student staff as part of their training.

o Post more instructions for common procedures on the wall, and create an online FAQ-style manual where staff can ask and answer questions about work procedures.

o Hire a full-time Audio-Visual specialist who can be involved in all steps of the workflow process.

12

Appendices Introduction We have included two appendices that we thought would be useful for the SPH’s Rich Media Library. In Appendix A we have shown a Consolidated Communication Flow Model. This model shows an abstract view of the flow of communication between the people working for the Rich Media Library. The circles denote the people who are communicating while the arrows depict the flow of communication between the people. The rectangles represent the artifacts generated or used during communication. The clouds containing red colored thunderbolts represent the breakdowns that occur during communication. In Appendix B we have included a Cultural Assessment of the organization. This assessment contains a written narrative of evidence of ten dimensions across which the culture of the organization is evaluated based on the contextual inquiry interviews which we conducted.

13

Appendix A: Consolidated Communication Model

14

Appendix B: Organizational Cultural Assessment

Member Identity

Job |---------X--------------------------------------------------------| Organization

The people most involved in the processes behind the Public Health Library and Informatics Rich Media Library, besides U5, are all part-time student workers (U4 and U6) who don’t work at the same times as each other, and who don’t work there for long, because they leave after they graduate. There is little formal organizational structure, as staff members determine their own schedules, and are not required to follow a strict procedure when performing their duties. The staff members are hired for the skills they already have, and are given little training in addition to that.

Group Emphasis

Individual |---X--------------------------------------------------------------| Group

As stated above, student workers work independently. They coordinate over email with U5 and sometimes U7 regarding what needs to be done. U6 has contact with U2 only when the latter contacts him to cancel a recording session. U9 is only involved in process at all when website needs fixing. U1 has had little involvement in the process since he launched it.

People Focus

Task |-------X-------------------------------------------------X---------| People

U2 U1 toward U2

U2 are more clients than management, but their decisions can carry a lot of impact, for example when they realize very late that they need a lecture to be recorded, or one that is already recorded to be produced, it causes U4, U5, and U6 to do a lot of last-minute work. When U2 decides not to rerecord a faulty recording, it affects the quality of material used by U3. As far as formal management decisions go, U1 has decided not to promote RML out of concern for their intellectual property rights.

15

Unit Integration

Independent |----------X-------------------------------------------------------| Interdependent

All units operate relatively independently. U4 and U6 coordinate with U5 regarding what needs to be done, but then complete it on their own schedules. The only contact between

U6 and U2 is when there is an exception to the regular procedure, such as when a class doesn’t have to be recorded because there is a test, etc. U9 is only contacted when there is a problem to be fixed with the website. Besides having his own lectures recorded, U1 has had little involvement in the process since he conceived of it.

Control

Loose |-----X------------------------------------------------------------| Tight

There is minimal managerial control on how the recording and editing processes are completed. U5 allows lab/recording assistants to determine their own schedules, as long as lectures are recorded, edited and posted by the time students need them. U4 does not need U5 to approve work before posting material to the server. There is no formal procedure for U4 to follow except file-naming protocol, which is posted on walls in editing lab. In lab, there is ad hoc organizational schema, such as tapes to be processed kept in piles in cardboard boxes.

Risk Tolerance

Low |-------------------------------------------X----------------------| High

As U4 says, “Any school that does this is testing the waters,” implying that the entire concept of having lectures available for view online is new and that the processes behind it are still in their formative stages. U5’s relaxed managerial style supports this – the only enforced protocol is the file-naming system – the staff are free to complete the rest of their tasks in their own ways.

Reward Criteria

16

Performance |-----------------------------------------------------------------| Other ‘’’

The people involved in the RML processes cannot be rated on this characteristic, because there is no evidence that any of them are rewarded by such things as salary increases or promotions. U7 was promoted, but there is no evidence that it has anything to do with her involvement in RML.

Conflict Tolerance

Low |----------------------------------X-------------------------------| High

U4 has a few suggestions on how to improve the accessibility of materials for U3, but unclear whether he has expressed these to U5. Staff members appear to have a comfortable relationship with U5.

Means-Ends Orientation

Means |--------------------------------------------X---------------------| Ends

There is no formal procedure to follow when recording or editing lectures, but staff strives to have quality work. Editing is a very time-consuming process – about 2 hours for every 1 hour of lecture. U4 and U5 informs U2 when there is a problem with a recording so that they may come in to re-record it, but U2 doesn’t always come in to re-record. Some lectures must be produced quickly because they must be available to U3 soon.

Open-System Focus

Internal |-----------------------------------------------------------X------| External

The RML project was created out of an acknowledgment of changes in the external environment, in the regard that it would be helpful for students to listen to lectures online and not just in the classroom. It now is developing into a tool for a distance-learning program, OJOC (On Job On Campus).

17

This assessment of the relationships among the people involved in the UM Public Health Library & Informatics’ Rich Media Library reveals that the culture of this organization, while casual, is not very intimate. The managers, staff members, and clients (faculty) only communicate when necessary. The only face-to-face contact occurs when staff or faculty needs to be trained to use equipment, or when a lecture is being recorded. The staff members (excluding U5, who serves as manager) are mostly college students who work part-time and don’t stay at the job long, and so there is little opportunity for a sense of attachment to develop among the members of the organization. However, the minimal communication among the staff members is not problematic. What appears to be a difficulty is the distant relationship between the faculty and the RML staff members. The faculty, because they are further removed from the processes that go into recording and editing, do not understand the how their actions impact the entire procedure. If there were a closer relationship between the faculty and the RML staff, perhaps the faculty would be more aware of the importance of timely scheduling of recordings, and then, in turn, there would be a greater opportunity to do any necessary re-recording and more thorough editing of the recordings.