1

JISC Grant Funding 18/11 Cover Sheet for Bids (All sections must be completed)

Name of JISC Initiative: HEA/JISC OER Programme: Rapid Innovation SECTION ONE Name of Lead Institution: The Open University

Name of Proposed Project:

Track OER: Tracking Open Educational Resources

Name(s) of Project Partners(s) Full Contact Details for Primary Contact: Name: Patrick McAndrew Position: Professor of Open Education Email: patrick.mcandrew@open.ac.uk Tel: 01908 652638 Address: The Open University, Institute of Educational Technology, Milton Keynes. MK7 6AA. Length of Project: 6 months (no longer than 6 months) Project Start Date: 1 April 2012 (no earlier than Monday 19th March)

Project End Date: 19 October 2012 (no later than Friday 19th October)

2

SECTION THREE

1. Outline Project Description 1. Open Educational Resources are released with permission to transfer to other contexts

and use, however often they are placed on a server that provides them with a “home” location. While the permission typically includes the rights to place on other servers and make changes, the need to provide an identified place for the content can mitigate against that transfer and indirectly inhibit reuse. Reasons for wanting to retain courses in one place include the wish to gather use data, build a critical mass of users around the course and to bring any changed versions back into view. These need to be balanced with the users potential wish to place them on local servers, manage groups of students and organise cohorts studying together, and the overheads inevitably involved in sharing back versions. The project aims to look at two ways to reduce tensions between keeping OER in one place and OER spreading and transferring. If we can find out more about where OER is being used then we can continue to gather the information that is needed and help exploit the openness of OER.

2. An important business reason is also emerging from the enhanced brand awareness that is one value of providing OER. However when content transfers the impact of the content may be reduced, and certainly the ability to measure that impact is limited. Knowing how much content does in fact go to other servers is an important measure to be able to report and understand if there is an actual issue. On the other hand if the content can be tracked information can be collated impact can be retained and help make the case for sustainable OER.

3. The action of the project will be to develop software that can help track open educational resources. This provides an enabling function to see the impact of releasing content as distinct from serving content without the option to take and remix. The software will be generic in nature and build from existing work developed by BCCampus and MIT, however a key step in this project is to provide an instantiation of the tracking on the Open University’s OpenLearn platform. The aim will be to understand the profile of use of materials from the OU’s OpenLearn and LabSpace systems. The two software solutions will be to:

a. Track content that is downloaded from OpenLearn and the uploaded elsewhere; b. Add tracking information to cut-and-paste extraction of materials from

OpenLearn. 4. The solution will build on earlier work, notably by OLnet fellow Scott Leslie (BCCampus)

and JISC project CaPRéT led by Brandon Muramatsu (MIT project partner in B2S).

3

2. Use Case 5. Summary: This use case considers the inhibitions and restrictions on OER projects and

the associated impact on the educator community from restricted tracking data.

6. Context: In the Bridge to Success (B2S.aacc.edu) project materials have been generated that take OER from the Open University and make them available as reworked OER with a focus on use in US Community Colleges. The content is released in OpenLearn’s LabSpace. This material is available for use on other systems under the CC-BY permissive form of the Creative Commons licence. Transfer to other servers is permitted and supported by the release of downloadable content packages and clear messages that material can be copied and reused as permitted by the CC-BY licence.

7. Requirements: The project is funded by the Next Generation Learning Challenge (nextgenlearning.org) with requirements that we research use in identified pilots and reach targets about the use and impact of B2S content. Some pilots need to manage their registered students through the content and would like to provide access to B2S content from institutional VLE. We also are actively investigating other platforms such as Peer-to-peer University, iTunesU and OERGlue.

8. Existing approach: Information on use is available as long as content remains on the LabSpace system through analytics and support for groups in the Learning Club. If content transfers we may then have no data about the use of the content and be unable to learn more about the use or report back to our funder.

9. Solution: The Track OER system provides information back to the originator of the content to show the content that is reused and the location in which it now operates. The scale of reuse can now be monitored and further information sought if needed.

10. Success criteria: A programme of testing will demonstrate the capability to track and provide accurate data into the system. Impact will be assessed during the period of the project through use of the tracking data within existing reporting of OpenLearn performance. Further benefits are expected for example identification of reuse cases and interfaces that reveal where alternative content can be found, these may occur after the project.

11. Community: The need for a tracking system appears repeatedly in OER projects and providing sites, particularly as it is common to need to show impact to funders and institutions. For the educator community the challenge to correctly attribute and meet the intent in sharing content is enhanced. We will specifically explore this with the Community College educator community. For the learner community the benefits will come from more flexible use of content and the potential to use the data provided to connect groups learning in different online spaces.

12. Engagement: the idea for tracking has been considered for some time, including in the context of JISC CETIS, OLnet and work by MIT. In addition Creative Commons have identified the need for tracking as important in extending use of the CC licence for Museums and Art works and are interested in this activity and how it could work with the embedded licence.

13. This use case is released under CC-BY-SA.

4

3. Proposal 14. In the JISC Rapid Innovations Call OLnet’s Key Challenges...( http://olnet.org/node/639)

are picked out as a list of issues for OER practitioners. Taking the form of exploratory questions one of these “challenges” is: “What evidence is there of Use (and Re-Use) of OER?” The pertinence of this issue, and the OER movement’s need to engage more fully with this question, was similarly highlighted recently on the JISC CETIS blog (http://blogs.cetis.ac.uk/lmc/2011/11/24/ukoer-3-technical-reflections/).

15. Central to OER is the ability to take and remix content provided by others. As recognised in the Call and elsewhere it remains difficult to gather data on the remixing of materials; content can be hosted on different servers and combined in different ways. Current standard practice, in which tools such as Google Analytics (via Javascipt) are used, loses track of OER use once it is removed from its original location. Other tools, such as CaPRéT (http://capret.mitoeit.org/) and OERGlue (http://www.oerglue.com/) identified by this call as “…useful in that they provide benefits to users (easy attribution) rewarded by benefits to content providers (analytics),” allow tracking of cutting and paste reuse, the insertion of information on the material’s origin and, in the latter instance, enable educators to individualise OER content with the integration of features such as social networking.

16. In CaPRéT the project built on the perceived deficit of existing “image attribution tools.” (http://blogs.cetis.ac.uk/lmc/2011/05/09/jisc-cetis-oer-technical-mini-projects-results/ ) However whilst CaPRéT needs no “end user” input to function this tool is limited to tracking reuse of sampled text and not, for example, images or videos. The need for an integrated tool which can track different types of content therefore remains relevant. This project seeks to address this in two ways:

a. Develop a no-Javascript/static web-bug based server-side approach using Piwik (http://piwik.org), Google Analytics and/or other analytics software.

b. Provide an extended CaPRéT++ by building on the CaPRéT Javascript libraries, to address its current limitations.

17. Taken together these approaches will provide an embedded tracker for content that is transferred as a whole by download, and to provide an enhanced code set to meet and extend the requirement for tracking and supporting the copy-paste model of taking OER content. Such data gathering may also be one way to support the work of the Learning Registry in collating events linked to OER content. While we will review parallel work on providing data to the Learning Registry, integration is beyond the scope of this project,

18. Track OER’s approach is both cost-effective as it will build on the successes of projects such as CaPRéT, prior work in OLnet and links with Bridge to Success. The work will be carried out by existing specialised staff whilst disseminating their skills outside of the remit of the project. The result will be a broad, multi-platform, self-contained, open source (released under a GNU General Public License-compatible licence in line with OSSWatch recommendations) solution to the problem of understanding and monitoring OER reuse. The flexible approach allows for integration with data analysis systems. Integration with analytics (both open source and proprietary) gives front-end support for the analysis and use of data by a variety of users: content owners, researchers, instructors and re-mixers. From a user perspective the tool will be able to track a multiplicity of different “cut and paste” reuses: “Save As,” “Print,” pasting into Word, the file type (e.g. video, text, image) etc.

19. More broadly, the ability to analyse data on the remixing and reuse of OERs would enable educators and researchers to provide clear evidence for OER re-use and potentially support the case for an increase in institutional OER provision and funding. It would also support OER best practice and the case for continued support for OER projects through identification of which materials are popular with re-mixers of content and how these are used. Originators of OER would also be able to better understand and track the multiple paths that OER content takes. Educators would be able to view

5

how OER is used and reused and the context in which it is reused; this would enable the sharing of best practice in meeting particular educational requirements. From a re-users perspective, this data could augment OER search engines or be used to create charts that highlight 'reuseful' resources.

20. Evaluation of the developed code will occur within the following different contexts: 21. Producing Demonstration Content: The two B2S courses will be integrated as a first

set of content to make use of the platform. The solution will then be extended to the Open University LabSpace. We also have the potential for comparison with other platforms such as OERGlue and would encourage take-up following proof of the system.

22. Producing Demonstration Data: We will encourage reuse of this content by working with a group of instructors and supporting them to do some remixing of content. This is expected to occur in the remit of the B2S project and the evaluation of this project will augment that. We will also promote remixing of the tracked content more widely, applying tracking to users outside the project to give sources for further data. This approach is both cost effective as institutions have been identified as part of another project, and maximises dissemination and user engagement opportunities.

23. Iterative Design of Visualisations and Views of Statistics for End-Users: Using the data produced through 1 and 2, we will set up trial visualisations through systems such as Piwik and Drupal. This will include statistics and visualisations of reuse for the benefit of a content creator - answering questions such as 'who has been using it and how?' and also for a potential remixer - answering questions such as 'what content are people reusing that might be relevant to me?'. An iterative, participatory design process will be used, walking through these visualisations with potential users from both groups. This will elicit data on what they would like to understand and how they would like to view it. We will revise our design in response to these suggestions, such that the final result is an appropriate set of tools to make the most of the collected data.

24. As specified in the call this project will disseminate its findings via regular blog posts and a final two-minute video. In addition testing via the B2S project will see the results disseminated through the project’s network of partners and affiliates, via the website, webinars etc.

3.1. Approach and evaluation 25. By developing two solutions the project is able to address both the direct challenge

identified in B2S and expanded in the Use Case, and to provide evidence from more casual cut-and-paste reuse. It is clear that both types need to be considered and with existing prototypes it is realistic to seek the combined solution and to integrate the reporting process. We can also compare the value and impact of the two approaches to consider: - Which of the alternative tracking systems provides the most utility for end-users? - Which of the alternatives is simplest to install/implement in the wild? - Which generates the most useful data? - What mechanisms are applied to use and re-use OERs? Eg. cut & paste, consuming

RSS feeds, SCORM packages, etc. - Whether a multi-pronged approach is needed to provide sufficient coverage?

26. It is proposed that testing and evaluation would take several forms: - An unsupervised trial of the alternative systems on The Open University’s

OpenLearn-LabSpace, specifically the Bridge to Success (B2S) project course units (http://labspace.open.ac.uk/b2s | http://b2s.aacc.edu). That is, the systems are made available, the trial is publicised, and the general public uses the system, with data being collected and analysed,

- A controlled laboratory evaluation of the systems on the B2S LabSpace and/or other content, with a variety of users, employing a think-aloud protocol and potentially using eye-tracking equipment available to the project.

6

- Building on the work of the Bridge to Success project, we have identified and built relationships with piloting institutions and instructors who are specifically interested in remixing OER content. We will use these contact to form an initial group with whom to evaluate Track OER in order to assess the benefits to, and use of the tracking system by, both OER producers of content and educators/instructors. This will enable direct feedback based on existing information gathering in B2S as well as assessment of the tracking system outputs.

3.2. Workplan for development 27. The overall development plan will start immediately the project is funded to establish the

code base, review use case and engage with the developer community. The majority of work in the development cycle will take place later to fit with planned availability of development effort which has been agreed with line management prior to submission of this proposal. There will be two parallel strands.

3.3. Strand 1: Track OER – web-bug tracker 28. At the heart of most Web analytics systems, including Google Analytics, Piwik,

ComScore and the analytics part of CaPRéT is what is termed a web-bug – code attached to an image, often a one-pixel by one-pixel ‘hidden’ image. By virtue of it’s inclusion or embedding in a host Web page it can be used to log data about the host page, the visitor, and the visitor’s browsing software. The image link can pass back information, taking the form for example:

29. <img src= "http://track.example.org/tracker.gif?host=host.example.com&path=path/to&paramA=X&..." alt="" />

30. Google Analytics, Piwik and other systems wrap a web-bug in a Javascript library, which makes the analytics system easier to implement, more flexible, and adds extra data about the visitor that is only available via Javascript (for example, screen resolution, support for Flash and Java, etc.).

31. The dis-advantage of the Javascript wrapper is that it is not available or is dis-allowed in interfaces like RSS feeds, Zip and packaging standards (eg. SCORM, IMS Common Cartridge), and is often stripped out because of security concerns when imported into content management and online learning systems.

32. Given this overwhelming restriction, no-Javascript web-bugs were researched in collaboration with Scott Leslie of BCcampus as part of the OLnet project (http://olnet.org/node/149), and code was released (http://snipplr.com/view/37647). This code was developed as an example prototype to demonstrate the principles. It is not currently functional and will form the basis for extension in this project. While web-bugs are usually linked to one-pixel hidden graphics we will associate the tracking with the embedded Creative Commons graphic e.g.

33. The code base will be extensively reworked to provide a working service linked to an

appropriate analytics system such as Piwik. Piwik is an open-source alternative to Google Analytics implemented using Javascript, PHP and MySQL. It provides similar tracking, visualization and reporting functionality, and is highly extensible. Whereas Google Analytics is a hosted solution, Piwik can be installed and customized on a local server (http://piwik.org). (Expected effort 15 days.)

34. Installing/implementing a web-bug in B2S LabSpace: It is suggested that test no-Javascript web-bugs can be manually added to the RSS feeds and downloads/ archives for the two B2S modules available via Labspace (http://labspace.open.ac.uk/b2s). This is a low-risk approach that will be useful for evaluation. The software will then undergo Full-scale implementation across OpenLearn would be a more complex proposition. (Expected effort 10 days/5 days testing.)

7

3.4. Strand 2 CaPRéT++ 35. As the name suggests Cut and Paste Reuse Tracking (CaPRéT) is designed to add

attribution and license meta-data, and to track the informal copying and pasting of OERs. It has been developed by Tatemae and MIT Office of Educational Innovation and Technology, and the code is available via Github (http://capret.mitoeit.org & http://github.com/tatemae/capret).

36. CaPRéT comprises two parts. The client-side consists of Javascript libraries built on top of jQuery. The server-side part is a Node.js application, which uses Hummingbird for realtime visualizations (http://nodejs.org).

37. Currently CaPRéT does not appear to track when an individual image or multimedia resource is copied from within an OER. This limitation can be reduced using HTML5 DOM extensions within CaPRéT. (10 days)

38. Installing/ implementing CaPRéT, in LabSpace and Moodle: The CaPRéT Javascript libraries have already been implemented in a Drupal 6.x blog (http://freear.org.uk) and a Moodle 2.0 system (http://freear.org.uk/moodle). In both cases, there was no need to install plugins or extensions on the server - they were implemented as custom blocks (the implementor needs super-user/administrator privileges on Drupal and Moodle). Given this initial success, it is thought that the CaPRéT Javascript can be added to LabSpace in a low-risk manner, and within the constraints of the OpenLearn development and release cycle, for evaluation with the B2S course content. (10 days)

39. Moodle version of CaPRéT: As part of the project, a CaPRéT plugin will be written for Moodle to go through acceptance testing for full inclusion in OpenLearn/LabSpace. (10 days/5days testing)

40. Overall testing and documentation in accordance with requirements (10 days)

3.5. Risk Assessment

Risk Potential Outcome Contingency Likelihood Staff availability Delay and pressure

to complete. Staff are in place to carry out the main development work.

Low

Alternative solutions Work on adoption of an alternative software base may be more desirable

The project aims to consider alternatives, however the benefits of the planned implementation are clear.

Medium

Embedding in OpenLearn/LabSpace is not viable

The scale of demonstrated use is lower than planned.

The software will be embedded in the Bridge to Success content as initial demonstrator and alternative OER platforms may be approached as demonstrators

Low

Technical difficulties in solutions

The planned implementation does not give sufficient information to be useful

Initial prototypes have illustrated feasibility and alternatives will be considered

Low

8

Rejection of tracking by the community

Take up is low and there is resistance to the automated provision of tracking data

Work with Creative Commons on the value of the approach and ethical ways to combine it with licensing.

Low/medium

3.6. Costing

Costing is omitted from the public version of the proposal. 3.7. Team The Principle Investigator for the project is Dr Patrick McAndrew, Senior Lecturer in Institute of Educational Technology and Director of the Bridge to Success and OLnet projects. Academic consultant is Dr Tony Hirst, Lecturer in Mathematics Computing and Technology. Tony has an established reputation at identifying viable solutions to technical challenges. The lead developer is Dr Nick Freear (IET). Nick works across project based software development specialising in accessibility and integration, Nick is also the author of Moodle 2 for Teaching 4-9 Year Olds Beginner's Guide. Acceptance testing will be carried out by Learning & Teaching Solutions Service Team led by Roger Moore. Implementation by the OpenLearn and LabSpace team led by Dr Guy Barrett and Jenny Gray. This team has extensive experience with OER, JISC and development of software to meet requirements.