Enterprise Sakai:Enterprise Sakai:Research and Development Research and Development for a National Learning Gridfor a National Learning Grid

Mr. Josh Baron, Director, Academic Technology, Marist College

Mr. Joseph Davey, Marist Graduate CS Student

MARIST COLLEGE• We are NOTNOT a large research university!

• Founded 1929 – small complex liberal liberal arts collegearts college

• Approximately 5700 FTE5700 FTE student population

• 200 full-time faculty200 full-time faculty, 500 part-time

• Long history of incorporating technologyincorporating technology into the teaching and learning process

Why do we need an Why do we need an enterprise solution?enterprise solution?

Learning from the past…

• In 1996 the digital horizondigital horizon was foggy…

• We have a better view today… watch outwatch out!

• Use and production of digital media will grow exponentiallygrow exponentially over next decade:– Electronic portfoliosElectronic portfolios (oh yeah…“for life”)– Rich media learning objectsRich media learning objects (IVT Demo)– IdentityQuest ProjectIdentityQuest Project (Podcasting Demo)– Next Up: Mobile authoring toolsMobile authoring tools?

Colleges and Universities

A National Learning Grid

K-12 DistrictState and

Local Consortia

Museums, science

centers, etc.

Certification Exams

Online Courses

“Raw” Video Content

Student content

Schema for Provisioning the Sharing of Resources/ToolsSchema for Provisioning the Sharing of Resources/Tools

Sakai Repository

Marist/IBM Joint Study• Well Established Relationship, First Project –

1988• Living Lab for New Technologies

– Research projects, IBM Academic Initiative support, Marist grant initiatives

• IBM program manager provides overall project management– IBM researchers, developers, business execs, and

Marist faculty, IT staff, and students collaborate on numerous projects

• Marist students hired as IBM interns– Opportunities for full-time positions

Recent Joint Study Projects• IBM Academic Initiative/Scholars Program

– z/OS Hub/Course Creation/Test Drive Images/Instructor Training – Mainframe Contest – 700 Students/85 Colleges/Universities– z/OS Certificate Program – Series of Online Courses (via IDCP)

• Extensive Use of Linux Virtual Servers for Course Instruction– 600+ Images on Marist Mainframe

• Open Source Development LabOpen Source Development Lab– Marist based lab for Linux on z projects

• 2003 SUR Grant – Project Greystone2003 SUR Grant – Project Greystone– Teacher and Student Portals – Grown to 11 Local School Districts– Several Rich Media Applications/CISCO Integration

• 2005 SUR Grant – Sakai Project2005 SUR Grant – Sakai Project – Open Source LMS Integrated with IBM’s WAS, DB2, RCI/Content Mgr

• Grid Computing– Course Creation/ NSF Grant Participation

• Marist Grant Initiatives – Support and Collaboration– NYSTAR Funded CART – Center for Collaborative and On Demand Computing– NSF Funded – Institute for Data Center Professionals

Global CampusStudents

EducatorsAdministrators

Customized Portals

App 1

App 1

App 2

App 2

App 3

App 3

Collaborative Learning Environment

Sakai Tools &DevelopmentEnvironment

Distributed Environment

Core Services: content, user, security, site

Core Infrastructure

Sakai Vision

Marist Sakai Goals

• How do we create a scalable, enterprise version of Sakai capable of handling future digital repository needs?– Port Sakai to IBM WebSphere – Content Integration with DB2 Content

Manager and RCI

Main Porting Issues

• Classloaders– Tomcat decided to go against specification

• Looks in local classloader first unless resource contains certain package keywords (javax, org.xml.sax, etc)

• If resource not found, request is delegated to parent classloader

– WebSphere is the opposite, but allows the administrator to adjust the classloading policy

Main Porting Issues

• Deployment Descriptors– Tomcat is much more lenient with invalid

deployment descriptors• If a web.xml file does not match the servlet

specification it overlooks the error and allows the web application to function normally

– WebSphere is much more strict• Will not allow an application to be installed unless

there are no errors in the web.xml file

Main Porting Issues

• Filters– The Servlet Specification states that any filter

that maps to a url-pattern should be applied before any filter that maps to a servlet name.

– Some Sakai components do not follow this spec

• Tomcat applies filters in the order in which the Sakai Application is expecting

• WebSphere applies the filters according to specification

Content Hosting

• The legacy Content Hosting service manages files and attachments for the Sakai 2+ tools.

LegacyService

Integration

Uses Content Hosting Does not use Content Hosting

Resources Tool Grade book

Attachments Test and Quizzes

Drop Box Schedule

Sakai V2.0 Implementation June, 2005

“Charon”(precursor to uPortal)

As the User Interface

Legacy Tools contributed by:-U of Mich -CHEF-MIT -OKI-U of Ind-Stanford

-X-Platform O/S -Fedora, other Linux, Windows-Tomcat Application Server-My SQL Database + Others

Content Hosting Services

Db Content Services

Relational Databases File Systems

Sakai RCI/Content Manager Integration

What is RCI?

• Rich Content Infrastructure (RCI) is middleware supporting the creation of rich content solutions.– High level APIs, tools, and enhanced media functions– Allows application providers to easily adapt their

applications to a common infrastructure– Simpler and more cost efficient way to create,

manage and distribute content.– Built on top of DB2 Content Manager

Benefits of RCI

• Common management and distribution utility for rich content solutions

• Provides a single repository for different types of data– Multiple views and use of the same content. – Simpler and easier set of interfaces into

existing CM repositories.– More efficient way to create or update

enterprise applications.

Sakai Content Hosting Servicemodified for Content Manager

•BaseContentService

•Storage Interface

•C - Create

•R - Retrieve

•U - Update

•D - Delete

Storage Interface

• Defined in BaseContentService

• Implemented in RciContentService

• 20 Methods defined

• 90% of time implementing

• Communicate with RCI

• Some trivial, some very tricky

 void cancelCollection(ContentCollectionEdit edit)

 void cancelResource(ContentResourceEdit edit)

 boolean checkCollection(java.lang.String id)

 boolean checkResource(java.lang.String id)

 void close()

 void commitCollection(ContentCollectionEdit edit)

 void commitResource(ContentResourceEdit edit)

 ContentCollectionEdit editCollection(java.lang.String id)

 ContentResourceEdit editResource(java.lang.String id)

 ContentCollection getCollection(java.lang.String id)

 java.util.List getCollections(ContentCollection collection)

 ContentResource getResource(java.lang.String id)

 byte[] getResourceBody(ContentResource resource)

 java.util.List getResources(ContentCollection collection)

 void open()

 ContentCollectionEdit putCollection(java.lang.String id)

 ContentResourceEdit putResource(java.lang.String id)

 void removeCollection(ContentCollectionEdit edit)       

 void removeResource(ContentResourceEdit edit)

 java.io.InputStream

streamResourceBody(ContentResource resour

ce)       

Storage Interface Implementation

Trivial Difficult

Open() commitCollection()

Close() commitResource()

CheckResource() getResource()

getLocks() getCollectionSize()

Development Process1. Requirements

a. RCI to run on Sakai 2. Architecture

a. Sakai b. WAS c. CM/DB2 d. SUSE Linux e. IBM X series server

3. Design a. What to change/create, and how b. Meetings and documentation

4. Implementation a. RAD (built on eclipse technology) b. Used RCI API c. Implemented Sakai Interfaces d. Weekly Status meetings

5. Test a. Sakai Extended Test cases b. Unit tests c. Stress tests

6. Deploy a. Production environment b. Currently Running z/OS Certificate Classes

7. Support a. New Sakai versions b. Migration to Sakai 2.2.2 planned completion

February 2007

JSR168 Portletas the UserInterface

Legacy Tools + LMS -(alternative to Educator Enhanced Functions) -Streaming Media

Greystone/RCI-SuSE-WebSphere -Portal Server -Web Service Remote Portal -Application Server-DB2 /Content Manager-Content Distribution Networks

Domain Independent Rich Content Infrastructure Web Services

Content Hosting Services Content Managing Services

RCI_Content Services RCI_Managing Services

Information and Content Integration

Relational DatabasesContent Repositories File Systems

Marist College/IBM/Sakai V2.1.2 Implementation Sept, 2006

Current Production Environment

Preparations for Production

• Migration from existing Sakai– MySQL pull from existing server– MySQL -> RCI– RCI -> MySQL– Verification

• Backup and Recovery– 4 Databases on three servers– Resource Data

Backup Plan

The Next Phase

• New Sakai Tools

• New Sakai Services - Like EMU

EMU: Extensible Media Utility• The Goals

– Create a Rich Media application capable of handling (uploading, streaming, serving) any type of media across many repositories.

– Make it extensible and dynamic so that new features may always be added.

– Give faculty and media personnel the capability to load and tag their own content.

– Give faculty and media personnel the ability to update meta-data on previously loaded content.

– Make content accessible through courseware systems.

EMU: Extensible Media Utility

• Why EMU?– The growing needs of professors called for an

application that could search many repositories and create media lists that would be available to students through their online courseware.

– This utility also needed to be able to be updated by faculty and media staff who were looking to create their own content.

EMU Design

EMU Design

EMU Transcode Video Grid Service

• Marist plans to maintain the grid environment and exploit the new grid applications.

• They recently used the Grid Transcoding application to encode their collection of MPEG-2 videos, including their Emmy videos.

• The transcoder application ran on the Marist grid of 6 servers.

• Nearly 78 hours saved over a single processor environment!

EMU Transcode Video Grid Service

Number of Videos (MPEG-2) 268 268 536 Length of Video (Sec) 163881 163881 327762 Length of Video (H:M:S) 45:31:21 45:31:21 91:02:42 Time to Encode (Sec) 149389 58113 207502 Time to Encode (H:M:S) 41:29:49 16:08:33 57:38:22 Ratio (Encode Time:Actual) 0.91157 0.354605 n/a *Estimated Ratio Single 2.116009 0.85846 n/a *Estimated Total Sec 346773.7 140685.3 487459 *Estimated Total (H:M:S) 97:40:26 39:04:45 136:45:11 % of time 43.08 41.31 42.57 Time Saved (H:M:S) 54:49:45 22:56:12 77:45:57 Legend * Estimated Time based off the average time of a random sampling of videos from the collection to encode on a single machine with the same specs **HQ High quality video encoded at 30 Frames/Sec and 320 kbps bitrate with 128 kbps audio *** LQ Low quality video encoded at 8 Frames/Sec and 208 kbps bitrate with 64 kbps audio (H:M:S) = hours:minutes:seconds Note: All videos 352 x 480

Colleges and Universities

A National Learning Grid

K-12 DistrictState and

Local Consortia

Museums, science

centers, etc.

Certification Exams

Online Courses

“Raw” Video Content

Student content

Schema for Provisioning the Sharing of Resources/ToolsSchema for Provisioning the Sharing of Resources/Tools

Sakai Repository

Contact InformationContact Information

Mr. Josh Baron Josh.Baron@marist.edu

Mr. Joseph Davey Joseph.Davey@marist.edu

Q & A

iLearnCM Implementation

• Began in September 2006September 2006 with System z Program (iLearnCM System)– Fully onlineFully online non-credit “cohorted” program– 25 students, IBM employees are instructors– Provided “real worldreal world” test of system

• Conducted extensive testingextensive testing before launch

• Had plan in place for troubleshootingtroubleshooting– “Double clickDouble click” issue in Test & Quizzes

iLearnCM Implementation

• Provided students with online orientationonline orientation• Trained instructorsTrained instructors in one-on-one sessions• User supportUser support is critical:

– MOTD used for support information– “Check-in” phone conferences

• Easy to blame technologyEasy to blame technology in “experimental” system

• Surveys being administered nowSurveys being administered now…