design and implementation of an open-source video … · needed quicker encode workflow add ......

2
Outcomes Support calls became almost non-existent Verified usage over wireless networks both on and off campus Increase in faculty playlists by about 150% (Fall ‘09 semester 74 courses had playlists) Increased media in asset library by about 50% (989 in Summer ‘09 to 1430 Spring ‘10) Fall ‘09 14% of Pomona College courses used the system up from 7% Saved about 2/3 in disk space, a typical 1GB movie is now 300 mb with better quality Quicker media encode turnaround time, went from one encode station to four. No more buggy plug-ins or proprietary software developers to deal with Built additional faculty confidence in ITS Faculty love the self-service playlist builder We have gained control over our video assets and are able to grow without fear Freed up lots of staff and student worker hours, we can get other stuff done Demonstrated that it is possible to “glue together” a lot of different technologies easily Unexpectedly, faculty began using Video47 in class while lecturing Why did Pomona College build Video47? For a number of years Pomona College has used a proprietary video-on-demand product to deliver academic video content over the web. After some evaluation we came to the conclusion that it could do a better job of delivering academic content with an in-house system built around open-source technologies. We were in search of a solution with some of the following characteristics: Had to be easier to use for students and faculty More flexible systems administration Ability to view video over wireless Did not want download special plug-ins Ability for faculty to self-manage playlists LDAP Authentication Better video quality Smaller file sizes / conserve disk space Ability to utilize multiple video servers and/or players Eliminate the need for duplicate media Ability to use or convert legacy video content Must have a fast forward and jump to function Needed meta-data and searchable video repository Content to be accessed inside Sakai and by cross-registered students Needed quicker encode workflow add new video content Ability to have multiple user types Wanted to use off-the-shelf hardware System Architecture Video47 has been designed to be extremely flexible. It is possible to distribute the system across hardware, use multiple streaming technologies, utilize LDAP authentication, and allow for multiple end user video players. Here is an example of our current configuration. Design and Implementation of an Open-Source Video-on-Demand System Jason Smith - Instructional Technologist Hardware Specifications Our current configuration uses one server, however the use of multiple physical servers is possible. Off-the-shelf hardware was used for all components. IBM X3650 Single Quad Core 8 GB memory Disk Space: 2 x 2 terabyte raid for media The video encode stations use four retired 1.4 Ghz Dual Core Mac Minis equipped with an Elgato USB video capture device. Additionally we use several 2 terabyte external disks to archive pre-transcode media for backup. Web Interface Login Account Types The web interface is divided into four categories of logons or views. Sakai View: A course playlist is embedded within Sakai using the Sakai web tool. Student authentication is accomplished through Sakai against Claremont consortium LDAP servers. When a playlist is requested within Sakai a secure token is exchanged allowing access to video content. For the student the content has an expiration date, can not be viewed outside of Sakai, and is not downloadable. Secure streaming is possible with a Flowplayer and Wowza server configuration. Student Stand Alone View: Pomona College students can use their campus login to view course playlists. Authentication is against the campus LDAP server. While this login method is not normally used it was designed as an alternative method of viewing. Faculty Administrator View: Pomona College faculty can build playlists for their current semester courses. Additionally, it is possible to clone playlists from previous semesters. Faculty can not see other faculty members’ playlists. However, faculty have full access to the entire media database and the ability to add any of its content to their course playlists. Faculty have read access to media meta-data and are not able to add media to the system. Faculty are also able to see the Sakai and student views above. Faculty data is loaded into the system with their courses each semester from Jenzabar. Faculty are authenticated using campus LDAP. Video47 Administrator: This account manages the video assets and their associated meta- data. The account also has full control over all playlist administration functions such as add, modify, delete, and clone. Administrators are kept in a configuration file on the Video47 server but authenticated using campus LDAP. Screen shots available on the other poster…. What did we come up with and how? During the summer of 2009 in an effort to figure out what features were need as well as those that were wished for we held a few sessions where the faculty and staff were able to provide input and give feedback on the project. These meetings were held during the design, prototype, and launch phases. Better work-flow for encoding content Automatic conversion for legacy content Asset management system with monolithic video repository (no duplicate media) Controlled access through LDAP/AD and Sakai Time expired playlist access and content Ability to embed into our Sakai LMS Self service playlist creation for faculty Ability to import course data from Jenzabar (our SIS) frequently Option to implement various media players on the user end / no funky plug-ins Option to choose multiple streaming technologies (in the event flv goes bad) Ability to implement multiple streaming servers and multiple media Design a modified Dublin Core metadata for video assets Create a faster media encode and transcode process, more efficient student labor Better fast-forward, timeline scrub, and a bookmark capability. Media Encoding and Transcoding Process Most new media can be encoded with the Elgato Video encoder which can capture analog video from any source, anything with an RCA output. The software included with the Elgato captures to mp4 and allows for a simple edit. The mp4 files are then converted into the flv (flash video) format using ffmpeg. Both the ffmpegX gui and the command line ffmpeg are used with some Perl scripts to batch process multiple files. Some content may arrive in a digital format. In this case only ffmpeg is used. Legacy content: we had about 1000 mv4 format titles from our legacy VOD system. To covert them into flv we used Perl and ffmpeg to batch the files. It took three Mac Minis about five days to complete the entire conversion. What about copyright? Please see: http://copyright.claremont.edu/ for our copyright policy. Video47 Application Server Wowza Media Server (Linux/Apache/MySQL/PHP) Campus LDAP/AD Server Additional RTMP server Media Encoding Stations Student User Consortial Sakai Returns Sakai Embedded Video47 Sakai class gets playlist CMS Course Data Faculty User Admin User Playlist Creation Samba/SCP How long did it take and what did it cost? The idea for Video47 was hatched in the Fall of 2009. Programming and systems development started in the Spring of 2009 and finished in the Summer of 2009. Legacy media conversion and a live production server was completed by August of 2009. In September, 2009 we were officially in production. Our legacy VOD system was mothballed in October, 2009. The project had one lead person who designed most of the systems architecture and did the programming. Student labor was involved in the meta-data input and media translation processes for about 100 hours. Some aspects of the project extended into other areas of departmental and institutional expertise and are therefore hard to quantify. We purchased about $4,000 in new hardware. We decided to use Redhat Linux and the Wowza media server which both required a license because they worked well together. Alternatively, it is possible to use CentOS and Red5 configuration which we did while prototyping. http://video47.pomona.edu [email protected] Samba/SCP Copyright Jason Smith, 2010. This work is the intellectual property of the author. Permission is granted for this material to be shared for non- commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Upload: dinhque

Post on 23-Aug-2018

223 views

Category:

Documents


0 download

TRANSCRIPT

Outcomes Support calls became almost non-existent Verified usage over wireless networks both on and off campus Increase in faculty playlists by about 150% (Fall ‘09 semester 74 courses had playlists) Increased media in asset library by about 50% (989 in Summer ‘09 to 1430 Spring ‘10) Fall ‘09 14% of Pomona College courses used the system up from 7% Saved about 2/3 in disk space, a typical 1GB movie is now 300 mb with better quality Quicker media encode turnaround time, went from one encode station to four. No more buggy plug-ins or proprietary software developers to deal with Built additional faculty confidence in ITS Faculty love the self-service playlist builder We have gained control over our video assets and are able to grow without fear Freed up lots of staff and student worker hours, we can get other stuff done Demonstrated that it is possible to “glue together” a lot of different technologies easily Unexpectedly, faculty began using Video47 in class while lecturing

Why did Pomona College build Video47?For a number of years Pomona College has used a proprietary video-on-demand product to deliver academic video content over the web. After some evaluation we came to the conclusion that it could do a better job of delivering academic content with an in-house system built around open-source technologies. We were in search of a solution with some of the following characteristics:

Had to be easier to use for students and faculty More flexible systems administration Ability to view video over wireless Did not want download special plug-ins Ability for faculty to self-manage playlists LDAP Authentication Better video quality Smaller file sizes / conserve disk space Ability to utilize multiple video servers and/or players Eliminate the need for duplicate media Ability to use or convert legacy video content Must have a fast forward and jump to function Needed meta-data and searchable video repository Content to be accessed inside Sakai and by cross-registered students Needed quicker encode workflow add new video content Ability to have multiple user typesWanted to use off-the-shelf hardware

System Architecture

Video47 has been designed to be extremely flexible. It is possible to distribute the system across hardware, use multiple streaming technologies, utilize LDAP authentication, and allow for multiple end user video players. Here is an example of our current configuration.

Design and Implementation of an Open-Source Video-on-Demand SystemJason Smith - Instructional Technologist

Hardware Specifications

Our current configuration uses one server, however the use of multiple physical servers is possible. Off-the-shelf hardware was used for all components.

IBM X3650 Single Quad Core8 GB memoryDisk Space: 2 x 2 terabyte raid for media

The video encode stations use four retired 1.4 Ghz Dual Core Mac Minis equipped with an Elgato USB video capture device. Additionally we use several 2 terabyte external disks to archive pre-transcode media for backup.

Web Interface Login Account TypesThe web interface is divided into four categories of logons or views.

Sakai View: A course playlist is embedded within Sakai using the Sakai web tool. Student authentication is accomplished through Sakai against Claremont consortium LDAP servers. When a playlist is requested within Sakai a secure token is exchanged allowing access to video content. For the student the content has an expiration date, can not be viewed outside of Sakai, and is not downloadable. Secure streaming is possible with a Flowplayer and Wowza server configuration.

Student Stand Alone View: Pomona College students can use their campus login to view course playlists. Authentication is against the campus LDAP server. While this login method is not normally used it was designed as an alternative method of viewing.

Faculty Administrator View: Pomona College faculty can build playlists for their current semester courses. Additionally, it is possible to clone playlists from previous semesters. Faculty can not see other faculty members’ playlists. However, faculty have full access to the entire media database and the ability to add any of its content to their course playlists. Faculty have read access to media meta-data and are not able to add media to the system. Faculty are also able to see the Sakai and student views above. Faculty data is loaded into the system with their courses each semester from Jenzabar. Faculty are authenticated using campus LDAP.

Video47 Administrator: This account manages the video assets and their associated meta-data. The account also has full control over all playlist administration functions such as add, modify, delete, and clone. Administrators are kept in a configuration file on the Video47 server but authenticated using campus LDAP.

Screen shots available on the other poster….

What did we come up with and how?

During the summer of 2009 in an effort to figure out what features were need as well as those that were wished for we held a few sessions where the faculty and staff were able to provide input and give feedback on the project. These meetings were held during the design, prototype, and launch phases.

Better work-flow for encoding content Automatic conversion for legacy content Asset management system with monolithic video repository (no duplicate media) Controlled access through LDAP/AD and Sakai Time expired playlist access and content Ability to embed into our Sakai LMS Self service playlist creation for faculty Ability to import course data from Jenzabar (our SIS) frequently Option to implement various media players on the user end / no funky plug-ins Option to choose multiple streaming technologies (in the event flv goes bad) Ability to implement multiple streaming servers and multiple media Design a modified Dublin Core metadata for video assets Create a faster media encode and transcode process, more efficient student labor Better fast-forward, timeline scrub, and a bookmark capability.

Media Encoding and Transcoding ProcessMost new media can be encoded with the Elgato Video encoder which can capture analog video from any source, anything with an RCA output. The software included with the Elgato captures to mp4 and allows for a simple edit. The mp4 files are then converted into the flv (flash video) format using ffmpeg. Both the ffmpegX gui and the command line ffmpeg are used with some Perl scripts to batch process multiple files. Some content may arrive in a digital format. In this case only ffmpeg is used.

Legacy content: we had about 1000 mv4 format titles from our legacy VOD system. To covert them into flv we used Perl and ffmpeg to batch the files. It took three Mac Minis about five days to complete the entire conversion.

What about copyright?

Please see: http://copyright.claremont.edu/ for our copyright policy.

Video47 Application ServerWowza Media Server

(Linux/Apache/MySQL/PHP)

Campus LDAP/AD Server

Additional RTMP server Media Encoding Stations

Student User Consortial SakaiReturns Sakai Embedded Video47

Sakai class gets playlist

CMS Course Data

Faculty User

Admin User

PlaylistCreation

Samba/SCP

How long did it take and what did it cost?

The idea for Video47 was hatched in the Fall of 2009. Programming and systems development started in the Spring of 2009 and finished in the Summer of 2009. Legacy media conversion and a live production server was completed by August of 2009. In September, 2009 we were officially in production. Our legacy VOD system was mothballed in October, 2009.

The project had one lead person who designed most of the systems architecture and did the programming. Student labor was involved in the meta-data input and media translation processes for about 100 hours. Some aspects of the project extended into other areas of departmental and institutional expertise and are therefore hard to quantify.

We purchased about $4,000 in new hardware. We decided to use Redhat Linux and the Wowza media server which both required a license because they worked well together. Alternatively, it is possible to use CentOS and Red5 configuration which we did while prototyping.

http://video47.pomona.edu [email protected]

Samba/SCP

Copyright Jason Smith, 2010. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Design and Implementation of an Open-Source Video-on-Demand SystemJason Smith - Instructional Technologist

Future Enhancements

The project will continue to evolve and we will most likely add new features. Here are some that have been requested.

Additional ways to search meta-data Some sort of integration with the IMDb Ability for faculty to create “virtual libraries” or “meta-collections” Folksonomy or some kind of tagging system Dual screen playback - the “Rick Blackwood” feature Ability to better handle various aspect ratios Digital stream for specialized faculty meta-data input / QC system More descriptive statistics Faster than real-time encoding

TABLE MEDIAu_id int(11) title varchar(150) creator varchar(70) subject varchar(70) description text director varchar(40) dateOriginal date type varchar(30) format varchar(15) digSpecs varchar(100) dateDigital date identifier varchar(70) source varchar(30) language varchar(5) rights varchar(70) contributingInstitution varchar(100) fileSize varchar(20) fileName varchar(70) duration varchar(20) comment text

TABLE PLAYLISTu_id int(11) hit_count int(11) name varchar(30) year year(4) term varchar(2) timestamp timestamp owner varchar(20) owner_name varchar(40)

notes text

TABLE PLAYLIST_ELEMENTu_id int(11) class_id int(11)

media_id int(11)

TABLE FACULTY_COURSESu_id int(11) login_id varchar(10) name varchar(40)

course_name varchar(30) TABLE STUDENT_COURSESu_id int(11) login_id varchar(10)

course_name varchar(30)

Database Table Design

Why the number 47?

In 1964, Professor of Mathematics Donald Bentley proved that all numbers are equal to 47. Since then, Sagehens in search of 47 trivia, have found that this quintessential random number turns up in the most unlikely places.

Pomona College is located off the San Bernardino Freeway. The sign reads: Claremont Colleges Next Right, Exit 47.Tolstoy's novel The Kreutzer Sonata is named after Beethoven's Opus 47.The New Testament credits Jesus with 47 miracles.The Declaration of Independence consists of 47 sentences.Pancho Villa was killed by a barrage of 47 bullets.Cesar proclaimed "veni, vidi, vici" in 47 B.C.The tropics of Cancer and Capricorn are located 47 degrees apart.If all this 47 trivia upsets your stomach, you'll be glad to know that Rolaids absorbs 47 times its weight in excess acid.

http://video47.pomona.edu [email protected]

Copyright Jason Smith, 2010. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.