down, set, hike! video integration with pac-12 networks

Down, Set, Hike!

October 26, 2013

Video Integration on Pac-12 Networks

Thursday, August 14, 14

Web Developer, Pac-12 Networks

Lance Geng

Project Team

Senior Developer, Phase2

Joshua Turton


Game PlanTechnical Requirements


Pac-12 networks has a vision to be one of the industry leaders in online sports media, integrating videos from all 12 conference schools as well as original content generated in their downtown San Francisco studio.

Phase2 was contracted to help implement their vision. One major component of this project was the video solution, which is the part we’re talking about today.

Key Video Requirements


[Read requirements]

Key Video Requirements

• High traffic, bullet-proof uptime

• Single-console editorial experience, with quick uploading

• Import of external content

• Tagging by sport, school, team, and type

• Integration with scheduled events: before, during and a"er

• Subscription-based access to streaming & premium content


[Read requirements]

The OffenseHow we did it


Pac-12 had an existing video site - built by Lullabot, who did some great work on it - which accomplished some of these goals, but not all of them. We expanded on their work, added features, and tied it all together with a big bow on top.

Here’s how...

Running GameIntegration with Ooyala & Drupal


Pac-12’s integration with Ooyala required some key functionality, each of which posed their own unique challenges.

These challenges included a way for users to dynamically add/edit/delete metadata fields to a video, an integration with services and chunked uploads for handling large file sizes, and keeping all of this in sync between The Network, The Schools & Ooyala.

Custom Field

A new, custom field type allows for direct uploads to Ooyala. That

same field type stores reference information for both Live

Broadcasts and On Demand videos.


The foundation of our integration with Ooyala is the custom Ooyala Video field type. Because both Video on Demand and Live Broadcasts share many of the same properties, we decided to only make one field that could be shared by both.

The actual field stores only the video’s Unique ID, provided by Ooyala. Ooyala is responsible for delivering video content and rendering the video player. We simply save a reference to the video in Drupal, and let the video provider handle the rest.

Asynchronous Uploads


Our videos are huge, so how can we best approach the upload issue?

[PRESS SPACE BAR]This is a second function of our custom field and handler. Once the user has entered a title and chosen a video file, the upload can be initiated.

We send an AJAX request to the services module which includes the title, the file name, the total size of the file and the desired size of each chunk to send to the server. Services initiates the calls to our Ooyala Video module which sends the provided information to Ooyala. Ooyala returns a serialized array of upload URLs for each chunk of video which we then pass back to the client to handle the upload.

On the client we take each URL, splice the file chunk that coordinates with that URL and then send each one to Ooyala. This allows us to asynchronously handle the file upload, greatly increasing the speed in which a large file can be transmitted to Ooyala for processing.


• Problem: How do you upload a 2 GB video quickly, without interrupting the editorial workflow?







• Problem: How do you upload a 2 GB video quickly, without interrupting the editorial workflow?

• Solution: use the Services module to implement a chunked Ajax upload, while the editor finishes filling in other fields.






Track Segments


Our Live Broadcast video type, “EPG”, has the extra requirement of a Track & a Track Segment. A Track is essentially a channel and a Track Segment is a slot in time on a given Track. A single EPG can have several Track Segments attached to it.

Tracks were implemented as a basic taxonomy.

For Track Segments, we implemented a custom database table. The reason we chose to do this was that Track Segments are never edited by users - they are all imported from scheduling data.

We did not want all of the overhead of a full blown entity just to attach this data to our Ooyala Video fields. All of this is tracked in a separate custom table, which is also queried by the events display code that Josh will talk about in a few minutes.

Metadata Configuration


Metadata is a key/value pair of custom data that is stored in Ooyala with a video. It tells us sport, team, date, and other important information about the video.

[PRESS SPACE BAR]

In the past, entry of this information has been a tedious task for our content team. Rather than creating individual custom fields for each type of metadata we needed to store, we decided to borrow Drupal’s Form API for presentation, and stored the data as a serialized blob in our database.

We built a custom configuration tool, pictured here, to manage these fields. This allowed us to assign field types to existing metadata fields.


• Problem: How to handle

tedious metadata entry and

configuration?



[PRESS SPACE BAR]




• Problem: How to handle

tedious metadata entry and

configuration?

• Solution: Leverage FAPI and

key to existing Ooyala

metadata.



[PRESS SPACE BAR]



Keeping it in Sync

A handful of custom

drush commands helps

keep the Network in

sync with their Schools

and Ooyala.


One of the biggest challenges was keeping all of our videos in sync. The main idea of this project is that Drupal would become the master point of record for all of our videos, including those uploaded through external platforms by our Schools.

This meant that video data would need to be pulled from Ooyala on a regular basis, and video nodes created. To do this, we created some custom Drush commands. These commands were responsible for pulling down either EPG or VOD videos and importing/updating them in Drupal. These commands can be scheduled to run in custom intervals by either Jenkins or Acquia CRON.

When a video is published, we push all of our meta-data in Drupal for the video back up to Ooyala using their PHP API.

Passing GameContent Prioritization on Event Nodes


Once all the videos are imported, uploaded, parsed, encoded, and ready to view... we need someplace to find and view them. That leads us to the Event Node.

Events are one of the most complicated pieces of content on the site.

Events display related content from any one of a number of other content types, defined using the Node Reference module. What content appears where and when is based on a complicated algorithm, incorporating time of day, content type, taxonomy, and freshness of content.

Content related to the event


Here is the order of prioritization for related event content. [PRESS SPACE BAR]

[Read bullets]

Within any content type if there are multiple items at the same level, they’ll be sequenced in reverse chronological order.

There are secondary rules, based on school and team, if the above doesn’t have enough results to fill out an event node.

• EPG (streaming video), if one is currently playing.

• VOD (Video on demand), tagged with Highlight taxonomy term

• VOD, tagged with Replay taxonomy term

• VOD, untagged

• Photo Gallery

• Article, tagged with Preview or Recap taxonomy term

• Article, untagged

• Podcast

Content related to the event


Here is the order of prioritization for related event content. [PRESS SPACE BAR]

[Read bullets]

Within any content type if there are multiple items at the same level, they’ll be sequenced in reverse chronological order.

There are secondary rules, based on school and team, if the above doesn’t have enough results to fill out an event node.


Here’s what an *incomplete pass* at that query looked like. It was an attempt to find an EPG node, based on the Event’s NID, and then loaded that node. If it didn’t find one, then it attempted to do the same with VOD.

If you dig into this query, you can see that it doesn’t do anything with the taxonomy terms, and that additional queries will be necessary to handle Galleries, Articles, and Podcasts.

It was quickly apparent that this methodology wouldn’t work; the database load from this operation would likely have killed the site.


The solution to this database issue turned out to be more database work. Specifically, a new table called event_priority, which allows us to store a weight for each node related to a given event. This weight is calculated once, when a content node is saved or updated, based on the rules shown earlier. Then, when an event it loaded, it queries this table with a tiny, super-fast, well-indexed query.

Which is much, much better.

More information about this process is available on Phase2’s blog.

The DefenseWhat we would do differently


All that said, there are definitely some things we have learned through this process.

Video Metadata Storage

• Metadata is stored as a serialized array in a database blob

• The intent was to keep it from the full field API overhead

• Doesn’t allow for filtering and sorting by views, or auto-complete fields

• End solution was to add capability to import / export process to transfer data into fields

• Better solution would have been to use fields originally


We made a decision very early on to store metadata in a database blob in a self-contained table. Our intention was to shield this data from the processing overhead of a full-fledged Drupal field.

This backfired on us.

We wound up later needing to use that data in views as a filter or sort parameter, and to allow it to inform auto-complete fields. Thus, we had to add a workaround to the import and export process that *also* stored most of those data items in fields.

The better solution would have been to use fields from the start.

Home-grown Video Solution

• Assembled using openly available components including:

• Brightcove’s Zencoder or Amazon Elastic Transcoder

• Amazon S3

• Drupal Video module, which supports multiple players

• Less expensive

• More possible customization


A possible avenue of exploration for Pac-12’s future exploration and development is to build their own video solution.

Phase2 has had some good results lately with another sports client creating a video hosting and encoding solution using an assortment of openly available components. These include Brightcove’s Zencoder or Amazon Elastic Transcoder, Amazon S3 hosting, and the Drupal video module, which supports multiple different players.

This would have multiple advantages:First and foremost, Drupal would be truly the source-of-truth for all video data. No more pushing metadata back and forth!

It’s also a less expensive solution, with a much higher possible degree of customization.

Post-game InterviewQuestions?


We have time for a few questions.

Senior Developer, Phase2

Email: [email protected]

Joshua Turton

Twitter: @sjinteractive

Web Developer, Pac-12 Networks

Email: [email protected]

Lance Geng


mailto:[email protected]






PHASE2TECHNOLOGY.COMPHASE2TECHNOLOGY.COMPAC-12.COM


down, set, hike! video integration with pac-12 networks

Technology

ooyala video module

chunk of video

video integration

video content

video solution

gb video

custom ooyala video

video provider