the wall of moments - amazon s3...social media integration the content of the wall heavily relies on...

The Wall of MomentsRik BauwensVRT, Belgium

rik.bauwens@vrt.be

Mike MattonVRT, Belgium

mike.matton@vrt.be

Tine DeboosereVRT, Belgium

tine.deboosere@vrt.be

Luk OvermeireVRT, Belgium

luk.overmeire@vrt.be

ABSTRACTIn this paper a prototype application developed by VRT –aspart of the ICoSOLE project– is described. The Wall of Mo-ments aims to combine user generated content with profes-sional content, in order to create an immersive experience forpeople who can’t attend an event. The demo uses a selectionof content recorded at the Marconi Moments, a small-scaletest shoot in the Marconi studio at the VRT campus. To il-lustrate the liveliness when used near-to-live, an early UGCcapture prototype will be showcased as well.

Author KeywordsICoSOLE; VRT; user generated content; the wall ofmoments

ACM Classification KeywordsD.2.2 Design Tools and Techniques: User interfaces

INTRODUCTIONWithin the ICoSOLE project [2], different (cost-effective)ways of capturing a spatially outspread event are considered.Collected content can either be user generated, captured bystandalone equipment or made by a professional crew. Acore research challenge is synchronizing all the content intime and space, and this is achieved by using various kindsof metadata. Time and location are obvious, but also videoanalysis is done to find reference points in different videostreams in order to synchronise them. This analysis is part ofthe ICoSOLE core system and is considered available througha JSON REST interface for this prototype.

A second challenge of the ICoSOLE project consists of usingthis content to create an immersive experience for people whocan’t attend an event. This prototype is a practical implemen-tation of this goal.

THE WALL OF MOMENTSThe Wall of Moments consists of three main parts: the Wallitself, Live and My Experience.

MomentsMoments are short video clips, maximum 30 seconds inlength, made by people attending an event. These clips should

Except otherwise noted, the author retains copyright of this work under a CC-BY 4.0license: http://creativecommons.org/licenses/by/4.0/TVX’15, June 3–June 5, 2015, Brussels, Belgium.TVX2015 DEMO 102

show something unique, catch something a professional cam-era would probably miss. Together, they tell a story throughthe eyes of friends and create a unique, personal experiencefor people at home.

These Moments are recorded using a dedicated app, whichdoes instantaneous quality control and adaptive uploading, tobe able to give hints or abort when the video doesn’t meetquality requirements (e.g. to save bandwidth).

The WallThe Wall is a mosaic representation of Moments, and acts asthe home screen. This mosaic –or matrix– is a constantlyrefreshing feed of the latest and most interesting Moments. It’highly personalized: in order to decide which Moments areshown, preferences, trending Moments and social networksof the users (friends, followers) are taken into account.

The size of the Wall depends on the device type. On mostsmartphones, a one-by-two matrix would be used. The largestpart of tablets allow for a two-by-two matrix (due to higherscreen resolution and more processing power) whilst a per-sonal computer can easily handle a three-by-three matrix.

Figure 1. Screenshot of the home screen on a personal computer.

LiveWithin the application, a section is reserved for watching alive performance. For now, it only shows a live performance.In later stages, a timeline will be added to this video, whichwill show Moments popping in near-to-live as they are pickedup by the ICoSOLE backend.

My ExperienceWhen a user opens a Moment, s/he can add it to My Experi-ence. Essentially, this is a favourites list. To make it a reallyinteresting feature, a personal aftermovie –based on this list–is created by adding an intro, outro and background music.This feature will be enhanced along the way: users will beable to mix in professional content, add titles and more.

SYSTEM DESCRIPTION

Combination of UGC and professional contentWe believe UGC is key in creating a near-to-real experi-ence, because users at home can relate to content recordedby friends and family attending the event. Unlike a profes-sional crew, however, a major issue with UGC is video andaudio quality. Notwithstanding, a professional crew is moreprone to missing the most emotional and exciting moments inthe crowd due to accessibility issues (among other things). Tocircumvent this problem, we take the best of both worlds andset the goal of combining user generated content (UGC) withprofessional content. All the content that has been capturedby a device that’s part of the ICoSOLE system, is annotatedin several ways (e.g. timecode, location, orientation). Thisway, content can be mapped precisely in time and place ofthe event in the ICoSOLE core.

SynchronisationA user can zoom into a Moment on the Wall by clicking or tap-ping. The Moment is then played fullscreen, along with pro-fessional content picture-in-picture (when available). Fromthere, a user can switch to this professional content. At thetop, a timeline is shown. Every other interesting Moment (forthis user) is annotated, and s/he can jump to a particular onein just a click or tap. Hence, a user experiences the event ina social way, rather than the classic linear way (e.g. song bysong).

Figure 2. Screenshot of an opened Moment.

Social media integrationThe content of the Wall heavily relies on social interaction.The app is connected to the user’s social media accounts, andgathers Moments made by friends, artists s/he liked, radio sta-tions s/he follows etc. Social media interaction is also deeplyintegrated into the app itself. Users can like and share Mo-ments directly from within the app with their followers andfriends.

TECHNOLOGIESWe developed the Wall of Moments as a web application tosupport as much (mobile) devices as possible, using HTML5,JavaScript and CSS3. Communication with the backend ishandled through a JSON REST interface. For full-screenvideo playback, a DASH-enabled video player by consortiummember Bitmovin (Bitdash [1]) has been integrated.

Figure 3. Screenshot of My Experience.

DEMO CONTENTProfessional content was captured at the Marconi Momentsusing a wide variety of devices, including TV cams (bothstatic and close-up, for a live director’s cut) and omnidirec-tional cameras (on stage and in the audience).

Everyone in the audience was able to record short clips withtheir own smartphone and send them in to a backend we pro-vided. After uploading was completed, they were placed in aqueue and shown on a screen next to stage a short time there-after.

On TVX 2015, we’re planning to host this application for at-tendees to test out, as well as to install a screen to show thelatest Moments that were sent in. Additionally, a range of mo-bile devices will be setup with an interactive version of theWall using synchronized demo content of the Marconi Mo-ments.

CONCLUSION AND FURTHER RESEARCHThe Wall of Moments is a prototype of a use case defined inthe ICoSOLE project. User tests and especially the MarconiMoments sparked a fair amount of interest. Furthermore, wehave a better understanding of problems we’ll run into whenattending larger events: copyright issues and filtering of in-appropriate content among them. Experiments during severaltest shoots in 2015 and 2016 shall provide further insights intoimprovements of applications for end users in terms of usabil-ity, design and features. Our methods follow an iterative ap-proach: user tests are organised and results are subsequentlyevaluated to enhance future prototypes. Further research hasto be performed in the back-end as well: content needs to beprecisely annotated and synchronised in order to make thisprototype work properly.

ACKNOWLEDGMENTSThis work is part of the ICoSOLE project, funded by the Eu-ropean Commission through the 7th Framework Programme(FP7/2007-2013) under grant agreement no. 610370.

REFERENCES1. Bitdash (by Bitmovin) is a suite of highly optimized

MPEG-DASH clients. http://www.bitmovin.net/bitdash-mpeg-dash-player.

2. Immersive Coverage of Spatially Outspread Live Eventsproject website. http://icosole.eu.