project 1:comediannotate: towards more usable multimedia ... · project 1:comediannotate: towards...

Post on 02-Jun-2020

10 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Project 1: CoMediAnnotate: towards more usablemultimedia content annotation by adapting the UI

“Final” Presentation: Friday August 6, 2010

Christian Frisson 1, Sema Alaçam 2, Emirhan Coşkun 2,Dominik Ertl 1, Ceren Kayalar 4, Lionel Lawson 1,

Florian Lingenfelser 5, Johannes Wagner 5

1 University of Louvain-la-Neuve (Belgium),2 Istanbul Technical University (Turkey),3 Technical University of Vienna (Austria), 4 Sabanci University, Istanbul (Turkey),

5 University of Augsburg (Germany)

Friday August 6, 2010eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 1 / 20

eNTERFACEbook (reminder)

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 2 / 20

Outline

1 ContextTeamAnnotation tools: current problems, proposed solutions

2 AchievementsImprovement of an annotation tool (SSI)Design and Prototyping of a new UIIntegration of input devices into OpenInterface

3 To DoFinalize our (OpenInterface) componentsProduce more pipeline prototypes towards usability testingPort CoMediAnnotate into the MediaCycle framework

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 3 / 20

Annotated Team

input devices integrator

Ceren

UI developer

Christian

UI developer

Emirhan

annotation core integrator

Dominik

UI designer

Sema

Ismail

interviewed expert user

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 4 / 20

Annotation tools: current problems

2 Scale: number and/or length of media elements in the database2 Reusability: isolated tools, rather than toolboxes/frameworks2 Genericity: different tools, different media, different languages...2 Accessibility: local media databases, rather than client/server apps2 Multimedia: better file formats support2� Interactivity: generally WIMP2� Workflow: lack of support of the full annotation workflow

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 5 / 20

Annotation tools: proposed solutions

Interactivity: a single visual user interface that allows:1 2� to monitor signal feeds while recording datasets,2 2� optionally to add annotations while recording,3 2� to edit or correct annotations;4 2� a more natural, usable, pleasurable user interface (pen and touch).

Workflow:1 2 one administrator prepares (design of a template and choice of

coders);2 2� several coders record;3 2� several coders annotate;4 2 the administrator analyses results (coder agreement...).

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 6 / 20

Outline

1 ContextTeamAnnotation tools: current problems, proposed solutions

2 AchievementsImprovement of an annotation tool (SSI)Design and Prototyping of a new UIIntegration of input devices into OpenInterface

3 To DoFinalize our (OpenInterface) componentsProduce more pipeline prototypes towards usability testingPort CoMediAnnotate into the MediaCycle framework

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 7 / 20

Timeline-based Annotation Tools: SSI winsA detailled comparison in 3 tables is on the wiki:

1 development (quantitative)2 context, usage (quantitative)3 eNTERFACE participants feedback (qualitative)

Advene

Python

AmiGram

Deprecated

Anvil

Not opensource

Elan

Finalist

Lignes de Temps

Flash/Flex

On The Mark

JavaFX, simple

SSI VCode/VData

OSX-only

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 8 / 20

Timeline-based Annotation Tools: why SSI?

close contact with its developers(also project participants)core separated from UIlightweight GUI:

1 simple to understand2 easy to replace

toolkit first aimed at multimodalsignals annotation, towardsadaptive multimodal interfacesby training

More info:http://mm-werkstatt.informatik.uni-augsburg.de/ssi.html

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 9 / 20

Improved SSI UI

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 10 / 20

Integration of SSI into OpenInterface

More info: http://www.openinterface.orgeNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 11 / 20

UI design proposal

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 12 / 20

UI design proposal (annotated)

Fixed Play Zone

Past Events Future Events

Audio Track: Waveform

Video Track: Keyframes

Sensor Tracks:

Frames sliding right-to-left

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 13 / 20

UI prototype in Processing

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 14 / 20

Integration of input devices into OpenInterface

Previously available:Ï Wii RemoteÏ 3D mice

In progress:Ï multitouch devices, with two options available:

F WM_TOUCH high-level events fromWindows 7 using MT4j

F cross-platform low-level HID usingGenericHID

Ï jog wheels (also reusing GenericHID)Ï pens

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 15 / 20

Outline

1 ContextTeamAnnotation tools: current problems, proposed solutions

2 AchievementsImprovement of an annotation tool (SSI)Design and Prototyping of a new UIIntegration of input devices into OpenInterface

3 To DoFinalize our (OpenInterface) componentsProduce more pipeline prototypes towards usability testingPort CoMediAnnotate into the MediaCycle framework

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 16 / 20

Finalize our (OpenInterface) components

Input modalities, towards:Ï finger-based navigation / annotationÏ pen-based annotationÏ manual navigation using jog wheels

“On-demand” gesture recognition and rawdevice event filteringGranularized SSI core componentsOur timeline-based multitrack audio/videovisual user interface

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 17 / 20

Produce more pipeline prototypes towards usability testing

3D Mice

Pen+Touch

Jog Wheels Multitouch screen

Mice

Keyboards

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 18 / 20

Port CoMediAnnotate into the MediaCycle framework

More info:http://www.numediart.org/projects/08-3-multimediacycleeNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 19 / 20

Thanks for your attention, once again!

We stayed okay! ™ →

← ...and liked the social events!

eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 20 / 20

top related