- List of Multimodal Libraries -

(EIF students only)

The MMI team MMI_01

Multimodal Interfaces Toolkits : review ( - EIA-FR)Toolkits : review ( - EIA-FR)

For EIF students:A) Small demo (« Hello World » type)B) Provide a short documentation (3 copies) and a CD

containing:The libraryHow to install the libraryWhat it does and how it worksQuick state of the art of the specific domain, if applicable

RRRR

The MMI team MMI_01

Multimodal Interfaces TK1: EyesWeb TK1: EyesWeb !! Taken !!!! Taken !!

Target : multimodal toolkit

« EyesWeb consists of a number of integrated hardware and software modules which can be easily interconnected and extended. »

Plate-forme : Windows

Suggested form : review

http://www.infomus.dist.unige.it/Index.html

RRRR

The MMI team MMI_01

Multimodal Interfaces TK3: HandVu TK3: HandVu !! Taken !!!! Taken !!

Target : gesture recognition « Detects the hand in a standard posture, then tracks it and recognizes key

postures - all in real-time and without the need for camera or user calibration. » Language : C / Platform : Linux, Win32 Suggested form : review http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html

RRRR

The MMI team MMI_01

Multimodal Interfaces TK5: Infovis + Prefuse ToolkitTK5: Infovis + Prefuse Toolkit

Target : visualisation toolkit

« Interactive Graphics Toolkit written in Java to ease the development of Information Visualization applications and components. »

Language : Java / Platform : all

Suggested form : review

http://ivtk.sourceforge.net/

http://prefuse.sourceforge.net/

RRRR

The MMI team MMI_01

Multimodal Interfaces TK7: iFace TK7: iFace !! Taken !!!! Taken !!

Target : 3D face animation, text-to-speech

« Integrated framework for working with Face Multimedia Object by defining and implementing the necessary functionality  »

Language : proprietary / Platform : Win32

http://ivizlab.sfu.ca/research/iface/

RRRR

The MMI team MMI_01

Multimodal Interfaces TK8: Sphinx 4.0 TK8: Sphinx 4.0 !! Taken !!!! Taken !!

Target : speech recognition

« state-of-the-art speech recognition system written entirely in Java »

Language : Java / Platform : all

Suggested form : review

http://cmusphinx.sourceforge.net/sphinx4

RRRR

The MMI team MMI_01

Multimodal Interfaces TK10: CLSU toolkit (2) TK10: CLSU toolkit (2) !! Taken !! !! Taken !!

Target : emotion recognition

« A comprehensive suite of tools to enable exploration, learning, and research into speech and human-computer interaction. »

Platform : Win32

Suggested form : review

http://cslu.cse.ogi.edu/toolkit/download/index.html

RRRR

The MMI team MMI_01

Multimodal Interfaces TK11: Nouse + Sonification TK11: Nouse + Sonification

Sandbox Sandbox !! Taken !!!! Taken !! Target : face tracking and sonification

Nouse : « Nose as a mouse » (face tracking)

Simple demo programs…/ Platform : Win32

http://synapse.vit.iit.nrc.ca/Nouse/index2.html

Sonification Sandbox : « simple, multi-platform, multi-purpose toolkit for sonifying data  »

Language : java / Platform : all

http://sonify.psych.gatech.edu/research/sonification_sandbox/sandbox.html

RRRR

The MMI team MMI_01

Multimodal Interfaces TK12: CMU CommunicatorTK12: CMU Communicator

Target : dialog manager

« A dialog management tool based on Sphinx. »

Language : C++ / Platform : Win32

Suggested form : review

http://fife.speech.cs.cmu.edu/Communicator/

RRRR

The MMI team MMI_01

Multimodal Interfaces TK13: FestVoxTK13: FestVox

Target : speech synthesis

« The Festvox project aims to make the building of new synthetic voices more systemic and better documented, making it possible for anyone to build a new voice. »

Language : C++ / Platform : *nix

Suggested form : review

http://festvox.org/

RRRR

The MMI team MMI_01

Multimodal Interfaces TK14: Other modalities review TK14: Other modalities review

Suggested form : review

Review of possible other modalities

… including, but not limited to : Eye gaze Face expressions Emotion recognition …

RRRR