multi-modal interfaces “ “the focus of ist in fp6 is on the future generation of technologies in...

Multi-modal InterfacesMulti-modal Interfaces

“The focus of IST in FP6 is on the future generation of technologies in which computers and networks will be integrated into the everyday

environment, rendering accessible a multitude of services and applications through easy-to-use human interfaces““

IST WP 2005-2006

Our Mission Our Mission

The challenge is:

• To make interfaces as simple as possible, and interaction as easy as possible

• For citizens to be able to access, receive and use information in their own language

We support the research, development and integration of:

• Advanced technologies for interfaces & interaction

• Integrated multilingual information systems and services

We focus on:

• Technology-mediated interaction and communication between people, between their devices, and between people and content

• Combining traditional human language services, like human translation and interpretation, and advanced language technologies and resources

E1 Interfaces

The Facts The Facts

Call Title: IST Call 5

Call Identifier: FP6-2004-IST-5

Strategic Objective: 2.5.7 Multi-modal Interfaces

Open: 17 May 2005

Close: 21 September 2005 at 17.00 (Brussels local time)

Budget: 54 M€

Instruments: IP’s, STReP’s

New/Traditional Instruments: 60/40

Main challengesMain challenges

• Need to provide information artefacts to be used by increasingly diverse user groups

• Expanding context of use, from scientific and business use to residential and nomadic use

• Increasing variety of media and devices used to access a community-wide pool of services and information resources

• Demonstrate added-value of multi-modality and performance advances through multi-modal interaction

• Go beyond system integration for demonstration purposes and target real world solutions

• Emphasise multi-user scenarios and non-business applications

• Stronger focus on defining the expected impact

• Need to promote cooperation between small European companies

• Promote research that could lead to spin-offs

An R&D agendaAn R&D agenda

• Key interaction technologies (speech, natural language input, haptics, vision, sensors, gesture, agents, …, including unconventional modalities) and their modelling and integration into interaction platforms

• Interface architectures, tools for cost effective integration, e.g. fusion, and toolkit interoperability

• Computational models of multi-modality, and advanced corpora for research

• Re-develop a serious effort on automatic translation

• Reinforce work on robustness and acceptability of speech recognition and research on speaker-independent software

• New learning paradigms in (spoken) language translation and portability to new languages, including greater focus on language resources as computer processable and reusable resources

• Design processes, methods and tools that support scalability and modality independence

• Empirical science base for understanding end-users, e.g. computational models, test suites and benchmarks

Multi-modal Interfaces Multi-modal Interfaces

Objective: natural and easy to use interfaces that use several modalities or

are multilingual

• Natural interaction between humans and the physical or virtual environment

• Multilingual communication systems

IST WP 2005-2006

Natural interaction between humans and the physical or virtual environment

• Interfaces that are autonomous, and learn and adapt to user intentions and behaviour, in dynamically changing environments

• Featuring unconstrained, robust interaction, recognise user reactions and respond to them in a natural way

• Selecting the right combinations of modalities according to the preferences and context

• Looking at both the fusion of information related to different modalities and their channelling to multiple modalities


IST WP 2005-2006


Multilingual communication systems

• For unrestricted domains, including task-oriented, real-time understanding of spontaneous spoken and gesture input

• Address novel learning paradigms that exploit contextual information, human and linguistic knowledge in a more effective way than currently done

• Portability of new languages taking advantage of methods and techniques developed for languages already covered is a further challenge to be addressed, e.g. in the context of new EU languages

IST WP 2005-2006


This covers:

• User modelling, system design, visual recognition and tracking, language understanding and spoken language translation

• Applications should focus on proof of concepts

• Includes consumer, nomadic, creative, artistic and gaming applications

IP - system-level objectives in natural interaction and multilingual communication

STReP – can also include work on language understanding and spoken language translation

IST WP 2005-2006

Instruments Instruments

draft

Integrated Project (IP):

• Result driven with strong industry/academic consortium and users well integrated

• Focus on impact, exploitation, and replication

• Multi-faceted with R&D, demonstration, training, awareness building, standardisation, …

• 10-15 partners, 5+ countries, 4 years, 5-10 M€

Targeted Project (STReP):

• Problem oriented with the right team for the S&T challenge

• Single-problem R&D with high innovation/risk/reward

• Investigating new research options, proof of concept, or competitive approaches

• 5-8 partners, 4+ countries, 3 years, 2-4 M€

The Evaluation Criteria The Evaluation Criteria

draft

Relevance:

• Clarity – make it a clear yes/no decision

• Reference national and European foresight results

Quality of consortium:

• Tell them what your principle qualities are – quality of R&D, industrial participation, exploitation potential, SME involvement, uniqueness of consortium, track record, …

• Well balanced in terms of expertise, effort and exploitation

• All partners must have substantial and well defined roles to play (no window dressing, no hangers on)

• Users and end-users are present and have clearly defined roles

Resources:

• Clear task allocation and no unnecessary overlaps

• Justify costs, effort allocation, and link explicitly to results


draft

Management and organisation:

• Clear work-plan and good progress indicators and tracking are key signs

• Indicate some valuable intermediate milestones

• Indicate how risks will be assessed and what are you back-up options or contingency plans

• How decisions are made, and can they be made in a timely way – conflict resolution and consensus building

• Good interlinking between objectives, activities and work-packages (Gantt)

Exploitation/dissemination:

• Protect and manage IPR and provide access to partners

• Start actions early and be precise about options

• Go beyond just “publishing results” – make an impact and above all be visible


draft

European added-value:

• Show added value to existing national and European programmes

• Compliment ongoing efforts of most partners

S&T excellence:

• Position the project vis-à-vis wider strategic objectives

• Define clear challenging but attainable objectives

• Describe state-of-the-art and go beyond (no catch up)

• Commit to some key deliverables/impact measures

• Justify balance between research and development

• Stress visible industry-scale validation through experiments, prototypes, demonstrators…

• Show that you can scale to industrialisation

• Don’t just be interesting, be manufacturable

Its all about generating, demonstrating and validating new knowledge

Be ambitious, but clearly define your scientific and technological objectives – think 2010+

Be unique, innovate but be industrially relevant - include key industry players and ensure broad industrial impact

Be evaluator friendly, make it a compelling read, easy to understand, a give them a reason to remember it at the end of the week

Be credible, one proposal, no “template” proposals, do what is needed to get the result, not more, but not less – remember 54 M€ represents ~1,200 person-years of effort

Check your ideas with us!

Remember…Remember…

IST infodeskIST infodeskemail: [email protected] email: [email protected] fax: +32 2 296 83 88fax: +32 2 296 83 88

General FP6:General FP6:

http://europa.eu.int/comm/research/fp6/http://europa.eu.int/comm/research/fp6/

http://www.cordis.lu/http://www.cordis.lu/

IST:IST:

http://www.cordis.lu/isthttp://www.cordis.lu/ist

Bernard SmithBernard Smithemail: [email protected] email: [email protected]

Further informationFurther information

multi-modal interfaces “ “the focus of ist in fp6 is on the future generation of technologies in...

Documents