majordome gérard chollet, richard croce, laurence likforman, dijana petrovska-delacretaz, pascal...
TRANSCRIPT
MAJORDOME
Gérard CHOLLET, Richard CROCE, Laurence LIKFORMAN,
Dijana PETROVSKA-DELACRETAZ,Pascal VAILLANT
(chollet,croce,lauli,petrovsk,vaillant)@tsi.enst.frENST/CNRS-LTCI
46 rue Barrault75634 PARIS cedex 13
http://www.tsi.enst.fr/~chollet/
Majordome Outline
What is it ?
What it does for you ?
Research and application topics:
The SIROCCO project The EUREKA !2340 MAJORDOME project VoIP, VoiceXML, Human-Computer Interaction
Perspectives
Majordome is a distributed Personal Digital Assistant
It is your digital slave. It is personal. It remembers everything that you told him.
It uses resources from you mobile (wireless) device, from your home, from your office, from the Internet, from the environment, …
You interact with him using voice, pen, graphics, …
Interactions with your Majordome
Majordome recognizes your identity, your voice, your handwriting, ...
His speech recognizer is adapted to your voice,
His handwriting recognizer is adapted to your writing style,
He can speak to you, He can display information for you, He can talk with other persons either locally or
over the phone.
What Majordome does for you ?
Answers your phone, Receives and interpret your faxes, your emails, … Supplements your memory (address book,
agenda, bookmarks, alarm clock, health record, bank account, documentation, …)
Serves as an interface between you and the (digital) world,
Searches the web, internet forums, … Controls your home, your car, your children, your
parents, …
A framework: A L I S P
A utomaticL anguageI ndependentS peechP rocessing
with applications in Speech Coding, Synthesis, Recognition,
Speaker Verification and Language Identification
SIROCCO project Unlimited Vocabulary Speech Recognition
INRIA (IRISA et LORIA), LIA, IRIT, ENST-LTCIhttp://www.irisa.fr/sirocco/
SIROCCO
Unlimited vocabulary speech recognition system
French lexicon (MathLex) with 64kwords (AUF task)
Feature extraction with Spro (G. Gravier) Context-dependent HMM phone models Word pronunciation graph Uses CMU-Toolkit for Language modeling Beam search for word hypothesis Rescoring of word hypothesis by A*
«MAJORDOME»
Unified Messaging System
Eureka Projet no 2340
EDFHolistique
D. Bahu-Leyser, G. Chollet, R. Croce, K. Hallouli , J. Kharroubi, D. Kofman, L. Likforman, E. Matta-Sanchez, D. Petrovska, M. Sigelle, P. Vaillant, F. Yvon
Participants
• speech : G. Chollet, R. Croce, J. Kharroubi, D. Petrovska
• fax : K. Hallouli, L. Likforman, Marc Sigelle
• language : P. Vaillant, F. Yvon
• platform : D. Kofman, E. Matta-Sanchez, R. Croce
• ergonomy : D. Bahu-Leyser
Majordome’s Functionalities
• Speaker verification
• Dialogue
• Routing
• Updating the agenda
• Automatic summary
Voice
Fax
Overview of Majordome
Background tasks (server-side only): sorting and filtering messages from different
sources (E-mail, voice, fax, SMS,…); extracting relevant information for reporting
to user (names of senders, subject,…).
Dialogue with the user: over phone or Web. The system presents the state of the mailbox,
the type of messages, their sender, subject, and may sum them up or read them on request;
The users access their mailbox, addressbook, time schedule, or URIs (Web addresses).
Voice technology in Majordome
Server side background tasks:continuous speech recognition applied to voice messages upon reception
Detection of sender’s name and subject
User interaction: Identification of the speaker (and Verification if
necessary) Speech recognition (receiving users’ commands
through voice interaction) Text-to-speech synthesis (reading text summaries, E-
mails or faxes)
Voice Over IP Platform
Network
192.168.223.0/1
1
Network 192.168.222.0/11
Visioconference
VTHD
Renater
UnisphereERX-700
1Gbps (FO Interne)
ENST-Paris
RTC/RNIS
Intranet
GK
PBX
GW IPVR
1Gbps
Cisco Catalyst
6507
Salle C-234
Salle C-234
Salle PBX
Salle C-234
Network192.168.111.0/11
VideoServer
DistanceLearningService
‘Majordome’ partners
Majordome / NetCentrex project
IP-VR NetCentrexRecorder Machine
Usual #NetCentrex #
Calling person
Is the called person here ?
Vocal E-mail
Usual user called
PABX /Gateway ENST-Call Control Server-Application Server
No response
NetCentrex user called
Majordome / NetCentrex project
Usual #NetCentrex #
IP-VR NetCentrex
Calling person
PABX /Gateway ENST-Call Control Server-Application Server
Usual user called
Voice Interactive call
• Speaker verification
• Dialogue
•Vocal e-mail
• Routing
• Updating the agenda
• Automatic summary
No response
NetCentrex user called
Perspectives
Add Vision, Hearing and Understanding to Mobile Terminals (UMTS)
Multimedia for Distance Education and Conference Indexing
Semantic Web,
‘Universal Networking Language’
‘Smart Home’, ‘Smart Car’, ‘Smart Office’
Perspectives
The application context of the Majordome project could be of interest to COST-278.
The Majordome/NetCentrex platform could be made available to interested partners.
HTK, ISIP and SIROCCO softwares are available as freeware. One of them will be used on the NetCentrex platform.