multimodal interaction in distributed and ubiquitous computing - iciw 2010
DESCRIPTION
Multimodal Interaction in Distributed and Ubiquitous Computing presentationTRANSCRIPT
![Page 1: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/1.jpg)
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc
Multimodal Interaction in Distributed and Ubiquitous Computing
Marc Pous and Luigi Ceccaronibdigital
ICIW 2010 - The Fifth International Conference on Internet and Web Applications and Services
May 9 - 15, 2010 - Barcelona, Spain
![Page 2: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/2.jpg)
• Why multimodality?
• Multimodal distributed services deployment
• User tests and conclusions
SUMMARY
![Page 3: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/3.jpg)
In our cities, urban services practically have not changed in a century
![Page 4: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/4.jpg)
1: Location-based services
MOTIVATIONS
![Page 5: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/5.jpg)
http://www.flickr.com/photos/nnova/4512346900/
![Page 6: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/6.jpg)
http://picasaweb.google.com/marc.pous/Xina2007#5102695479944027890
![Page 7: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/7.jpg)
http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415374502175450370
![Page 8: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/8.jpg)
http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415371534810675298
![Page 9: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/9.jpg)
http://www.flickr.com/photos/loquat73/3385335980/
![Page 10: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/10.jpg)
2: Interactions and multimodality
MOTIVATIONS
![Page 11: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/11.jpg)
http://www.flickr.com/photos/nnova/4423756547/in/set-72157623598200316/
![Page 12: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/12.jpg)
http://www.flickr.com/photos/nnova/4424521014/in/set-72157623598200316/
![Page 13: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/13.jpg)
3: Accessibility
MOTIVATIONS
![Page 14: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/14.jpg)
![Page 15: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/15.jpg)
![Page 16: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/16.jpg)
http://www.dynamiclanguageblog.com/2009/12/technology-grows-for-hard-of-hearing.html
![Page 17: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/17.jpg)
MOTIVATIONS
What is an ICD?(Interactive Community Display)
![Page 18: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/18.jpg)
![Page 19: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/19.jpg)
INREDIS
3G = ACCESSIBILITY + UBIQUITY + INTEROPERABILITY
http://www.inredis.es
![Page 20: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/20.jpg)
INREDIS ICD
![Page 21: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/21.jpg)
Management of distributed services that offer the capability of processing and synthesizing multiple modalities of interaction
![Page 22: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/22.jpg)
DISTRIBUTED INTERACTION SERVICES
• Voice services– Voice recognition
• Via microphone, real-time streaming converting the voice signal into text
– Voice synthesis
• Sign language service– Spanish sign language recognition
• Via webcam, real-time streaming converting the image into text
• Emotion service– Emotion recognition (voice + image)
• Via webcam and microphone, real-time streaming
• Avatar service– Emotional avatar– Sign-language speaker avatar
![Page 23: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/23.jpg)
DISTRIBUTED SERVICES
SIGN LANGUAGE SERVICES
![Page 24: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/24.jpg)
DISTRIBUTED SERVICES
VOICE SERVICES
![Page 25: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/25.jpg)
How to consume multimodal distributed services offered by an ICD?
![Page 26: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/26.jpg)
W3C MMI Architecture
• W3C MMI Architecture
http://www.w3.org/TR/mmi-arch/
![Page 27: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/27.jpg)
W3C MMI Framework
http://www.w3.org/TR/mmi-framework/
![Page 28: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/28.jpg)
Proposed multimodal interaction framework
![Page 29: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/29.jpg)
• ICD Architecture
ICD ARCHITECTURE
![Page 30: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/30.jpg)
(1) Interaction identification
(2) Main page- Map or avatar- Command buttons- Content- Scenario options
(3) Contextualized map with services
(4) Avatar with map and Street View service
ICD DESIGN and INTERFACES
![Page 31: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/31.jpg)
USER–CENTERED DESIGN
• User profiles (Personas)– Blind users– Deaf users
• Unable to read– People living or visiting a city
• Scenarios– Context-aware informative scenario– Emergency scenario
![Page 32: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/32.jpg)
IMPLEMENTATION
![Page 33: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/33.jpg)
USABILITY TESTING
• PLI (People Led Innovation) methodology
• Preliminar usability tests with a limited number of users
• But the ICD implementation received good feedback!
![Page 34: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/34.jpg)
CONCLUSIONS and FUTURE WORK
• Implementation based on the W3C MMI Architecture idea• Platform able to integrate multimodal interactive distributed
services and offer interaction in real-time• Web-based applications and device-independent
implementation• Improvement of the user experience and accessibility of
people with special needs
What are we working on now?– Real-time interaction: delay reduction between interactions– Browser-based apps not being able to easily access device hardware– Synchronous interaction vs. Asynchronous technologies– Improvement of the distributed services orchestration– Enhancement of the mobile usability
![Page 35: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/35.jpg)
MOBILE PHONES?
http://www.slideshare.net/nokiaconversations/younghee-jung-and-jan-chipchase-presentation-for-nokia-design-london-studio-event-london-29th-april-2008
![Page 36: Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010](https://reader035.vdocument.in/reader035/viewer/2022070302/548728b8b4af9fe8528b45ec/html5/thumbnails/36.jpg)
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc
Thank you!
Moltes gràcies!
¡Muchas gracias!
Marc Pous and Luigi [email protected] and [email protected]