user interface composition with usixml
DESCRIPTION
This paper presents novel ongoing works on user interfaces composition. These works have emerged with the problematic of component software composition transposed to theHuman-Computer Interaction domain. Some software architectures indeed allow components assembling at the final design step. Our work, based on UsiXML, aims at proposing a composition/decomposition of user interfaces. These works begin with the concrete level of UsiXML dedicated to the graphical modality and continue with higherabstraction levels. This article provides a positioning of the proposal related to composition compared to the seven dimensions related to the "μ7" concept of UsiXML projectTRANSCRIPT
User Interfaces Composition with UsiXML
Sophie LEPREUX (1), Jean VANDERDONCKT (2), Christophe KOLSKI (1)
(1) LAMIH“Automated Reasoning and Human-Computer Interaction” Research Group
University of Valenciennes and Hainaut-Cambrésis, France
(2) BCHIUniversity of Louvain-La-Neuve, Belgium
UsiXML-EICS 2010 - Berlin, Germany - June 20, 2010
- To take the user caracteristics, the platform used, the environment into account, to allow an universal access to information, knowledge, services… in numerous different contexts
Introduction
New UI design problems, focus on UI Composition
Brain-Computer Interface
- Tree-algebra based UI Composition
-UsiXML framework
-Composition/decomposition Operators
- Compose to support the 7 Dimensions
- Conclusion
Summary
Tree-algebra based UI Composition
Window (id=window, name=window, width=« 500 » height=« 350 »)
Box (type=« vertical »)
Button (DefaultContent = Save)
Button(DefaultContent=Close)
Output (DefaultContent =« Fiche Client »)
Box (type = horizontal)
Box (type = horizontal)
Output(…)
Input(…)
Box (type = horizontal)
Output(…)
Input(…)
tag content
<cuiModel id="Client_registration-cui_31" name="Client_registration-cui"> <window id="window_component_0" name="window_component_0"
defaultContent=« Nouveau client" width="500" height="350">
<box id="box_0" name="box_0" type="vertical"><outputText id="output_text_component_2"
name="output_text_component_2" defaultContent=« Fiche Client"
isVisible="true" isEnabled="true" isBold="true" textColor="#000000"/>
<box id="box_1" name="box_1" type="horizontal"><outputText id="output_text_component_2"
name="output_text_component_2« defaultContent=« Fiche Client"
isVisible="true" isEnabled="true" isBold="true" textColor="#000000"/>
… </window> </cuiModel>
- Tree-algebra based UI Composition
-UsiXML framework
-Composition/decomposition Operators
- Compose to support the 7 Dimensions
- Conclusion
Summary
UsiXML framework
Task & Concepts
Abstract UI
Concrete UI
Final UI
Source = < User, Environnement, Platform >
Target = < User, Environnement, Platform >
Task & Concepts
Abstract UI
Concrete UI
Final UI
Summary
- Tree-algebra based UI Composition
-UsiXML framework
-Composition/decomposition Operators
- Compose to support the 7 Dimensions
- Conclusion
Composition/decomposition Operators
Cut( T1, )
T1 T2Unary operators Binary operators
Similarity
Equivalence
Left difference
Right difference
Intersection
Unique union
Normal union
Fusion
∩Complementary( T1, )
Projection( T1, )
Selection( T1, )
T1
Composition/decomposition OperatorsExample: Normal Union operator applied on two user interfaces
Uhorizontal
=
Composition/decomposition OperatorsIllustrations of operators on CUI models
T1 = tourist application UI T2 = event management application UI
(a) Selection (T1, set of elements)
(b) Projection (T1, {outputText, Button})
(c) Intersection (T1, T2)
- Tree-algebra based UI Composition
-UsiXML framework
-Composition/decomposition Operators
- Compose to support the 7 Dimensions
- Conclusion
Summary
Compose to support the 7 Dimensions
- Use of (de)composition operators to adapt the UI according to the 7 dimensions:
(1) Multi-device Usage and/or(2) Multi-User interface,(3) Multi-linguality,(4) Multi- organization,(5) Multi-context usage,(6) Multi-Modality usage,(7) Multi-platform Usage
Compose to support the 7 Dimensions(1) Multi-device usage
Task & Concepts
Abstract UI
Concrete UI
Final UI
Source = < User, Environnement, Platform = 1 device >
Task & Concepts
Abstract UI
CUI’1
FUI’
1
Target = < User, Environnement, Platform = n devices >
FUI’n
CUI’n
…
…
CUI’1 = Selection (set elements of Music controller task, Concrete UI)CUI’2 = Selection (set elements of Sounder player task, Concrete UI)CUI’3 = Selection (set elements of volume controller , Concrete UI)
Adaptation 1: decomposition of the CUI Source in several
CUIs
Adaptation 2: Fusion of several decompositions
of the CUI source
CUI’1 = Selection (set elements of Music controller task, Concrete UI)CUI’2 = Selection (set elements of Sounder player task, Concrete UI)Union Selection (set elements of volume controller , Concrete UI)CUI’3 = Selection (set elements of volume controller , Concrete UI)
- Illustration: control of a music player
Compose to support the 7 Dimensions(2) Multi-user interface
- Two interesting cases: Case #1: a shared UI is co-localized, the users can simultaneously interact on one UI
Ex.: tactile multi-touch interface or tangible objects used
Case #2: a user arrives in a collective work situation (or leaves it): the user needs a work space, with personal data and/or functions
- Other interesting problems to consider :
Distant shared UI
Consideration of the user rights or roles (ex: moderator UI in a brainwriting session at a distance: composed differently as the UI of the other participants)
Compose to support the 7 Dimensions(3) Multi-linguality
- if two similar UI are modeled at the CUI level (or higher) with specifications given for one (or more) language and another UI modeled with others languages, it can be possible to compose these UI to obtain one UI modeled for all these languages
- Importance to consider translation considerations leading to necessities about composition/decomposition due to socio-cultural aspects
Ex: source language read from left to right, and downwards (english) target language read from right to left, and downwards (arabic), or upwards (for instance: Chinese)
Consequence: different zones of the UI have to be composed/decomposed differently
Compose to support the 7 Dimensions(4) Multi-organization
T1 & C
T’& C
T2 & C
T’ = T1 U T2
Source = < User, Environnement, Platform >
Target = < User, Environnement, Platform >
Source = < User, Environnement, Platform >
- Importance to consider how the tasks have to be performed in different organizations or groups
- Importance to consider the organization models described in the literature (ex. in social sciences or multi-agent system domain)
Compose to support the 7 Dimensions(5) Multi-context usage
- Several composition/decompositions strategies are possible:
Ex.: development methodology based on patterns, business component and machine learning (plastic UI approach, [PhD Hariri 08])
Ex: integration of the task model in the business component in order to compose complete application and to facilitate the co-evolution of the system [Bourguin G., Lewandowski A. & Tarby J.C., TAMODIA’2007] - A context modification can generate evolutions at each abstraction level
Ex: a lighting change acts on the luminosity of the device (FUI) whereas the background noise change acts (or not) on the choice of the vocal modality.
Ex: a modification of the user work brings modification of tasks and/or on the tasks planning Interesting composition/decomposition problems to consider
Compose to support the 7 Dimensions(6) Multi-Modality usage
- The AUI layer allows specifying the UI independently of the modality According to needs brought by context change, a UI composition at a high level allows merging UI which will be concretized by different modalities in the lower levels
- Ex : Food order realized with several modalities
AUI1
AUI’
AUI
T’ = T1 U T2
Order Chinese food
AUI03: Choose meal
Chooseappetizer
Choosestarter
Choosesoup
Chooserice
AUI02: Give delivering address
Givename
Givephone
Giveaddress
AUI’ =AUI1 U AUI2
Normal Union (AUI (order a pizza), AUI (Order Chinese food))
Compose to support the 7 Dimensions(7) Multi-platform Usage
- Analogy: example of multi-platform language is: Java Execution on several platforms without changing model or implemention
- Our objective: to do the same from the UI point of view
to generate code from CUI model to FUI model
For example, graphical editor GraphiXML allows generating java or XHTML code
So the graphical UI which is modeled with GraphiXML is multi-platform
The composition operators: not needed into this process
- Tree-algebra based UI Composition
-UsiXML framework
-Composition/decomposition Operators
- Compose to support the 7 Dimensions
- Conclusion
Summary
Conclusion- Presentation of operators developed initially to compose at design time and at the CUI level
- Currently: operators used to each abstraction level of UI
- Operators: could be used at the runtime
- These operators: a support to UI adaptation
- Illustrations in the paper for the 7 dimensions: Multi-device Usage, Multi-User interface, Multi-linguality, Multi- organization, Multi-context usage, Multi-Modality usage, Multi-platform Usage
But: adaptations to criterion (dimension) can be done to the detriment of the others
As a result, the adaptation must be global i.e. the adaptation has to take into account all the criteria simultaneously and not considered one by one
- Research questions are open concerning: (1) the one by one or simultaneous consideration of these dimensions, (2) the order in taking them into consideration …
User Interfaces Composition with UsiXML
Sophie LEPREUX (1), Jean VANDERDONCKT (2), Christophe KOLSKI (1)
(1) LAMIH“Automated Reasoning and Human-Computer Interaction” Research Group
University of Valenciennes and Hainaut-Cambrésis, France
(2) BCHIUniversity of Louvain-La-Neuve, Belgium
UsiXML-EICS 2010 - Berlin, Germany - June 20, 2010
Ajout logo1 JVD
Ajout logo2 JVD
THANK YOU FOR YOUR ATTENTION