© tamodia’061 the comets inspector: towards run time plasticity control based on a semantic...

1© TAMODIA’06

The Comets Inspector: Towards Run Time PlasticityControl based on a Semantic Network

Alexandre Demeure, Gaëlle Calvary, Joelle Coutaz, Jean Vanderdonckt

Université de Grenoble, FranceUniversité catholique de Louvain, Belgium

2© TAMODIA’06

The problem• UI plasticity: adaptation of a UI

– to its context of use <User, Platform, Environment>

– while preserving usability

• E.g: Choice of a month - Distinctive criteria

- Various

- Variable

- Unforeseeable

Task Navigation Genericity

3© TAMODIA’06

Objectives• Provide models, methods and tools for

– Describing and capitalizing UIs• In an extensible way• incorporating tailored UIs

– Supporting “plasticity” questions both at design and run time• Which UIs are HTML versions equivalent to this UI?• Is this UI decomposable into smaller ones? • Is there a tailored UI for choosing a month? (pattern?)• What is the common functionality supported by these 2 UIs?• …

4© TAMODIA’06

State of the art• Description

Domain & Task

Abstract UI

Concrete UI

Final UI

||| with navigation

DS0 DSi

|||

Cond

Choice in a known set Type: a TYPE S_poss: set of Type S_eff: set of Type min, max : IntegerConstraints: S_eff S_poss #S_eff [min; max]User Task: Specify S_eff

Linear |||

A SPACE name

A SPACE

A SPACE name

A SPACE

ScrollListPie |||

LanguagesXAML, XUL, …UsiXML, …

5© TAMODIA’06

State of the art• Capitalization

– Taxonomy of interactors• Rarely explicit

• Abstraction levels not explicit

• Not easily extensible

– Semantic network• “a graphic notation for representing knowledge in patterns of interconnected nodes and arcs.

Computer implementations of semantic networks were first developed for artificial intelligence and machine translation, but earlier versions have long been used in philosophy, psychology, and linguistics” (Sowa)

• Explicit the UI meta models

• Appropriate for extension

• Easy to represent abstraction levels

Interactor

Choice

Simple Choice

Multiple Choice

Specifyer

Radio Button

Combo Box

6© TAMODIA’06

Approach• Based on a semantic network for

– Capitalizing UIs

– Supporting plasticity both at design and run time

• Semantic network for plasticity– Node UI models

– Edge(arcs) Relationships between UI models

7© TAMODIA’06

Semantic network for plasticity• Nodes

– Describe an interactive system at any level of abstraction

• Relationships– Describe how a node is related to another one

C&T AUI CUI FUI: Is deduced from

Choice in a known set

Choice of months

8© TAMODIA’06

Semantic network for plasticity• Choice at C&T level


Simple choice Constraints: min = max = 1

Restriction ofpartial/non exclusive

Choice a month Constraints: Type = MONTH

|||Marks *

||| S_IS: set of OBJECT

Encapsulates

Marker Type: a TYPE Value: TYPE Obj: OBJECT

Specializespartial/exclusive

Choice by ||| and marks Constraints: m:Marks() m.Type = Bool #Marks() = #L_poss U{m:Marks() | m.Value = true m.Obj} = L_eff U{m:Marks() | m.Value = false m.Obj} = L_poss \ L_eff |||().S_IS = Marks()

Accumulator Constraints: Eff().Type = Poss().Type Eff().S_poss = S_eff Poss().S_poss = S_poss\S_eff If Poss().TaskDone() then L_eff U= Poss().L_eff If Eff().TaskDone() then L_eff() \= Eff().L_eff

9© TAMODIA’06

Semantic network for plasticity• Interleaving at AUI level


||| without navigation

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond

Specializestotal / exclusive

Cont *Container Dialog level

Constraints: Represents(DS, S_IS))

DS

Encapsulates

Concretizespartial/non exclusive

||| Dialog level

Constraints: DS = {c:Cont() c.DS} is:S_IS (c:Cont() | c.S_IS = {is}) c:Cont() (is:S_IS | c.S_IS = {is})

|||

DS0 DSi

||| monospaceConstraints: Cond #{i:Integer|DSi.active}=1

Specializespartial / non

exclusive

||| sequenceConstraints: Cond i1,i2,i3:Integer | i1<i2<i3 DSi1.active DSi2.active DSi3.active

||| sequential accessConstraints: Cond i:Integer | DSi.active at t DSi.active at t-1 DSi-1.active at t-1 DSi+1.active at t-1

10© TAMODIA’06

Plasticity questions

TC

AUI

CUI

FUI

11© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Choice in a known set Type: a TYPE S_poss: set of Type S_eff: set of Type min, max : Integer Constraints: S_eff S_poss #S_eff [min; max] User Task: Specify S_eff

Choice in a partially known set If Specifyer.TaskDone() then Choice.S_eff U= {Specifyer.elmt} Constraints: Spec().Type = Choice().Type

Specifyer Type: a TYPE elmt: Type Task: Specify elmt

Is composed of Simple choice

Constraints: min = max = 1

Restriction of partial/non exclusive



Specializes partial/exclusive

Encapsulates



Extends

Encapsulates


TC

AUI

CUI

FUI

12© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Cont *










Encapsulates



Extends

Encapsulates


Concretizes partial/non exclusive


||| Dialog level Constraints: DS = {c:Cont() c.DS} is:S_IS (c:Cont() | c.S_IS = {is}) c:Cont() (is:S_IS | c.S_IS = {is})

|||

DS0 DSi

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond

||| monospace Constraints: Cond #{i:Integer|DSi.active}=1

Specializes total / exclusive

Specializes partial / non

exclusive

||| sequence Constraints: Cond i1,i2,i3:Integer | i1<i2<i3 DSi1.active DSi2.active DSi3.active

||| sequential access Constraints: Cond i:Integer | DSi.active at t

DSi.active at t-1 DSi-1.active at t-1 DSi+1.active at t-1

Container S_IS : set of OBJECT

Container Dialog level Constraints: Represents(DS, S_IS))

DS

Concretizes total/exclusive

Encapsulates

TC

AUI

CUI

FUI

13© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Cont *










Encapsulates



Extends

Encapsulates





|||

DS0 DSi

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond




exclusive






DS


Encapsulates

TC

AUI

Scrollable monospace

Scattering |||

Scrollable listbox Constraints Typeof(|||) = ScrollList m : Marks() typeof(m) = Hignlighter

Linear |||

Matricial |||

A SPACE name

A SPACE

A SPACE name

A SPACE

ScrollList

Pie |||

CUI

FUI

Concretizes partial / non

exclusive


exclusive


exclusive


exclusive


exclusive


exclusive

14© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Cont *










Encapsulates



Extends

Encapsulates





|||

DS0 DSi

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond




exclusive






DS


Encapsulates

TC

AUI

TK torus month chooser


Implements partial / non

exclusive

TK Scrollable listbox


Scattering |||


Linear |||

Matricial |||

A SPACE name

A SPACE

A SPACE name

A SPACE

ScrollList

Pie |||


exclusive

CUI

FUI


exclusive


exclusive


exclusive


exclusive


exclusive


exclusive

Is there a choice tailored for months?

?n : Choice : NODE()<-REL(type~=GDD_inheritance)*

<REL(type==GDD_restriction&&constraint~=“type==MONTH”)<-$n()

15© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Cont *










Encapsulates



Extends

Encapsulates





|||

DS0 DSi

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond




exclusive






DS


Encapsulates

TC

AUI




exclusive



Scattering |||


Linear |||

Matricial |||

A SPACE name

A SPACE

A SPACE name

A SPACE

ScrollList

Pie |||


exclusive

CUI

FUI


exclusive


exclusive


exclusive


exclusive


exclusive


exclusive

Are there pie menu equivalent UIs without navigation?

?n : InterleavingNoNav :Node()<-REL(type==GDD_inheritance)*<-$n()

16© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Cont *










Encapsulates



Extends

Encapsulates





|||

DS0 DSi

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond




exclusive






DS


Encapsulates

TC

AUI




exclusive



Scattering |||


Linear |||

Matricial |||

A SPACE name

A SPACE

A SPACE name

A SPACE

ScrollList

Pie |||


exclusive

CUI

FUI


exclusive


exclusive


exclusive


exclusive


exclusive


exclusive

What are the possible TK choices?

?n : Choice : NODE()<-REL(type~=GDD_inheritance)*<REL(type==GDD_implementation)<-$n()

Cartesian product on Marker [TK] and Interleaving [TK]

17© TAMODIA’06

Demonstrator• Comet Inspector

– Comet = Polymorphic interactors

18© TAMODIA’06

Conclusion• Contribution : a semantic network that

– supports plasticity questions both at design and run-time

– is opened to tailored UIs

– Can be located inside and outside interactors

• Strength for plasticity– Easy to extend: Just add nodes and edges

– Enlarges plasticity without extra programmation : The effort is set on the exploration mechanisms

– Preserves plasticity questions

• Drawbacks– People must agree on a common vocabulary and structure

– Who controls the network?

19© TAMODIA’06

Perspectives• Theoretical

– Agreement on• The UI levels of abstraction• The UI relationships• The structure of the semantic network

– Language to describe UIs and UI transformations

– Language to build plasticity queries

– How to include patterns

• Practical– Semantic network server

– Visualization techniques for guiding the user (designer and/or final user)META-UI

– Integration to Comets (plastic interactors)

– Usability of the approach

21© TAMODIA’06

Semantic network for plasticity• Who use?

– Conceptors• design time : Build different version• Run-time : Express substitution

– Users• Personalize the interface (change this radio button choice with…)

• Who extend?– Conceptors : new systems or new way to present

– Users : End user programming (on the fly application building)

22© TAMODIA’06

Semantic network for plasticity• Nodes

– Describe an interactive system at any level of abstraction

• Example; Choice at the C&T level

C&T AUI CUI FUI: Is deduced from


23© TAMODIA’06

Y is deduced from X

Semantic network for plasticity• Relationships

– Describe how an interactive system is related to an other.

x and y are nodes

x has a X level of abstraction

Y has a Y level of abstraction

It can be

• total/partial

• Exclusive or not

24© TAMODIA’06


Eff

Poss

Spec

|||

Marks *

Cont *










Encapsulates



Extends

Encapsulates





|||

DS0 DSi

DS0 DSi

|||

||| with navigation

DS0 DSi

|||

Cond




exclusive






DS


Encapsulates

TC

AUI




exclusive



Scattering |||


Linear |||

Matricial |||

A SPACE name

A SPACE

A SPACE name

A SPACE

ScrollList

Pie |||


exclusive

CUI

FUI


exclusive


exclusive


exclusive


exclusive


exclusive


exclusive

26© TAMODIA’06

Semantic network for plasticity• What are all implementation of a choice?

• Is there a choice tailored for months?

• What is the closest version of a TK pie-menu in HTML?

• What is the greatest common functionality between this 2 systems?

• What are equivalents to pie menu ensuring no navigation?

• Could this system be decomposed into smaller ones?

• What are the possible TK choices for choosing a month?

© tamodia’061 the comets inspector: towards run time plasticity control based on a semantic...

Documents