model-based ui design - ptolemy project · 1! user interface modeling" model-based ui design!...
TRANSCRIPT
![Page 1: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/1.jpg)
1
User interface modeling���Model-based UI design
Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences Norwegian Univ. of Science and Technology
![Page 2: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/2.jpg)
2
Trondheim, Norway
![Page 3: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/3.jpg)
3
User interface modeling���Model-based UI design
1. Background and���framework for classifying design representations
2. (Examples of) Models for development of UIs 3. Diamodl 4. ptui – ptolemy-based tool for development of UIs
![Page 4: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/4.jpg)
4
Many models capture our knowledge about the world
Information
Actions
Tools
People
![Page 5: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/5.jpg)
5
Roles representations play
• Semantic – accurately and completely capture knowledge
• Communicative – support communication among designer and end-users
• Constructive – stimulate, guide and constrain further design
• Analytic – support interpretation and evaluation
• Engineers and designers focus on different roles
![Page 6: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/6.jpg)
6
From informal���representations...
... to models
![Page 7: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/7.jpg)
7
Canonical Abstract Prototypes [Constantine] – semi-formal sketching
![Page 8: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/8.jpg)
8
What aspects of a UI do we want to capture?
• Structure – hierarchical structure of interaction elements
• Information – what information is accessible in which parts of the UI – what is the relationsship between information in various parts of
the UI • Behavior
– when are the various interaction elements active – how are changes in the UI triggered by the user
• Style – non-functional aspects, like layout, use of colors, fonts etc.
![Page 9: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/9.jpg)
9
Four phases of MBDUI 1. Model and generate
– model your domain – generate UI from canned knowledge and pre-compiled rules
2. Computer-Aided Design of UI – abstract models/representations of UI – explicitly represent design knowledge – model editors and tools for applying design knowledge
3. Task-based UI design – can’t design usable interfaces without knowing the user and tasks – base design of UI on task model (goals, structure and dependencies)
4. Contextualizing and adapting design models – focus on context of use – target multiple devices
![Page 10: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/10.jpg)
10
Design representation���classification framework
perspective
formality
granularity
level of formality
problem vs. solution abstract vs. concrete
level of detail
![Page 11: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/11.jpg)
11
Perspective and granularity dimensions
![Page 12: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/12.jpg)
12
Dutch [van der Veer] -���task models as activity charts
![Page 13: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/13.jpg)
13
ConcurTaskTrees [Paternò] – task hierarchies with temporal operators
![Page 14: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/14.jpg)
14
Dialog graphs [Forbrig] –���Relating tasks to dialog
![Page 15: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/15.jpg)
15
UsiXML [Vanderdonckt] – A family of XML-based notations for UI elements
![Page 16: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/16.jpg)
16
Pet shop [Palanque] – Modeling safety critical UIs with ICO PetriNets
![Page 17: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/17.jpg)
17
Cameleon framework – targeting multiple devices
![Page 18: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/18.jpg)
18
Lots of pragmatic approaches ���(read: non-academic and useful) • XML-based formats for describing user interface layout and style
– XHTML (W3C) , XAML (Microsoft), JavaFX (Oracle), XUL (Mozilla) – template languages for web pages
• DSLs – Ecore-based: Eclipse 4’s workbench model, Wazaabi – Xtext-based: APPlause, MOBL, Agentry
• Application modeling – Esito’s Genova – business applications for the desktop and web – WebRatio - business applications for the web
• Standardization – WebML – IFML (in progress) – Model-Based User Interfaces (MBUI) Working Group
![Page 19: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/19.jpg)
19
IFML – Interaction Flow Modeling Language • OMG RFP • Proposal by
WebRatio++ • Abstract UI model • Functional units and���
view containers • Dataflow and���
control/activation signals
![Page 20: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/20.jpg)
20
Dialog modelling with DiaMODL
• Based on Pisa���interactors and���Harel’s Statecharts – interactors, gates and
connections – hierarchical states – transitions, events/actions,
conditions • Abstraction of IO function • Composition in terms of
– interactor structure – state hierarchy (and, or)
![Page 21: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/21.jpg)
21
Generic interactor abstraction
• Notation for generic input og output components
• Dataflow-oriented • Interactor mediates
information in two directions: – output: system to user – input: user to system
![Page 22: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/22.jpg)
22
Scalable notation
• Specification of concrete interaction object’s functionality – output and input interface
• Description of construction of concrete interaction objects – composition of sub-interactors – string input combined with���
parsing and unparsing
• Same abstract description,���many alternatives
![Page 23: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/23.jpg)
23
More complex interaction objects
• Functionality defined in terms of configuration of domain objects
• Utilise power of���domain modelling language – Output: set – Input: subset
Alternative implementation
![Page 24: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/24.jpg)
24
Configuration of larger elements
folder list mailbox content single message
![Page 25: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/25.jpg)
25
Interactor-���based���GUI-builder
![Page 26: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/26.jpg)
26
Integrating domain and dialog modeling • Eclipse-based editor [CADUI’06]
![Page 27: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/27.jpg)
27
Prototyping with Diamodl
![Page 28: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/28.jpg)
28
Prototyping with Diamodl
![Page 29: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/29.jpg)
29
Application architecture
• The whole runtime state is captured as coordinated graphs of data • The widget hierarchy is continuously rendered on a device
Domain data Widgets
events, bindings, actions & activation logic
Interactors
![Page 30: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/30.jpg)
30
Rendering widgets • Ecore model of toolkit, with instances rendered in Eclipse view
![Page 31: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/31.jpg)
31
Rendering widgets across platforms
![Page 32: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/32.jpg)
32
Moveable application
![Page 33: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/33.jpg)
33
Shareable application
updates
![Page 34: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/34.jpg)
34
Distributed application
?
![Page 35: Model-based UI design - Ptolemy Project · 1! User interface modeling" Model-based UI design! Hallvard Trætteberg, Associate Professor Dept. of Computer and Information Sciences](https://reader030.vdocument.in/reader030/viewer/2022040901/5e71aa8f7878993204694efb/html5/thumbnails/35.jpg)
35
ptui –���ptolemy-based tool for UI development • Diamodl
– concepts are very close to Ptolemy’s – interactors, computations and variables can all be modeled as actors – its weakness, the (lack of) semantics, is Ptolemy’s strength
• Ptolemy can provide – a (set of possible) semantics – a solid runtime platform
• Ptolemy – describes the behavior of a cyber-physical system, but – has poor support for modeling user interaction
• Diamodl can provide – an approach to integrating UI elements – runtime support for rendering widgets locally or in a browser