semanticlife ui, visualizations and accessibility institute of software technology & interactive...
DESCRIPTION
SemanticLIFE Day UI Design for SemanticLIFE In first step to kick start the activity, general UI standards were defined so as to generate suggestions from group members working on different components of the system, and at the same time maintaining homogeneity between UI for these components.TRANSCRIPT
SemanticLIFE UI, Visualizations and Accessibility
[email protected] of Software Technology &
Interactive Systemshttp://www.ifs.tuwien.ac.at/
Vienna University of Technology
13.6.2006 SemanticLIFE Day 2006 2
Outline• UI for SemanticLIFE• Visualization• Accessibility
13.6.2006 SemanticLIFE Day 2006 3
UI Design for SemanticLIFE
• In first step to kick start the activity, general UI standards were defined so as to generate suggestions from group members working on different components of the system, and at the same time maintaining homogeneity between UI for these components.
13.6.2006 SemanticLIFE Day 2006 4
The initial UI requirements – 1/2
1. The user interface is to be interactive2. The user interface should be stable3. Ability to generate query from the same user
interface4. Ability to do annotation on the same user interface5. Both graphical and textual display6. Ability to view classes (primitive + user defined)
from the ontology
13.6.2006 SemanticLIFE Day 2006 5
The initial UI requirements – 2/2
7. Ability to view associated classes, annotations and properties as described in the ontology
8. Ability to see context menu for each selected item
9. Display mechanism for each item specified in ontology, and thus under user control
10.The layout of the display under user control by drag and drop style
13.6.2006 SemanticLIFE Day 2006 6
Initial UI Standards – 1/4
Screen is divided into the following main parts. (The display mechanism for this one would also be specified in the ontology).
• Main menu bar at the top• Status bar at the bottom• A tabbed pane (eclipse style tabbed panes) at the left
showing the categorized class hierarchy. One of the categories can be for system management covering user profile, query, annotation, ontology management, etc.
13.6.2006 SemanticLIFE Day 2006 7
Initial UI Standards – 2/4
4. A tabbed pane at the right for filtering / control options. The default is time based filtering managed by a GUI component, text entry fields and horizontal time range sliders.
5. A collection of tabbed panes in upper middle part for visualization of contents in graphical mode, query generation, annotation, user profile management etc.
13.6.2006 SemanticLIFE Day 2006 8
Initial UI Standards – 3/4
5.1. A tabbed pane in lower half of each of the above tabbed pane for visualization of contents in text mode (rdf/xml), whether query in xml, annotation in xml or the results in xml (editable except for contents visualization pane
5.2. In turn, each of these tabbed panes would have the same default layout (from point 1 to 6)
13.6.2006 SemanticLIFE Day 2006 9
Initial UI Standards – 4/4
6. Upon selecting the appropriate operation in main contents visualization pane, the corresponding tab will also become active. For example, right click on a class will display a context menu (generate new query, new annotation, etc.). Selecting one of these options (say “generate new query”) will activate the query tab as query UI. The changes made on query tab will take effect on main contents visualization. Similar will be the case for other tabs.
13.6.2006 SemanticLIFE Day 2006 10
Next step - Designing the questionnaires for gathering requirementsTask Analysis: It is important to determine
what tasks user needs to perform by our system. Then we can decide where to perform those tasks distributed in UI and business components, or entirely in business components. Later, the detailed design about UI and business components would follow. At least, try to fill 2
13.6.2006 SemanticLIFE Day 2006 11
data collection forms (one for component of your interest like annotation / query / classes & instances / ontology management / user profile / dynamic associations / privacy & access control / …, and the other for over all system Slife in context of Scenario 2 – Researcher working on multiple projects). The following list is numbered according to importance of each item.
13.6.2006 SemanticLIFE Day 2006 12
1. List of tasksa. Name the tasks or operations to be performed by
this component. For example, annotate one class to another class; annotate one instance to another instance, etc.
b. Mark the task as Mandatory or Non-Mandatory. By mandatory tasks we mean those tasks without which the component is considered to be Non-Functional?
c. Mark the task as executing on individual items, or in batch mode. For example, by seeing an email item we may try to annotate it instantly, instead of dragging and dropping many email items on ANNOTATION icon, and then do the annotation sometime later.
13.6.2006 SemanticLIFE Day 2006 13
d. Task frequency; how many times per day, per hour, etc
e. Task priority; High, Medium, Low. High priority tasks can be very crucial and urgent for the user.
f. Is it a management task such as saving, importing annotation, or core functionality like doing the annotation itself?
g. Task dependency; Describe dependency on other task(s) maybe in other components
h. Any other specificity about nature of task
13.6.2006 SemanticLIFE Day 2006 14
2. Concerned information objectsa. Type of information object; the concerned
information objects can be class, instance, property …or the ontology. For example, the named queries can also be treated as first class objects for query.
b. Nature; By nature we mean if it is an email, web page, contact, picture, blob etc
c. Size; the estimated size will vary depending upon the concerned information objects for your component
d. Frequency; Try to give an estimate about the frequency of incoming information objects to your component
e. Any other specificity about nature of information items
13.6.2006 SemanticLIFE Day 2006 15
3. Filtering / control optionsWhat are the options for user control? For example to filter the items according to date, popularity, or some other criteria. Describe the result fields after applying filtering, if possible.
4. Tracking of task stateOne task may consist of several steps. Is it beneficial to track those steps for replaying later?
5. Status messagesIs it necessary to display the status message for this task? If yes, what status information is to be displayed?
6. Visible / invisible navigationAlthough a task can be made automatic as far as possible, however, it is sometimes beneficial to display the task execution steps to the user for developing a mental image
13.6.2006 SemanticLIFE Day 2006 16
7. Reversible actionsLike undo functionality. During execution of a sequence of queries, is it beneficial to undo the selected queries instead of undoing all.
8. Copy / paste functionalityFor example, apply one type of recently made annotation on another collection of items. Or copy one sub-query from a named query to another named query.
9. Scratchpad functionalityIs it useful for the task to drop the items upon scratchpad, for taking action later?
10. Active Items list functionalityIs it useful for the task to have frequently or recently addressed items on the Active Items list
11. What should be the default view in your opinion?12. Any other thing you feel necessary to describe.
13.6.2006 SemanticLIFE Day 2006 17
Paper prototypes – 1/2 (after 3/4 iterations)
13.6.2006 SemanticLIFE Day 2006 18
Paper prototypes – 2/2 (after 3/4 iterations)
13.6.2006 SemanticLIFE Day 2006 21
Sample screenshot
13.6.2006 SemanticLIFE Day 2006 22
13.6.2006 SemanticLIFE Day 2006 23
Items categorization
13.6.2006 SemanticLIFE Day 2006 24
Visualizations for SemanticLIFE (In progress)• In principle, multiple visualizations best
suited to the task at hand, are to be supported for visulaizing the personal information evolving over time
• The currently planned visualizations are timeline view for Slife information items, graph view for ontology and ontology instances. Later the items or concepts will be visualized in the most suitable metaphor.
13.6.2006 SemanticLIFE Day 2006 25
Sample timeseries visualizations
13.6.2006 SemanticLIFE Day 2006 26
Sample timeseries visualizations
13.6.2006 SemanticLIFE Day 2006 27
Implementation• Timeseries plugin implemented for displaying
information from ontologies• Visualization pipeline
– Implemented using successive XSL stylesheets for transfroming the queried data for presenting to user
– Currently primitive operations introduced like sorting, grouping, aggregating, etc.
– Later more comprehensive transformations are planned according to scenario requirements
13.6.2006 SemanticLIFE Day 2006 28
Planned Interactive Timeline view• Display multiple timeseries charts (for different
infromation items, and/or concepts in ontology related with particular scenario) on the same plane with the possibility to interact with each other
• Possibility to select a portion of timeline and zoom-in for detailed view of items
• Possibility to search for discrete items using control panel
• Possibility to interact with individual items for further operations like query and annotation with other items/concepts
13.6.2006 SemanticLIFE Day 2006 29
Accessibility (In Progress)Ability to store user’s personal information items
in our prototype with the facility to make annotations leads to our next step – “make the system accessible for people with special needs”
A separate accessibility plug-in is planned which could be used by visualization plug-in and the visualization pipeline.
13.6.2006 SemanticLIFE Day 2006 30
The existing situation
There are different types of:• User impairments (blindness, hearing
problems, motor problems, cognitive problems, etc)
• Interface characteristics (text, sound, UI widgets and their behavior, etc)
• Accessibility guidelines – recommends UI according to impairments
13.6.2006 SemanticLIFE Day 2006 31
– Providing alternate text for the pictures– Avoiding certain colors for the color blind– Avoiding complex table layout for the blind– Difficulty in interaction by
mouse/keyboard due to motor problems– Difficulty in system behavior recall due to
cognitive problems etc.
Every impairment requires a renewed effort for providing accessibility
13.6.2006 SemanticLIFE Day 2006 32
Impairments severity may change over time
– Voice feedback (beep, alarm) would no more work if hearing impairment is introduced or becomes more severe
– Response of tactile feedback will change with severity of motor problems
– Cognitive problems may increase over time etc.
A corresponding change of system behavior over time is required in these cases
13.6.2006 SemanticLIFE Day 2006 33
One impairment may have consequences on another
– Color blind + motor problems– Color blind + cognitive problems (for
example difficulty in comprehension)Consider the case when a human operator
is suffering from either of the above combinations, and he/she has to take quick action to respond (for example in call centre scenario)
13.6.2006 SemanticLIFE Day 2006 34
User Interface characteristics
• Text – size, color, font• Sound – volume, pitch, ...• UI components – text field, button, scrollbar,
...• Certain UI components go well with each
other and not with all• Interface characteristics evolve over time
13.6.2006 SemanticLIFE Day 2006 35
Accessibility guidelines• WCAG make web content (Document Content, Structure,
and Presentation) accessible to people with disabilities1. Provide equivalent alternatives to auditory & visual
content. 2. Don't rely on color alone. 3. Use markup and style sheets and do so properly. 4. Clarify natural language usage 5. Create tables that transform gracefully. 6. Ensure that pages featuring new technologies
transform gracefully. 7. Ensure user control of time-sensitive content changes.
13.6.2006 SemanticLIFE Day 2006 36
8. Ensure direct accessibility of embedded user interfaces.
9. Design for device-independence. 10. Use interim solutions. 11. Use W3C technologies and guidelines. 12. Provide context and orientation information. 13. Provide clear navigation mechanisms. 14. Ensure that documents are clear and simple.
Focus mainly on vision and motor impairments. Other impairments are not significantly addresses which are equally important in today‘s interactive systems.
13.6.2006 SemanticLIFE Day 2006 37
Our system is not just about web content! The primary information items in our prototype SemanticLIFE are:– Emails
– Browsed web pages– Chat sessions– Process state data– Appointments– Documents– File system monitor log
13.6.2006 SemanticLIFE Day 2006 38
Associations between the primary information itemsSome examples are:• Email e1 relatedWith File f1• File f1 relatedWith ProcessStateData p1• File f1 collaboratedWith Contact c1• WebPage w1 relatedWith Email e1• File f1 sharableWith Contact c1
13.6.2006 SemanticLIFE Day 2006 39
Accessibility requirements for our system• Present the primary information items and the
already existing associations between those• Explore and make new associations• Users may vary from normal users to a user
having some special needs• System should be accessible on multiple
devices• Conformance with WCAG guidelines
13.6.2006 SemanticLIFE Day 2006 40
Human Disease
TreatmentCause
Chemotherapy
Drug Therapy
Surgery
Psychotherapy
Physiotherapy
Type
EnvironmentalGenetic
Symptom
Extension of ontology by “Hadzic M. and Chang E., Ontology-based Support for Human Disease Study, HICSS’05”
13.6.2006 SemanticLIFE Day 2006 41
Approach followed – 1/4
• Purpose & scope: Provides mapping information bewteen the user‘s physical and cognitive impairments, and the user interface characteristics
• Possible interaction with other ontologies: DOID, UMLS, ICD9CM, MeSH
13.6.2006 SemanticLIFE Day 2006 42
Approach followed – 2/4Sample user scenarios• To avoid the confusing colors on an interface for
a user with particular type of color blindness• Font size adjustment according to user‘s visual
acuity• Information presentation on the better part of
the screen for a user suffering from Hemianopsia (absence of vision in half of visual field)
13.6.2006 SemanticLIFE Day 2006 43
Approach followed – 3/4Competency questions• What is suitable font size for the user?• Which colored control buttons are suitable?• What is the most suitable screen area for
displaying information?• Which UI components are appropriate?
13.6.2006 SemanticLIFE Day 2006 44
Approach followed – 4/4Impairments Taxonomy• Collection of important terms realted to
impairments• Find the relation between those terms and their
interconnections• A sample impairments taxonomy was prepared
by consulting different resources and by interviewing some physicians
• Determining class hierarchy, properties,... and creation of ontology in OWL-DL using Protege 3.1
13.6.2006 SemanticLIFE Day 2006 45
Part of proposed Im
apirment-U
I ontology
13.6.2006 SemanticLIFE Day 2006 46
Consequences of impairments-user interface ontology
– Helpful in automatically adapting the UI for a specific user
– Helpful in deducing the best match of UI characteristics for a user with multiple impairments
– Historical data would be beneficial for studying the cause-effect relationship between the impairments and the computer interfaces
– Could be useful for rehabilitation purposes
13.6.2006 SemanticLIFE Day 2006 47
Conclusions• Basic user interaction strategy is in place• Some interfaces and visualizations are
imlemented• Initial Accessibility ontology is developed,
and its integration is in progress
13.6.2006 SemanticLIFE Day 2006 48
• Thanks a lot!