Download - 1a8 parrallel ui rendering
1/16Kris Van Hees & Jan Engelen
Parallel User Interface Rendering~
Accessibility for Custom Widgets
Kris Van Hees & Jan EngelenKatholieke Universiteit Leuven
Department of Electrical EngineeringESAT - SCD - DocArch
2/16Kris Van Hees & Jan Engelen
● Introduction● Parallel User Interface Rendering● Widgets● Cosmetic widgets● Conclusions● Future...
Agenda
3/16Kris Van Hees & Jan Engelen
● GUIs in all aspects of life: computers, MP3 players, home appliances, mobile phones, ...
● Inherent visual interaction model poses a complication for users with disabilities (especially blindness)
● Mixing graphical toolkits within a single environment complicates matters even more (Linux: Qt, GTK, Athena, …)
● Custom widgets: often not supported in assistive technology without fancy scripting (if available)
Introduction
4/16Kris Van Hees & Jan Engelen
● Abstracting the user interface offers advantages:– Unified user interaction semantics across toolkits and
modalities– Retaining flexibility in rendering for multiple modalities– Assistive technology as 1st generation rendering
● Parallel User Interface Rendering (PUIR) provides for equivalent concurrent renderings of the UI
Introduction (cont...)
5/16Kris Van Hees & Jan Engelen
● User interface defined in abstract form (XML file)– Effectively defines data and how to present data
● Abstract UI description interpreted by an AUI engine– Handles user interaction semantics
● UI presentation rendered by modality-specific agent(s)● Rendering agent depends on system specific toolkits● Rendering agents provide presentations concurrently
– Provide both visual and non-visual rendering
Parallel User Interface Rendering
6/16Kris Van Hees & Jan Engelen
● Interprets an abstract UI description (XML file)● Implements user interaction semantics● Encapsulates data in the UI● Provides focus management● Dispatches events to the application and rendering agents● Can be called from application and rendering agents● Has no knowledge of implementation details of
application and rendering agents
PUIR: The AUI engine
7/16Kris Van Hees & Jan Engelen
● Java Swing based in-process rendering agent● Complex changes to bypass user interaction in widgets● Reduced to provide merely the presentation of widgets● Some widgets operate slightly differently● Pointer device UI interaction interpreted in the rendering
agent:– Necessary because position is tied to GUI context– Semantic operations mapped to AUI operations
PUIR: GUI rendering agent
8/16Kris Van Hees & Jan Engelen
● Remote rendering agent● Basic speech output presentation as proof-of-concept● Proxy objects for transparency and caching● Equivalent with GUI rendering agent:
– Receives the same events– Synchronized with other rendering agents
PUIR: Non-visual rendering agent
9/16Kris Van Hees & Jan Engelen
● Keyboard:– Textual input– Navigation, text selection, editing, …– Processed directly by AUI (for proof-of-concept)
● Mouse:– Processed by rendering agent– Position to be interpreted within graphical context– Semantic user interaction delegated to AUI engine
PUIR: Input processing
10/16Kris Van Hees & Jan Engelen
● Braille keyboard:– Coded textual input translated into text– Text processed as if entered on regular keyboard– Limited cursor addressable positional input– Positional input interpreted in context– Semantic user interaction delegated to AUI engine
PUIR: Input processing (cont...)
11/16Kris Van Hees & Jan Engelen
● Two entities per widget:– Abstract widget:
● Operates entirely within the AUI engine● Implements user interaction semantics● Encapsulates data
– Widget presentation:● Usually one per rendering agent● Implements rendering in output modality
Widgets
12/16Kris Van Hees & Jan Engelen
● Design:– Define UI in abstract UI description– Typically by means of design tool– Rendering specific information can be included
Widgets (cont...)
13/16Kris Van Hees & Jan Engelen
● Runtime construction:– Instantiation of UI based on AUI description– For every widget:
● Find built-in abstract widget class● If none, search in alternate locations● If none, UI cannot be instantiated
Widgets (cont...)
14/16Kris Van Hees & Jan Engelen
● Rendering:– One-to-one correspondence between abstract widget
class and presentation class– For every widget:
● Find built-in widget presentation class● If none, search in alternate locations● If none and cosmetic widget, use core alternate
(more on this 2 slides from now...)
● If none, use default place-holder widget
Widgets (cont...)
15/16Kris Van Hees & Jan Engelen
● Rendering (cont...):– Rendering agent knows what widgets are supported– Rendering agent can alert user appropriately– Rendering agent can query information from AUI
about the unsupported widget– Accidental user interaction is guarded against
Widgets (cont...)
16/16Kris Van Hees & Jan Engelen
● Widgets with a very specific presentation
● Typically derivative of an existing core widget
● User interaction semantics remain the same
● AUI description specifies equivalence between cosmetic widget and core widget
● Core widget can act as default presentation when no custom presentation class is found
Cosmetic widgets
17/16Kris Van Hees & Jan Engelen
● Left-to-right:
– Four regular buttons
– VU-meter style widget: equivalent with numeric label
– Four knobs: equivalent with discrete slider
– Four indicator lights: equivalent with label
– Four regular buttons
Cosmetic widgets (cont...)
18/16Kris Van Hees & Jan Engelen
● Parallel User Interface Rendering– Powerful technique for Design-for-All– Coherence between concurrent equivalent renderings– Generic: not just for non-visual renderings– Robust handling of custom widgets– Support for custom widgets independent from
applications and specific uses– Can be used in automated application testing as well
Conclusions
19/16Kris Van Hees & Jan Engelen
● Completion of proof-of-concept● More testing● Public release
● Default presentation of custom widgets merely based on abstract widget information
● Remote operation of applications
At all stages, feedback from real users is be crucial!
Future...
20/16Kris Van Hees & Jan Engelen
Kris Van Hees & Jan EngelenKatholieke Universiteit Leuven
Department of Electrical EngineeringESAT – SCD – DocArchKasteelpark Arenberg 10
B-3001 HeverleeBelgium