56 o oo ccf_final

AEGIS Workshop and International Conference, Brussels

Graphic Symbol Support in Open/LibreOffice Shaping Up Graphic Symbol Server and Inline Symbol Font

Display Based on the CCF

(Mats Lundälv, SU-DART)


Symbol Supported Writing and Reading in LO/OO Writer

● The rationale for this work and how it's done● Primary and secondary target user needs● Current state: How the needs are met● The technology behind it● What the pilot testers say● Where we're going from here – within and

beyond AEGIS – conclusions


Symbol Supported Writing and Reading in LO/OO Writer – the Rationale

Inclusive Multi-modal Language SupportThe Concept Coding Framework (CCF) multi-modal and multilingual support in a free standard office suite offers a truly inclusive productive environment for a wide range of users and needs, e.g.:

● Early literacy or second language learning● Cognitive impairments resulting in reading and

writing difficulties and/or need for AAC● Supporting the facilitators


Target Personas – Jane, an AAC User

Jane uses a voca with a symbol display and voice output to communicate. She'd like to use a standard office application to create and read documents based on her symbol communication skills.


Target Personas – Adam, a User with CIAdam reads and writes some, but is limited to short and simple texts. Graphic symbol representation – in addition to speech – will help him interpret and type more demanding words and text – and may also help to give him basic access to the content of some English text .


Symbol Supported Writing and Reading in LO/OO Writer

Writing and reading with symbols displayed outside text helps comprehension and spelling

No symbol insertion in text: The CCF Symbol Server displays looked-up concepts and symbols (ARASAAC + Bliss) as words are written, and/or as the text cursor is moved in the text


Writing with full AAC (Blissymbol) support

Writer with CCF Symbols and SAW


Helpers producing symbol material

Mixed mode document – b&w symbol fonts, as well as colour graphics insertion in text


Concept Coding Options dialog


The CCF Symbol Support Desktop Package1) The CCF Symbol Server application (with its language

and symbol database resources), and 2) The CCF Symbol Support Extension for Writer


CCF – the Concept Coding Framework… what is that?


CCF - Architecture

CCF CCF Assisted Technologies(Proprietary)

AssistiveDatabase

AssistiveOntologyComplementary

ReferenceOntology (CRO)

Concept Code Definitions (CCD)

BaseReference

Ontology (BRO)

CCF Bridge Assistive BridgeWordNet


CCF - Architecture


Pilot Testers and Experts - Feedback

Appreciated as an already fully usable and valuable application for a wide range of users and user needs. Those suggested by users, tutors and experts include:● AAC symbol users, as a tool for communication, writing and

reading – typically supported by additional AT such as on-screen symbol charts for input, and text-to-speech (TTS) feedback

● Users with cognitive disabilities of different kinds, rendering a need for anything from extensive to occasional graphical symbol feedback for functional reading and writing – typically in combination with TTS

● Temporary support in early stages of second language learning● Mainstream use for early literacy introduction


Advantages and disadvantages compared to competitive alternatives

Main listed advantages and disadvantages in relation to existing commercial software with corresponding functionality (e.g. the Widgit Communicate: series) include:

Advantages: ● Standard office software base with freely available extended

functionality – accessible across a wider range of socio-economic circumstances and environments (school, home, public, private)

● Cross-platform availability (Windows, Mac OS X and Linux/Unix)● Based on openly defined technologies and open-source software

(allowing novel integration and innovation in a longer perspective)


Advantages and disadvantages compared to competitive alternatives

Main listed advantages and disadvantages in relation to existing commercial software with corresponding functionality (e.g. the Widgit Communicate: series) include:

Disadvantages: ● Functionality and UI design less tailored (and tailorable) to some

special needs requirements● Less sophisticated language technology and vocabulary support

compared to the most advanced proprietary alternatives (though in parity to, or beyond, several other)

● Possible issues in relation to future maintenance and support


Remarks on existing or missing features of the second alpha version

General comments: ● Good progress since previous first alpha vesion● Need for simplifications of management – e.g. installation, manual

handling of the separate CCF Server window etc. ● Saving, restoring and sharing of Open Document Text (ODT)i

documents with concept coded content and included symbol representations.

● Limitations of the vocabulary resources, requested improvements. ● Also expressed an understanding and acceptance of limitations as

a trade-off for the advantages of the free and wide availability.● Stressed importance of clear information about functionality and

limitations, to create realistic expectations among potential users.



Specific comments: ● Highly appreciated the ”out-of-text” symbol display. However, they requested

improvements in the management of the CCF Symbol Server display. Suggested solutions included:

● The CCF server display window floating on top of all other windows – ideally with an option to let the symbol display follow the text insertion caret in the Writer document

● The symbol display being moved into the Writer window as a floating or docked display panel (similar to the ”Navigator” etc. panels)

● Appreciation the Ruby Annotation display of symbols over their corresponding words. However, they regret the loss of colour in the ARASAAC

● A specific request has been:● Option to control the background colour of the symbols, which by some evaluators was

now thought to be in a slightly too dark shade of blue-gray



Specific comments: ● Great with Ctrl+G keyboard shortcut to select preferred representation, but users

and experts also wanted to be able to select by pointing and clicking● Swap the original word to a new custom one accompanying the same symbol,

e.g. replacing the word ”dog” under a dog symbol with ”Pluto”● Specific requests for improvements of the word-concept-symbol look-up

included:● Better priority (based on frequency and probability) for first choice among concept

alternatives● Capability to handle two word (and potentially multi word) i.e. Suggestion of matching

concepts and symbols while typing a word (and not only after completing the word with Space, as now)

● The obvious requests for support of more text languages and other preferred symbol libraries


Conclusions and Future Plans

Based on the input from the first and second pilot tests, and the revised user needs and requirements specifications, a number of issues will be addressed:● Usability and UI enhancements:

● Smoother integration of the CCF Symbol Server and the OO/LO Writer extension, including:

● Automatic start of the CCF server, if it's not running, when called from the extension

● CCF server symbol display window always floating on top, possibly with text cursor/caret following, alternatively adding a symbol display panel to the extension for presentation within the Writer window

● Selection of preferred concept-symbol representation by point and click (in addition to the current shortcut key command)

● Easier access to the settings panel, better grouping of the settings options, and improved design of the the toolbar panel


Conclusions and Future Plans

… more issues and requests to be considered:● Functional enhancements:

● Providing an option for the user to change text word for a symbol represented concept

● Support for two word (possibly three word) concept-symbol look-up (e.g. “all right” and “to go” to be joined to one concept representation)

● Possibly providing word-concept-symbol “prediction” (while typing)


Further perspectives (some beyond the Writer extension and AEGIS)

Within the scope of AEGIS:● The CCF ontologies and databases used by the CCF Symbol Server will be

in need for continuous upgrades on several levels:● Current resources need to be maintained in short term perspective with:

● improvements of the representational databases (in particular for ARASAAC symbols)

● improvements of lexical support for English and Swedish with better coverage, higher hit rate for most probable concept, support for multi-word concepts, etc.

● Additional basic support for Spanish and Dutch is underway ● A more strictly controlled and longer term re-build of the CCF ontologies as part of

the employment of the CCF technology for the AAC app for Android ● Special Access to Windows (SAW) will be upgraded to support integration

with the CCF Symbol Server ● The CCF Symbol Server is already used as an online cloud server for the

provision of vocabulary resources for the first prototype of the Android AAC app


The same open and free CCF technology applied for AAC on the mobile device


Further perspectives (some beyond the Writer extension and AEGIS)

The longer term CCF perspectives:

● The work to integrate the CCF technology, as well as the graphic language of Blissymbolics, as part of the ISO TC 37 multi-modal terminology standards, has been initiated within the AEGIS project. As this is a longer term task, it will continue far beyond the duration of the project.

● The general maintenance of the CCF vocabulary resources is basically a never-ending task of refinement of the support for the current languages and representational resources, and the subsequent addition of new ones.

● The above tasks will in turn require maintenance and further development of the exploitation and business plans which are in progress as part of the AEGIS work.


Final conclusions

● The CCF Symbol Support Extension for OpenOffice/LibreOffice is in preparation for its beta release in early 2012.

● This will provide a new basic level support for access to text content, and a new and wider range of opportunities to communicate for people with cognitive, speech and language impairments, and for those with multiple impairments.

● Likely to be appreciated by wider groups of users in general, in particular in connection to early literacy and second language introduction.

● Opens up new perspectives for providing more inclusive accessible ICT environments – for people in general, across socio-economic conditions, and for users with more challenging needs due to a wide range of impairing conditions.

● This will not replace the need and space for dedicated ICT products in this area, but will serve as a complementary basic and more widely available entry level for this kind of language support.

●


Users in actionFrom Anna (tutor – in translation)“Hi!I've tested the OpenOffice ext with one of our students who comes to our school here once a week. She was so happy when she could use her SAW on-screen-keyboard to write with blissymbols...”

To the right, the attached first produced blissymbol and text document “Natalie goes to a party” (variation over Cinderella)

User at home with SAW + LO/OO Writer

56 o oo ccf_final

Technology