north carolina s&h pdf · 2016-08-15 · sample alp graphics - animated grids and scenes...

4/7/16

1

Visual Immersion System™: Communica5on Enhancement for

Au5sm Spectrum Disorders Howard C. Shane

North Carolina Speech & Hearing Associa8on April 27, 2016

Harvard Medical School Boston Children’s Hospital

Visual Immersion System™: Communica5on Enhancement for Au5sm Spectrum Disorders

•  This short course will overview a communica?on strategy for children with a moderate to severe au?sm spectrum disorder known as the Visual Immersion System™. This uses visual supports and state of the art technology and takes advantage of the strong visual processing skills and interest in electronic media of children on the au?sm spectrum.

Released 2008 Available on Amazon as of October, 2014

PREFACE

Purposes

1.   Overview the principles and Framework of the Visual Immersion System

2.   Differen8ate between grids, visual scene and scene cue displays

3.   Describe the ra8onale and methodology for combining visual graphics into the building blocks of language.”

4/7/16

2

Purposes

Overview the principles and framework of the visual immersion system

‘Visual Immersion System’ (VIS) •  A visual instruction system for teaching language

concepts that tend to be difficult for people with moderate to severe autism (e.g., verbs, prepositions and attributes)

•  VIS is a closed visual language, limited to the essential

vocabulary and syntax needed to support the comprehension and expression of practical, everyday communication exchanges

•  Training includes instruction in the virtual environment (video clips) and tabletop environment (photographs, toy figurines and miniature objects), then extends to the natural environment to enable functional communication at home, school and community

Key Ingredients of VIS •  Visual language for both language comprehension and

expression

•  Both mentors and learners use the same visual symbols to communicate with one another

•  Targets language comprehension: viewed as foundation for expression

•  Advanced computer and video technologies attract and maintain children’s attention, provide compelling multimedia language instruction, and enable the use of dynamic visual symbols

• Immersion is Immersion – Sign Language Analogy

• Immerse for VIS or other options

Three Modes of Visual Support Addressed in VIS

I. Visual Instructional Mode (VIM): Visual cues used for the purpose of comprehension, which are imposed as an alternative to, or in conjunction with, speech.

II. Visual Expressive Mode (VEM): Visual cues used for the purpose of expressive communication.

III. Visual Organizational Mode (VOM): Visual cues used to represent the organization of an activity, routine, script, or schedule.

Overview of Visual Immersion Program (VIS)

Seven Communicative Operations

4/7/16

3

Communicative Functions Communication instruction for moderate to

severe ASD:

1. Protesting and Refusal 2.  Organization and Transitions (1 & 2 language that supports organization and control) 3. Requests 4. Directives 5. Comments 6. Questions 7.  Social Pragmatics 3 - 7 language for everyday functions

Communicative Functions

Instruction of the seven functions should not be viewed as hierarchical – not a serial process

Today will cherry pick* from each of the functions

* Cherry Pick

Selec8vely choose (the most beneficial items) from what is available.

cherry-‐pick something Fig. to choose something very carefully. (As if one were closely examining cherries on the tree, looking for the best.)

Cherry Pick Selec8vely choose (the most beneficial items) from what is available.

Communicative Functions Communicative Functions do not include abstract language

(…with liberty and justice for all), passive voice (The book was

read by the boy), complex syntactic structures (If he hadn’t

checked the weather in the morning, then he would have

forgotten to bring his umbrella), figurative language (She flew

to the bookstore) or humor (Why did the chicken cross the

road?).

Purpose

Differen8ate between grids, visual scene and scene cue displays

So we’re all on the same page

4/7/16

4

Display Formats

Grid Display – A grate-‐like screen arrangement containing a target areas into which symbols or text are placed. Targets serve as either a vocabulary end point or lead to rela8onal targets (symbols) on other screens.

Grid based Display

Grid based Topic Display – Direc8ve or Comment -‐ singing

Display Formats Visual Scene Display -‐ Visual scene displays (VSDs) portray events,

people, ac8ons, objects and ac8vi8es against the backgrounds within

which they occur or exist. These scenes are used as an interface to

language and communica8on. A VSD may represent:

• a generic context (e.g., a drawing of a house with a yard, an

office with workers or a school room with a teacher and students.)

• a personalized context (e.g., a digital photo of a child playing in

his bedroom or a picture of the family on a beach while on

vaca8on.)

ACN NEWS, August 2004

Recogni8on of de-‐constructed Scene

Visual Scene Deconstructed Elements

Research / Literature Background

Summary of Drager et al. (2003) •  Purpose: To compare children’s ability to locate vocabulary on grid vs. visual scene display

•  Par8cipants: 2 ½ year old children, n = 30 •  Results: Children did significantly befer with the VSD than the grid display

4/7/16

5

Scene Based Display

Companion (Circa 1995) – Commercialized as screen format op8on on the Freestyle

Companion Home Screen

Companion Sub-‐Level Screen Companion Sub-‐Level Screen

Display Formats

Visual Scene Cue -‐ symbols that represent an en5re event/ac5vity. They contain the relevant background context, agents, objects, ac8ons and the interac8ons of the agents and objects. Visual scenes are either: •  Dynamic scenes –full-‐mo8on video clips that illustrate how the ac8vity/event unfolds over 8me, or

•  Sta8c scenes –digital photographs that capture a single prototypical moment in an ac8vity/event.

Visual Immersion System: A Guide to Communica5on Enhancement for Au5sm Spectrum Disorders)

Display Formats

1.   Rela8vely recent display format 2.   Expanded due to technological advancements 3.   Are recognized earlier than grid displays at earlier

developmental age 4.   Seemingly more intui8ve -‐ -‐ when compared to de-‐

constructed scene 5.   Scenes phrase or concept based ; grids element based 6.   Scenes – circumvent Language

Visual Scene Displays and Scene Cues

4/7/16

6


Summary of Schlosser, R. W., Laubscher, E., Sorce, J., Koul, R., Flynn, S., Hotz, L., Abramson, J., Fadie, H., & Shane, H. (2013) •  Purpose: Assess effect of visual scenes and scene cues on comprehension

•  Par8cipants: 10 subjects with ASD •  Results: Significant improvement in comprehension when scene cue provided


Summary of Remner, R, Baker, M., Karter, K., Kearns, K., & Shane, H. (In Press) Use Of Augmented Input To Improve Understanding Of Spoken Direc5ves By Children With Moderate To Severe Au5sm Spectrum Disorder •  Purpose: Assess effect of visual scenes and scene cues on

comprehension & replicate Schlosser et al, 2013 •  Par8cipants: 10 subjects with ASD •  Results: Significant improvement in comprehension

when scene cue provided


•  Limited research founda8on for use of visual scene, scene cue or grid displays despite considerable popularity as founda8on for AAC.

•  The limited research seems to favor VSDs with respect to vocabulary iden8fica8on.

•  When compared to speech direc8ves, VSCs improve understanding the intent of preposi8onal direc8ves

Developmental Trajectory

Clinical observa8on (and limited research) suggests a developmental con8nuum that extends from Noun Elements, Scene Cues and Visual Scene Display comprehension during early developmental periods to Grid Display made up of elements (and Text) at later stages

Clinical / Developmental Trajectory (Profile of Poten8al Consumers of Visual Graphic Language)

_____________________________________________________

Six Consumer Groups

Pre-‐sceneà Visual Scene àTransi8on (to Grid) àGrid àTransi8on (to Text) àText

1 2 3 4 5 6

Display Display Display

4/7/16

7

Developmental Trajectory

Vast majority of individuals with moderate to severe ASD primarily employ the communica8ve func8on of reques8ng, likely because: 1.   Not mo8vated to share 2.   Difficulty with joint afen8on 3.   Inability to combine the visual elements of language –when do

combine, based on scripted sentences

Clinical Decision Making to Enhance Visual Communication

An Evolving Clinical Exercise

Grid and Scene Displays

Seven Operations* •  Protest & Refusal •  Organization & Transition •  Requesting •  Commenting •  Directives •  Questions •  Social Pragmatics

Based on: Enhancing Communication in Individuals with Autism: A Guide to the Visual Immersion System


Function 1: Protest & Refusal – Definition -‐Protest: Behavior that expresses objec8on or disapproval of an ac8vity, event, or person. -‐ Refusal: Behavior that expresses rejec8on of an object, ac8vity, or event suggested or ini8ated by another person (Shane, O’Brien & Sorce, 2009).

Means:

•  Behavioral / physical •  Conventional gesture •  Symbol


Protest & Refusal – Behavioral / physical: can be both dangerous and destructive Goal to move from behavioral to symbolic expression

Likely Causes of Protestation

•  Transitions •  Demands or directives to perform a task or activity •  Removal or cessation of an enjoyable task or activity •  Expecting participation in a distasteful task or activity when introduced •  Lack of ability to request goods and services •  Desire to escape •  Desire for attention

4/7/16

8

Nature of Visual Supports

•  Exert control over Environment •  Language of Control

•  Stop •  No •  More •  Finished •  Help

Sample ALP Graphics - Animated

Grids and Scenes

Protest & Refusal

⏎ Visual Scene Display

Grid Display

à

à

Protest & Refusal – Protest Bar Imbedded into Scene

One possible difference and likely advantage of Scene Display is placement of a “protest strip” within the seqng in which a symbolic communica8ve replacement might be needed

Function 2: Organization & Transitions

Definitions: Organization - arranging elements (e.g., tasks, events) into an orderly, functional, structured whole. Transition - the process of changing from one state, stage, or activity/environment to another.

•  Traditionally considered a VOM

• Not a speech act per se

Assessment of Organiza8on and Transi8ons

Part 1: Transi8ons Ques8on 1: Does the learner have difficulty transi8oning between ac8vi8es? Ques8on 2: Does the learner have difficulty with novel environments? Ques8on 3: Does the learner have difficulty with changes in rou8ne? Ques8on 4: Does the individual exhibit aggression, non-‐compliance, or passivity during /in Transi8ons?

Assessment of Organiza8on and Transi8ons

Part 2: Organiza8on Ques8on 1: Does the learner have difficulty sustaining par8cipa8on in learning tasks? Ques8on 2: Does the learner have difficulty understanding sequences? Ques8on 3: Does the learner have difficulty with appropriate expecta8ons? Ques8on 4: Does the individual exhibit aggression, non-‐compliance, or passivity during learning situa8ons?

4/7/16

9

Aligning With Characteris5cs of Au5sm

Behavior Control & Aggression Problems

Method: First – Then Displays: Countdown Boards: Social

Stories: Timer


•  First/Then Displays

•  Countdown Boards

•  Timers

•  Visual Schedules (daily, calendar, and activity schedules)

School Work

Then

Playground

First

10 minutes

First-Then Low Tech - Expanded •  Built in duration & trial information – static scene cues

10 Trials

Math

Next Then

Basketball Videos

10 Trials

First

5 Baskets 15 minutes

First-Then – Low Tech - Expanded •  Built in multiple steps

Traditional Daily Schedule – Low Tech

Traditional LayoutØ Typically refers to Where or Location

Ø Can refer to “who” as marker


Daily Schedule - Expanded

Event: The theme of the event Who: The participants in the task

Where: The location where the task will be performed What: Activities that will be performed

How Long/Many: Number of repetitions required or duration of task

Reward: Reinforcement for participating in the activity, if applicable

4/7/16

10

Additional Innovations in Visual Organization

•  If I -Then Display

•  Preparation for Uncomfortable Circumstances

Organiza8on & Transi8on Final Thoughts

•  Be certain the level of representa8on is understood -‐-‐ Content of grid can be a who, what or where

•  Grids oten use scenes as content – scene enhances comprehension

•  Most schedules, calendars, first then displays, etc. are grids, yet content is oten scene based


Requesting

Expressing a desire for objects (e.g., toys, food), ac5vi5es, people, affec5on, aSen5on, recurrence, assistance, and informa5on/clarifica5on

Requesting: Goal •  An important goal of AAC / reques8ng is to provide a symbolic rather than physical means of expression

Physical Requests vs Symbolic Requests

-‐ Limited -‐ Easily understood

-‐ Needs instruc8on

Cookie

Func8on 3: Thoughts about Reques8ng •  Communication should be easy •  Don’t reduce spontaneity with artificial syntax •  Motivation •  Level of Representation •  Mands work •  Practice across environments and partners •  Stilted requesting speech is hard to break

• What is the real benefit of: Name + Want + Item + Please

Func8on 3: Thoughts about Reques8ng Overwhelming Clinical Focus on the Requesting Communicative Function Reasons? Thoughts?

4/7/16

11

Display Formats that Enable Reques8ng

•  Noun element reques8ng displays •  Visual scene displays •  Linked visual scene displays •  Hybrid visual scene displays •  Grids •  Text

Picture Exchange Communica8on System (PECS)

Classic Example of Noun Element Reques8ng

PECS (Likely Most Popular Grid Display)

Behaviorists approach to communica8on Enhancement Ubiquitous

– Why à it works – Why à easy to understand steps

But

–  is it a Glass Ceiling? –  It is the an8thesis of teaching genera8ve language –  Really only about reques8ng ( I + want + __ + please)

Strengths of PECS •  Demonstrable success - - program comprehensible • 

•  Mand is teachable

•  Early focus on requesting -- an achievable skill •  ‘Tear and put’ selection

– Primary difference from past AAC requesting approaches (although not necessary for all children).

– Establishes relationship between learner and mentor and provides a visual model of syntactic construction.

– Electronic version of ‘tear and put’ is symbol selection to message window.

•  Children learn to use symbols to communicate –  learn that others can understand them –  learn that they can specify their reinforcement

•  Reduces early behavioral problems caused by decreased ability to request goods and services

Visual Scene Display Reques8ng

Use familiar, natural background setting

Goldfish in a lunchbox Goldfish on a shelf

Linked Visual Scene Displays

•  Launches learner from one scene to another •  Supports independence, naviga8on skills

4/7/16

12

Grid Displays

•  For combining symbols to create sentences •  Message window •  Expanded naviga8onal demands

Reques8ng Summary

•  Many display types •  Reques8ng should be highly mo8va8ng •  Reques8ng is a good founda8on for acquisi8on of other func8onal opera8ons

•  Move the learner along the con8nuum of visual supports when possible

Four Types of Directives

1. Control – regulates behavior “stop” “stand up”

2. Rou8ne – par8cipa8on in daily rou8nes “open” “put” “get”

3. Instruc8onal – needed for educa8onal ac8vi8es “circle” “point” “show”

4. Play-‐based – enable par8cipa8on in games, play “roll” “throw” “pop”

Function 4: Directives Direc8ves: Ra8onale for Instruc8on

Recep*ve: •  Direc?ves make up a large por?on of the spoken language received by learners

•  Difficult due to distrac?bility, mo?va?on, non compliance, poor comprehension

•  Breakdowns result in frustra?on and weariness for caregivers

Direc8ves: Ra8onale for Instruc8on Expressive: •  Learners with ASD usually give few direc8ves

–  lack variety of verbs –  have reduced syntax –  limited vocabulary

•  Poor expression of direc8ves can lead to communica8on breakdowns and helplessness

•  * Language Comprehension is Essen8al

Teaching Directives: Understand and use informa8on from Dynamic Scene Cues (videos), Sta8c Scene Cues (photographs), and Isolated Elements (Noun, verb, etc) to:

–  Improve understanding of noun labels, verbs – Teach single-‐step direc8ons

4/7/16

13

Teaching Directives (Noun & Verb Elements)

Early stage show noun and verb elements. Model expected reaction. e.g., KICK

Verb (Animated) element: anima8on showing a familiar Ac8on (verb)

Direc8ves: Verb Elements

Teaching Direc8ves: Verb Element Purposes

Describe the ra8onale and methodology for combining visual graphics into the building blocks of language

Syntax

Rule based order / organiza8on of words in a spoken sentence Symbol Syntax refers to construc8on of a sentences using the elements of language in a visual form

Why is Syntax Important?

1.   For persons with moderate to severe au8sm the use of genera8ve language, where symbols are combined to form novel sentences, is rare

2. I + “desired item” is a scripted sentence and not novel, genera8ve language

4/7/16

14

Prizant (1983)

“The reproduction of memorized multiword units would seem to be devoid of the creative and generative linguistic processes typically associated with the spontaneous production of multiword utterances.” (p. 303)

79

Reality of Expressive Language in Moderate to Severe Au8sm

•  Spoken Output Predominantly Scripted Phrases •  Reques8ng Predominant Communica8ve Func8on •  Social Interac8ons Frequently Require Promp8ng •  Instruc8on oten Focused on Expression •  PECS Predominant Instruc8onal Strategy

Clinical Reality of Language Comprehension in Moderate to Severe Autism

•  Greater Noun Comprehension (Spoken And Visual)

•  Limited Understanding (Verbs, Prepositions, Adjectives, Question Forms)

•  Spoken Language Within Routines Often Understood

•  Context Facilitates Understanding

AUGMENTED INPUT

AUGMENTED INPUT DEFINED

The Introduc5on of a Visual En5ty (Graphic or Object), Typically in Conjunc5on with Speech, that is Intended to Improve Understanding. (Allen & Shane, 2014) The Synchronized Presenta5on of Graphic Symbols or Objects Cues while Speaking with the inten5on of improving understanding and / or expression

Improving Comprehension

APPROACHES DESCRIBED IN THE LITERATURE

Augmented Communica8on Input •  First use of term Augmented Communica5on Input •  Purpose to improve the comprehension of spoken language •  Augmented communica?on input “….communica?on partners]

wri?ng or typing key elements of their messages onto some sort of system that provides a visual display to the individual” (Beukelman & Garrea, 1988, p. 119)

•  Used with aphasia adults

4/7/16

15

Improving Comprehension APPROACHES DESCRIBED IN THE LITERATURE Aided Language S8mula8on (ALgS) •  Aided language s*mula*on (ALgS) (Goossens, 1989) •  Simple, short phrases, a slowed rate of speech, expand on language produced by the learner, and emphasize keywords.

•  “When providing aided language s?mula?on, the facilitator points out picture symbols on the child’s communica?on display in conjunc?on with all ongoing language s?mula?on” (Goossens’, 1989, p. 16)

•  Used with children with Cerebral Palsy

“Aided Language S8mula8on”

Does simply poin8ng to symbols while speaking improve understanding?

“Aided Language S8mula8on”

Would A Message / Accumulator Window Improve Understanding?

Message window Message Window

Improving Comprehension

APPROACHES DESCRIBED IN THE LITERATURE Augmented Input (with visual scene cues) Scene cues Defined

Visual Language Instructional Phases:

•  Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:

Dynamic Scene Cues: full-motion video clips of action

scenes

Animated Verb Graphics - Scene

4/7/16

16

Visual Language Instructional Phases


1. Dynamic Scene Cues: full-motion video clips of action scenes

2. Static Scene Cues: photographs that capture a prototypical moment in the action scene

Func8onal Scene Cue -‐ Play

Toy cars

Put man in car Drive car up/down ramp Make the car go fast/slow Build a track (add a piece)

Ac8vity Possible Scene Cue

Func8onal Scene Cue -‐ ADL

Bed8me

Socks in the hamper Put on pajamas Toothpaste on toothbrush Brush teeth Climb in bed

Ac8vity Possible Scene Cue

Role of Scene Cues

Enables Func8onal / Prac8cal Opportuni8es Serves as Founda8on for Symbol Syntax Learning

Implications of Scene Cue Mastery Use of dynamic and static cues bypasses language

processor –  Due to load on language processor when using elements, some

children may use static scenes as their communication system

Mastery of static scene cues is a significant accomplishment –  Can be used to promote general understanding and communication

related to: •  Daily Living Skills •  Play •  Transitions •  Requesting •  Commenting •  Clarifying •  Directives

Visual Scene Cues: Limitations

Difficult to flexibly apply visual scene cues to spontaneous situations and messages

–  Highly context-dependent –  Not a cohesive, rule-bound language system

4/7/16

17

Learners Needs

•  Be aware that linguistic elements can be combined to make unique meanings

•  Understand rules for combining these disparate linguistic elements into meaningful phrases

Visual Language Instructional Phases


1. Dynamic Scene Cues: full-motion video clips of action scenes

2. Static Scene Cues: photographs that capture a prototypical moment in the action scene

3. Language Element Cues: graphic icons representing each of the individual linguistic components that comprise an action scene (e.g., subject, object, verb, preposition, adjective, etc.)

In Search of a Visual Language

Symbolate: Language Comprehension?

•  Stringing symbols together does not automatically

result in comprehension •  Learners must first have a knowledge of language

elements and semantic relationships •  Paradoxically, stringing symbols together may

actually interfere with comprehension

What’s Known About AUGMENTED INPUT

•  Literature offers some evidence that comprehension and expression can be improved through AUGMENTED INPUT

•  Clinical observa8on suggests that exposure to symbol sequencing during an immersive experience can lead to improved comprehension and expression (Op8mal approach/es and candidacy remains unstudied)

What’s Known About AUGMENTED INPUT (Based on experiences in Au8sm Language Program at

Boston Children’s Hospital and Monarch School for Children with Au8sm )

•  Considerable clinical outcomes from exposure to Mixed Displays leads to improved understanding of symbol syntax

•  Pilot research and clinical outcomes from exposure to Teaching Language Concepts leads to improved understanding of symbol syntax

•  Considerable clinical outcomes from exposure to Topic Displays leads to improved understanding of symbol syntax

4/7/16

18

Empirical Ques8on

Can technology facilitate the acquisi8on of symbol syntax (the linking of element cues /sequencing of graphic symbols)?

Teaching Language Concepts

Teaching Language Concepts – Sotware Applica8on Licensed

to SpecialNeedsWare

Overview and Use of Topic Displays

Tool: Topic Display

•  Language elements organized for one topic •  Contains various parts of speech •  Color coded

Topic Display

4/7/16

19

Tool: Topic Displays

Use a mo8va8ng theme Including mul8ple pronouns, verbs, adjec8ves and/or objects Program words in order of sentence structure Begin with two contras8ng concepts

Overview and Use Mixed Displays

Apps To Teach Symbol Syntax Sentence Key

Sentence Match Puddingstone Adventure: Introduction

4/7/16

20

Other Language Approaches

Suskind – Language Acquisi8on through Disney Video viewing Anima5on offers a medium of story telling and visual entertainment which can

bring pleasure and informa5on to people of all ages everywhere in the world

Walt Disney

“She’d do it if Winnie the Pooh told her to do it”

Quote from Mother from ALP Clinic of child with Au5sm

Suskind’s Impact

•  Raised Awareness of Disney Anima8ons •  Anima8on has a place in language instruc8on •  Outcome based on anecdote not Behavioral Science •  Poten8al for Nega8ve Consequences – Obviously an Unintended Consequence

Final Thoughts •  Direc8ves can be given and received with Visual Scene Displays at earlier age

•  Unscripted direc8ves (using genera8ve language) requires knowledge of symbol syntax

•  Technology assists in the acquisi8on of symbol syntax – Developmental Trajectory (scene cues to element cues)

4/7/16

21

Function 5: Commenting Commen8ng Defini8on:

A behavior, gesture, vocaliza8on, or other modality that conveys informa8on about observa8ons, experiences, knowledge, opinions, emo8ons, and reac8ons

•  Connect shared experiences •  Give informa8on, thoughts, feelings •  External and observed vs. internal •  Essen8al to conversa8on and social exchanges •  External and observed vs. internal •  Essen8al to conversa8on and social exchanges

Commen8ng: Ra8onale

Commen8ng is challenging because: •  Learners may have low mo8va8on for social exchange •  Limited vocabulary and concept knowledge

–  Hot, cold, shiny, loud, big, preSy, ugly •  Restricted interests •  Difficulty with reciprocity

Commen8ng: General Goals of Instruc8on

•  Build social mo8va8on, connectedness •  Expand language

–  Vocabulary –  Syntax –  Pragma8cs –  Conversa8on

•  Promote genera8ve language

Emerging Commenters

Different Levels of Commen8ng Competence

Emergent Commenters: Goals

•  Label -‐ items in familiar environments •  Build apprecia8on for joint afen8on and informa8on sharing

•  Ini8ate symbolic commen8ng

4/7/16

22

Labeling

The Training Wheels of Comments

A Label Is a Generally a Noun

Power of Learning a Label

Ability to Label Enables “spoon”

Requesting --> ‘Want spoon’ Commenting --> ‘Silver spoon’ Directives --> ‘Get the spoon’ Wh questions --> ‘Where is the spoon?’

Tools for Emergent Commenters

•  Visual Scene Displays with voice hotspot

•  Animated graphics – Especially Descriptors & Verbs

Commen8ng Goals: Intermediate Communicator

Build concept knowledge Expand vocabulary, syntax, linguis8c knowledge

Emergent Communicators: Tools

Mixed display

Emergent Commenters: Tools

Teaching combina8on of symbols that describe –  “Red truck”, “funny boy”

4/7/16

23

Advanced Communicators: Goals

Syntax, Syntax, Syntax Increase genera8ve language and decrease scripted phrases

Pain & Illness

Other Application Suggestions

Preferred video routine

Window routine

Items in unexpected locations

I hear, see, feel (PECS)

Function 6: Questions Ques8ons Defined: Sentences or phrases that are spoken, wrifen, gestured or otherwise expressed as an interroga8ve and directed to someone in order to receive informa8on in reply. Ques8ons have both a recep8ve and an expressive component (Shane, O’Brien, & Sorce, 2009).

Thoughts about Ques8ons

Valuable means of gaining informa8on and socially engaging with others Lay founda8on for conversa8onal exchange Paradox -‐ those with reduced language capabili8es are frequently bombarded with ques8ons – perhaps to compensate for reduced capacity with other forms of expression Context and key words can occasionally serve as a means to answer some ques8ons – not always prac8cal Yes – No – 50% right = chance

Thoughts about Ques8ons Usually begin with specific ques8on markers “who,” “what,” “where,” “how” and so on Elicit informa8on about

•  Time (“when” ques8ons) •  Place (“where” ques8ons) •  people / par8cipants (“who” ques8ons) •  Choice (“which” ques8ons) •  Uncertainty (“how” and “why” ques8ons)

Can serve as way to compensate for inability to ini8ate requests. “What do you want?” or “Do you want this?” or “Where should we go?”

4/7/16

24

Ques8on Types Addressed in VIS

•  Yes/no ques8ons pertaining to labels (e.g., “Is this a train?”) •  Yes/no ques8ons pertaining to preferences (e.g., “Do you want some raisins?”)

•  Which ques8ons pertaining to choice (e.g., “Which one do you

want?”)

•  Who ques8ons

•  What + do ques8ons (e.g., “What is he doing?”)

•  Where ques8ons

•  Certain aspects of when, how and why ques8ons

Ques8ons: Developmental Perspec8ve Typical age of acquisition (comprehension) for question forms

targeted in the VIP (after Chapman, 1981).

Question forms addressed in

the VIS

Typical age of acquisition (year;

month)

1. Yes/no 2:0

2. What + be (What is that?) 2:0

3. What + do (What is he

doing?)

2:6

4. Where (place) 2:6

5. Who 2:6

6. Whose 2:6

7. Why 3:0

8. How 3:6

9. Which 5:6

10. When (temporal) 5:6

Ques8on Graphics

Graphic symbols representing the question words who, what and where as portrayed by Mayer-Johnson Company, PixWriter, (SymbolStix) and the Autism Language Program (ALP) graphics

Text Mayer-Johnson PixWriter SymbolStix ALP Graphics

Who?

What?

What-be

What-do

Where?

Ques8on Graphics

Examples graphic images for Yes and No from three commercial developers of graphic symbols.

ALP Animated Graphics PCS: Mayer-Johnson SymbolStix (xxx)

No

Yes

Ques8on Graphics

Paucity of research to suggest if one graphic or an en8re graphic

set is superior to another or if any one set can actually improve

or facilitate the understanding of ques8ons

Learning Approaches to Ques8ons

Buildable Scenes

4/7/16

25

Instruction for Who, What , Where (non-literate learners)

Instruction for Who, What , Where (literate learners)

Example: Color-coding WH Questions

Which (Question response or behavioral selection from choice set ) Calendar for Teaching ‘When’

Calendar for Teaching ‘When’ Anima8on to Teach Ques8ons

Elusive Language Concepts : (Wh Questions)

Method: Puddingstone Ac5on Factory App

4/7/16

26

Apps to Teach Ques8ons Aligning With Characteris*cs of Au*sm

Elusive Language Concepts : (Wh Questions)

Languagelearning Apps, LLC $24.99

Aligning With Characteris*cs of Au*sm Elusive Language Concepts : (Wh Questions)

SNAKEY

Function 7: Social Pragmatics

Social Pragma8cs: a qualita8ve func8on of communica8on that

reflects an individual’s ability to ini8ate, sustain, terminate,

par8cipate and direct interac8ons with others via verbal or non-‐

verbal communica8ve means and in socially appropriate ways.

Shane, et al Visual Immersion System: A guide to Communica*on

Enhancement in Au*sm

Defini8on

Social Pragmatics

Social pragma8cs allows for the appropriate expression of the six communica8ve func8ons previously reviewed Address social pragma8cs within key contexts that are oten problema8c for severe end of ASD Address those pragma8c skills that most likely respond to systema8c use of visual supports

Social Pragmatics

Social pragma8cs in normal development are both learned and taught Observa8onal deficits and interac8ons by those with ASD frequently lead to un-‐or under-‐developed social pragma8c skill development Language & Cogni8ve limita8ons reduce pragma8c ability

4/7/16

27

Social Pragmatics

Social pragma8cs can also be affected by: •  Mo8va8on •  Joint afen8on •  Cogni8ve ability •  Language ability •  Afen8onal deficits •  Limited TOM (perspec8ve of others)

Newman – Social Pragma8c Language Acquisi8on through Interac8on with Siri

hap://mobile.ny?mes.com/2014/10/19/fashion/how-‐apples-‐siri-‐became-‐one-‐au?s?c-‐boys-‐bff.html?_r=1&referrer=

Personal Assistants

“……they will also be able to carry on more complex conversa5ons about a person’s area of interest. “Your son will be able to proac5vely get informa5on about whatever he’s interested in without asking for it, because the assistant will an5cipate what he likes,” said William Mark, vice president for informa5on and compu5ng sciences at SRI.”

Personal Assistants

Personalize as therapy assistants? Speech improvement Social Pragma8cs: •  Store informa?on on friends, family and acquaintances to be recalled prior

to a visit or a phone call to aid in conversa?on development (Alexa, tell me what Aunt Joann’s hobbies are).

•  Program to prac?ce basic social nice?es with Avitar Timer Func8on Other

4/7/16

28

Function 7: Social Pragmatics Common Social Exchanges affected by SP deficits:

• Engaging in simple gree8ngs, par8ngs and small talk (a new voca8onal site,

mee8ng new people)

• Sharing and taking turns (wai8ng in line, opening presents, wai8ng for food

to be served)

• Traveling to and tolera8ng new places (church, a mall, a party)

• Mee8ng new people (at a store, family gathering)

• Modifying and managing atypical behaviors according to the environment

(knowing what is appropriate, where)

Function 7: Social Pragmatics Three areas of Social Pragma8cs targeted in VIS: 1.   Social language and conversa8on

2.   Play skills 3.   Challenging Behaviors

Assessment of SP

Part 1. Assessment of Conversa?on Skills 2.  Does the learner ini?ate or respond to small talk (e.g., “How was

your weekend?” “What have you been doing?” “What did you do at school?”)

3.  Does the learner engage in conversa?on around a shared topic (including responding to and asking ques?ons across mul?ple turns)?

4. Does the learner ini?ate conversa?on with a familiar partner regarding a topic relevant to the partner (e.g., recent past events, interests, family members)?

Assessment of SP

Part 2. Assessment of Play Skills 1. Does the learner engage in sensorimotor games with others (e.g., chasing, 8ckles)? Yes – no interven8on required No – refer to Interven8on sec8on 2A: Sensorimotor Play 2. Does the learner engage in pretend play (e.g., feeding a baby doll)? Yes – no interven8on required No – refer to Interven8on sec8on 2B: Pretend Play 3. Does the learner engage in rule-‐based games with others (e.g., board games)? Yes – no interven8on required No – refer to Interven8on sec8on 2C: Rule-‐based game play

Assessment of SP

Part 3. Assessment of Challenging Social Behaviors 1. Does the learner exhibit persevera8ve behaviors? 2. Does the learner engage in conversa8onal turn-‐taking? 3. Does the learner exhibit overuse of scripted phrases? 4. Does the learner exhibit inappropriate, obtrusive, or emo8onal responses (outbursts)? 5. Does the learner have difficulty understanding and honoring personal space and boundaries? 6. Does the learner exhibit an aberrant or distrac8ng voice or speech?

1.   Social language and conversa8on

•  Social gree8ngs

•  Extended gree8ngs

•  Engaging in Small Talk

•  Conversa8on Around a Shared Topic

•  Ini8a8ng Conversa8on Based upon Prior Knowledge

4/7/16

29

Social language and conversa8on

Tools

Social gree8ngs

•  Low Tech -‐

•  Hi Tech – Video Modeling

Social Pragmatics – Tools (Greetings)

Low Tech

(Echoic Greeter)

Social Pragmatics – Tools (Greetings)

Hi Tech

Video Modeling (can be self)

Social Pragmatics – Tools (Expanded Greetings)

Social Story

“When I see someone I know I say “hi,” look at them, and wave/shake hands. If they say “How are You?” I can say “Good/fine.” I can say “how are you?” or I can give them a compliment.”


Turn taking visual


Ini8a8ng and responding to compliments

4/7/16

30

Social Pragmatics – Tools (Small Talk)

Scripted

Last week I….. Ac5vity With (who picture) At (where picture)

Social Pragmatics – Tools (Small Talk)

Ini8a8ng

Function 7: Social Pragmatics Three areas of Social Pragma8cs targeted in VIS: 2. Play skills

•  Exploratory

•  Rou8ne based

•  Rule Governed

•  Pretend

Function 7: Social Pragmatics Three areas of Social Pragma8cs targeted in VIS: 3. Challenging Behaviors

Topic Shifting - Preservative Topics

Waiting: Low Tech

4/7/16

31

Waiting: Hi-Tech

Observational Learning - Virtual Character/s 1. Demonstrate Appropriate Behavior 2. Virtually Countdown

Next Genera8on Apps & Tools for Learning & Communica8on

Trends in Applied Technology

Wearable Technology

Scene Cues & Wearable Technology

Primary Aim To determine the effect of “Just-‐In-‐Time” (JIT) visual supports delivered through wearable technology (e.g., the Apple Watch) on language and behavior of individuals with Au8sm Spectrum Disorder (ASD)

Principal Investigator : Amanda O’Brien, MS

Just-‐in-‐Time Supports in AAC

Based on: – Schlosser, R. W., Shane, H. C., Allen, A., Abramson, J., Laubscher, E., & Dimery, K. (2015). Just-‐in-‐?me supports in augmenta?ve and alterna?ve communica?on. Journal of Physical and Developmental Disabili*es. Advance online publica?on.

– DOI. 10.1007/s10882-‐015-‐9452-‐2

4/7/16

32

Just-‐in-‐Time

•  Origin: –  Japanese business strategy to streamline inventory –  Signals, or “Kanban,” inform produc?on line exactly when next part is needed: i.e., "Just-‐in-‐Time”

–  Informa?ve Video

•  Adapta?on to other fields: – Medicine –  JIT text messages –  Just-‐in-‐Time Teaching (JITT)

Applica8on to AAC

•  Currently: –  Research with SLI and word produc?on shows promise of JIT for treatment purposes

–  Technological advances support current use of JIT in AAC •  Examples:

–  Situa?on-‐specific vocabulary by GPS –  Shane’s applica?ons:

•  symbolTalk •  M-‐STARR

–  Scene-‐Cues –  Light et al. (2012) ASHA paper –  Drager et al. (2014) ISAAC paper

Proposal

•  Proposal: –  Possibility of providing supports that tradi?onally are not provided on an “as needed” basis

•  JIT can: – Deliver prompts, reminders, rewards, encouragement, memory aids and error correc?on from a distance

–  Improve independence across serngs –  Provide more efficient storage, retrieval, and delivery compared to low tech materials

– Allow for mul?ple formats of delivery that can be tailored to an individual’s processing strengths

Organiza?onal Framework

Classifications

Intended Purpose Modalities Source Delivery

Methods

Framework -‐ Intended Purposes •  Prompts:

–  Used to bring about an ac?on by the recipient •  Reminders:

–  Given at exact moment when ac?ons should occur –  Repeated behaviors will establish a new rou?ne

•  Rewards: –  Allows for reinforcement is associated with desired behavior

•  Encouragement: –  Show instantaneous support of successful behavior

•  Conversa?on: –  Aid in star?ng, maintaining, or ending a conversa?on –  May also assist in repairing communica?on breakdowns

•  Vocabulary: –  Selec?on, instruc?on, organiza?on, and generaliza?on.

•  Generaliza?on: –  Assist in transfer of skills to natural environment

•  Organiza?on: –  Clarify organiza?on for confused learners

Framework -‐ Modali8es

•  Auditory Modality: –  “Earcons” –  Spoken language or non-‐linguis?c environmental sounds –  Pr-‐recorded or created at ?me of delivery

•  Visual Modality: –  “Eyecons” –  Sta?c/animated graphic symbols, tradi?onal orthography, photos, sta?c/dynamic scene cues, video, and flashing screens/indicators

•  Vibrotac?le Modality: –  “Vibrons” – Wearable, pulsa?ng mobile technology

4/7/16

33

Framework -‐ Sources

•  Self-‐Ini?ated: – The learner becomes aware of a need for support

•  Automated: – Supplied when needed without human assistance – Guided by preprogrammed ?mes, environment detec?on, or partner speech analysis

•  Mentor-‐Generated: – Supplied by a family member, peer, or support personnel

Framework -‐ Methods of Delivery

•  Face-‐to-‐Face: – Mentor-‐Generated JITs

•  Wireless Transmission: – Mentor-‐Generated JITs

•  Preprogrammed: – Automated JITs

•  Teleprac?ce: – Mentor-‐Generated JITs

•  Direct: –  Self-‐ini?ated JITs

Benefits of JIT

•  Decrease demands on Working Memory: – Dynamic displays place addi?onal demands on working memory.

–  JIT as a memory aide or to introduce vocabulary/symbols in context

– Less burden on naviga?on = more focus on understanding symbols or ongoing conversa?on

Benefits of JIT

•  Situated Cogni?on & Ac?on – Both JIT and Language in general are situa?on specific

– Face-‐to-‐face conversa?ons depends on context –  JIT can further clarify the rela?onships between symbols and objects in a par?cular situa?on

– Thus JIT cues of situa?onal features work in conjunc?on with speech to aid in language comprehension

Benefits of JIT

•  Teachable Moments: – Opportuni?es for a child to learn when they are excited, engaged, and primed for learning

– Requires ac?ve role of teacher –  JIT supports may enhance likelihood of learning due to implica?ons for communica?on partners


Primary Aim To determine the effect of “Just-‐In-‐Time” (JIT) visual supports delivered through wearable technology (e.g., the Apple Watch) on language and behavior of individuals with Au8sm Spectrum Disorder (ASD)

Principal Investigator : Amanda O’Brien, MS

4/7/16

34


Secondary Aims 1. Assess opera5onal competencies

•  Percep8on of images on screen. •  Tolerance for wearing •  Raise arm to view •  Tap screen view dynamic cue

2. To explore mul5modality of JIT delivery and comprehension.

•  Visual only (e.g., sta8c scene cue, dynamic scene cue). •  Auditory cue presented with dynamic scene cues. •  Afen8on to, and comprehension of, hap8c prompts.

Scene Cues & Amazon Echo Technology

Primary Aim To determine the effect of “Just-‐In-‐Time” (JIT) visual supports delivered by the Amazon Echo Technology (e.g., on language and behavior of Individuals with an Au8sm Spectrum Disorder (ASD)

Trends in Applied Technology

Mobile Device Interac8vity

Symboltalk

202

SymbolTalk

Unique features – “next genera8on” App 1.   Voice driven Speech to Graphic Transla8on 2.   Two-‐way transmission between devices 3.   Intelligent Q & A tool -‐ AKA Smart Ask

203

SymbolTalk – iPad App

Finn pushes the shopping cart.

Mentor’s device

I understand

!

Learner’s device

Transmit

Next Genera8on Communica8on Apps

DEMO

4/7/16

35

Evolu8on of the Synthesized Computer Voice

Robo8c à Intelligible à Natural à Personalized

north carolina s&h pdf · 2016-08-15 · sample alp graphics - animated grids and scenes...

Documents