itechnology and communicative drawing for individuals with aphasia jennifer a. ostergren, phd,...

iTechnology and Communicative Drawing for Individuals with Aphasia Jennifer A. Ostergren, PhD, CCC-SLP Jennifer Raminick, B.A.

Welcome Drawing on an iPad by an individual with aphasia (Ostergren, 2012)

Thank You To our clients (our wonderful teachers) Thank you for your inspiration and willingness to participate in our research To my students (future SLPs) Thank you for your help in preparing this presentation. In particular thank you Kristen Churney for her guest appearance in describing the CDP

House Keeping A handout for this presentation, including links to the technology discussed is available at: __________ (insert CD page web address once loaded)

Agenda Multi-Modality Communication and Aphasia Correlations with Neuroplasticity Drawing and Aphasia iTechnologies for Communicative Drawing Examples of available iApps Features of available iApps Recommendations for Selection Communicative Drawing and iTechnology in a Clinical Setting New Directions and Future Research Questions and Answers

What is Multimodality Communication? Using different modes of communication to meet the needs of the individual and to allow them the opportunity to participate in their communities (ASHA, 2004) Writing Gesturing Drawing AND Speech

Total Communication Use in PWA (Rautakoski, 2011) Many people with aphasia (PWA) use nonverbal methods of communication spontaneously, including: Pointing Gesturing Facial expressions Pantomime Drawing Concrete signs Some PWA also use low-tech devices Not used as much as the spontaneous nonverbal communication Some PWA need special training in use of total communication

Key Elements to Multimodality Communication (Beukelman & Mirenda, 2012) Co-Construction Partner Training Varied Approaches Given: Partner Dependent Communicators Partner Independent Communications Direct training and generalization to real- world settings is KEY!

AS: Example of Multimodality Communication What did you do yesterday? Drew a hamburger Said, Hamburger. Big Put up four fingers Wrote : UCLA, DDS, 2 Viet, 1 Japan, and 2 white The client had gone out to eat hamburgers with four of his friends from dental school at UCLA; two friends were Vietnamese and two were Caucasian, and the 1 Japan was referring to himself.

Neuroplasticity Rich body of research on neuroplasticity in the past decade (Saur & Hartwigsen, 2012) Neuroplasticity is defined as, ... functional reorganization/ compensation within residual neural tissue, mediated by changes in neural circuitry (Gonzalez Rothi, Musson, Rosenbeck, & Sapienza, 2008)

Educating Clients about Neuroplasticity Empowering Individuals with Aphasia with Neuroplastic Principles: From Theory to Practice. Birtler, E., Hayes, M., Kim, J., Sibby, K., Weltz, K., Navarro, A., & Ostergren, J. (2012). California Speech and Hearing Association (CSHA) Magazine, 42 (1), July 2012.

Explaining Neuroplasticity to PWA WHAT YOUR BRAIN IS DOING FOR YOU. Researchers have identified four strategies your brain uses to compensate for damaged areas: 1. Homologous area adaptation 2. Cross-modal reassignment 3. Map expansion 4. Compensatory masquerade No matter how long since your stroke, your brain is still working to get the job done! Grafman (2000). Conceptualizing Neuroplasticity

Explaining Neuroplasticity to PWA WHAT YOU CAN DO FOR YOUR BRAIN. Repeat, Repeat, Repeat o Why does your clinician make you keep repeating the same thing over and over? Raymer et. al (2008). Translational research in aphasia

Explaining Neuroplasticity to PWA Raymer et. al (2008). Translational research in aphasia

Explaining Neuroplasticity to PWA Pulvermller & Berthier (2008). Aphasia therapy on a neuroscience basis

Outcome of Education Clients and family members are very receptive to information about neuroplasticity Visual aids and reducing jargon critical Qualitative changes were noted immediately: Increased motivation/participation More attempts to verbalize More willingness to try alternative modalities Seeking areas of change in their own environments EDUCATING CLIENTS ABOUT DRAWING, as part of multimodality communication, is critical!

Effects of Brain Damage on Drawing Left hemisphere lesions May have difficulty depicting internal details of a target picture, but little difficulty depicting external shape Some are unable to generate a mental image of the target items Some may have difficulty choosing the correct color for images Right hemisphere lesions May retain ability to produce internal details but fail to correct spatial relations or relative size May omit all or the left half of external configuration May interfere with ability to discriminate between colors (Helm-Estabrooks & Albert, 2004)

Left or Right Hemisphere Lesion? ?? Right hemisphereLeft hemisphere (Helm-Estabrooks & Albert, 2004)

History of Drawing and Aphasia Long history in the research literature (Lyons, 1995) Two primary areas of emphasis in the research: Realm of AAC and improving functional communication (Beeson & Ramage, 2000; Sacchett, 2002; Sacchett & Lindsay, 2007; Helm-Estabrook & Albert, 2004 ) For its impacts on verbal output as a source of de- blocking and semantic access (Davis, Farias, Baynes, 2005; Farias, Davis, Harrington, 2006)

Drawing and AAC Drawing can be used to compensate when a PWA is not able to use written or verbal language (Lyons, 1995) Advantage - no limitations caused by deficits in short-term memory and/or sequential ability (Sacchett, 2002) A move toward interactive (Sacchett, 2002) Supplementing Language Communication is enhanced by the drawings, so the emphasis is on the exchange of information and ideas Substituting Language Drawings are the communication, so emphasis is on producing recognizable content units.

De-blocking, Semantic Access, and Drawing Drawing vs. writing as a cue for verbal expression Drawing more effective in increasing verbal naming (Davis, Farias, Baynes, 2005; Farias, Davis, Harrington, 2006) Quality of the drawings has no impact on the ability to access the semantic features of a word (Farias, Davis, Harrington, 2006) Possible Explanation?

Communicative Drawing Program (CDP) The functional goal of the CDP is for patients with aphasia who are unable to convey desired messages through speech or writing to communicate instead through drawing. (Helm-Estabrooks & Albert, 2004, pg. 273)

CDP Candidate Profile (Helm-Estabrooks & Albert, 2004) Inability to communicate information through speech or writing Ability to use a medium-point felt-tip pen to copy one dimensional shapes Relatively intact visual memory Good visual attention skills Alert, cooperative, and willing to pursue a drawing program to improve functional communication Pre-morbid artistic skills are not required, nor it is necessary to draw with the dominant hand

Overview of Steps 1.Basic semantic-conceptual knowledge 2.Knowledge of object color properties 3.Outlining pictures of objects with distinct shape properties 4.Copying geometric shapes 5.Completing drawings with missing internal and external features 6.Drawing objects with characteristic shapes from memory 7.Drawing objects to command from stored representations 8.Drawing objects within superordinate categories 9.Generative drawing- animals and modes of transportation 10.Drawing cartooned scenes (Helm-Estabrooks & Albert, 2004)

Circle the objects that belong together. Criterion: 100% identification of the 5 items in 5 different superordinate categories Step 1 Basic Semantic-Conceptual Knowledge

Select the correct color (from 12 colored markers) and color that item. Criterion: Correct color choices for all nine items. Step 2 Knowledge of Object Color Properties

Take this pen and draw an outline around each object without touching the lines of the pictures Criterion: Each object outline conforms to shape and does not touch or trace the lines of printed stimuli Step 3 Outlining Pictures of Objects with Distinct Shape Properties

Take this pen and copy each figure. Your drawings should not touch the examples. Criterion: For each shape, all lines are present, in correct proportion and in proper relation to one another Step 4 Copying Geometric Shapes

Show me what is missing in this pictureOkay, now take this pen and fill in the missing part. Criterion: All completions are successfully rendered, complete, and conceptually reasonable Step 5 Completing Drawings with Missing External and Internal Features

Look at this object. (Present single card.) Remember what it looks like, because Im going to ask you to draw it. Okay, are you ready? Take this pen and draw the object here. Criterion: For each shape, all lines are present, in correct proportion and in proper relation to one another Step 6 Drawing Objects with Characteristic Shapes from Memory

Step 7 Drawing Objects to Command from Stored Representations Look at this word. It says____ Can you draw____right here? ice cream Criterion: Drawings of each object can be correctly identified by a nave judge who is not shown the target words

Step 8 Drawing Objects Within Superordinate Categories See this word? It says ___. I want you to think of a _____ and draw it right here. tool Criterion: All responses belong to the named superordinate category and can be identified correctly by a nave judge not provided with the target words.

Step 9 Generative Drawing: Animals and Modes of Transporting People On this page, I want you to draw as many types of animals/modes of transportation as you can. Make your drawings clear enough so that anyone could name the animals/modes of transportation that you draw. Criterion: All responses are correct exemplars of the superordinate categories and are correctly identified by a nave judge.

Step 10 Drawing Cartooned Scenes Show me what is funny about this pictureNow study the picture well because Im going to ask you to draw it from memory. Criterion: Each panel should be accurate enough for a nave judge to identify all elements necessary for understanding and stating the message (joke) of each cartoon.

CDP: Case Study (A.S) 45-yr-old male Status post left basal ganglia intracranial hemorrhagic CVA 10/03/2008 Dentist Bilingual in English (primary) and Japanese Mixed, non-fluent aphasia - Verbal expression characterized by: - 1-word utterances - verbal apraxia - decreased loudness and pitch range - confrontational naming relatively intact - Written expression characterized by 1-word phrases - Severe auditory and reading comprehension deficits (1-2 word phrases) - Used writing and gestures to supplement verbal expression

Sample CDP Goals (A.S.) Goal: To improve multimodality communication, the client will draw three-panel drawings from memory with enough detail so that a naive judge can identify the message. Objective 1: In the clinical setting, when given the name of an object with distinct visual features (i.e., ice cream cone, palm tree, etc), the client will draw the object with enough detail so that a naive judge can identify the object with 100% accuracy in a minimum of 10 trials. Objective 2: In the clinical setting, when given a superordinate category (e.g., tool, animals, vehicle, etc.) and no prompts, the client will draw at least one object within the specified category with enough detail so that a naive judge can identify the object with 100% accuracy in a minimum of 10 trials. Objective 3: In the clinical setting, when given a three-panel cartooned story, the client will draw the story from memory with enough detail so that a naive judge can identify the message of the cartoon in 2 out of 3 trials.

CDP Training/Tasks (A.S.) Began with Step 3 of the CDP During Step 7 1.Modeled adding more detail to unrecognizable drawings 2.Client began to spontaneously add more detail 3.Clinician provided the client with a puzzled look whenever his drawings were not recognizable Homework given for Steps 8-9 CDP supplemented by multimodality communication Barrier tasks encouraged all modalities

CDP Outcome Measure The client will draw one-, two-, and three-panel cartooned stories from memory. Panel should be accurate enough for a nave judge to identify all elements necessary for understanding and stating the message (joke) of each cartoon. (Helm-Estabrooks & Albert, 2004)

Post-Therapy Pre-Therapy

Maintenance/Generalization Data Caregiver interview and the Communicative Effectiveness Index (CETI) indicate that A.S. primarily communicates via verbal and written expression and drawing 1 2 3

Potential Supplemental Practice - Game Apps Draw Something Social drawing and guessing game Whiteboard Collaborative drawing app

New Frontier The role of new technologies in multimodal communication, including drawing. More questions than answers Is new technology applicable for drawing purposes? What technology is best? Any hidden pitfalls? Any potential advantages? Where to begin??????

Technology for PWA Most often technology is used in speech therapy for: stimulus material, data tracking, and AAC (ASHA, 2011) Technology-aided therapy effective, due to the increased intensity of treatment (van de Sandt- Koenderman, 2011) Likely due to potential for use with multiple communication partners and use across multiple settings

What is a Smartphone? Smartphone is a mobile phone that includes software that a user is able to modify and update. The user controlled software must be able to transfer information to and from external systems (Toyysy & Helenius, 2006, p. 110).

Windows Phone $400-$500 Android $300-$400 Blackberry $250-$500 Smartphone Options

iPhone $400-$600 HP Palm $400-$600 Nokia $400-$600

Alternate Formats - Tablets iPadAndroid Tablet $500 - $750$480-$750

Alternate Formats iPod Touch $299-$399

What is an App? Apps is an abbreviation for application Piece of software that can run on the Internet, on a computer, on a phone or other electronic devices Variety of available on different smartphones Some require internet connectivity, while others do run off-line

What is iTechnology? Technology manufactured by Apple, collectively referred to as: iTechnology or iDevices For example: iPads iPod Touch iPhones

Potential Benefits of Drawing Apps Unlimited numbers of pages accessible without erasing Save and organize drawings created for future access/modification Enhance drawings with colors, stored symbols, and backgrounds. Incorporate real photographs Possibility? GPS Picture Prediction (e.g., Locabulary for picture) Export and send drawings via email Some apps include keyboard function to augment drawing with typed letters, words, or phrases. Possibility? Text-to-speech Possibility? Interactive Alphabet Board/Phoneme cue

iTechnology: A Pilot Study Methods 4 PWA Ages 20 80 years old Non-fluent forms of aphasia Participants were evaluated on their ability to: Access and navigate within a drawing app (NotesPlus) Intelligibility of their drawings (as compared to traditional paper and pencil methods) Ostergren, J., & Raminick, J. (n.d.) Drawing and iTechnology: A pilot study in the use of a drawing iApp by individuals with aphasia (n.d.). Unpublished manuscript.

Paper/Pencil vs. iPad Intelligible to a naive judge using pen and paper Intelligible to a naive judge using the iPad PT1 HPR One-PanelYes Two-PanelYes Three-PanelNo PT2 One-PanelYes Two-PanelNo Three-PanelNo PT3 One-PanelNo Two-PanelYes Three-PanelYes PT4 HPL One-PanelNo Two-PanelYesNo Three-PanelYesNo

Example (PT 1): Paper/Pencil vs. iPad

App Use: Ease of Use After Training

PT 4: Words of Caution Still obstacles to over-come Non-dominant hand use/Limited control Registering drawing contact (false starts) No different in stylus vs. finger-tip Difficulty and frustration with touch screen use (with and without stylus) No previous touch screen use Lower levels of use and confidence with computers and technology in general

Finding the Right App: Recommended Feature Analysis - ASHA defines feature matching as: devices are selected based on relationships between an individual's strengths capabilities and communication needs in relation to various features of a device (ASHA, 2004, page 9).

Best predictor of long-term success with AT is careful selection of aids to ensure they are well- matched to the user and the environment (Scherer et al., 2007)

Which Apps to Use: App Analysis Ostergren, J. & Gastelum, M. (n.d.). An Exploration iTechnology Drawing Apps for Individuals with Aphasia. Unpublished manuscript. Evaluation of Drawing and Writing apps for communicative drawing purposes, given: Phase 1 Phase 2 Phase 3

Phase 1 App Identification (Part A) METHODS Search Terms: Apps with drawing or handwriting in search terms Consumer Ratings: Apps highly rated by users on the Apple Market place as a 4 or higher stars Device Mode: Apps that could be utilized on an iPad. Category: Productivity apps RESULTS 22 drawing apps 112 writing apps identified TOTAL = 134 apps

Phase 1 App Identification (Part B) METHODS Line Sizes. Apps with at least 8 line size options Color Choices. Apps with at least 8 color options Background Choices. Apps with the ability to modify the background in some fashion beyond strictly a white canvas were selected. Saving Capacity. Apps with the ability to save drawings for future retrieval and easily modification. Ability to Email. Apps with an email function. RESULTS 5 drawing apps 9 writing apps identified TOTAL = 14 apps

Phase 3 App Evaluation METHODS Apps from Phase 2 rated using 5 point scale, as follows Ease of Navigation: 1 = Complex navigation5 = Minimal/Easy Navigation Visual Distraction 1 = Maximal Distraction....................5 = Minimal Distraction Workspace: 1 = Small Icons/Workspace..5 = Large Icons/Workspace Iconicity: 1 = Opaque Icons5 = Transparent Icons

Drawing Apps

Writing Apps

Looking Deeper: Additional Trends Uncovered Options for setting hand use (right vs. left) Moveable palm rests Icon knowledge required Layers in saving Less is more Remote access/additional accounts (e.g., EverNote) Start position important Access knowledge required Tapping, swiping, and finger expansion

SketchTime $1.99 Language: English Developer: Hansol Huh 2012 Hansol Huh Rated 4+

Absolute Board Free Languages: English, Korean Seller: Hyeoseong Hwang ibluegene Apps Rated 4+

$6.49 Languages: English, Chinese, Japanese Seller: Rama Krishna 2012 Fluid Touch PTE LTD Rated 4+ Noteshelf

Jotter $0.99 Languages: English, Korean Seller: groosoft 2010 groosoft Rated 4+

NotesPlus $8.49 Category: ProductivityProductivity Language: English Seller: Viet Tran Viet Tran Rated 4+

Recent Replication: Pilot Study 13 additional PWA Varying forms of aphasia (fluent and non-fluent) Varying ages All in chronic stage of recovery from a CVA All receiving services at CSULB for communication impairments Asked to draw a 1-panel CDP cartoon using paper and pen and one (1) of the following apps: Sketch Time Jotter Absolute Board Noteshelf

Sketch Time Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1YesNo PT 2 HPL No PT 3No

Jotter Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1No PT 2 HPL YesNo* PT 3 HPL No

Absolute Board Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1YesNo PT 2 HPR NoYes PT 3 HPL No PT 4YesNo

Noteshelf Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1 HPL No PT 2 HPR No PT 3 No

RESULTS 5/17 PWA had intelligible drawings in the paper and pencil mode (29%) Two (2) had equally intelligible app drawings (Notesplus) Three (3) had less intelligible app drawings (Jotter, Absolute Board, and Sketch-Time)*

Ad Hoc and Qualitative Motor control and app use: 9/17 individuals had hemiparesis (52%) 3 of these individuals had intelligible drawings in one mode (33%) 1 = equally intelligible drawings in both modes 1 = less intelligible app drawing 1 = less intelligible paper/pencil Most individuals in our study with hemiparesis had unintelligible drawings in both modes (6/9, 67%)

Conclusion Interpret with CAUTION small sample! For some individuals, drawing using an app was less intelligible, with some apps But

The Devils in the Details Aim for Total Communication

Pointed to spot highlighted and said: Water and Hot. Also, gestured drinking.

Big Picture: Lessons Learned Aim for communication, not perfection in drawing. Incorporate within multimodality system (e.g., speech, writing, gestures, AND drawing) Provide education in neuroplasticity and the positive impact of drawing on verbal expression Consider pre-morbid technology use and comfort Increase acceptance by minimizing client frustration

Additional Lessons Learned: Experiment/Match to Client Compare Drawings on Different Apps Explore Palm Rest/Hand Use Settings Evaluate Stylus vs. Finger-tip Individual preference a factor Assess with different pen settings on the app (fine vs. large) Assess Case Withstand pressure from resting palm while drawing Adjust to different positions

Additional Lessons Learned: Therapy Tips Incorporate within Multimodality/Total Communication Training Speaking Gestures Drawing Writing Use Barrier Tasks Train Partners Use CDP to Enhance Details (Internal/External Features) Assess with Referential Communication Tasks

Referential Communication Task Based on Purdy and VanDyke (2009) CSULB Modifications: 15 real color photographs of common everyday activities Unfamiliar listener: Instructions: A PWA is going to be describing to you pictures shown to them. You will not be able to see these pictures. The PWA may speak, write, draw, gesture (or use a variety of methods) in describing these pictures to you. Please write down what you think the picture represents.

Future Research: Information Needed Develop and test frameworks for effectively and efficiently evaluating apps Explore communicative drawing and technology Influence of technology in communication dynamics Collect actual efficacy data Just because it is new, doesnt make it better. Move toward dedicate apps for PWA that allow for fluidity in modalities PWA input critical

Q & A For additional information about this presentation contact: Dr. Jennifer A. Ostergren Assistant Professor California State University, Long Beach Department of Communicative Disorders [email protected]

Reference ASHA. (2004). Roles and responsibilities of speech-language pathologists with respect to augmentative and alternative communication: technical report [Technical Report]. Available from www.asha.org/policywww.asha.org/policy Beeson, P.M., & Ramage, A.E. (2000). Drawing from experience: The development of alternative communication strategies. Topics in Stroke Rehabilitation, 7(2), 10-20. Beukelman, D., & Mirenda, P. (2012). Augmentative and Alternative Communication: Supporting children and adults with complex communication needs (4 th Ed.) Baltimore, MD: Brookes Publishing Co. Davis, C.H., Farias, D., Baynes, K. (2005). Understanding the effects of cuing strategies through error analysis. Brain and Language, 95, 183-184. Farias, D., Davis, C., & Harrington, G. (2006). Drawing: Its contribution to naming in aphasia. Brain and Language, 97, 53-63. Gonzalez Rothi, L. J., Mussi, N., Rosenbeck, J. C., & Sapieza, C. M. (2008). Neuroplasticity and rehabilitation research for speech, language, and swallowing disorders. Journal of Speech, Language, and Hearing Research, 51(1), S222-S224. doi: 10.1044/1092-4388(2008/017).

Reference Grafman, J. (2000). Conceptualizing functional neuroplasticity. Journal of Communication Disorders, July-August, 345-356. (Posted on Beachboard) Helm-Estabrooks, N. & Albert, M. (2004). Manual of Aphasia and Aphasia Therapy (2 nd Ed.). Austin, TX: Pro-Ed. Koul, R. K, & Corwin, M. (2011). Augmentative and alternative communication intervention for persons with chronic severe aphasia: Bringing research to practice. EBP Briefs 6(2), 18. Bloomington, MN: Pearson Lyons, J. (1995). Drawing: Its value as a communication aid for adults with aphasia. Aphasiology, 9, 33-94. Purdy, M., & VanDyke, J. (2009). Intermodal training to facilitate communication in aphasia: A pilot study. In Clinical Aphasiology Conference: Clinical Aphasiology Conference (2009 : 39th : Keystone, CO : May 26-30, 2009. Retrieved from: http://aphasiology.pitt.edu/archive/00002083/ http://aphasiology.pitt.edu/archive/00002083/

Reference Pulvermller, F., & Berthier, M. L. (2008). Aphasia therapy on a neuroscience basis. Aphasiology, 22(6), 563-599. doi: 10.1080/02687030701612213 Raymer, A. M., et. al (2008). Translational research in aphasia: From neuroscience to neurorehabilitation. Journal of Speech, Language, and Hearing Research, 51, S259-S275. Rautakoski, P. (2011). Training total communication. Aphasiology, 25, 344-365. Sacchett, C., & Black, M. (2011). Drawing as a window to event conceptualization: Evidence from two people with aphasia. Aphasiology, 25(1), 3 26. Sacchett, C. (2002). Drawing in aphasia: moving towards the interactive. International Journals of Human-Computer Studies, 54(4), 263-277. Sacchett, C., & Lindsay, J. (2007). Revealing competence and rethinking identify in severe aphasia using drawing and a communication book. In S. Byng, J. Duchan, & C. Pound (Eds). The Aphasia Therapy File, Volume 2. Hove, UK: Pyschology Press

Reference Saur, D., & Hartwigsen, G. (2012). Neurobiology of language recovery after stroke: Lessons from neuroimaging studies. Archives of Physical Medicine and Rehabilitation, 93(1), Supplement, S15-S25 Toyssy, S. and Helenius, M. (2006). About malicious software in smartphones. Journal in Computer Virology, 2, 109-119. van de Sandt-Koenderman WM. (2011). Aphasia rehabilitation and the role of computer technology: can we keep up with modern times?. International Journal of Speechlanguage Pathology 13(1):21-7.

itechnology and communicative drawing for individuals with aphasia jennifer a. ostergren, phd,...

Documents

neuroplasticity drawing

speech slide

cdp slide

aphasia pwa

welcome drawing

aphasia ostergren

total communication

aphasia correlations