otap data mf november 2016 · november 17, 2016 gayl bowser: [email protected] 2 guide informed...
TRANSCRIPT
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 1
Using Evidence Using Evidence Using Evidence Using Evidence to Strengthen to Strengthen to Strengthen to Strengthen Your AT ProgramYour AT ProgramYour AT ProgramYour AT Program
OTAP Moving Forward Series
November 18, 2016
11:30-12:30Instructor: Gayl Bowser
1
Discussion:Discussion:Discussion:Discussion:
:
What information and data do you currently collect about the work of your AT team?
How is it collected?
2
Three Types of EvidenceThree Types of EvidenceThree Types of EvidenceThree Types of Evidence
1. Data about general use of the intervention (AT Research)
2. Data about how AT works for an individual student
3. Data about what your team does
3
Using Research About Using Research About Using Research About Using Research About Assistive TechnologyAssistive TechnologyAssistive TechnologyAssistive Technology
Part 1:
Data About Use of the Intervention
(AT Research)
4
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 2
Guide informed decision making
Guide purchasing
Set programmatic goals
Compare solutions
Why we need to know Why we need to know Why we need to know Why we need to know about researchabout researchabout researchabout research
5
1. Use of audio text alone
2. Use of text to speech software
3. Use of audio text + print text + training in note taking
What would you choose?
Example: How might you Increase Example: How might you Increase Example: How might you Increase Example: How might you Increase Reading Comprehension?Reading Comprehension?Reading Comprehension?Reading Comprehension?
6
300.320 (a) (4) A statement of the special education and related services and supplementary aids and services, based on peer-reviewed research to the extent practicable, to be provided to the child, or on behalf of the child, and a statement of the program modifications or supports for school personnel that will be provided to enable the child--
7
From Final Regulations:
Teacher surveys about knowledge, attitude, etc.
AT user surveys
Reviews/synthesis of research
Application research
Outcomes research
Types of ResearchTypes of ResearchTypes of ResearchTypes of Research
8
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 3
Teachers who receive 40 or more hours of training about AT report that AT is essential to students’ daily activities.
Teacher who have not received training about AT report that AT is not important and are not confident in identifying and using AT with students.
◦Ashton, Lee & Vega (2005)
Survey: Need for trainingSurvey: Need for trainingSurvey: Need for trainingSurvey: Need for training
9
Students place importance on experiencing immediate benefit from use of AT;
AT was disliked if it interfered with peer relations;
The need to “take care of” the AT sometimes perceived as a burden;
Influential aspect: whether AT was accessible when needed and integrated into every day activities.
◦Hemmingsson, Lidstrom, Nygard. (2009).
Consumer SurveysConsumer SurveysConsumer SurveysConsumer Surveys
10
Reviewed 256 research articles about AT for VI
Found only two that used true research design. They were both single subject design.
They found, “A large percentage of the literature consists of anecdotal evidence of the impact of AT without evaluating the effectiveness of AT”.
◦ Kelly, S. & Smith, D., (2011). Impact of AT on Educational Performance of Students with
Visual Impairments.
Synthesis of Research: Synthesis of Research: Synthesis of Research: Synthesis of Research: Quality of Research in a specific areaQuality of Research in a specific areaQuality of Research in a specific areaQuality of Research in a specific area
11
AT not provided to majority of students with VI who could benefit from it.
AT more likely to be provided if parents are highly involved.
AT more likely to be provided if student is in residential placement.
◦ Kelly, S. (2009). Use of AT by Students with VI: Findings from a National Study.
Secondary Analysis Secondary Analysis Secondary Analysis Secondary Analysis SSSShows: hows: hows: hows: Big PictureBig PictureBig PictureBig Picture
12
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 4
Can use of cognitive mapping improve grasp of critical information?
High school students with behavior outbursts and LD in reading
Use of Inspiration software changed scores on chapter quizzes from <15% to >75%
Decreased off task behavior and reliance on teacher for help
• Blankenship, T.L., Ayers, K. M., & Langone, J. (2005).
Application StudiesApplication StudiesApplication StudiesApplication Studies
13
Studied how students in special education are impacted by the inclusion of AT as part of an intervention;
Repeated-measures pretest–posttest quasi-experimental design;
AT’s contribution as an intervention strategy is greater than nine other possible intervention strategies.
◦ Watson, Ito, Smith & Anderson (2010)
Impact of ATImpact of ATImpact of ATImpact of AT
14
Augmentative and Alternative Communication-www.isaac-online.org/en/publications/aac.html
Assistive Technology-www.resna.org
Journal of Research on Technology in Education-www.iste.org
Journal of Special Education Technology-www.tamcec.org or jset.sagepub.com
Journals of your professional organizations
15
Journals with AT ResearchJournals with AT ResearchJournals with AT ResearchJournals with AT Research
16
MMMMore about ore about ore about ore about Research Research Research Research and how to and how to and how to and how to use ituse ituse ituse it
https://pantherfile.uwm.edu/edyburn/www/what/index.html
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 5
Part 2:
Data About How AT Works for an Individual Student
17
www.educationtechpoints.org
http://[email protected]
Quality Indicators for the Evaluation of Effectiveness of AT Services
http://[email protected]
Evaluation of Effectiveness of AT Services
1. Shares responsibilities for data collection, evaluation, and
interpretation
2. Collects data on specific student achievement and related to goals
3. Includes quantitative and qualitative measurement of changes
4. Evaluates across environments during naturally occurring and
structured activities
5. Provides teams with a means for analyzing student achievement
6. Makes changes when indicated by data
7. Views evaluation as a dynamic, responsive, ongoing process
Student Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection Opportunity
Trial period plan
20
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 6
Trial PeriodsTrial PeriodsTrial PeriodsTrial Periods
�Trial Period form
� Specifies the tool(s) on trial
�Describes the environment for use
�Dates of trial
� Indicators of effectiveness
�Identify staff/student training needs
�Identify staff person responsible for follow-up
21
Trial Period Plan
www.otap-Oregon.org
Results of Completed Trial Data
23
24
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 7
Student Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection OpportunityStudent Data Collection Opportunity
Trial Period Follow-up
25
Completed Trial Completed Trial Completed Trial Completed Trial
Scan
26
Student FollowStudent FollowStudent FollowStudent Follow----up up up up DocumentationDocumentationDocumentationDocumentation
Data/Work samples
What worked
AT considerations documented on the student’s IEP
What didn’t work
Feedback about the support you provide
27
Example Data Set:Example Data Set:Example Data Set:Example Data Set:Tools used by studentsTools used by studentsTools used by studentsTools used by students
28
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 8
29
HIAT HIAT HIAT HIAT Online Online Online Online Effectiveness Effectiveness Effectiveness Effectiveness SurveySurveySurveySurveyHandoutHandoutHandoutHandout
Example Data Set:Example Data Set:Example Data Set:Example Data Set:Student changes as a result Student changes as a result Student changes as a result Student changes as a result of the trial Periodof the trial Periodof the trial Periodof the trial Period
Strategies were...
A. AT strategies not
implemented at this
time
B. AT trial stategies
ineffective
C. Some Trial stategies
effective
D. All trial strategies
effective
30
Student Outcomes of Student Outcomes of Student Outcomes of Student Outcomes of the Trial Periodthe Trial Periodthe Trial Periodthe Trial Period
�Student has or is developing use of a new tool or strategy
�Student does not require AT
�No changes for student
31
Data about changes in Data about changes in Data about changes in Data about changes in student performancestudent performancestudent performancestudent performance
�Quality of student work improved
�Student is more independent with identified task
�Productivity increased
�Time on task increased
�Time on task decreased
�Motivation improved
�Behavior improved
32
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 9
Changes in Student Changes in Student Changes in Student Changes in Student BBBBehaviorehaviorehaviorehavior
33
If you don’t communicate the value of what you do in evidence- based ways, then no one can appreciate what you do.
Part 3:
Data About What Your Team Does
34
Commonly Used AT Teams DataCommonly Used AT Teams DataCommonly Used AT Teams DataCommonly Used AT Teams Data
Number of staff
Number of students served
35
Implementation Process Data
Training provided
Primary Contact Role
Hours of AT Team Support
Kinds of AT Support
36
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 10
Keep a Team Data File!Keep a Team Data File!Keep a Team Data File!Keep a Team Data File!Be ready with answers!Be ready with answers!Be ready with answers!Be ready with answers!
How many students have you served so far this year?
What are the reasons that IEP teams request your services?
What types of services have you provided?
How many full assessments have you completed?
How much time do you spend with individual students?
How do your numbers for this year compare to last year’s numbers?
37
Opportunities for BuiltOpportunities for BuiltOpportunities for BuiltOpportunities for Built----in Data Collectionin Data Collectionin Data Collectionin Data CollectionHIAT Model of School SupportHIAT Model of School SupportHIAT Model of School SupportHIAT Model of School Support
School completes an online survey
describing
the type of support needed
Contact case
manager at school to
gather
additional relevant
information
Determine level of support required
Tool Consult
Phone Consult
SETT Meeting
Staff Training
Trial period with
training
Follow up/
documentation
38
Services Data Collection OpportunityServices Data Collection OpportunityServices Data Collection OpportunityServices Data Collection Opportunity
School completes an online referral form or survey describing the child and the type of support needed
39
40
Traditional Traditional Traditional Traditional Referral FormReferral FormReferral FormReferral Form
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 11
An Alternative An Alternative An Alternative An Alternative Request for Support SurveyRequest for Support SurveyRequest for Support SurveyRequest for Support Survey
Reason for Request
What prompted you to make a request for AT services?
◦ (My idea, parent request, advocate request, administrator suggestion, central office suggestion, mediation or due process agreement, conversation with an AT Team member)
Information on the student for whom you are considering AT
◦ (Name, ID, grade, disability, program, reading level, math level, case manager)
41
OnlineSurvey
42
Questions about team’s Questions about team’s Questions about team’s Questions about team’s consideration/assessment skillsconsideration/assessment skillsconsideration/assessment skillsconsideration/assessment skills
How would you rate your team’s ability to meet and complete the following? (Scale: not comfortable, somewhat comfortable, comfortable, very comfortable)
◦ Identify and prioritize critical barriers to learning for a specific student
◦ Collaboratively problem solve as a team
◦ Identify a clear diagnostic question that can be answered during an AT trial period
43
Questions about team’s trial Questions about team’s trial Questions about team’s trial Questions about team’s trial period skillsperiod skillsperiod skillsperiod skills
How would you rate your team’s ability to do the following tasks related to AT trials?
(Scale: not comfortable, somewhat comfortable, comfortable, very comfortable)
◦ Identify available technology tools at your school and how they can support student needs
◦ Match potential AT tools to the identified student needs
◦ Develop a plan to use technology strategies to answer your diagnostic questions
◦ Ensure that staff and students involved can use the features of the tool(s) on trial for the identified student
◦ Use a team process to monitor the trial and make adjustments as needed
44
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 12
Questions About Team’s Ongoing Questions About Team’s Ongoing Questions About Team’s Ongoing Questions About Team’s Ongoing Implementation SkillsImplementation SkillsImplementation SkillsImplementation Skills
How would you rate your team’s ability to do the following tasks related to ongoing AT implementation?
(Scale: not comfortable, somewhat comfortable, comfortable, very comfortable)◦ Use a team process to integrate technology tools into instructional
routines
◦ Document the rationale for the AT on the IEP and other student records
What is your knowledge of the following tools? (List of tools commonly used in the district)◦ (Scale: Do not know this well, know how this can help students, use
this regularly with students, can train other staff on this)
◦ List applications or other tools most commonly used in the district(s)
45
46
Agency Data from Your Agency Data from Your Agency Data from Your Agency Data from Your Online Request SurveyOnline Request SurveyOnline Request SurveyOnline Request Survey
Underlying Student Issues Identified by Teams.
�Math
�Language
�Spelling
�Reading
�Organization
�Behavior
�Composition
�Communication
47
Services Data Collection OpportunityServices Data Collection OpportunityServices Data Collection OpportunityServices Data Collection Opportunity
Determine course of action with case manager.
48
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 13
Initial ContactInitial ContactInitial ContactInitial Contact
Frame the question
Examples of items which you might request to better serve the process:
◦ Current levels of performance
◦ Written Productivity Profile (WPP)
◦ Protocol for Accommodations in Reading (PAR)
Determine course of action
49
Your Action OptionsYour Action OptionsYour Action OptionsYour Action OptionsProvide Resources
◦ Technology-options Handout
◦ Link to “Strategies for supporting struggling students in the classroom”
◦ Quick Guides
◦ Road Maps
◦ HIAT Help Blog
◦ Newsletter
Loan necessary tools when appropriate
Identify training needs
Attend IEP meeting or assessment meeting
Complete traditional multidisciplinary assessment
And……
50
Tracking the Actions Your AT Team Takes
• Led an assessment meeting
• Participated in assessment meeting
• Regular face-to-face with student
• Consult to IEP team
• Loaned equipment for trial period
• Loaned equipment for long-term use
• Training provided
• Consultation about implementation plan
51
Automated Referral: Automated Referral: Automated Referral: Automated Referral: Tracking services in new waysTracking services in new waysTracking services in new waysTracking services in new ways
52
Reasons for Request for Support School Contact
� Composition � Math Name:
� Handwriting � Language Delay Phone:
� Organization � Communication Email:
� Behavior � Mobility
� Spelling � Vision Tools Used/Added
� Reading � Hearing Tool #1
Services ProvidedTool #2
Tool #3
� Full SETT Meeting
Support
� Trial Period Consult Provided Level of Support
� Tool Consult Provided � Equipment Loaned
� Training Provided � Implementation Plan Consult
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 14
Adapted from DeCoste, D. (2013) HIAT program data report
53
Intensity of services provided
Adapted from DeCoste, D. (2013) HIAT program data report54
Services Data Collection OpportunityServices Data Collection OpportunityServices Data Collection OpportunityServices Data Collection Opportunity
Follow-up documentation
55
Interview feedback about Interview feedback about Interview feedback about Interview feedback about the support you providethe support you providethe support you providethe support you provide
◦ review goals of support relationship
◦ reach consensus on efficacy of process for the student
◦ Reach consensus on efficacy of process for staff
◦ decide to continue or discontinue support
56
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 15
57
HIAT Online effectiveness SurveyHandout
Staff OutcomesStaff OutcomesStaff OutcomesStaff Outcomes
Staff learned our consideration and assessment and trial period process and will be able to complete it independently in the future.
Staff exposed to our consideration, assessment and trial period process but will need further support.
Staff learned new strategies to support the student.
Staff learned new strategies to incorporate and will be able to support multiple students in the future.
Consideration and Assessment process was not useful to the staff at this time
58
Common Issues Affecting Common Issues Affecting Common Issues Affecting Common Issues Affecting EffectivenessEffectivenessEffectivenessEffectiveness
• Strategies too dependent upon adult support
• Not enough opportunities to use strategies
• Strategies took too much time to implement
• Student did not use strategies appropriately or see them as useful
• Strategies did not result in improvement
• Strategies did not prove necessary
• Absenteeism
59
Staff OutcomesStaff OutcomesStaff OutcomesStaff Outcomes
Staff learned our consideration and assessment and trial period process and will be able to complete it independently in the future.
Staff exposed to our consideration, assessment and trial period process but will need further support.
Staff learned new strategies to support the student.
Staff learned new strategies to incorporate and will be able to support multiple students in the future.
Consideration and Assessment process was not useful to the staff at this time
60
OTAP Moving Forward Series: #2
Using Evidence
November 17, 2016
Gayl Bowser: [email protected] 16
Staff outcomes as a result of SupportStaff outcomes as a result of SupportStaff outcomes as a result of SupportStaff outcomes as a result of Support
61
What are three types of data that you will plan to collect during this school year to help you understand and improve the effect that your AT team is having on AT use in your district?
MAKE A MAKE A MAKE A MAKE A PLAN !PLAN !PLAN !PLAN !
YOUR TURNYOUR TURNYOUR TURNYOUR TURN
62
63
Ideas? Questions? Comments?