using date to target implementation supports in a multi ... · using date to target implementation...

CHAPTER 9

Using Date to Target Implementation Supports in a Multi-Tiered Model

What Will This Chapter Tell Me?Previous PRIME Manual chapters have described how to collect and

graph implementation and student data. This chapter describes how to integrate treatment integrity, student progress-monitoring, and Implementation Beliefs Assessment (IBA) data to make decisions in the PRIME Model. After reading this chapter, you will be prepared to integrate these data sources to make appropriate data-based deci-sions and, if treatment integrity levels are low, decreasing, or highly variable, identify an appropriate PRIME Implementation Support.

What is Data-Based Decision Making in PRIME?PRIME is designed to support student outcomes through a focus

on intervention implementation. As such, PRIME data-based deci-sions involve reviewing implementation data (i.e., treatment in-tegrity and IBA data) alongside progress-monitoring data to deter-mine next steps for the intervention and implementation supports. Graphing and interpreting data are critical first steps for making data-based decisions (see Chapter 8). Next, treatment integrity and progress-monitoring data are reviewed together and one of four possible data-based decisions is made. Depending on whether im-plementation is deemed sufficient and whether the student is on

95

track to meet his or her goal, the appropriate next step will involve continuing current practice, conducting further evaluation, chang-ing the intervention, or providing PRIME Implementation Supports. Specific action steps are described below. If PRIME Implementation Supports are deemed appropriate, treatment integrity and IBA data will help you identify what strategy is appropriate to support the implementer.

How to Make PRIME Data-Based Decisions PRIME data-based decisions involve incorporating treatment in-

tegrity and progress-monitoring data. In Chapter 8, you graphed these data sources based on the questions of interest and devel-oped summary statements to describe the data. The next step is to integrate these summary statements to make a data-based decision. Use Appendix I to complete this three-step process.

1. Use the progress-monitoring summary statement to deter-mine whether the student is on track to meet his or her goal. That is, you will review the trend, level, and variability of the progress monitoring data to evaluate if the progress-mon-itoring trend line (i.e., the student‚Äôs overall progress) is aligned with the aim line (i.e., the line from baseline to the intervention goal). Based on this interpretation of the prog-ress-monitoring graph, decide whether (a) yes, the student is on track to meet his or her goal, or (b) no, the student is not on track to meet his or her goal.

2. Use the treatment integrity summary statement to determine whether the implementation is sufficient or insufficient. That is, is the intervention implemented to an extent where it could be expected that the student would benefit from the intervention? There is no specific level of implementation suggested across all interventions to be sufficient. Rather, review the current level of implementation overall and by

USING DATE TO TARGET IMPLEMENTATION SUPPORTS IN A MULTI-TIERED MODEL 96

How to Make Action Plans Based on Data-Based DecisionsIn the preceding section, you integrated the progress-monitoring

and treatment integrity summary statements to identify the pres-ent scenario. With these two data sources, there are four potential scenarios. The following section describes these scenarios and their associated action steps. Use Appendix I, the PRIME Data-Based Decision Making Worksheet, to walk through these decision-mak-ing steps. Record the appropriate scenario and action steps on the PRIME Intervention Implementation Tracking Form (Appendix D).

Scenario A: On Track to Meet Student Goal & Sufficient Treatment Integrity

In Scenario A, the student is making appropriate progress toward his or her intervention goal and the intervention is being imple-mented as planned. The intervention can be considered effective, based on the student‚Äôs progress. As the student is moving toward his or her intervention goal, it is not necessary to make any changes to the intervention or its implementation at this time. The action steps that correspond to this scenario, listed below, are related to continuing the current plan to support the student and implementer.

Scenario A Action Steps Continue to deliver the intervention at its current level of implementation.

Regularly evaluate progress monitoring and treatment integrity data to ensure the current progress and level of implementation is maintained over time.

Scenario B: On Track to Meet Student Goal & Insufficient Treatment Integrity

In Scenario B, the student is making appropriate progress to-wards his or her intervention goal, but the intervention is not be-ing implemented as planned. In this case, it is unclear whether the


intervention is effective. There are three possible reasons for this pattern of progress mon-

itoring and treatment integrity data: 1. It is possible that something outside of the intervention is re-

lated to the improvement in student outcomes. For example, it is possible that a seemingly unrelated change in class-room management, family activities, or academic/behavior support may have led to improvements.

2. It may be necessary to update the intervention goal and the corresponding expected rate of student progress (i.e., aim line). Data may indicate that, with appropriate support, the student can more quickly reach higher levels of improve-ment than expected.

3. There may be issues with the treatment integrity assess-ment method. For example, the treatment integrity tool may include several steps that are not directly related to the improvements in student outcomes and, if the implementer skips these steps, it would result in a low treatment integ-rity score, but actually sufficient implementation to support student outcomes. Or, the treatment integrity observations may not have accounted for implementation across the full session (e.g., day) so may be an underestimate of treatment integrity.

The action steps that correspond to Scenario B, listed below, and provided in a flowchart on Scenario B Action Steps worksheet in Appendix I, involve identifying the reason for the pattern of prog-ress monitoring and treatment integrity data and then taking corre-sponding next steps.

Scenario B Action Steps (see also Appendix I) Talk with the implementer and other stakeholders (e.g., classroom teacher, parent, paraprofessional, student) to


identify if any changes have been made outside of interven-tion implementation.

Review research on the intervention to evaluate if the initial projection for the intervention goal was appropriate.

Review the intervention step treatment integrity data, cover-age of the treatment integrity assessment method, and the representativeness of the observation sessions to identify if the intervention steps directly related to improvements in student outcomes were appropriately captured. Note: It may be useful to review intervention research to identify what steps most impact student outcomes.

Decide which of the three possible reasons for the data pat-tern most likely explains the current scenario.

If the data pattern is explained by reason 1, systematically discontinue implementation of the intervention, as it may not be necessary for the student to reach his or her goal. Continue to regularly evaluate progress-monitoring data to ensure that the improved student outcomes are maintained.

If the data pattern is explained by reason 2, review the in-tervention goal and adjust the aim line for student progress. Reevaluate the progress-monitoring and treatment integrity data to make an appropriate, data-driven decision about next steps. Continue to regularly evaluate progress-monitor-ing and treatment integrity data.

If the data pattern is explained by reason 3, adjust the treat-ment integrity assessment method and/or observation ses-sions to appropriately account for the intervention. Reevalu-ate the progress-monitoring and treatment integrity data to make an appropriate, data-driven decision about next steps. Continue to regularly evaluate progress-monitoring and treatment integrity data.


Scenario C: Not On Track to Meet Student Goal & Sufficient Treatment Integrity

In Scenario C, the student is not making appropriate progress to meet his or her intervention goal, however, treatment integrity data show the intervention is being implemented as planned. It appears the intervention is not effective for the specific student and their intervention goal.

Before changing the intervention, consider whether the treatment integrity data accurately account for current implementation. The assessment method or sessions observed may have inadvertently suggested a higher level of implementation than actually occurred regularly. For example, during the session observed, the implement-er could have delivered the intervention with higher than usual treatment integrity or the implementer could have overestimated their implementation on the self-report measure.

If it is believed the intervention is appropriate for the problem and student, the consultant and implementer could increase the intensi-ty of the intervention. For instance, you may increase the frequency and duration of the intervention (e.g., delivering an academic inter-vention four times per week instead of two) or change the format of intervention delivery to decrease the ratio of students to teachers (e.g., delivering an academic intervention to a student one-to-one rather than in a small group). Make modifications to that interven-tion that are research supported and theoretically sound.

If changes to the treatment integrity assessment plan or modifica-tions to the intensity of the intervention do not result in changes for the student, the intervention may not be appropriate for the student. In this case, review the baseline data, hypotheses developed, and identify an alternative evidence-based intervention.

The action steps that correspond to Scenario C, listed below, and provided in a flowchart on Scenario C Action Steps worksheet in Appendix I, involve identifying the reason for the pattern of prog-ress-monitoring and treatment integrity data and then taking corre-sponding next steps.


Scenario C Action Steps (see also Appendix I) Review the intervention step treatment integrity data, treatment integrity assessment method, and the repre-sentativeness of the observations session to identify if the intervention steps directly related to improvements in student outcomes were appropriately captured. Note: It may be useful to review intervention research to identify what steps most impact student outcomes. If appropriate, revise the treatment integrity assessment method. Reevaluate the progress monitoring and treatment integrity data to make an appropriate, data-driven decision about next steps.

Consider increasing the intensity of the intervention. If such modifications are made, update the Implementation Plan to account for new implementation behaviors and poten-tial barriers. After subsequent implementation, evaluate the progress monitoring and treatment integrity data to make an appropriate, data-driven decision about next steps.

Decide to change interventions. To do so, begin the PRIME process again, beginning with Problem Identification (see Chapter 2).

Scenario D: Not On Track to Meet Student Goal & Insufficient Treatment Integrity

In Scenario D, the student is not making sufficient progress to meet his or her intervention goal and the intervention is not being implemented as planned. In this case, treatment integrity should be promoted through the delivery of a PRIME Implementation Support. To identify the appropriate PRIME Implementation Support, you need to decide (a) what Tier of support is appropriate and (b) what support is appropriate for the specific situation.

In general, provide PRIME Implementation Supports beginning at the Tier 1 level before delivering strategies at Tier 2 and, last, pro-viding ongoing intensive support at Tier 3. To decide what Tier of


PRIME Implementation Support is appropriate for the specific situa-tion, consider the following (see also flowchart on Scenario D Action Steps worksheet in Appendix I):

1. Review implementation history. What PRIME Implementation Supports have been provided to support the implementer previously? For example, if an implementer is struggling and received Tier 1 supports months ago, it might make sense to again deliver Tier 1 supports or deliver Tier 2 strategies. If an implementer has received several Tier 2 strategies, but continues to deliver the intervention with low treatment integrity, it may be appropriate to provide Tier 3 supports.

2. Consider the intensity of the intervention. At what tier is this intervention? What decisions may be made based on the student‚Äôs response to this intervention? In most cases, there should be a relationship between intensity of the intervention provided to the student and the intensity of implementation support provided to the implementer. That is, for a student intervention that is more intensive (e.g., stu-dent spends much time outside of the classroom, it‚Äôs re-source-intensive, or significant decisions will be made based on these data), more quickly deliver increasingly intensive PRIME Implementation Support.

PRIME Tip• If the student is receiving a less intense intervention, provide

Tier 1 Implementation Supports and deliver Tier 2 Implementation Support Strategies several times, before providing Tier 3 Implemen-tation Support, if needed.

• If the student is receiving an intensive intervention, the consul-tant should more quickly move from Tier 1 and Tier 2 Implemen-tation Supports to Performance Feedback at Tier 3 of the PRIME Model. For example, if an implementer experiences challenges


implementing an intensive behavior support plan for a student (i.e., Tier 3 student intervention), the consultant may provide a Tier 2 Implementation Support Strategy once before moving to Tier 3, if treatment integrity support is still warranted.

Once you have decided what Tier of PRIME Implementation Sup-port is warranted, it is time to determine what specific support is appropriate. To do so, use treatment integrity and IBA data to target what Implementation Support best addresses the challenges the implementer is experiencing.

For Tier I Implementation Supports, decide whether to provide Di-rect Training, Implementation Planning, or both supports. If these supports have not yet been provided, we suggest both are delivered to facilitate high levels of treatment integrity. At the very least, Di-rect Training should be provided due to its foundational steps. If these supports have been provided, but specific intervention steps are regularly missed, consider providing ‚Äúbooster‚Äù Direct Train-ing, updating the Implementation Plan for those steps, or both. This plan would also be an appropriate strategy if the IBA data indicate low levels of self-efficacy. If the IBA data indicate low outcome ex-pectations or overall treatment integrity is low, consider providing Direct Training again.

For Tier 2 Implementation Supports, decide which of the four strat-egies is appropriate to address the implementer‚Äôs treatment in-tegrity. To do so, review treatment integrity and IBA data and, based on these data, identify what strategy is appropriate.

• Role Play or Participant Modeling are appropriate strategies if (a) intervention step treatment integrity data indicate some steps are not implemented at all; (b) session treat-ment integrity data indicate low levels of quality, even if adherence is overall adequate; or (c) the IBA data indicate the self-efficacy subscale is low. To decide between these two PRIME Implementation Support strategies, review the


strategy protocol and consider the implementer and context. For example, some implementers may welcome a consultant into their classroom (necessary for Participant Modeling), while others would prefer to practice in a meeting format (i.e., Role Play).

• Motivational Consulting or Raising Awareness are appropri-ate strategies if (a) treatment integrity is inconsistent (e.g., fully implemented some days, little implementation the next); (b) treatment integrity has been high, but has de-creased over time; (c) treatment integrity data indicate low levels of exposure, even if adherence is overall adequate; or (d) the IBA data indicate the outcome expectations subscale is low. To decide between these two PRIME Implementation Support strategies, review the strategy protocol and consider the implementer and their feedback about the intervention and its implementation. For example, if the implementer sees the benefit of the intervention, but has low self-effica-cy about implementing consistently or frequently enough, Motivational Consulting may be more suited, while if the implementer needs support to increase their understanding of expected outcomes as well as support to implement more consistently and comprehensively, Raising Awareness may be best.

For Tier 3 Implementation Supports, provide Performance Feedback. As described in Chapter 14, use the intervention intensity, treatment integrity data, and IBA data to decide how frequently to provide Per-formance Feedback.

The action steps that correspond to Scenario D, listed below, in-volve identifying what Tier of support is appropriate, determine what specific support is appropriate for the situation, and preparing to do deliver the PRIME Implementation Support.


Scenario D Action Steps Determine what Tier of PRIME Implementation Support is appropriate to support the implementer

Review implementation support history, treatment integrity and IBA data, and contextual/implementer factors to deter-mine the most appropriate PRIME Implementation Support.

Prepare to deliver the PRIME Implementation Support by reviewing the appropriate PRIME Manual chapter (Chapter 3-4 or 10-14), the General Implementation Support Guide-line (Appendix E) and the specific PRIME Implementation Support protocol (Appendices F-G or J-N).

Continue to regularly evaluate progress-monitoring and treatment integrity data.

What Did I Learn About PRIME? In the PRIME Manual, treatment integrity data, progress-moni-

toring data, and IBA data should be collected, graphed, and used to make data-based decisions. This chapter addressed how to in-tegrate data to make appropriate decisions and next steps. One of four data patterns can be found when treatment integrity and prog-ress-monitoring data are reviewed together. Depending on whether implementation is deemed sufficient and whether the student is on track to meet his or her goal, the appropriate next step will involve continuing current practice, conducting further evaluation, changing the intervention, or providing PRIME Implementation Supports.

Chapter 9 Key Terms Data-based Decision Making


using date to target implementation supports in a multi ... · using date to target implementation...

Documents