![Page 1: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/1.jpg)
Data-based Decision Making Cultures:Four Assumptions
Ronnie Detrich
Wing Institute
Association for Positive Behavior SupportSt. Louis
March 26, 2010
![Page 2: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/2.jpg)
Goals for Today
• Describe the four assumptions necessary for data-based decision making to be effective.
• Explore the degree to which these assumptions are currently true.
![Page 3: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/3.jpg)
Context for Data-based Decision Making
• Influence of problem solving model (Deno, 2005). Problem: Discrepancy between expected and actual
performance. Data help identify discrepancies.
![Page 4: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/4.jpg)
Why Data-based Decision Making?
• Data is only one basis for making decisions. Intuition Experience Authority Persuasion Preference
![Page 5: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/5.jpg)
Why Data-based Decision Making?
• Advantages: Replicable Transparent Objective
• Disadvantages Difficult to implement Loss of certainty May not confirm our
preferences
![Page 6: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/6.jpg)
Function of Data-based Decision Making
Increase the probability that socially valued
outcomes will be achieved.
• Applies to all levels of a system: students staff resource allocation organizational
![Page 7: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/7.jpg)
Data-based Decision Making as Ethical Obligation
• DBDM central in many ethical standards for: selecting interventions
evaluating interventions
![Page 8: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/8.jpg)
Data-based Decision Making as Ethical Obligation: Selecting Interventions
• American Psychological Association Ethical Standard 2.04: Psychologists’ work is based on the established
scientific and professional knowledge of the discipline.
![Page 9: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/9.jpg)
Data-based Decision Making as Ethical Obligation: Selecting Interventions
• National Association of School Psychologists Standard III F 4.
School psychology faculty members and clinical or field supervisors uphold recognized standards of the profession by providing training related to high quality, responsible, and research-based school psychology services.
![Page 10: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/10.jpg)
Data-based Decision Making as Ethical Obligation: Selecting Interventions
• National Association of School Psychologists Standard IV 4.
School psychologists use assessment techniques, counseling and therapy procedures, consultation techniques, and other direct and indirect service methods that the profession considers to be responsible, research-based practice.
![Page 11: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/11.jpg)
Data-based Decision Making as Ethical Obligation: Selecting Interventions
• Behavior Analyst Certification Board Standard 2.09a
The behavior analyst always has the responsibility to recommend scientifically supported, most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society.
Standard 2.09bClients have a right to effective treatment (i.e., based on the
research literature and adapted to the individual client).
![Page 12: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/12.jpg)
Data-based Decision Making as Ethical Obligation: Evaluating Interventions
• National Association of School PsychologistsStandard IV C 1b.
Decision-making related to assessment and subsequent interventions is primarily data-based.
Standard IV 6.School psychologists develop interventions that are
appropriate to the presenting problems and are consistent with the data collected. They modify or terminate the treatment plan when the data indicate the plan is not achieving the desired goals.
![Page 13: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/13.jpg)
Data-based Decision Making as Ethical Obligation: Evaluating Interventions
• Behavior Analyst Certification Board Standard 4.04
The behavior analyst collects data or asks the client, client-surrogate, or designated other to collect data needed to assess progress within the program.
Standard 4.05The behavior analyst modifies the program on the basis of
data.
![Page 14: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/14.jpg)
Data-based Decision Making as Ethical Obligation
• Services funded by tax dollars. Dollars are for public benefit. Fiduciary responsibility to assure educational benefit
to everyone. Data-based decision making provides the best
chance of assuring benefit. If DBDM is an ethical responsibility then it follows that
assuring there is a system to support DBDM is also an ethical responsibility.
![Page 15: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/15.jpg)
Treatment
EvidenceValues
Basis for Choosing TreatmentSzatmari (2004)
![Page 16: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/16.jpg)
Do Nothing
NoneUnethical
Clinical Paralysis
![Page 17: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/17.jpg)
Do Nothing
Clinical Paralysis
Unethical
None
![Page 18: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/18.jpg)
Toss a Coin
NoneUnethical in light of evidence
![Page 19: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/19.jpg)
Do Nothing
Clinical Paralysis
Unethical
Toss a Coin
Unethical in light of evidence
None None
![Page 20: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/20.jpg)
Training
NoneOutdated
lotsCurrent
![Page 21: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/21.jpg)
Do Nothing
Clinical Paralysis
Unethical
None
Toss a Coin
Unethical in light of evidence
None
Training
Outdated
![Page 22: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/22.jpg)
Etiology
limitedDifficult
![Page 23: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/23.jpg)
Do Nothing
Clinical Paralysis
Unethical
None
Toss a Coin
Unethical in light of evidence
None
Training
Outdated
Etiology
limitedDifficult
![Page 24: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/24.jpg)
ABA
lots
Not very humane
effective
![Page 25: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/25.jpg)
ABA
Not very humane
Do Nothing
Clinical Paralysis
Unethical
None
Toss a Coin
Unethical in light of evidence
None
Training
Outdated
Etiology
limitedDifficult
ABA
Not very humane lots
![Page 26: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/26.jpg)
Developmental Social cognitive
Not yetLove it
![Page 27: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/27.jpg)
Developmental social cognitive
Love it Not yet
Inform Parents of Options
Do Nothing
Clinical Paralysis
Unethical
None
Toss a Coin
Unethical in light of evidence
None
Training
Outdated
Etiology
limitedDifficult
To be Ethical:
ABANot very humane
Effective
lots
![Page 28: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/28.jpg)
Four Assumptions
• Data are reliable and valid.• Decision-makers have the skills to base
decisions on data.• Decision-makers know what to do when a
problem is identified.• Those responsible for implementing a solution
have the skills, supports, and resources to implement with integrity.
![Page 29: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/29.jpg)
Assumption 1: Reliable and Valid Data are Available
• Reliable: Two measures of the same phenomenon result in the same score. Across recorders (inter-observer agreement). Same instrument measuring same phenomenon on
different occasions (test-retest). Consider a thermometer:
two thermometers measuring the same temperature should obtain the same result.
the same thermometer should obtain the same result in back to back measurements.
Reliability is not necessarily accuracy.
![Page 30: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/30.jpg)
Assumption 1: Reliable and Valid Data are Available
• Valid: assessment is measuring what it purports to measure.
• Validity is contextual. A measure can be valid for one purpose but not for
another.
![Page 31: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/31.jpg)
Validity and Inference
Low InferenceBehavior of interest in its own right.
No attempt to generalize to other times, settings, behaviors.
Changes in behavior have no predictive value.
Example: Talking out
High InferenceBehavior interesting as a measure beyond its immediate context.
Generalize from behavior within a context to other times, settings, and behaviors.
Changes in behavior predict changes in risk status.
Example: Words read correctly/minute
![Page 32: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/32.jpg)
General Outcomes Measures (GOMS)
• Both RtI and PBS are prevention approaches.• In many instances, we want to know the
predictive value of a measure. Level of risk for academic and social difficulties.
provide appropriate intensity of intervention.
![Page 33: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/33.jpg)
Reliability and Validity
• CBM valid for (Stecker, Fuchs, & Fuchs, 2005): identifying both successful and unsuccessful
students in reading, math, and spelling. identifying which students are more likely to have
later academic difficulties if unaddressed. how well students will perform on high stakes tests.
• CBM is valid GOM for academic difficulties. Level of intensity of intervention can more precisely
be determined rather than going from Tier 1-2-3.
![Page 34: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/34.jpg)
Office Discipline Referrals and Reliability
• Same event may result in a referral on one occasion but not another. Lack of consistency within and across teachers is an
issue.• Same event may not always be coded in the same
way within and across teachers. Makes it difficult to identify what behaviors are being
identified through referrals.• Usually high agreement that a referral has
occurred.• Use of SWIS increases consistency:
of reporting what is reported how it is coded
![Page 35: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/35.jpg)
Office Discipline Referrals and Validity
• Office Discipline Referrals may measure student behavior but may also measure teacher behavior.
• ODR valid for making statements of level of risk for other socially relevant behaviors (delinquency, academic difficulties, etc.) (Irwin et al., 2004 for review). Nelson (2002) ODR may not be valid for identifying
students at risk. Too many false negatives
Especially, among students with internalizing problems
![Page 36: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/36.jpg)
Assumption 1: Reliable and Valid Data are Available
• More likely to be the case for academic performance than social behavior. Especially, in elementary school.
Measures such as words read correctly per minute, digits correct, and letters correct are well established measures.
Students doing well on these measures are more likely to meet literacy goals.
![Page 37: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/37.jpg)
Reliable and Valid Data Are Available
• To date, no valid general outcome measures for “social literacy.”
![Page 38: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/38.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Most student level data displayed in graphs.• Several studies suggest that persons with
advanced training in behavior analysis are not reliable at interpreting graphic data displays. DeProspero & Cohen (1979)
Reviewers of behavioral journals IOA = 61% (mean). How will less skilled decision makers perform under the
same conditions?
![Page 39: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/39.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Hagopian, et al., (1997)Developed decision rules for interpreting visual
display of functional analysis data and trained predoctoral interns to apply the criteria. IOA increased from a baseline of .46 to .81
![Page 40: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/40.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Fischer, et al., (2003) Evaluated effects of training and visual aids on
accuracy of visual inspection. Interpretation accuracy increased from 55% to 94% with full
training. Interpretation accuracy increased from 71% to 95% with
brief (15 minute training) and visual aids.
![Page 41: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/41.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Stecker, Fuchs, & Fuchs (2008) Teachers increased accuracy of interpreting CBM
data when: trained to interpret graphs graphical judgment aids incorporated into data-display decision rules provided computer assisted graphing and data analysis
![Page 42: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/42.jpg)
Grade Level Standard
AimAim Line
Trend Line
Visual Aids Can Enhance Data Interpretation
![Page 43: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/43.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Codding, Skowron, & Pace (2005) Teachers implemented CBM decision-making
protocols only after training: Modeling Practice Performance feedback
![Page 44: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/44.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Bartels & Mortenson (2005) Compliance with problem-solving protocol improved
only with feedback and checklist.
![Page 45: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/45.jpg)
Assumption 2: Decision Makers Have Skills and Supports to Base Decisions on Data
• Assumption valid under certain conditions: decision-makers trained to interpret graphic display. graphical judgment aids are provided for data
interpretation. explicit training and feedback provided for following
decision-making protocols.
Take home message: System must be in place to support decision making.
![Page 46: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/46.jpg)
Assumption 3: Decision Makers Know What to Do When a Problem is Identified
• Knowing there is a problem and knowing what to do about it are very different things.
• It is not clear that we are preparing educators to identify and implement evidence-based interventions.
![Page 47: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/47.jpg)
![Page 48: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/48.jpg)
Are We Training Educators to Know What to Do?
Survey of School Psychology Directors of Training
29%
Evidence-based interventions
Knowledge
(Shernoff, Kratochwill, & Stoiber, 2003)
Training41% programs
directors
![Page 49: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/49.jpg)
Are We Training Educators to Know What to Do?
550 named interventions for children and adolescents
BehavioralCognitive-behavioral
Empirically evaluated
Evidence-based interventions are less likely to be used than interventions for which there is no evidence or there is evidence about lack of impact.
Kazdin (2000)
![Page 50: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/50.jpg)
Sources of Information for Practitioners
• Journals Practitioners spend about 1 hour/week reading
research. Often practitioners are not trained to evaluate
research.
• Internet Quality of information unreliable.
• Supervisors/Colleagues Not necessarily informed about research base.
![Page 51: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/51.jpg)
Sources of Information for Practitioners
• Evidence-based Clearinghouses What Works Clearinghouse
Addresses primarily academic issues Rigorous standards
Relatively few research-based interventions meet standards.
Currently does not evaluate single subject research. Practice guides are good resource.
National Autism Center Guidance only for students on Autism Spectrum.
Best Evidence Encyclopedia
![Page 52: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/52.jpg)
Sources of Information for Practitioners
• Best Practice Guidelines Intended to inform consumers and service provides
about optimal care guidelines as compared to generally accepted practice for conditions or disorders (Romanczyk 2008) Guidelines set a higher standard for care. Many groups purport to having BP guidelines but they fail to
follow accepted methodology for evaluating treatments. Systematic review of literature; experts base
recommendations on literature and expand to more general guidelines for practice.
The “gold standard” for Best Practice Guidelines
![Page 53: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/53.jpg)
Sources of Information for Practitioners
• Research-based Principles Function of Principles in Practice
Create Individualized Interventions Adapt Interventions
![Page 54: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/54.jpg)
Basic Principles of Behavior
• Reinforcement• Punishment• Extinction
![Page 55: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/55.jpg)
Principles and Tactics of Intervention
• Shaping• Chaining• Prompting• Differential reinforcement• Functional assessment• Monitoring through direct measurement• Providing immediate feedback• Increasing number of opportunities to respond
![Page 56: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/56.jpg)
Assumption 3: Decision Makers Know What to Do When a Problem is Identified.
• Validity of this assumption is tenuous.• Training programs are often failing.• Evidence-based clearinghouses are becoming
better sources of information. Considerable work to be done.
![Page 57: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/57.jpg)
Assumption 4: Adequate skills, supports, and resources to implement with integrity.
• The effectiveness of interventions is a function of the integrity with which they are implemented.
• The quality of decisions regarding an intervention is directly linked with the integrity of implementation.
![Page 58: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/58.jpg)
Assumption 4: Adequate skills, supports, and resources to implement with integrity
• Estimated that evidence-based drug prevention programs are implemented with integrity only 19% of the time. (Hallfors & Godette, 2002)
This may be a generous estimate.
• No reason to assume that PBS is immune to poor implementation.
![Page 59: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/59.jpg)
Positive NegativeH
igh
Low
Continue Intervention Change Intervention
Unknown reason Unknown reason• Intervention problem?
• Implementation problem?
• Other life changes?
• Unknown intervention?
• Intervention is effective?
OutcomeIn
tegr
ityPositive Negative
Hig
hLo
w
![Page 60: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/60.jpg)
Facilitating Treatment Integrity
• Contextual fit• On-going technical assistance• Resources available
materials staff time
• Systematic monitoring of integrity
![Page 61: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/61.jpg)
OUTCOMES
(% of Participants who demonstrate knowledge, demonstrate new skills in a training setting,
and use new skills in the classroom)
TRAININGCOMPONENTS
Knowledge SkillDemonstration
Use in the Classroom
Theory and Discussion
10%
5% 0%
..+Demonstration in Training 30% 20% 0%
…+ Practice & Feedback in Training
60% 60% 5%
…+ Coaching in Classroom 95% 95% 95%
Joyce and Showers, 2002
Effects of Training
![Page 62: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/62.jpg)
Assumption 4: Adequate skills, supports, and resources to implement with integrity
• Assumption true only when a system in place to actively promote and facilitate treatment integrity. Poor treatment integrity is common across both
academic and social interventions. It is common for systems to invest very few
resources in assuring treatment integrity.
![Page 63: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/63.jpg)
Determining the Impact
Additive
MultiplicativeData reliable and valid +Decision makers have necessary skills
+
Decision makers know what to do
+
Interventions implemented with integrity
-
Score 3/4
Data reliable and valid +Decision makers have necessary skills
+
Decision makers know what to do
+
Interventions implemented with integrity
0
Score 0
![Page 64: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/64.jpg)
Summary
• Data-based decision making offers great promise for assuring educational benefit for all students.
• To realize promise requires a culture that supports data-based decision making. Primary function of all elements of a system are
defined as enhancing student outcomes.
• Comprehensive system is required.
![Page 65: Data-based Decision Making Cultures: Four Assumptions Ronnie Detrich Wing Institute Association for Positive Behavior Support St. Louis March 26, 2010](https://reader030.vdocument.in/reader030/viewer/2022032714/56649cd65503460f9499d696/html5/thumbnails/65.jpg)
Thank youCopies may be downloaded at:
www.winginstitute.org