Action Research Action Research Deployment TeamsDeployment Teams
November 29/30, 2006November 29/30, 2006
Please Sit By Please Sit By SIP Goal AreaSIP Goal Area
Table Tents Indicate Goal AreaTable Tents Indicate Goal Area
Welcome!Welcome! Introductions around the tableIntroductions around the table Sharing PDSA progressSharing PDSA progress How are PDSA efforts going at your How are PDSA efforts going at your
school?school? Place a sticky dot in section of Place a sticky dot in section of
PDSA that represents where you PDSA that represents where you are at in the process (review the are at in the process (review the Action Research Overview PDSA Action Research Overview PDSA diagram from session 1/binder)diagram from session 1/binder)
Where did we Where did we leave off again?leave off again?
A quick review of last A quick review of last sessionsession
Pre Planning: Identification of Need1. Develop/Review Student Learning Expectations2. Examine alignment of learning expectations with assessments3. Review assessment data4. Identify areas of need based on assessment
Plan:1. Describe the current process for addressing the identified area of need
(flow chart) 2. Review data to determine baseline performance in the specific area
identified (Run Chart/Pareto Diagram)3. Identify potential root causes contributing to the identified area of need (Cause & Effect Diagram, 5 Why’s, Relations Diagram)4. Study research-based best practice/improvement theory addressing areas
of need
DO1. Plan for implementation of improvement theory (Force Field Analysis, Action Plan)2. Implement research-based best practices improvement theory based on root causes according to the Action Plan3. Monitor the implementation of research-based best practice/improvement theory to insure integrity and fidelity4. Assess student learning
Study:1. Examine student assessment results (compare to baseline)2. Assess the impact of research-based best practice/improvement theory on
student achievement
Act:1. Standardize the implementation of research-based best practice (improvement theory) that improved student learning (revise the flow chart to reflect changes made to the system)2. If improvement theory was unsuccessful continue the PDSA cycle (try another improvement theory based on the next identified root causes)
Action ResearchOverview
for Professional
Learning Communitie
s
Pre Planning: Identification of Need1. Develop/Review Student Learning Expectations2. Examine alignment of learning expectations with assessments3. Review assessment data4. Identify areas of need based on assessment
Plan:1. Describe the current process for addressing the identified area of need
(flow chart) 2. Review data to determine baseline performance in the specific area
identified (Run Chart/Pareto Diagram)3. Identify potential root causes contributing to the identified area of need (Cause & Effect Diagram, 5 Why’s, Relations Diagram)4. Study research-based best practice/improvement theory addressing areas
of need
DO1. Plan for implementation of improvement theory (Force Field Analysis, Action Plan)2. Implement research-based best practices improvement theory based on root causes according to the Action Plan3. Monitor the implementation of research-based best practice/improvement theory to insure integrity and fidelity4. Assess student learning
Study:1. Examine student assessment results (compare to baseline)2. Assess the impact of research-based best practice/improvement theory on
student achievement
Act:1. Standardize the implementation of research-based best practice (improvement theory) that improved student learning (revise the flow chart to reflect changes made to the system)2. If improvement theory was unsuccessful continue the PDSA cycle (try another improvement theory based on the next identified root causes)
Action ResearchOverview
for Professional
Learning Communitie
s
Pre Planning: Identification of Need1. Develop/Review Student Learning Expectations2. Examine alignment of learning expectations with assessments3. Review assessment data4. Identify areas of need based on assessment
Plan:1. Describe the current process for addressing the identified area of need
(flow chart) 2. Review data to determine baseline performance in the specific area
identified (Run Chart/Pareto Diagram)3. Identify potential root causes contributing to the identified area of need (Cause & Effect Diagram, 5 Why’s, Relations Diagram)4. Study research-based best practice/improvement theory addressing areas
of need
DO1. Plan for implementation of improvement theory (Force Field Analysis, Action Plan)2. Implement research-based best practices improvement theory based on root causes according to the Action Plan3. Monitor the implementation of research-based best practice/improvement theory to insure integrity and fidelity4. Assess student learning
Study:1. Examine student assessment results (compare to baseline)2. Assess the impact of research-based best practice/improvement theory on
student achievement
Act:1. Standardize the implementation of research-based best practice (improvement theory) that improved student learning (revise the flow chart to reflect changes made to the system)2. If improvement theory was unsuccessful continue the PDSA cycle (try another improvement theory based on the next identified root causes)
Action ResearchOverview
for Professional
Learning Communitie
s
In-Process Measure Example
Estimation Check Sheet Weekly
Class Errors
Hour 1 - Mr. Albert
Strategy 1 2 3 4 5 6
Total1-6
Week12-Sep
19-Sep
29-Sep 3-Oct
10-Oct 17-Oct
Addition 2 2 1 1 0 0 6
Subtraction 2 2 3 1 2 1 11
Multiplication 7 8 7 8 7 7 44
Division 10 11 10 11 12 10 64
Application 11 11 10 11 12 12 67
28 students
Total Errors 32 34 31 32 33 30 192
Number of Students 28 28 28 28 28 28
Estimation Check SheetHour 1 - Class Errors
15
20
25
30
35
40
9/12 9/19 9/26 10/3 10/10 10/17
To
tal N
um
ber
of E
rro
rs
Estimation Check Sheet SkillsWeeks 1- 6
67 64 44 11 6 192
35%
68%
91%
97%100%
0
20
40
60
80
100
120
140
160
180
Application Division Multiplication Subtraction Addition Total N
Item
Nu
mb
e o
f E
rro
rs
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Per
cen
tag
e
This bar represents the total (N
) of your pareto.
More on “In-Process” Measures
• Assessment of learning Vs. assessment for learning–Rick Stiggins
From High Stakes Assessment to In-Process Measures
• Mistaken Belief: “It’s the adults who use assessment results to make the most important instructional decisions…”
• Mistaken Belief: “The most important decisions are made annually based on annual high-stakes tests”
What Do Your In-Process Measures Look Like?
• Talk about what in-process measures your team is using
• How are things going?–What is working well?
–What is your team struggling with?
What are we doing today?
Continuing the PDSA Cycle
Pre Planning: Identification of Need1. Develop/Review Student Learning Expectations2. Examine alignment of learning expectations with assessments3. Review assessment data4. Identify areas of need based on assessment
Plan:1. Describe the current process for addressing the identified area of need
(flow chart) 2. Review data to determine baseline performance in the specific area
identified (Run Chart/Pareto Diagram)3. Identify potential root causes contributing to the identified area of need (Cause & Effect Diagram, 5 Why’s, Relations Diagram)4. Study research-based best practice/improvement theory addressing areas
of need
DO1. Plan for implementation of improvement theory (Force Field Analysis, Action Plan)2. Implement research-based best practices improvement theory based on root causes according to the Action Plan3. Monitor the implementation of research-based best practice/improvement theory to insure integrity and fidelity4. Assess student learning
Study:1. Examine student assessment results (compare to baseline)2. Assess the impact of research-based best practice/improvement theory on
student achievement
Act:1. Standardize the implementation of research-based best practice (improvement theory) that improved student learning (revise the flow chart to reflect changes made to the system)2. If improvement theory was unsuccessful continue the PDSA cycle (try another improvement theory based on the next identified root causes)
Action ResearchOverview
for Professional
Learning Communitie
s
Identifying Root Cause!
• 3 Quality Tools–Cause and Effect Diagram
–Relations Diagram
–5 Whys
Cause & Effect DiagramHow is it made?
1. Identify the problem.
2. Record the problem statement.
3. Draw and label the main bones.
4. Brainstorm for problem causes.
5. Identify the most likely cause candidates.
People InstructionalMaterials
State/NationalRequirements
Methodology
Low Integration
of State Test
Objectives Across
Curriculum Areas
Internal resistance to change
Lack
of
know
ledg
eLac
k of
in
form
atio
n
Teachers have no knowledge on how to integrate
Time
Worksheets & games isolate skills
State tests divided by content area
Identified Problem (effect)
Backbone
Prim
ary
Cau
sal
Cat
egor
ies
Cause
Sub Cause
What’s Next?
• After completing the Cause and Effect Diagram, the Relations Diagram can help identify root causes
Relations Diagram
Relations DiagramHow is it made?
1. Clearly define the issue or problem.
2. Construct the diagram layout.
3. Analyze the relationships.
4. Count the arrows.
5. Identify the root causes and effects.
6. Study the final diagram.
Problems in Physical Education)
The Teacher Was Mean
Some Students Were Not Listening To The Teacher
The Teacher Yelled At Us
Some Students Were Not Behaving
#In / #Out
2 / 1
3 / 0
0 / 3
1/ 2•Root causes are those factors or
aspects of a problem which primarily influence other factors (arrows out)
Root Cause Identified
• Once root cause is identified… it is time to select a research-based improvement theory
Pre Planning: Identification of Need1. Develop/Review Student Learning Expectations2. Examine alignment of learning expectations with assessments3. Review assessment data4. Identify areas of need based on assessment
Plan:1. Describe the current process for addressing the identified area of need
(flow chart) 2. Review data to determine baseline performance in the specific area
identified (Run Chart/Pareto Diagram)3. Identify potential root causes contributing to the identified area of need (Cause & Effect Diagram, 5 Why’s, Relations Diagram)4. Study research-based best practice/improvement theory addressing areas
of need
DO1. Plan for implementation of improvement theory (Force Field Analysis, Action Plan)2. Implement research-based best practices improvement theory based on root causes according to the Action Plan3. Monitor the implementation of research-based best practice/improvement theory to insure integrity and fidelity4. Assess student learning
Study:1. Examine student assessment results (compare to baseline)2. Assess the impact of research-based best practice/improvement theory on
student achievement
Act:1. Standardize the implementation of research-based best practice (improvement theory) that improved student learning (revise the flow chart to reflect changes made to the system)2. If improvement theory was unsuccessful continue the PDSA cycle (try another improvement theory based on the next identified root causes)
Action ResearchOverview
for Professional
Learning Communitie
s
Introduction To Research-Based
Improvement Theories
NCLB and the Quest for Scientifically Based Research
NCLB encourages -
• Instructional strategies shown to be effective through sound scientific research.
• Systematic, empirical methods
• Rigorous data analysis
• Meeting the highest standards of research in your discipline
Iowa Department of EducationContent Network
Best practice research in:
– Reading
– Math
– Science
Pyramid Continuum
LEVEL 5—Gold Standard
Randomly assigning subjects to treatments yields findings that generate the greatest confidence in treatment effect.
LEVEL 4—Strong Evidence
Research designs that do not randomly assign subjects to treatments. With these designs a logical rather than a statistical argument is used to support the validity of the results.
LEVEL 3—Promising Studies
A combination of studies, consistent across time and subjects, that shows some promise even though the individual studies do not meet the requirements for levels 4 and 5.
LEVEL 2—Marginal
These are case studies, conducted by individual investigators. Such reports are by no means worthless, but it is difficult to assess the validity and generalizability of the treatment.
LEVEL 1—No Empirical Evidence
Reports are anecdotal, testimonial, etc.
IOWA Content Network
http://www.iowa.gov/educate/prodev/main.html
ASCD is the Association for Supervision
& Curriculum Development• Mission statement: ASCD, a
community of educators advocating sound policies and sharing best practices to achieve the success of each learner.
• http://www.ascd.org/portal/site/ascd
Educational Leadership Magazine
• “As legislators contemplate reauthorizing and revising the No Child Left Behind Act (NCLB), the authors in this issue of Educational Leadership offer the following recommendations: use multiple assessments to inform instruction; design valid accountability measures for schools; support teachers with resources and training; make sure regulations don't have the unintended consequence of limiting the curriculum; and, finally, consider how NCLB is affecting the students - it is meant to help.”
ASCD• “ASCD SmartBrief is a FREE daily news
briefing delivered directly to you through e-mail. Each ASCD SmartBrief takes you straight to the most salient news and trends affecting education today. This FREE service will help you break through the information overload and provide quick, easy-to-read summaries of top news in areas important to you.”
• http://www.smartbrief.com/ascd/
National Science Teachers Associationwww.nsta.org
State Approved Research Based Strategies Web Resources for Reading/Thinking Strategies
Excellent starting point:• http://www.netc.org/focus/strategies/
PreP Prereading Plan• http://literacy.kent.edu/eureka/strategies/prep.pdf• http://www.justreadnow.com/strategies/prep.htm
Advanced Organizer• http://scied.gsu.edu/Hassard/mos/2.10.html• http://www.glnd.k12.va.us/resources/graphicalorganizers/• http://www.ncrel.org/sdrs/areas/issues/students/learning/lr100.htm
Anticipation Guides• http://www.indiana.edu/%7el517/anticipation_guides.htm• http://www.greece.k12.ny.us/instruction/ela/6-12/Reading/
Reading%20Strategies/anticipation%20guide.htm
Mathematics - Great Resources• National Council of Teachers of Mathematics• NCTM Principles and Standards (free)• Curriculum Focal Points (free)• Journals (some articles are free down loads)• Iowa Council of Teachers of Mathematics• State meetings• State contacts• AEA contacts• U.S. Dept. of Math and Science Professional
Development Reports• Math Forum• Ask Dr. Math• Your friendly math facilitator
Learning From Dramatically
Improved Schools!
40%
50%
60%
70%
80%
90%
100%
91–92 92–93 93–94 94–95 95–96 96–97 97–98 98–99 99–00
Reading Test Scores - Brazosport ISD
AfricanAmerican
All Students
Hispanic
White
EconomicallyDisadvantaged
98%96.6%93.9%93.8%92.3%
82%
70%
64%
60%
50%
40%
50%
60%
70%
80%
90%
100%
91–92 92–93 93–94 94–95 95–96 96–97 97–98 98–99 99–00
Writing Test Scores - Brazosport ISD
AfricanAmerican
80%
70%
60%59%57%
98%96.7%95.6%94.1%93.2%
EconomicallyDisadvantaged
Hispanic
WhiteAll
Students
Math - Aldine ISD
93–94 94–95 95–96 96–97 97–98 98–99 99–0030%
40%
50%
60%
70%
80%
90%
100%
AfricanAmerican
White
56.2%
49.4%47.2%42.2%
55.0%
93.5%90.9%89.1%88.1%84. 4%
EconomicallyDisadvantaged
Hispanic
All Students
What The Successes Have in Common
A total systems framework
Aligned processes
Data-driven decisions
Strong leadership
A customer focus
Implementation of PDSA
When school districts apply When school districts apply the principles and the principles and
techniques of continuous techniques of continuous improvement over time, improvement over time,
positive and lasting positive and lasting changes in all parts of the changes in all parts of the system (including student system (including student
learning) are dramatic.learning) are dramatic.
Why Quality and Why Quality and Continuous Continuous
Improvement?Improvement?Answer: It’s Answer: It’s Research Based!Research Based!
Next steps…• Ensure that you’ve completed work on
Define the System (Flow Chart and Assess the Situation (In-Process Measures, Run Chart, Pareto Diagram)
• Once baseline data has been collected (in-process measure), identity Root Cause (Cause & Effect Diagram and Relations Diagram), ready for improvement theory
• Set and communicate direction!
Team Time & Table Evaluation