leading effective data meetings through collaborative teams kathryn catherman nancy lindahl...

Leading Effective Data Meetings

Through Collaborative TeamsKathryn CathermanNancy LindahlKalamazoo RESA

Reflections• Goal: Improve Reading Outcomes for all students• Objective: SMARTThe reading achievement gap between minority and non-minority students

will decrease by 20% as measured from 2010-2011 on the MEAP (reword.)• Include more formative assessment just not DIBELS.• Make sure participants understand that they are writing an action plan for

the day’s work i.e., teaching meeting mechanics to staff, monitoring that, sharing SIP with staff, teaching what data should be used at each type of meeting etc.

Setting Group Norms

• To make this day the best possible, we would appreciate your assistance and participation– Please allow others to listen

• Please turn off cell phones and pagers• Please limit sidebar conversations• Please do not use email

– Share “air time” – Active participation – Take care of your own needs– Attend to the “Come back together” signal

Credits

• Learning by Doing Richard DeFour

• Center for Performance Assessment: Doug Reeves

• Success Line Inc: Deborah Wahlstrom

• Portage Public Schools: Haverhill Elementary

Today’s Agenda

• Why collaborate• Keeping your focus with School Improvement• Meeting Mechanics• Types of Effective Data Meetings• Action Planning

Frequency of Collaboration and Closing the Gap

0

10

20

30

40

50

60

A few times ayear

A few times amonth

A few times aweek

Gap closers

Non-Gap closers

Center for Performance Assessment-2006

Two examples of effective data meetings

APOLLO 13

Discussion PointsClip #1

• What did you notice about the culture that allowed for all viewpoints to be adequately heard?

• What did you notice about the leadership?

Discussion PointsClip #2

• What did this scenario say about focus and the use of resources to accomplish a tough task?

Purpose of the PLC

“Members of a professional learning

community recognize they cannot accomplish their fundamental purpose of high levels of learning for all students unless they work together collaboratively. The collaborative team is the fundamental building block of a PLC.”

DuFour, DuFour, Eaker, & Many, 2006

Climate of a Collaborative Team

• Celebrative• Action-oriented• Risk-taking• Accountable• Supportive (data is not used to punish)• Focused on mission

Keeping your Focus

Use your School Improvement PlanTo Create Focus

13

One Common Voice – One Plan

Michigan Continuous School ImprovementStages and Steps

Implement Plan Monitor PlanEvaluate Plan

Develop Action Plan

Getting ReadyCollect School DataBuild School Profile

StudentAchievement

Analyze DataSet Goals

Set Measurable ObjectivesResearch Best Practice

Do Study

Plan

Gather

Expectations of a Collaborative Team

• Norms – Agreements and commitments• Roles and Responsibilities – How to organize• Agenda – Always written• Data – Use for decision-making • Objectives – Be SMART• Action Plan – written, reviewed, and revisited

Meeting Norm

Norms are not intended to serve as rules but rather as commitments –

public agreements shared among the members.

s

Establishing Norms

Tips for Establishing Norms

• Create own• State as commitments• Review frequently• Evaluate semi-annually• Focus on a few• Address violations• Establish a Parking Lot

An Example

• Begin and End on Time (9:00 – 9:45)• Limit Sidebar Conversations• Stay focused on the critical questions• Leave other work outside of the meeting

Roles and Responsibilities

Examples of Roles/Responsibilities

• Moderator/Facilitator – facilitates meeting content and–flow according to agenda

• Norms monitor ensures adherence to the agreed upon meeting commitments

• Time keeper – keeps meeting moving toward agenda

• Data keeper – organized individual who makes sure the appropriate data is available

• Scribe/Recorder – takes notes during the meeting especially regarding action plan

Your Turn

• Within your team, assign the following roles:• Moderator/Facilitator – facilitates meeting content and–flow

according to agenda• Norms monitor ensures adherence to the agreed upon meeting

commitments• Time keeper – keeps meeting moving toward agenda• Data keeper – organized individual who makes sure the

appropriate data is available• Scribe/Recorder – takes notes during the meeting especially

regarding action plan

• Finally, establish your norms and write them on the provided tent

The Agenda

Distribute written agenda before the meeting.

Include norms as part of the agenda.

Include roles & responsibilities.

HaverhillLeadership Team Meeting

Tuesday, December 14th, 2010 4:00Norms:

• Begin and End on Time• Limit Sidebar Conversations• Focus on Critical Questions• Maintain Confidentiality

Outcomes for today’s meeting:• Look at literacy needs for our building• Schedule/organize how we will be re-teaching our behavior expectations• Discuss building behavior needs/interventions• We will discuss the Buzz Bash

Roles and Responsibilities:• Facilitator-Jen/Tajia• Norms Monitor-Cindy• Time Keeper-Erin• Scribe-Dawn• Data Keeper-Jena and Susan K.• Attendance Keeper: Susan L.

Agenda:• Reminder of Group Norms - 1 minute• Go over action plan from November meeting-5 minutes• Review Literacy Data – 20 minutes• Histograms• Professional Development needs• Review SWIS data -10 minutes• Discuss building behavior needs/interventions – 10 minutes• Discuss Buzz Bash – 14 minutes

Essential Elements of Action Plans

• Identify/review the SMART objective (how much, how well, & by when)

• What strategy will be used to achieve the objective?• What steps do we need to take to implement the

strategy? How will we carry out the steps, i.e., – Who will be responsible for the action items?– What is the timeline for completing the action item?– What resources are necessary?– What will be our “results indicators”?

WHAT do we need to do? HOW will we do it? WHO is responsible?

By WHEN?

What RESOURCES do we need?

Provide parents with information on reading progress

Print off progress monitoring graphs and gather classroom unit tests.

JennyTeachers

Week of 4/20

Paper

Paraprofessional expectations will be created

Several different sources will be compiled to create the “checklist”

Jenny with teacher feedback

Before the end of the year

Various checklists

Re-evaluate the use of interventions

Using a 4 Square Sort student need will be determined. Need will be matched with interventions program. Additional pts addressed; pacing, progress monitoring. Specifically RMI and II and REWARDS.

MichaelJennyTeacher

After DIBELS testing in the spring in prep-aration for fall

DIBELS sort worksheet

Improve application of silent “e” rule

Instruction through small group using word lists and application

JaneTeachers

4/20 Word listsDecodable Text

ACTION PLAN

Grade: 3 Date: 4/10/09Faculty Present: Jean, Michael, Jenny, Andre, Sally, Kelly, Karen

WHAT do we need to do? HOW will we do it? WHO is responsible?

By WHEN?

What RESOURCES do we need?

Gather trade book sets for upper end readers.

Look at East. June 3/9

Start RN with strategic readers in Carl’s room. Students would also remain in RM.

Gather materials and complete placement test. Figure a possible change to Paul’s schedule. Also look at Carla’s schedule.

June 2/23Completed

Materials

Discuss the possibility of holding 2 half day GLM meeting instead of 1 full day.

Kathryn and Mike will look at the school schedule

KateMichael

3/6Completed

Houghton Mifflin training-more in-depth, differentiated

Choice PD time. Michael will email Sophie about this.

Michael Schedulefor January

PD money to pay for H/M rep (if necessary)

Carlto observe in Stephanie’s room for ½ day.

Michael will get a sub teachers . Carl and Stephanie will let Michael know date

MichaelStephanieCarl

3/6 Money for sub

ACTION PLANGrade : 2nd Date : 2/20/09 Faculty Present: Sharron, Stephanie, June, Michael, Kate, Carl

27

Activity to implement

the strategy

Staff responsible to

implement

Timeline

Begin End

Resources neededAmount Source

Monitoring Activities

Evidence of

Success

One Common Voice – One Plan Step 9: Implement

Plan in Sufficient Detail

Using Data to Make Decisions

Four Critical Questions

• What is it we want our students to learn?• How will we know if each student has learned

it?• What will we do when some students do not

learn it?• How can we extend and enrich the learning

for students who have demonstrated proficiency?



• What is it we want our students to learn?


STOP

32

• Identify a local issue.• Conduct primary research.• Gather data.• Analyze data (convert

numerical data to statistical information including means, trends, correlations).

• Create graphs, charts, and maps.

• Create software that includes a community digital database.

• Present a position via a PowerPoint, Website, or multi-media presentation.

• Makes a list of possible issues and chooses one to study.

• Identifies different points of view for the issue.

• States reasons for different points of view.

• Makes a list of research questions. Writes an appropriate number of questions to thoroughly conduct the research.

• Uses a variety of resources to gather information.

• Chooses resources that will help answer the question.

• Evaluates sources for the quality of their information.

• Takes good research notes, identifying directly copied materials and recording the source.

• Gathers enough information to answer the questions or solve the problems.

UNPACKED

UNPACKED21ST Century Skill Example

These become objectives.

These become enabling objectives.

Deborah Wahlstrom ♦ Unpacking Standards ♦

Unpacking the Standard Essential Questions

or Learning Targets

INSTRUCTIONAL STRATEGIES,

ACTIVITIES, andINTERVENTIONS

CLASSROOMASSESSMENT STRATEGIES

State Standard(s) Words to Know

Deborah Wahlstrom ♦ Unpacking Standards ♦

Analyzing Data

Unpacking the Standard Essential Questions

(Critical Question)or

Learning Targets

INSTRUCTIONAL STRATEGIES,

ACTIVITIES, andINTERVENTIONS

CLASSROOMASSESSMENT STRATEGIES

State Standard(s) Words to Know

D.AN.07.03

Calculate and interpret relative frequencies and cumulative frequencies for given data sets.

CONTENT WORDSaccuratebar graphcumulative frequencydata setDistributionerrorfrequencyfrequency tablehistogramline plotpie chartrelative frequencystem-and-leaf plottally

ACADEMIC WORDSas manycalculatedatadisplaysgreater thanhow manyinterpretshows

D.AN.07.03.00Organize and explain frequencies.

Performance Assessment: Students collect, organize, and interpret data.

Multiple Choice: 12 questions (3 from each enabling objective)

D.AN.07.03.01Organize data with tools such as charts, tallies, bar graphs, and line plots.

• Comparison Matrix: Ways to Represent Data

• Word Sort – Ways to Represent Data• Activity: Going Nuts• Activity: Ways We’re Different

Formative: Students construct different graphical tools for data.

charts

tallies

bar graphs

line plots

D.AN.07.03.02Interpret the distribution of numbers in a chart or graph.

• Analysis Questions for each graph that is used.

• Activity: Ways We’re Different

Formative: Students answer questions related to the graphic. Have students write a summary of the data.

D.AN.07.03.03Check the accuracy of data in a frequency distribution.

• Comparing Data Sets (e.g., tallies to graphs)

Formative: Students match tallies with their graphs. (Have students also explain why something matches or doesn’t.)

D.AN.07.03.04Calculate cumulative frequency.

• Activity: Ways We’re Different Formative: Students compute the cumulative frequency from a variety of graphs and data sets.

D.AN.07.03.05Calculate relative frequency.

• Activity: Ways We’re Different Formative: Students compute the relative frequency from a variety of graphs and data sets.

General Success Strategies

• Success Sequence

• Analysis Questions

• Vocabulary – Frayer’s Model

• Here’s How I Do It (modeling)

• Explain Your Thinking (Tell me how

you do it)

• Manipulatives

Your Turn

• Using the Unpacking Standards Form by Deborah Wahlstrom of Success Inc:

Know and use various text features (e.g., captions, bold print, subheadings, glossaries, indexes, electronic menus, icons) to locate key facts or information in a text efficiently. (RI.2.5)

Compare and contrast two or more characters, settings or events in a story or drama, drawing on specific details in the text (e.g. how character interact.)(RL.5.3)

• With your team, unpack the standard

Types of Assessment Use

Formative (for learning) • Multiple Choice• Short Answer• Exit Slips• Quick writes• Benchmark• Progress Monitoring

Summative (of learning) • MEAP• IOWA• Chapter tests• Benchmark

Summative Assessment

• Purpose: to determine level of proficiency in relation to norm or criterion

• When: Typically administered annually or at end of an instructional unit. Can be administered pre/post to assess overall growth.

• Who: All students

• Relation to instruction: Provides index of overall efficacy not intended to provide timely instructional information

Screening Assessment• Purpose: To determine children who are likely to

require additional instructional support (predictive validity)

• When: Early in the academic year or when new students enter school

• Who: All students

• Relation to Instruction: Most valuable when used to identify children who may need further assessment or additional instructional support.

Progress Monitoring Assessment

• Purpose: Frequent, timely measures to determine whether current instruction is meeting student’s needs to close the gap

• When: one to four times per month• Who: Students who are receiving targeted or

specific instruction to close a learning gap• Relation to Instruction: Indicates student’s rate

of progress and adequacy of instruction


• What is it we want our students to learn?• How will we know if each student has learned

it?• What will we do when some students

do not learn it?• How can we extend and enrich the learning

for students who have demonstrated proficiency?


42:

Effective Data Meetings in an RtI Framework

Continuum of Supports

Benchmark Meetings •Instruction:• Students in Whole Class

and Differentiated Instruction

•Data• Core Program

Assessments• Formative assessments • Screening measures• Classroom Observations

Progress Monitoring Meetings• Instruction• Students in Small Group

Instruction and Supplemental Intervention Programs

•Data• Supplemental program

assessments• CBM Progress monitoring

probes• Frequent targeted formative

assessments

Student Study Team Meetings•Instruction• Student s in 3:1 or smaller Group

Instruction and Intensive Intervention Programs

•Data• Diagnostic assessments• Progress monitoring of specific skills

Your Turn

• Using the previous slide as an example, create your school’s Pyramid for all the different data meetings that you currently use at each tier

Data Interpretation at the School Level

Box PlotsWhat Decisions?

Have we increased the percent of students at benchmark since the previous assessment

period? What is the range of skill level across the grade and over time?

Who?School Improvement Team and Grade level

teachers.How often?

Three times per year

Box Plots(with whiskers)

Box Plot(with whiskers)

Median Score 50th percentile (The score of the middle student.)

80th percentile (80% of thestudents scored above thisscore.)

20th percentile (20% of the students scored below this score)

95th %ile

5th %ile

Example

126102

63

31

15

5% of the studentsscored above 126

20% of the studentsscored above 102

50% of the students scored above 63

50% of the studentsscored below 63

20% of the studentsscored below 31

5% of the students scored below 15

KEY

KEY

Minimum Benchmarkgoal

Target Zone

If you remember nothing else about Box Plots

• The entire box and whiskers should be at the target zone or higher. The DIBELS goal is a minimum standard!

• The box plot shows you at a glance the range of student performance and their progress over time.

An Example: A Tale of Two Schools

Comparing the Schools

School A

Northwestern Elementary• 31% Free and Reduced

Lunch• 32% Ethnic Minority• New Principal started three

years ago.

School B

Sunnyvale Elementary• 24% Free and Reduced

Lunch• 3% Ethnic Minority

First Grade

Northwestern Elementary

Sunnyvale Elementary

Second Grade

Northwestern elementary

Sunnyvale elementary

Third Grade


Sunnyvale elementary

Fourth Grade


Sunnyvale Elementary

A Tale of Two Schools

• What makes the difference? Discuss how the following can help or hinder school improvement efforts:

– Instructional Time– Teacher Training– Belief– Intensity of Focus

• Be prepared to share out

Establishing Purpose for

BenchmarkProgress Monitoring

Meetings

Benchmark (Tier I) Meetings• Purpose: Evaluate screening data and core

(Tier I) instruction

• Based on this purpose:– Who should be involved?– How often should you meet?– How long should you meet?– Who should facilitate?

Guiding Questions

• What is your screening data telling you?• Do the results confirm what you know about the

students?• What does the data tell about Tier I instruction.• What classroom instructional groups should be

formed?• What are the instructional goals for each group?• What materials should be used to meet these groups?• What frequency of progress monitoring needs to

occur for each?

Progress Monitoring (Tier II) Meetings

• Purpose: Evaluate progress monitoring data and intervention instruction (Tier II)

• Based on this purpose, – Who should be involved?– How often should you meet?– How long should you meet?– Who should facilitate?

Guiding Questions

• Is the core program (academic and/or behavior) maintaining or accelerating skills for your students performing at or above expectations?

• Does supplemental instruction exist for students who are not on track?– Is instruction targeted, specific to student need, and

intensive?

• Is the supplemental instruction/intervention bringing students up to expectations?

Guiding Questions

• 30 – 60 minutes• Review of formative assessment data• Effectiveness of targeted

instruction/intervention for groups of students who have been identified as needing more instructional support• Avoid elaborate individual student

discussions

Progress Monitoring Out of Grade Level

Procedures Using DIBELS ORF Goal Setting and Progress Monitoring for Students Requiring

Intensive Support

• Out of grade goals and progress monitoring are often needed for students requiring intensive level of support.

• To find the appropriate goal and progress monitoring level, begin with the level of the curriculum in which the student’s grade level peers are instructed. Administer a minimum of 3 probes in the student’s grade level material. Calculate and graph the median score for that time of year (fall, winter, or spring).

• If the student scores at or above benchmark, stop testing. If not, drop down one level and give 3 probes. Calculate and graph the median.

Procedures Using DIBELS ORF Goal Setting and Progress Monitoring for Students Requiring

Intensive Support

• If the student has not met benchmark, repeat the procedure, dropping down level by level until the student’s median score falls at or above the benchmark. We test downward to ensure that we have the highest level at which the student meets benchmark.

• Set the student’s goal for the end of the year, one year above the level at which the student met benchmark. Both the goal and progress monitoring will be at this level.

Survey Level Assessment - Progress Monitoring with DIBELS ORF Data

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

1st 1st 2nd 2nd 2nd 3rd 3rd 3rd 4th 4th 4th 5th 5th 5th 6th 6th 6th

Grade Level of Text

Corr

ect

Wo

rds P

er

Min

ute

40+S1

90+S2

110+S3

118+S4

124+S5

125+S6

20+W1

44+F2

68+W2

77+F3

92+W3

93+F4

105+W4 104+

F5

115+W5 109+

F6

120+W6

The boxes represent minimal CWPM scores needed to meet 1st thru 6th grade fall, winter, and spring ORF benchmarks.

Additional Guiding Questions

• Are the learning targets for intervention clearly specified?

• Based on the data, are the students who are not meeting the learning standards identified?

• Are these identified students receiving appropriate support and interventions?

• Are these students being progress monitored on the skills and learning necessary? How often?

• Is the progress monitoring data being used to make instructional decisions (and adjustments if necessary) on an ongoing basis?

Quick Evaluation of Data

Sort Progress Monitoring Booklets into three categories:

1. Those who are on track or exceeding expectations.

Quickly share, then celebrate!

2. Those who are clearly not making adequate progress

Consider groups and review alterable variables

3. Those who are borderlineMonitoring and recheck plan.

Final Thoughts• What do you do when no one knows what to do?– Knowledge issue

• Seek info, set timelines, try something• What do you do when one or a few people refuse to

do?– Belief or attitude issue

• Go back to whole-staff consensus, establish behavioral expectations

• What do you do when many people refuse to do?– Community issue

• Stop and process as whole staff

leading effective data meetings through collaborative teams kathryn catherman nancy lindahl...

Documents