data tools for nancy love’s sessions · data-driven dialogue adapted from b. wellman and l....

Data Tools for Nancy Love’s Sessions

Research for Better TeachingActon, Massachusetts, USA

Near East South Asia Council of Overseas SchoolsBangkok, Thailand

5-6 April 2014

Research for Better Teaching, Inc. · One Acton Place, Acton, MA 01720 · +1-978-263-9449 · [email protected]

Copyright © 2014 by Research for Better Teaching, Inc.

The material in this handout from or adapted from Nancy Love, Katherine E. Stiles, Susan Mundry, & Kathryn DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry (Thousand Oaks, CA: Corwin Press, 2008) is used with the permission of Corwin Press.

Research for Better Teaching, Inc.One Acton PlaceActon, MA 01720

President: Jon SaphierExecutive Director: Sandra SpoonerProgram Director: Nancy Love

© 2014 Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • +1-978-263-9449 • [email protected]

i

Data Tools for Nancy Love’s Sessions

Table of Contents

After Analyzing Formative Assessment Data, Now What? ................................................................... 1

Building Blocks of Collaborative Inquiry Self-Assessment ................................................................... 2

Choice-Points for Effective Feedback ....................................................................................................... 4

Criteria Analysis Data Display Tool .......................................................................................................... 6

Data-Driven Dialogue ................................................................................................................................ 8

Engage in Data-Driven Dialogue with Item Data: Checklist ................................................................ 9

Engage in Task Deconstruction and Data-Driven Dialogue with Student Work: Checklist ........... 11

Error Analysis Protocol ............................................................................................................................ 13

Error Analysis Template ........................................................................................................................... 14

No-Because Sign ........................................................................................................................................ 15

Norms of Collaboration ............................................................................................................................ 16

Short-Cycle Action Plans for Grade-Level or Content Teams ............................................................ 18

Short-Cycle Action Plans for Grade-Level or Content Teams [Template] ........................................ 19

Stoplight Highlighting .............................................................................................................................. 20

Student Error Analysis .............................................................................................................................. 24

Verifying Causes ........................................................................................................................................ 26

Verifying Causes Graphic ......................................................................................................................... 27

Verifying Causes Template ....................................................................................................................... 28

Data Tools Organizer ................................................................................................................................ 29

Page


ii


1

After Analyzing Formative Assessment Data, Now What?

Findings from Formative Assessment

1. Based on these findings, what is our next step as a team? q Feedback: Will we give feedback to students? If so, how and to whom? q Investigation: Do we need to further investigate causes of students’ errors and misconceptions? If

so, how will we do so? q Reteaching/re-engaging/regrouping: Do we need to do one or all of these? If so, how will we do

so? How will we manage time and tasks to do so? q Moving on: Can we move on? Based on what criteria? q Extension: Are there students ready for an extended learning opportunity? How will we create it?

2. Which of the above actions would we most like to learn more about as a team?

3. Which is a priority for taking action?

4. What will we do at our next team meeting?


2

(cont. next page)

© 2012 Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • 978-263-9449 • [email protected]

17

© Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • (978) 263-9449 • www.RBTeach.com

Building Blocks of Collaborative Inquiry Self-Assessment Assess the status of collaborative inquiry in either your school or your district, as applicable, with the following questions. 1. Leadership and Capacity How many teachers and administrators have the knowledge, skills, and dispositions to use data effectively, collaboratively, and continuously to improve teaching and learning? 1 2 3 4 Few Some Most All 2. Structured Collaboration a. How often do teachers engage in collaborative inquiry with their colleagues during the school day? 1 2 3 4 Once a year Quarterly Monthly Weekly b. To what extent is collaborative team time productive and focused, making use of processes, tools, and protocols? 1 2 3 4 Not productive Somewhat productive Productive Highly productive 3. Frequent and In-Depth Data Use How often are teachers using common formative assessments to analyze students’ strengths and immediate needs? 1 2 3 4 Annually 2-4 times a year Monthly Weekly 4. Instructional Improvement For how many teachers does collaborative inquiry have an immediate and direct impact on improvements in curriculum, instruction, and assessment practices? 1 2 3 4 None Few Some All

2.1 Collaborative Inquiry Self-Assessment

(cont. next page)


3© 2012 Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • 978-263-9449 • [email protected]

18

© Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • (978) 263-9449 • www.RBTeach.com

5. School Culture a. How many teachers and administrators act in alignment with the following value: “We are collectively responsible for the learning and achievement of each and every student and adult in our system—no excuses”? 1 2 3 4 Few Some Most All b. To what extent is constructive dialogue about race, class, and culture a norm in your setting? 1 2 3 4 Not at all Somewhat Mostly All the time c. To what extent are relationships among staff characterized by trust, candor, openness, and collaboration? 1 2 3 4 Not at all Somewhat Mostly All the time

Questions for Reflection:

1. What are your areas of strength?

2. What areas are in need of improvement?

3. What are the most important next steps you can take to strengthen the bridge between data and results?


4

Cho

ice-

Poi

nts

for

Eff

ecti

ve F

eedb

ack

Que

stio

ns t

o C

onsi

der

Opt

ions

to

Con

side

rT

imin

gA

re th

e st

uden

ts s

till m

indf

ul o

f th

e le

arni

ng ta

rget

?Is

ther

e st

ill ti

me

for

them

to a

ct o

n yo

ur f

eedb

ack?

q

Imm

edia

te:

For

basi

c ri

ght/w

rong

que

stio

ns q

As

soon

as

feas

ible

: Fo

r m

ore

com

plex

pro

duct

s lik

e pa

pers

or

proj

ects

q

Cum

ulat

ive:

Aft

er o

bser

ving

a p

atte

rn o

f er

rors

or

mis

conc

eptio

ns

Qua

ntit

y/Q

ualit

yH

ow m

uch

feed

back

can

the

stud

ent

abso

rb a

nd a

ct u

pon

at o

nce?

If th

ere

are

mul

tiple

are

as n

eedi

ng

impr

ovem

ent i

n th

e w

ork

can

they

be

pri

oriti

zed

to m

ake

the

feed

back

m

anag

eabl

e fo

r th

e st

uden

t?Is

the

feed

back

spe

cific

eno

ugh

to

mak

e th

e st

uden

t aw

are

of h

ow h

is/

her

wor

k co

mpa

res

to th

e cr

iteri

a?

Will

the

feed

back

hel

p th

e st

uden

t to

kno

w w

hat t

o do

to im

prov

e th

e w

ork?

You

r de

cisi

ons

abou

t qua

ntity

and

qua

lity

mig

ht b

e di

ffer

ent f

or d

iffe

rent

stu

dent

s an

d di

ffer

ent a

ssig

nmen

ts:

Qua

ntit

y: (

see

“Sca

ffol

ded”

abo

ve)

q

Smal

ler

chun

ks q

All

crite

ria

Qua

lity:

(M

ake

sure

that

you

r fe

edba

ck m

eets

all

of th

e C

hara

cter

istic

s of

Eff

ectiv

e Fe

edba

ck)

q

Cle

ar q

Rel

evan

t q

Non

judg

men

tal

q

Supp

ortiv

e q

Tim

ely

q

Use

ful

q

Scaf

fold

edM

ode

Whi

ch m

ode

of f

eedb

ack

will

bes

t su

ppor

t the

stu

dent

in b

eing

abl

e to

un

ders

tand

and

act

on

it?W

ill th

e st

uden

t be

able

to r

ead

and

unde

rsta

nd w

ritte

n fe

edba

ck?

Will

stu

dent

s ne

ed to

ref

er b

ack

to th

e fe

edba

ck in

ord

er to

mak

e ch

ange

s?Is

ther

e a

need

or

a w

ay to

mod

el

wha

t the

fee

dbac

k is

foc

usin

g on

?

Whi

ch f

orm

of

feed

back

mig

ht b

est

mat

ch th

e st

uden

t’s le

arni

ng s

tyle

?

q

Wri

tten

: St

uden

ts c

an m

ore

easi

ly r

efer

bac

k to

it a

s th

ey w

ork

beca

use

it is

m

ore

perm

anen

t. C

an b

e w

ritte

n in

key

pla

ces

dire

ctly

on

stud

ent w

ork,

or

on a

ru

bric

or

assi

gnm

ent c

over

she

et.

q

Ora

l: W

orks

bes

t for

ver

y yo

ung

stud

ents

or

thos

e w

ho m

ay li

kely

not

rea

d w

hat i

s w

ritte

n. A

lso

good

whe

n th

e te

ache

r ha

s so

muc

h to

say

that

it m

ay b

e in

timid

atin

g if

wri

tten.

Thi

s is

a g

ood

chan

ce to

let t

he s

tude

nt d

ecid

e w

hich

fe

edba

ck h

e/sh

e w

ill a

ct o

n. E

asy

to g

ive

in s

mal

ler

dose

s as

a c

ompl

emen

t to

wri

tten

feed

back

, as

stud

ents

mak

e re

visi

ons.

q

Dem

onst

rati

on: A

nyth

ing

that

invo

lves

a p

hysi

cal s

kill

lend

s its

elf

wel

l to

dem

onst

ratio

n (e

.g.,

hold

ing

an in

stru

men

t or

usin

g a

tool

). I

t is

also

a g

ood

way

to “

show

” a

stud

ent h

ow to

use

hig

her

cogn

itive

ski

lls s

uch

as ju

stif

ying

an

answ

er in

mat

hem

atic

s. S

tude

nts

will

be

mor

e lik

ely

to c

ompa

re th

eir

own

wor

k to

wha

t you

mod

el if

you

r de

mon

stra

tion

is c

oupl

ed w

ith o

ral f

eedb

ack.

(cont. next page)


5

Aud

ienc

eW

hat d

o m

y st

uden

ts n

eed

feed

back

ab

out?

D

o I

see

a pa

ttern

of

sim

ilar

mis

take

s ac

ross

my

clas

s?A

re th

ere

smal

l gro

ups

of s

tude

nts

who

wou

ld b

enefi

t fro

m g

ettin

g th

e sa

me

feed

back

?

q

Indi

vidu

al: W

orks

bes

t whe

n st

uden

ts n

eed

spec

ific

feed

back

that

oth

ers

may

not

nee

d or

if y

ou k

now

that

a s

tude

nt m

ay b

e em

barr

asse

d by

it. I

f yo

ur

clas

sroo

m c

limat

e is

stil

l dev

elop

ing

in a

ccep

tanc

e of

cri

tique

, thi

s m

ay b

e th

e be

st p

lace

to s

tart

. Thi

s al

so h

as th

e ad

ded

bene

fit o

f le

tting

the

stud

ent k

now

th

at th

e te

ache

r ha

s re

view

ed a

nd th

ough

t car

eful

ly a

bout

his

/her

wor

k an

d va

lues

and

car

es a

bout

his

/her

pro

gres

s. q

Smal

l gro

up: W

hen

seve

ral s

tude

nts

can

bene

fit f

rom

hea

ring

the

sam

e fe

edba

ck, i

t can

be

good

tim

e fo

r pu

lling

them

toge

ther

in a

flex

ible

gro

up a

nd

prov

idin

g a

“min

i-le

sson

.” W

hen

deliv

ered

wel

l, sm

all-

grou

p fe

edba

ck c

an in

m

any

case

s le

ad to

stu

dent

s fe

elin

g le

ss a

lone

in th

eir

conf

usio

ns a

nd b

enefi

t fr

om le

arni

ng to

dis

cuss

them

with

thei

r pe

ers.

q

Who

le g

roup

: The

re a

re s

ome

occa

sion

s w

hen

the

who

le g

roup

nee

ds to

hea

r th

e sa

me

thin

g. T

his

is a

goo

d tim

e to

beg

in a

less

on w

ith f

eedb

ack

from

the

prev

ious

less

on (

or f

rom

exi

t tic

kets

!). B

e ve

ry c

aref

ul th

at it

is r

elev

ant t

o al

l; it

can

turn

off

stu

dent

s w

ho k

now

they

don

’t n

eed

it, c

onfu

se s

tude

nts

who

are

n’t

sure

if th

ey n

eed

it, a

nd o

ften

be

easi

ly ig

nore

d by

the

ones

you

are

inte

ndin

g to

re

ach.

Ada

pted

Mos

s an

d B

rook

hart

, 200

9, p

p. 4

8-50

.


6

Criteria Analysis Data Display ToolExperiment 4: Criteria Analysis

Data Display Tool

What product or performance are you using as a formative assessment? ___________________________ Criteria for success for this formative assessment (What evidence will the product or performance have as confirmation of student mastery of the objective?) 1

2

3

4

5

Student Name Criteria for Success 1 2 3 4 5

Objective for the lesson: By the end of the lesson students will be able to….

Data Display P= Proficient performance -- = Not yet proficient

(cont. next page)


7

Identify several possible root causes for the student learning problem you identified above: Criteria Not

Yet Met Possible Root Causes

(Why do you think the students did not meet standard on this particular criteria?) I wonder if they didn’t meet standard because…

I wonder if they didn’t meet standard because…

I wonder if they didn’t meet standard because…

How could you gather additional data to verify the hypothesized root cause(s). Criteria Not yet

Met Further Data Collection to Test Hypotheses

(How could you gather additional data to verify the accuracy of your hypothesized root cause[s])?

(same as above)

Based on the analysis of the data, identify your next instructional steps for those students who have met the criteria as well as those students who have not yet met the criteria. Consider: • Different presentation • Different materials • Providing exemplars and models

After performing your criteria analysis, identify a student learning problem that needs attention:

Next instructional steps for students who have met the criteria:

Next instructional steps for students who have not yet met the criteria:

-

-

Yet

?)

?


8

89

89

Dat

a-D

riven

Dia

logu

e

Ada

pted

from

B. W

ellm

an a

nd L

. Lip

ton,

Dat

a-D

riven

Dia

logu

e: A

Fac

ilita

tor’

s G

uide

to C

olla

bora

tive

Inqu

iry, S

herm

an, C

T:

Mira

Via

LLC

, 200

4. A

s fo

und

in N

. Lov

e, K

.E. S

tiles

, S. M

undr

y, a

nd K

. DiR

anna

, The

Dat

a C

oach’s

Gui

de to

Impr

ovin

g Le

arni

ng fo

r All

Stu

dent

s, T

hous

and

Oak

s, C

A: C

orw

in P

ress

, 200

8. A

ll rig

hts

rese

rved

.


9

©2012 Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • Phone 978-263-9449 • www.RBTeach.com

Engage in Data-Driven Dialogue with Item Data: Data Coach Checklist

Preparation

¨ Collect item data for the content area, grade level, and time frame being analyzed from state or local assessments. You may also want to collect item data for a particular strand, standards within a strand, and/or multiple years of data to establish trends over time.

¨ Check data for accuracy. ¨ Prepare the data in table form as illustrated below:

¨ Prepare a predictions template (see sample in Institute Handouts). ¨ Provide a copy of the test blueprint or item map and relevant standards for each team member. ¨ Provide released items that correspond to item data being analyzed. ¨ Provide meeting agenda to team in advance. ¨ Prepare necessary materials (e.g., chart paper, pink, yellow, and green highlighters, markers, LCD

projector). Meeting Protocols

¨ Review purpose/agenda. ¨ Assign group roles (e.g., timekeeper, recorder, dialogue monitor, materials manager). ¨ Agree to norms on which the team will focus. ¨ Start and end on time. ¨ Review tools or protocols being used (e.g., Data-Driven Dialogue, Stoplight Highlighting). ¨ Review criteria for effective Data Team meetings (see last section below).

Item Data Analysis

¨ State questions that guide inquiry into item data: o What kinds of items are on the test? In what content strands? At what level of difficulty? o What knowledge, skills, and concepts are required for students to be successful with a

particular item? o What specific skills and understandings are our students’ strengths? Which pose

difficulties for them? o For which items are students frequently giving the same incorrect answers? o On what types of questions, such as short answer, extended response, or multiple-choice,

do our students perform well? Which pose difficulties? o Why are our students doing well or missing points on their open-response questions?

Engage in Data-Driven Dialogue with Item Data: Checklist

(cont. next page)


10

©2012 Research for Better Teaching, Inc. • One Acton Place, Acton, MA 01720 • Phone 978-263-9449 • www.RBTeach.com

¨ Ask team to study released test items. ¨ Ask team to Predict (Phase 1) based on the following questions:

o What are our predictions about students’ performance on these items (standards)? o Which items (or standards) do we think they will do well on? Which will they have

difficulty with? o What trends will we see over time (using multi-year data)? o Based on what assumptions?

¨ Record predictions on chart paper or on the predictions template provided. (Note: predictions can be quantitative or descriptive.)

¨ Go Visual (Phase 2): o Provide team with a poster-size (paper) graph (for percentage correct and patterns over

time) and/or table (for percentage correct and distractor patterns) or electronic graph or table projected onto a screen or Smartboard.

o Ask team to check data for clarity and accuracy. o Ask team to determine criteria for Stoplight Highlighting (e.g., cut points to distinguish

urgent areas, team’s vision of an excellent school, or comparisons with state, district, or similar schools) for percentage correct and for distractor (incorrect responses) patterns

o Have team Stoplight Highlight their graph or table accordingly. ¨ Ask team to Observe (Phase 3) based on the questions below. Observations are best made without

looking at the released test items, just the table or chart. o What important points seem to pop out? o What is surprising or unexpected? o What are items of relative strength? Weakness? o What trends do we observe over time (if analyzing multiple years of data)?

¨ Record observations on chart paper. ¨ Ask team to refer back to released items that they are most interested in studying further and

Infer/Question (Phase 4) based on the following questions: o What would students need to know/do to be successful at this task? o Why might so many of our students have done well at a particular item? o What might students have been thinking to make the errors they did? o How can we find out which of our hypotheses is right? o What questions do we have? o What additional data might we need?

¨ Reflect on next steps and implications for actions. Reflect on the Criteria for Effective Data Team Meetings

¨ Did we follow protocols (e.g., Data-Driven Dialogue)? ¨ Did we observe our norm/s? ¨ Did we avoid blame and culturally blind or destructive behaviors? ¨ Did we “look for love in the all the right places,” that is, look for possible explanations and

actions in those areas that impact student learning: curriculum, instruction, assessment, equity practices, and critical supports?

¨ Did we determine clear next steps that will impact students and their learning? ¨ How can we improve our Data Team meetings in the future?


11

Engage in Task Deconstruction and Data-Driven Dialogue with Student Work: Checklist

(cont. next page)

(See also The Data Coach’s Guide to Improving Learning for All Students, pp. 218-226, and Unleashing the Power of Collaborative Inquiry: A Program for Data Coaches, Course Handouts, Task 10.)

Preparation q Ask team members to bring the task or item and the student work they will be analyzing (one set

for each team member) or collect and prepare student work yourself. q Bring copies of relevant standards and rubrics related to the task. q Provide meeting agenda to team in advance. q Prepare necessary materials (e.g., chart paper, markers, Post-its).

Meeting Protocols q Review purpose/agenda. q Assign group roles (e.g., timekeeper, recorder, dialogue monitor, materials manager). q Agree to norms on which the team will focus. q Start and end on time. q Review tools or protocols being used (e.g., Data-Driven Dialogue). q Review criteria for effective Data Team meetings (see last section below).

Task Deconstruction with Student Work Analysis q State questions that guide inquiry into student work:

o What evidence are we seeing of student mastery of the knowledge and skills required by the task?

o What errors are students making? o What knowledge and skills seem to be missing? o What additional insights into student thinking are we gaining?

q Deconstruct the task. Ask team to: o Do the task and share solutions or strategies. o Brainstorm, drawing on our own experience doing the task:

• What do students need to know and be able to do to be successful at this task? • Write each piece of knowledge and each skill on a large Post-it, one item per Post-it.

o Refine what we have generated based on:• Consulting relevant standards and rubrics.• Focusing on the three to six key concepts/skills in the content area being assessed.• Focusing on ideas and skills that would inform reteaching and extension.

q Ask team to Predict (Phase 1) based on the following questions:• What do students need to know and be able to do to be successful at this task?• How do we think our students performed?• What do we think they had trouble with?• What kinds of errors or misconceptions do we anticipate?• Based on what assumptions?

q Record predictions on chart paper. q Pass out samples of student work.


12

Go Visual (Phase 2): Recreate the table below on chart paper.

Student(list below)

Know/Can Do

Adapted from Research for Better Teaching, Sstudying Skillful Teaching Course Handouts, Acton, MA: Research for Better Teaching, 2011.

o Record students’ names in left-hand column. o Place Post-its with the knowledge and skills identified in task deconstruction in the top row,

over the words “Know/Can Do.” o Next to each student’s name, place a check in each column where there is evidence that the

student has demonstrated the requisite knowledge or skill identified. o Note student errors or misconceptions in the last column.

q Ask team to Observe (Phase 3) based on the following questions: o What patterns or trends do we observe across several pieces of work (examine the table by

columns)? o What patterns in errors and misconceptions are emerging (examine last column)? o What do we notice about individual students (examine the table by rows)?

q Record observations on chart paper. q Ask team to Infer (Phase 4) based on the following questions:

o What new insights have we gained about the student-learning problem? o What might be contributing to students’ lack of understanding or skill? What errors are we

noticing? What misconceptions are we seeing evidence of? o What additional questions are raised by the student work? o What additional data could be helpful? o If relevant, consider if examination of student work confirms or refutes the tentative conclusions

we drew from other data analysis o Reflect on next steps and implications for action

Reflect on the Criteria for Effective Data Team Meetings q Did we follow protocols (e.g., Data-Driven Dialogue)? q Did we observe our norm/s? q Did we avoid blame and culturally blind or destructive behaviors? q Did we “look for love in the all the right places,” that is, look for possible explanations and actions

in those areas that impact student learning: curriculum, instruction, assessment, equity practices, and critical supports?

q Did we determine clear next steps that will impact students and their learning? q How can we improve our Data Team meetings in the future?


13

Error Analysis Protocol

Advanced Preparation• To select an item for error analysis, identify a frequently missed item in a benchmark, matching pre-

post, or short readiness or diagnostic assessment, or in a quiz given after instruction. If the item is multiple-choice, bring a report that indicates that percentage of students who answered each of the distractors as well as the correct answer. You can use the Error Analysis Template as you step through the process.

• Alternatively, bring in student work from any of the other pre-assessment (or post-assessment) sources. Provide 6-10 samples (or more if the student work is short) that illustrate a range of student thinking and proficiency.

At the Meeting • Do the problem or item first individually or in pairs. Share solutions and strategies.• Engage in Data-Driven Dialogue, guided by the questions below. Note that your team might need to

gather more data (question 4) before moving to questions 5–7. Be sure to record team responses to the questions on newsprint or projected on a screen or white board.

1. What will students need to know and be able to do in order to be successful at this item? What kinds of errors or misconceptions do we anticipate students will make? (Predict)*

2. What errors are students making? (Observe)

3. What might they have been thinking to make these errors? (Infer)

4. How can we find out which hypothesis is true? (Investigate/Verify Causes)

5. What different teaching (reteaching) strategies could we use to help students understand their errors, unravel their confusion, and/or correct a misconception? (Generate Solutions)

6. How can we manage time, tasks, and student groups to assure that students receive the instruction they need? How can the team help? (Generate Solutions)

7. What individual and collective action do we commit to?

*Note: If you have already determined how students performed on the item through a previous item analysis, skip the prediction step.


14

Error Analysis TemplateNote: This template can be used for error analysis on multiple-choice assessment items. Note that before progressing to “Verified Hypotheses” on the template, teams might need to collect additional data, e.g., ask students to provide a written or verbal explanation of why they chose the answer they did and why they did not choose the others.

Assessment Date

Item # A B C DPercentage Responding to Each ChoiceIndividual Students Who Selected Each Response

Hypotheses About Student Thinking: How Can We find Out?

Verified Hypotheses

Strategies for Teaching, Reteaching, Grouping, and/ or Extending Learning


15

BECA

USE


16

�

A DAtA CoACh’s GuiDe to improvinG LeArninG for ALL stuDents CopyriGht © 2008 Corwin press

Handout H1.6Norms of Collaboration

Two sets of collaborative norms that have been useful to a variety of school teams are the Seven Norms of Collaboration (Garmston & Wellman, 1999) and the Four Agreements of Courageous Conversations (Singleton & Linton, 2006). Each is described briefly below. After discussing these sets of norms, the Data Team can decide to (1) adopt one, the other, or both, (2) adapt one or both, or (3) generate your own new set of norms for working together.

The Seven Norms of CollaborationPausing: Pausing slows down the “to and fro” of discussion. It provides for “wait time,” which has been shown to dramatically improve thinking. It signals to others that their ideas and comments are worth thinking about, dignifies their contributions, and implicitly encourages future participation. Pausing enhances discussion and greatly increases the quality of decision making.

Paraphrasing: To paraphrase is to recast into one’s own words, to summarize, or to provide an example of what has just been said. It helps members of a team hear and understand each other as they evaluate data and formulate decisions, and it helps to reduce group tension by communicating the attempt to understand. Signal your intention to paraphrase (“So, you’re suggesting…”), and choose a level for the paraphrase: (1) acknowledge and clarify; (2) summarize and organize; or (3) shift the focus to a higher or lower level.

Probing for specificity: Probing seeks to clarify something that is not yet fully understood. More information may be required or a term may need to be more fully defined. Clarifying questions can be either specific or open ended, depending upon the circumstances. Ask for clarification of vague nouns and pronouns (e.g., “they”), action words (e.g., “improve”), comparators (e.g., “best”), rules (e.g., “should”), and universal quantifiers (e.g., “everyone”).

Putting ideas on the table and pulling them off: Ideas are the heart of a meaningful discussion. Members need to feel safe to put their ideas on the table for discussion. To have an idea be received in the spirit in which you offer it, label your intentions: “This is one idea…” or “Here’s a thought….” The other half of this norm is equally important: knowing when an idea may be blocking dialogue or “derailing” the process and therefore should be taken off the table.

Paying attention to self and others: Collaborative work is facilitated when each team member is explicitly conscious of self and others—not only aware of what he or she is saying, but also how it is said and how others are responding to it. We need to be curious about other people’s impressions and understandings but not judgmental. As we come to understand someone else’s way of processing information, we are better able to communicate with them.

Presuming positive intentions: This is the assumption that other members of the team are acting from positive and constructive intentions, even if we disagree with their ideas. Presuming positive presuppositions is not a passive state; rather, it needs to become a regular part of one’s verbal responses. The assumption of positive intentions is an aspect of the concept of a “loyal opposition,” and it allows one member of a group to play “the devil’s advocate.” It builds trust, promotes healthy disagreement, and reduces the likelihood of misunderstanding and emotional conflict.

(cont. next page)


17

A D

ata

Coac

h’s

Gui

de to

Impr

ovin

g Le

arni

ng fo

r All

Stud

ents

�

Ch

apte

r 3, T

ask

�: H

ando

ut H

�.6

Pursuing a balance between advocacy and inquiry: Both advocacy and inquiry are necessary components of collaborative work. The intention of advocacy is to influence others’ thinking; the intention of inquiry is to understand their thinking. Highly effective teams consciously attempt to balance these two components. Inquiry provides for greater understanding. Advocacy leads to decision making. Maintaining a balance between advocating for a position and inquiring about the positions held by others helps create a genuine learning community.

Adapted from Robert J. Garmston and Bruce M. Wellman, The Adaptive School: A Sourcebook for Developing Collaborative Groups. 1999. pp. 37-

47. Norwood, MA: Christopher Gordon. Used with permission.

The Four Agreements of Courageous ConversationsStay engaged: Staying engaged means “remaining morally, emotionally, intellectually, and socially involved in the dialogue” (Singleton & Linton, 2006, p. 59).

Experience discomfort: This norm acknowledges that discomfort is inevitable, especially in dialogue about race, and that participants make a commitment to bring issues into the open. It is not talking about these issues that creates divisiveness. The divisiveness already exists in the society and in our schools. It is through dialogue, even when uncomfortable, that healing and change begin.

Speak your truth: This means being open about thoughts and feelings and not just saying what you think others want to hear.

Expect and accept nonclosure: This agreement asks participants to “hang out in uncertainty” and not rush to quick solutions, especially in relation to racial understanding, which requires ongoing dialogue.

Adapted from Glenn E. Singleton & Curtis Linton, Courageous Conversations about Race: A Field Guide for Achieving Equity in Schools. 2006. pp.

58-65. Thousand Oaks, CA: Corwin.

Source: Nancy Love, Katherine E. Stiles, Susan Mundry, and Kathryn DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008, CD-ROM, Handout H1:6.


18

Ann

ual S

MA

RT

Stud

ent-L

earn

ing

Goa

l: B

y Ju

ne, 2

014,

impr

ove

6th

grad

e st

uden

ts’ p

erfo

rman

ce in

com

preh

ensi

on o

f inf

orm

atio

nal t

ext a

s ev

iden

ced

by in

crea

sed

perc

enta

ge o

f stu

dent

s sc

orin

g pr

ofic

ient

from

50

% to

75%

ann

ual a

sses

smen

ts…

(lis

t act

ions

). In

terim

SM

AR

T St

uden

t Lea

rnin

g G

oal:

In s

econ

d qu

arte

r ben

chm

ark,

in

crea

se p

erce

ntag

e pr

ofic

ient

in c

ompr

ehen

sion

of i

nfor

mat

iona

l tex

t to

70%

. In

stru

ctio

nal o

r Ret

each

ing

Stra

tegy

We

Agr

ee O

n M

onito

ring

Impl

emen

tatio

n: H

ow W

ill

We

Kno

w W

e D

id It

?

Mon

itorin

g S

tude

nt L

earn

ing:

H

ow W

ill W

e K

now

It

Wor

ked?

• Tea

ch u

se o

f gra

phic

or

gani

zers

• P

rovi

de s

tude

nts

with

at

leas

t fou

r pra

ctic

e op

portu

nitie

s th

is m

onth

• Brin

g st

uden

t ex

ampl

es o

f gra

phic

or

gani

zers

to o

ur

next

mee

ting

• Ana

lyze

info

rmat

iona

l te

xt c

ompr

ehen

sion

on

open

-res

pons

e as

sess

men

t

Shor

t-Cyc

le A

ctio

n Pl

ans

for G

rade

-Lev

el

or C

onte

nt T

eam

s


19

Shor

t-Cyc

le A

ctio

n Pl

ans

for G

rade

-Lev

el o

r C

onte

nt T

eam

s

Ann

ual S

MA

RT

Stu

dent

-Lea

rnin

g G

oal:

Inte

rim S

MA

RT

Stu

dent

-Lea

rnin

g G

oal:

In

stru

ctio

nal o

r R

etea

chin

g S

trate

gy

We

Agr

ee O

n

Mon

itorin

g Im

plem

enta

tion:

How

Will

W

e K

now

We

Did

It?

Mon

itorin

g S

tude

nt

Lear

ning

: How

Will

We

Kno

w It

Wor

ked?


20

�

A DAtA CoACh’s GuiDe to improvinG LeArninG for ALL stuDents CopyriGht © 2008 Corwin press

Purpose To focus the Data Team on data that indicate a need for urgent attention.

Overview Stoplight Highlighting helps Data Teams analyze data they have represented in the Go Visual phase of Data-Driven Dialogue. Based on relevant criteria, Data Teams use highlighters to mark positive data as a “green light,” data that represent caution as a “yellow light,” and data that demand immediate attention as a “red light.”

Audience Data Team.

Use Primary Tasks: Tasks 6–9 and 11.

Advance Preparation1. In addition to the data noted above, gather information about the Data

Team’s, school’s, or district’s student-learning and achievement criteria for growth and improvement. Examples include your Data Team’s vision of a great school; any national, state, or local criteria already established for expected percentage of students reaching proficiency, of items correct, or of annual improvement.

Procedure 1. Direct the Data Team’s attention to the data on the chart they have

created for Phase 2: Go Visual. Depending on the task, this chart may focus on aggregated, disaggregated, strand, or item-level data.

2. Introduce Stoplight Highlighting as a process that enables the Data Team to highlight student-learning needs and successes. Explain the analogy of a stoplight: some of the data is “good to go” and is noted as green; some represents “caution” and is noted as yellow; some is in need of “immediate attention” and is noted in red. Stoplight Highlighting is a tool to guide and inform the team’s observations.

3. Share the school/district criteria for student learning and achievement growth and improvement. Facilitate a discussion about realistic criteria and bring the group to consensus about which criteria they will use. Write this information on a wall chart similar to Resource TR1 (Example of a Stoplight Highlighting Criteria Table). (Note that aggregated and disaggregated data are often reported as a percentage in each proficiency

29

Time �0-�5 minutes with each data set.

Materials Resources

TR�—Example of a Stoplight Highlighting Criteria Table

TR2—Stoplight Highlighting Vertical Plot Example

Data

Tables or graphs representing aggregated, disaggregated, strand, or item-level student-learning data

General

Chart paperHighlighters (green, yellow, red)Masking tape

Stoplight Highlighting

(cont. next page)


21

A D

ata

Coac

h’s

Gui

de to

Impr

ovin

g Le

arni

ng fo

r All

Stud

ents

2

To

olki

t: St

oplig

ht H

ighl

ight

ing

level, while strand or item-level data may be reported as a percentage correct; adjust your criteria accordingly.)

4. Ask team members to use the criteria to highlight their data. What is the range for “green,” for “yellow,” and for “red”?

5. Ask the team members to continue in their Data-Driven Dialogue, making observations and inferences about the highlighted data. In what areas can they celebrate student progress? What areas are in need of improvement?

6. If there are several “red” areas, have members discuss/determine a priority for addressing these areas. Do some naturally align themselves with others? If some were addressed, would others fall into place?

7. Use the “red” areas to target areas of focus for use in subsequent tasks and data sets. For example, if the Data Team is engaged with aggregated data and sixth-grade science is in the “red” zone, home in on disaggregated sixth-grade science in the next task.

Adapted from The ToolBelt: A Collection of Data-Driven Decision-Making Tools for Educators. Copyright ©

2004, Learning Point Associates. All rights reserved. Used with permission.

Facilitation Note

Stoplight Highlighting works best with line graphs for aggregated and disaggregated data. Bar graphs can be confusing because it will appear that some of the proficient students are in the red or yellow zone.

(cont. next page)


22

A D

ata Coach’s Guide to Im

proving Learning for All Students

� Toolkit: Stoplight H

ighlighting

TR1

Example of a Stoplight Highlighting Criteria Table: CRT Aggregated and Disaggregated Data

HigHligHT coloR Meaning PRoficiency level

green go! Meets expectations above 70%

yellow caution! Below expectations

Between 60% and 69%

RedUrgent!

in immediate need of improvement

Below 60%

Adapted from The ToolBelt: A Collection of Data-Driven Decision-Making Tools for Educators. Copyright © 2004, Learning Point Associates. All rights

reserved. Used with permission.

(cont. next page)


23

A D

ata

Coac

h’s

Gui

de to

Impr

ovin

g Le

arni

ng fo

r All

Stud

ents

�

To

olki

t: St

oplig

ht H

ighl

ight

ing

Stoplight Highlighting Vertical Plot Example: Percentage Proficient on Eighth-Grade Mathematics

Strand on State CRT, Year 3

PERCENT

MATHEMATICS STRANDS AND LEVELS OF UNDERSTANDING

Number of Students: 191

100

90

80 Geometry 78%

70

60 Data analysis/probability 61%sis/pMeasurement 58%

50Number sense 54%

40 Algebra 40%Computation 38%Knowledge and sk ills 35%

30 Conceptual understanding 30%Application/problem solving 27%

20

10

0

Go! (green)

Caution! (yellow)

Urgent! (red)

TR2

Source: Nancy Love, Katherine E. Stiles, Susan Mundry, and Kathryn DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008, CD-ROM, Toolkit.


24

Student Error Analysis

Name _________________

STUDENT ERROR ANALYSIS Assessment: ___________

Directions: Please review your assessment. For any questions you are correcting, please complete the following:

1. Write the number of the question you are correcting 2. Classify your error 3. Re-do the problem correctly 4. Explain how to do the corrected problem

A =

Arithmetic (You made a

calculation error - added, subtracted,

multiplied, or divided incorrectly)

C = Careless

(You made a silly mistake)

D = Directions (You didn’t follow the directions)

E= Explanation

(Your explanation was incomplete)

U = Understanding

(You did not understand how to do the problem)

Problem # _________ Error Classification: _______

Re-do the problem correctly

Explain the error you made

Note: This tool is to be used with students as a way for them to analyze and correct their own errors.

(cont. next page)


25







©2013 Michelle A. Savage This document is for individual use only. For any other purposes, please contact Michelle Savage

at [email protected] for permission.


26

Verifying Causes

Advance Preparation• Make a copy of the Verifying Causes Graphic to illustrate the process or show it on a slide.

• Make one copy of Verifying Causes Template for the team to fill out together.

At the Meeting(s)• Generate a hypothesis or possible cause for patterns of errors, misconceptions, or gaps in student

understanding that you observed in your data analysis (using any of the protocols provided).

• If the team determines that they do not have enough information from which to draw a sound conclusion about student thinking, use this Verify Causes protocol. Use the Verify Causes graphic provided to illustrate and explain the process. Note: It may be important to explain to the team why this extra step of verifying causes is important. If so, point out the danger of rushing to a premature conclusion without adequate data. This may lead to wasted time and effort attempting to solve the wrong problem.

• Using the Verifying Causing Template provided, insert the team’s hypothesis in the first section of the template.

• Determine what additional evidence is needed in order for the team to have confidence in their hypothesis and to generate appropriate solutions. For example, you might ask students to explain their answers verbally or in writing, observe students as they are working, or look to research on misconceptions.

• Insert what additional data will be collected and how in the next section of the template labeled “Additional Data/Research to Collect” and “How, By Whom, and When.”

• Collect evidence between meetings and come to the next meeting ready to analyze evidence, draw conclusions, and plan next steps.

• Use the “Findings (Observations)” section of the template to record observations of the data.

• Use the “Verified Hypothesis” section of the template to record the conclusion drawn from the data analysis. If the data confirm the team’s hypothesis, they can proceed to generating solutions. If the data refutes their hypothesis, the team revises their hypothesis, collects additional data if needed, and proceeds to generating solutions. (Note: Initial data collection may be sufficient to support a revised hypothesis.) And, if the data are inconclusive, they may decide to collect additional data.

• Record next steps in the final row of the template.


27

Verifying Causes Graphic

This is a graphic representation of the Verify Causes process.

• “Hypothesize Possible Cause” represents the team’s inference about what is causing the errors, misconceptions, or gaps in student thinking observed through data analysis.

• “Collect Additional Data” represents the various data sources the team might use to verify their hypothesis.

• Once they have collected the data or research, the team analyzes the data to determine if the data confirms or refutes their hypothesis or is inconclusive.

• The green arrow labeled “Data Confirms” illustrates that if the team has verified their hypothesis, they move on to generating solutions.

• The red arrow labeled “Data Refutes” shows that if the data do not support the hypothesis, the team revises their hypothesis.

• The yellow arrow labeled “Data Inconclusive” illustrates that if the data neither confirm nor refute their hypothesis, the team undertakes additional data collection.

Hypothesize Possible Cause

Collect Additional Data, e.g. • student interviews • observations • additional work • research

Generate Solutions

DATA CONFIRMS

DATA REFUTES

DATA IN

CO

NC

LU

SIV

E

Verify Causes – Short Cycle

© 2012 Research for Better Teaching, Inc. www.RBTeach.com

Verify Causes


28

Verifying Causes Template

Hypothesis to Test:

Additional Data/Research to Collect (e.g., student work, interviews, research on misconceptions):

How, By Whom, and When:

Findings (Observations):

Conclusion:____Hypothesis confirmed (go to template sections below)____Hypotheses refuted (write revised hypothesis below or repeat sections above as needed)____Hypothesis inconclusive (redo sections above as needed)Verified or Revised Hypothesis:

Next Steps:


29

Data Tools Organizer

Tool Where Can I Find It?

Notes


30

data tools for nancy love’s sessions · data-driven dialogue adapted from b. wellman and l....

Documents