Using Action ResearchTo Empower North Carolina Educators

A Race to the Top Initiative

NC Department of Public Instruction

Educator Effectiveness Division

Increasing the Teacher’s Effectiveness Toolbox

Joni AllisonHenderson County Public Schools

Rugby Middle and West Henderson High

Who is in the audience?

What is your professional role / title in education?

http://answergarden.ch/159042

Session Objectives● Participants will explore the topic of data literacy.

● Participants will analyze and discuss benchmark data using

SchoolNet Reports from a teacher and a student perspective.

What is Action Research?

Systematic inquiry conducted by teachers and other educators to find solutions for critical, challenging, relevant issues in their classrooms and schools.

Mills, Geoffrey E, Action Research: A Guide for the Teacher Researcher, 2014

What is Action Research?

Main Goals Include:•Positively impact student outcomes•Identify and promote effective instructional practices

•Create opportunities for teachers to become reflective practitioners

•Share research results with other educators

Mills, Geoffrey E, Action Research: A Guide for the Teacher Researcher, 2014

What is Action Research?A systematic research process to:● Identify an area of focus (critical, challenging

issue)● Develop an action research plan● Implement action research plan in

classroom/school● Collect, analyze, and interpret data● Share findings to inform practice

Mills, Geoffrey E, Action Research: A Guide for the Teacher Researcher, 2014

Go Soap Box

Event Code

ccsa2015Enter this code at

app.gosoapbox.com

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Focus Statement• For close to 9 years, our district has

asked all teachers of a subject or course with a state End-of-Grade or End-of-Course test to administer interim benchmarks.

• Some teachers, especially ELA, have not found the data generated from district interim benchmark assessments useful in designing and informing instructional units and interventions.

• Many middle and high school students do not see the benefit or purpose of the multiple benchmark assessments they take each year and often do not give their best efforts.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Purpose of the Study

The purpose of this study is to describe the impact of a

benchmark analysis tools on a teacher or student’s ability

to use a district-created reading and math

benchmarks in the formative assessment process.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Study Participants

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Research QuestionsHow can targeted professional development increase a teacher's sense of efficacy in data-driven decision making?

What the Peer-Reviewed Research says about…TEACHERS’ CONCERNS

“Like other constructs, such as intelligence and self-concept, a teacher’s concerns about an innovation cannot be directly observed and measured. It can only be inferred from the measurement of some predefined affective dimensions.”

Cheung, D., Hattie, J., & Ng, D. (2001). Reexamining the Stages of Concern Questionnaire: A Test of Alternative Models. Journal Of Educational Research, 94(4), 226.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Key Vocabulary

self-efficacy: refers to beliefs concerning one's capabilities to learn or perform behaviors at designated levels (Bandura, 1986)

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Key Vocabulary

data-driven decision making (DDDM):systematic collection and use of many forms of data from a variety of sources in order to improve student performance (Choppin, 2002; Marsh, Pane, & Hamilton, 2006).

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Key Vocabulary

classroom level DDDM: the use of a variety of data (e.g., state-mandated exams, formative assessments) to adjust instruction in order to improve learning for all students (Dunn 2013)

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Dat

a Li

tera

cy

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an Data Literacy

For the purpose of this action research project, data literacy is defined as the ability to gather, interpret, and use multiple data sources effectively to improve student learning.

What the Peer-Reviewed Research says about…

TEACHERS’ DATA LITERACY“Given that ‘[a]n aim of data-driven decision making is to link the results of summative testing to formative information systems that teachers can use to improve instruction across school’, it appears vital to have teachers who are data-literate and collaboratively involved in the DBDM process.”

Crum, K. (2009). Building the Foundation for Data-Based Decision Making: Creating Consensus on Language and Concepts. International Electronic Journal For Leadership In Learning, 13(5),

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Survey Question: I am confident in my ability to gather data from various and multiple sources.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Survey Question: I am confident in my ability to interpret data from various and multiple sources.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Survey Question: I am confidant in my ability to use data analysis to make effective instructional changes.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Survey Question: I am confident in using assessment data to provide targeted feedback to students.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Survey Question: Data analysis improves my instructional practice.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Survey Question: Data analysis results in improvements in student learning.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

What conclusions would you draw from these results?

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: I understand how EVAAS scores are determined.

Survey Question: I understand my EVAAS scores.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: EVAAS gives my valuable information about my instructional practice.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: EVAAS helps me adjust my instruction.

Survey Question: EVAAS is a tool for me to use to reflect and improve on my own effectiveness.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: EVAAS is a tool I can use to improve student learning.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: EVAAS makes more likely to collaborate with my peers.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: EVAAS makes me more competitive.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: EVAAS makes me more likely to take instructional risks.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

Survey Question: The district benchmark assessments provide useful data about the content knowledge of my students.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - Benchmarks

Survey Question: I use the data from the district benchmark assessments to inform my classroom instruction.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - Benchmarks

What the Peer-Reviewed Research says about…

TEACHERS’ SENSE OF EFFICACY“Teacher-centered instructors with low efficacy fail to adjust to the individual needs of students, a concept at the heart of DDDM.”

“Thus, teachers in this study who were more confident in their ability to successfully engage in DDDM were more likely to be working with colleagues to improve and increase use of DDDM in their classrooms. ”

Dunn, K. E., Airola, D. T., Lo, W., & Garrison, M. (2013). Becoming Data Driven: The Influence of Teachers' Sense of Efficacy on Concerns Related to Data-Driven Decision Making. Journal Of

Experimental Education, 81(2), 222-241.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

EVAAS Treasure Map

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy - EVAAS

EVAAS Mini-EdCamp Topics

Reflective Assessments• School Reports• Teacher Effectiveness ReportsProactive Assessments• School Academic Preparedness

Report• Custom Student Report• Student Search

Discussion Question

Event Code

ccsa2015Enter this code at

app.gosoapbox.com

What successes or challenges have you experienced with

using EVAAS?

What the Peer-Reviewed Research says about…EFFECTIVE DATA USE

“First, we use a broad and encompassing definition of data, meaning anything that helps educators know more about their students.”

“Second, data use means the actions in which educators engage as they collect these data, organize, and analyze data, and draw meaning from them to inform practice. ”Wayman, J. C., Cho, V., Jimerson, J., & Spikes, D. D. (2012). DistrictWide Effects on Data Use in the

Classroom. Education Policy Analysis Archives, 20(25),

What the Peer-Reviewed Research says about…EFFECTIVE DATA USE

“Third, we often use the term effective data use to distinguish between data use practices that benefit educators in their practice (and which thus benefits student learning) from other data use practices that have been to shown to actually hinder educational work.”

Wayman, J. C., Cho, V., Jimerson, J., & Spikes, D. D. (2012). DistrictWide Effects on Data Use in the Classroom. Education Policy Analysis Archives, 20(25),

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

4 Types of Data Haiku Deck

4 Types of Data in 4 Corners

Rank the 4 types of data. Which type of data is most important for teacher to understand about their students.

Vote with your feet. As a group, make a case for your selection.

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Data Literacy

Discussion Question:What could be the possible impact on teachers of a broader definition of data?

Event Code

ccsa2015Enter this code at

app.gosoapbox.com

What the Peer-Reviewed Research says about…

BENCHMARK ASSESSMENTS“District administrators often cited intentions that the assessments would be used to inform instruction. However, the realization of instructional purposes was limited by the type of information provided by predominantly multiple choice items, a lack of substantive professional development, and minimal coherence with respect to shared understandings of assessment purposes and uses across district, school, and classroom levels.”

Davidson, K. L., Frohbieter, G., & National Center for Research on Evaluation, S. T. (2011). District Adoption and Implementation of Interim and Benchmark Assessments. CRESST Report 806. National

Center For Research On Evaluation, Standards, And Student Testing (CRESST),

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Benchmark Revision

Revision Process

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Discussion Guide

1st Draft Benchmark Discussion Guide 8th Grade Science

Revised Benchmark Discussion Guide

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy

Which items were the most challenging for students and what can you learn about their misunderstandings by looking at the distractors? What instructional actions will you take to differentiate instruction based on the data?

Discussion Guide:School and District Data -

Item Analysis

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy

Sort by total score. Order by Standards Performance and Percent Correct. Which items were the most challenging for students and what can you learn about their misunderstandings by looking at the distractors? Which standards are you most concerned about for your students? What instructional actions will you take to differentiate instruction based on the data?

Discussion Guide:Student Performance -

Item Analysis

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy

Drill down to different levels of domains, standards, and substandards (clarifying objectives). Which skills are you most concerned about for your students? What instructional actions will you take to differentiate instruction based on the data? Look at the instructional materials available in SchoolNet. Look at the build express test option. Refer back to you EOG/EOC Test Specifications.

Discussion Guide: Student Performance Skills Analysis

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy

+Δ

Benchmark Discussion Guide Debrief

Stud

ent S

choo

lNet

Acc

ess

Scho

olN

et B

ench

mar

k D

iscu

ssio

n

Key

Voc

abul

ary

Act

ion

Res

earc

h Pl

an

Dat

a Li

tera

cy Student Reflection

Student Benchmark Reflection Questions

Student Benchmark Reflection Graphic Organizer

Session Evaluation

Google Form - http://goo.gl/forms/fMxOwUQKVt

CCSA Evaluation Form - http://www.cvent.com/d/hrq907

Conclusion of Presentation

Thank you !

Contact Information:Name: Joni AllisonDistrict: Henderson County Public SchoolsEmail: kjallison@hcps.orgWebsite: http://bit.ly/joniallison23Twitter: @joniallison23

References Cheung, D., Hattie, J., & Ng, D. (2001). Reexamining the Stages of Concern Questionnaire: A Test of Alternative Models. Journal Of Educational Research, 94(4), 226.

Corkett, J., Hatt, B., &Benevides, T. (2011). Student and Teacher Self-Efficacy and the Connection to Reading and Writing. Canadian Journal Of Education, 34(1), 65-98.

Crum, K. (2009). Building the Foundation for Data-Based Decision Making: Creating Consensus on Language and Concepts. International Electronic Journal For Leadership In Learning, 13(5),

Davidson, K. L., Frohbieter, G., & National Center for Research on Evaluation, S. T. (2011). District Adoption and Implementation of Interim and Benchmark Assessments. CRESST Report 806. National Center For Research On Evaluation, Standards, And Student Testing (CRESST),

Dunn, K. E., Airola, D. T., Lo, W., & Garrison, M. (2013). Becoming Data Driven: The Influence of Teachers' Sense of Efficacy on Concerns Related to Data-Driven Decision Making. Journal Of Experimental Education, 81(2), 222-241.

Morrison, Jennifer L. Morrison. (2013) Seize the Data [Online Course]. Retrieved from https://canvas.instructure.com/courses/830570

Schunk, D. H. (1985). PARTICIPATION IN GOAL SETTING: EFFECTS ON SELF-EFFICACY AND SKILLS OF LEARNING-DISABLED CHILDREN. Journal Of Special Education, 19(3),

Wayman, J. C., Cho, V., Jimerson, J., & Spikes, D. D. (2012). DistrictWide Effects on Data Use in the Classroom. Education Policy Analysis Archives, 20(25),