using data to improve student learning misa regional workshop march 24, 2006 damian cooper...

Using Data to Improve Student Learning

MISA Regional Workshop

March 24, 2006Damian Cooper

Plan~Teach~Assess

Tel: (905) 823-6298

Email: [email protected]

www.damiancooperassessment.com

mailto:[email protected]

Understand the importance of evidence-based decision making

Know how to use data effectively in a decision-making inquiry process

Understand the importance of using multiple and appropriate sources of data in an accountability framework

Understand basic principles and concepts pertaining to the use of data

Know how to act upon data to improve student learning

Outcomes

Terminology

Data?

Information?

Evidence?

“We live in a society that is data rich but information poor.”

Robert H. Waterman

“What gets measured

gets done.”Tom Peters

•Teach to the test?

•What gets tested gets taught.

•“Test” what we teach.

“Why do we avoid data? The reason is fear – of data’s capacity to reveal strengths and weaknesses, failure and successes. Education seems to maintain a tacit bargain among constituents at every level not to gather or use information that reveal where we need to do better, where we need to make changes. Data almost always point to action – they are the enemy of comfortable routines. By ignoring data, we promote inaction and inefficiency.”

Schmoker

The effective use of data depends on simplicity and economy.

Schmoker, 2003

Why Do We Use Data in our Schools?

Individually, jot down 1-3 reasons

Share with a partner

As a group, what are your top 3 reasons?

Guide to using Data in School Improvement Efforts,

Learning Point Associates, 2004

Connecting Mission Statements and Evidence

Mission Statement Evidence We Have Evidence We Need

Love of learning in a safe caring environment.

-discipline referrals

-vandalism

…

-student survey data

…

Our Speer-it gives us strength to grow, to learn, and to succeed.

-EQAO data

-report card data

…

-parent attitudinal data

…

Your turn…

Demographic Data

•Enrollment by grade level

•Racial/ethnic composition

•Student mobility rate

•Percentage of limited English proficient students

•Percentage of students receiving special education services

Student Data

•Student participation rates in nonacademic services and programs

•Student participation in co-curricular activities and the degree to which it is representative of the overall composition of the student body

•Percentage of students who exceed graduation requirements

Data on Teaching and Learning

•Usage data from library, information, and media services

•Number of students moving among ability groups, especially into groups with more challenging course work.

•Participation in professional development and evidence of impact on improved teaching

•Evidence of how analysis of data at the school level has resulted in specific improvements

•Evidence of how technology has contributed to increased use of data for decision making

Adapted from Getting Excited About Data • Edie L. Holcomb • 1999 Corwin Press, Inc.

Evidence We Have or Evidence We Need?

Data on School, Family, and Community Partnerships

•Data on family involvement in school activities and the degree to which the families involved are representative of the overall student body

Indicators of Success

•Results of standardized tests for each of the last 5 years, disaggregated according to the largest and most significant subgroups in the school

•Results of non-standardized, or alternative, assessments developed at the school level.

•Percentage of students in various educational and employment categories a year following graduation

•Daily student attendance

•Student dropout rate

•Daily teacher attendance

•Teacher turnover rate

•Results of climate surveys

•Percentages of students involved in various types of safety, discipline, and drug issues

Adapted from Getting Excited About Data • Edie L. Holcomb • 1999 Corwin Press, Inc.

Evidence We Have or Evidence We Need?

Student Database

Who are these students?

(Demographics)

What are they

experiencing?

(School Processes)

What are they

perceiving about the learning

environment?

(Perceptions)

What do they know?

(Student Learning Results)

Multiple Measures / Data Elements

Data: Inputs and Outputs

DemographicsAffectReadiness to

learnOpportunities

to learnRelationships Instruction

GrowthProgressAchievement

Interventions

Triangulation of Data

Use of different data sources to maximize validity and reliability

Data from different participantsData from different stages in a process

Triangulation of Data: Student Achievement

Valid & Reliable

Picture of Student

Achievement

Standardized Test Data (norm-referenced)

Standards-based Test Data (criterion-referenced)

Classroom Assessment

Data

Reliability and Validity in Assessments

Reliability Validity

Working with Data

1) Identify Problem & Formulate Question

3) Gather Data

4) Examine Data

5) Translate Data into

Information6) Develop Improvement

Plan

2) Check Assumptions

Checking Our Assumptions

Before examining a set of data, we often

suggest an hypothesis, based on certain assumptions, to explain a problem or current state.


Examine these Gr. 3 EQAO Mathematics

data.What trends do you notice over time?Suggest some hypotheses to account for

these trends.

4 C’s of Examining Data

Concealing

Complete

Consistent

Comparisons

From Assessment Training Consortium/Ottawa-Carleton District School Board


Complete?



Consistent?

What observations can you make about this school’s Gr. 3 EQAO Mathematics data in terms of “consistency”?



Comparisons?

What observations can you make about this school’s Gr. 3 EQAO Mathematics data in terms of board and provincial comparisons?



Comparisons?

What observations can you make when you compare 2004-05 Gr. 3 EQAO Mathematics data with report card data?


Gr. 3 Mathematics Overall Report Card Achievement

0

5

10

15

20

25

30

35

40

45

1st Rpt. 2nd Rpt. 3rd Rpt.

RDCBA

%-age of students at each report card grade.


Examine the same Gr. 3 Report Card

Mathematics data, disaggregated by strand.Suggest some hypotheses to explain the

data.What further classroom level data do you

need to gather?

Gr. 3 Mathematics Report Card Achievement by Strand

0

10

20

30

40

50

60

Number Measmt. Geometry Pattern. DataMgmt.

RDCBA

%-age of students at each report card grade level.

How Do We Improve Student Learning…and Achievement?

Planning

Instruction

Assessment

Classroom Management

A Safe and Orderly Environment

A Clear, Focused Vision for Learning

A Climate of High Expectations for Success

A Focus on High Levels of Student Achievement

Instructional Leadership

Frequent Monitoring of Student Progress

Strong Home-School Relations

Characteristics of Effective Schools

School Improvement Planning: A Handbook for Principals, Teachers, and School Councils • Education Improvement Commission • http://eic.edu.gov.on.ca

“Strategic planning presumes that the most vital, high-leverage thinking is done primarily by “planners” before the school year begins, rather than by teaching practitioners throughout the school year.”

Schmoker, 2004


The primary purpose of analyzing data: improving instruction to achieve greater student success.

Schmoker, 2003


1. What kinds of data am I able to gather about each of these classroom variables?

2. How can I gather such data?

Planning Instruction Assessment Classroom Management

DifferentiatingInstruction & Assessment

Instruction Students bring different

knowledge & experience to school

Students learn at different rates

Students learn in different ways

Assessment Not all students are

able to demonstrate their learning in the same way

Not all students respond the same way to test pressure

Some students need more scaffolding than others

Goal: success for all.

How Do We Differentiate Instruction and Assessment?

Instruction Vary pace of instruction Vary depth & complexity

of materials Vary teaching strategy:

direct/indirect, guided, whole-class, small group, individual

Vary amount of intervention/independ-ence

Assessment Provide choice of

assessment task Provide choice of

assessment environment Vary complexity of

assessment task Adjust scaffolding of

assessment task

A Model Lesson

1. Identify Learning Goal

2. Activate Prior Knowledge/Experience – hook, engage, personal experience

3. Diagnostic Assessment of Prior Knowledge

4. Present New Learning – teacher, small groups, etc

5. Check for Understanding – orally

6. Practice New Learning/Scaffold New Learning

7. Self/Peer Assess Practice Work

8. Review and Consolidate Learning – tie back to learning goal

9. Apply New Learning to New Context (Differentiated)

10. Assess Learning

Differentiated

Research on Effective Assessment

•The provision of effective feedback to students

•The active involvement of students in their own learning

•Adjusting teaching to take account of the results of assessment

•Recognition of the profound influence assessment has on motivation and self-esteem

•The need for students to be able to assess themselves and understand how to improve

Crooks, 1988; Black & Wiliam, 1998

Assessment for Learning

“Assessment for learning is any assessment for which the first priority in its design and practice is to serve the purpose of promoting students’ learning. It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence.”

Black, Wiliam et al. 2004

Assessment of Learning

“Assessment of learning includes those tasks that are designed to determine how much learning has occurred after a significant period of instruction. The data from such assessments is often used to determine report card grades.

Assessment for Learning: am I already doing it?

Do teachers routinely share learning goals with students so they know where they are heading?

Do teachers routinely communicate to students the standards they are aiming for before they begin work on a task?

Do teachers routinely have students self and peer assess their work in ways that improve their learning?

Assessment for Learning:am I already doing it?

Do teachers’ questioning techniques include all students and promote increased understanding?

Do teachers routinely provide individual feedback to students that informs them how to improve?

Do teachers routinely provide opportunities for students to make use of this feedback to improve specific pieces of work?

“The breakthrough strategy focuses on small but immediate improvements and operationalizes and accelerates the effectiveness of teamwork, goals, and data … The key is to pay attention to already existing approaches that work, and work fast.”

Schmoker

Rapid Results: the Breakthrough Strategy

“ Celebration, recognition and reward only require a varied repertoire of methods and a sincere heart. Teachers must believe that the methods are more than a bald attempt to manipulate or control behaviour. They must see praise and recognition as an extension of a leader’s character.”

Schmoker

Celebrating Success

Planning for Improvement

Working with Data

1) Identify Problem & Formulate Question

3) Gather Data

4) Examine Data

5) Translate Data into

Information6) Develop Improvement

Plan

2) Check Assumptions

Professional learning communities use evidence and intuition in order to work and talk together to review their practices and to increase their success…In a professional learning community, the culture changes – everyone sees the big picture and works for the good of the whole community. Professional learning communities bring together culture and contract. They value both excellence and enjoyment.

Hargreaves, 2004

School and System Improvement

The Big Ideas of Professional Learning Communities DuFour, 2004

Ensuring that students learn

A Culture of Collaboration

A Focus on Results


Ensuring that students learn:

-Focus on learning, (as well as) on teaching

-What do we want each student to learn?

-How will we know when each student has learned it?

-How will we respond when a student experiences difficulty in learning?


A Culture of Collaboration

-move beyond “Yes but…” and “If only…”

-partnerships

-sharing knowledge and learning

-team work

-community

-conversations


A Focus on Results

-establish baseline data

-set improvement goals

-work together to achieve goals

-gather frequent evidence of progress

-identify strategies that lead to the greatest gains

-share these with colleagues

Goals give teamwork meaning.

Goals allow educators to analyze and adjust practice.

Goals and the commitment that they generate are the glue that holds the team together.

A few specific, clear and measurable goals are the most vital ingredient to purpose.

Goals

Connecting Our Goals to the Desired State

Goal Student Achievement

Desired State

Results

Actions

Student Achievement

Current State

1. What specifically needs to be done? 2. How does it need to be done? 3. What is the timeline for doing it? 4. Who is responsible for getting it done? 5. What resources are needed? 6. How will we monitor progress? 7. How often and by what means will we

communicate progress? 8. How will we celebrate success?

Committing Our Plan to PaperSchool Action Plan

Improvement Goal: Current State,

including Evidence

Indicators of Desired

State (may

include targets)

Specific Measures (tools used to gather

data)

Strategies (specific actions that will be taken)

Resources (improvement

teams, materials)

Responsibility Timelines Communicating Progress

Celebration of Success

Effective Goals

Poor Goal Statement:

Support high expectations for student achievement with a specific focus on literacy and numeracy.

Revised Goal Statement :

Improved student achievement with a specific focus on literacy and numeracy.

S.M.A.R.T. Goals

SpecificMeasurableAttainableResults-orientedTime-bound

Leader

Do I have the teams in place? ___

Are the right structures in place to facilitate team work? ___

Am I creating a passion for system improvement? ___

Do I model evidence-based decision making? ___

Do I regularly assess the progress/implementation of our improvement plans? ___

Do I ensure that our budget aligns with our goals? ___

Administrator’s Role

Adapted from School Improvement Planning: A Handbook for Principals, Teachers, and School Councils • Education Improvement Commission

Administrator’s Role Professional Developer

Do I provide time for staff to ask questions and dialogue about evidence? ___

Do I know what is available from the Board to help us with gathering data? ___

Do our PD activities focus on achieving school improvement goals? ___

Do I know the strengths of my staff that will assist with our system improvement plan process? ___

Do I provide appropriate professional development opportunities for staff? ___


Administrator’s Role

Communicator

Do I clearly explain the process to staff, school councils, parents and other community members? ___

Can my staff articulate the sources of evidence we are using ? ___

Am I using evidence when I am communicating about the system? ___


Resources

1. Assessment Training Consortium, Linking Achievement Data to School Improvement, 1996, Toronto, Ontario.

2. Educational Leadership, February, 2003

3. Holcomb, Edie L. Getting Excited About Data: How to Combine People, Passion,

and Proof, Corwin, 1999. ISBN. 0-8039-6739-X

4. Learning Point Associates, Guide to Using Data in School Improvement Efforts, 2004 (available on-line)

5. Northwest Regional Educational Laboratory, Evaluating Whole-School reform

Efforts: A Guide for District and School Staff, 2000 (available on-line)

6. Schmoker, Mike. Results: the Key to Continuous Improvement, ASCD, 1996. ISBN. 0-87120-260-3

7. Schmoker, Mike. The Results Fieldbook, ASCD, 2001. ISBN. 0-87120-521-1

8. Tomlinson, Carol Ann. How to Differentiate Instruction in Mixed Ability

Classrooms, 2nd. Edition, ASCD, 2001. ISBN. 0-87120-512-2

using data to improve student learning misa regional workshop march 24, 2006 damian cooper...

Documents

analysis of data

data edie

data rich

terminology data

student survey data

increased use of data

effective use of data

learning usage data