implementing student-involved assessment: what does it look like in the classroom? bruce herzog...

Implementing Student-Involved Assessment: What Does It Look Like In The Classroom?

Bruce Herzog & Joni Heutink

Grade 5 Teachers/Trainers/Consultants

Nooksack Elementary

Nooksack Valley School District

herzogmail@mac.com

joniheutink@excite.com

It’s the economy, Stupid!

It’s the classroom assessment practices, Stupid!

Parents: Please

DO NOT DISTURB

Teachers are in a staff meeting

Improving communication with parents.

Who are we?Who are we?

What are we selling?What are we selling?

Is it relevant for you?Is it relevant for you?

We are both currently 5th grade teachers.We are both currently 5th grade teachers.

Between us, over the course of the last 25 years, we Between us, over the course of the last 25 years, we have taught every grade from K-5. have taught every grade from K-5.

Together, we have been teaching for over 50 years.Together, we have been teaching for over 50 years. We working in a school that has figured out how use We working in a school that has figured out how use student-involved assessment practices to get incredible student-involved assessment practices to get incredible achievement results from virtually all students.achievement results from virtually all students.

We have been working as trainers, presenters, and We have been working as trainers, presenters, and consultants across our district and throughout the consultants across our district and throughout the Northwest for the past 7 years.Northwest for the past 7 years.

What is student-involved assessment?What is student-involved assessment?

CurriculumCurriculum InstructionInstruction

AssessmentAssessment

CurriculumCurriculum InstructionInstruction

AssessmentAssessment

Clear TargetsClear Targets Goal-SettingGoal-Setting

Formative Formative AssessmentsAssessments

Summative Summative AssessmentsAssessments

Fair Grading Fair Grading Practices Practices

CommunicationCommunication

Reinstruction & Reinstruction & Reassessment Reassessment

CurriculumCurriculum InstructionInstruction

AssessmentAssessment

Clear TargetsClear Targets Goal-SettingGoal-Setting

Formative Formative AssessmentsAssessments

Summative Summative AssessmentsAssessments

Fair Grading Fair Grading Practices Practices

CommunicationCommunication

Reinstruction & Reinstruction & Reassessment Reassessment

STUDENTSSTUDENTS

Old: Assessment means testing.

New: Assessment is a continuous and dynamic process of communication between teachers, students, and parents regarding students’ achievement and is embedded in everything the teacher and students do all day, every day.

Old: Administrators control achievement data and filter it down to teachers, parents, and students.

New: Achievement data is directly available to teachers, parents, and students.

Old: Teachers set standards. Students may not know or understand grading practices.

New: Teachers set standards together with students and students have a clear understanding of grading practices.

Old: The target moves from grade to grade and teacher to teacher.

New: Teachers work together within and across grade levels to ensure that standards are consistent.

Old: The teacher grades papers. Students get papers back with a grade, marks, and/or comments days, or weeks later (or never).

New: The teacher only grades summative assessments and discusses results with students in a timely manner. Students assess their own and each other’s formative work together with the teacher.

Old: Attitude, effort, late or missing assignments, group work, homework, and extra credit are all factored into a student’s grades.

New: Only individual, subject-specific assessment data is used to determine grades. Other factors are reported separately.

Old: Assessment is used to sort students.

New: Assessment is used to identify student strengths and weaknesses and to help students set goals for what they need to do next.

Old: Year after year the same students are successful and the same students aren’t. Unsuccessful students seldom close the achievement gap that separates them from those who are performing at grade level.

New: Effective assessment practices can improve grade equivalent scores by several grade levels or up to 15%. Low performing students benefit most!

A New Vision of Assessment

An unrelenting emphasis on student accountability for learning.

• All work is expected to be done on time.

• All work is expected to meet a standard.

• All students are expected to be prepared for all

summative assessments.

• Immediate and natural consequences for failing to

meet learning expectations.

A New Vision of Assessment• Daily discussions about the purpose for learning and the connection between effort and achievement.

• Daily reinforcement of the idea of “no excuses” and that it is OK to fail, but not OK to choose failure.

• Frequent goal-setting, reflection, and communication about achievement by students and by the teacher to parents.

A New Vision of Assessment• Weekly teacher-created assessments (CBAs) in virtually all subjects with mandatory reinstruction, extra practice, and scheduled retesting for any student who fails to meet grade level standards (repeated as many times as necessary).

• The use of a simple grading system that is transparent to all students.

• Frequent teacher-generated achievement progress reports (at least one each month).

A New Vision of Assessment

The expectation that every student knows his or her

current level of achievement in each subject and also

knows what he or she needs to do next to maintain

or improve his or her level of achievement.

A shift in focus from teaching to learning.

Some kids come to school to slay the dragon…

some come to be slain.

Rick Stiggins

Why should you believe that

there is a connection between

student -involved assessment

practices and student achievement?

Black & Wiliam (1998) International Research ReviewBlack & Wiliam (1998) International Research Review0.5 to 1.0 Standard Deviation Score Gain0.5 to 1.0 Standard Deviation Score GainLargest Gain for Low AchieversLargest Gain for Low Achievers

Bloom (1984) Mastery Learning ResearchBloom (1984) Mastery Learning Research1.0 to 2.0 S.D. Gain1.0 to 2.0 S.D. GainRivals Impact of One-on-One TutoringRivals Impact of One-on-One Tutoring

Rodriguez (2004)Rodriguez (2004)0.5 to 1.8 S.D. Gain0.5 to 1.8 S.D. Gain

Effect of Reducing Class SizeEffect of Reducing Class Size0.2 S.D. Gain0.2 S.D. Gain

1.0 Standard Deviation Equals:

35 Percentile Points on ITBS 35 Percentile Points on ITBS

2 to 4 Grade Equivalents2 to 4 Grade Equivalents

100 SAT Score Points100 SAT Score Points

5 ACT Score Points5 ACT Score Points

U.S. TIMMS Rank from 22 of 41 to Top 5U.S. TIMMS Rank from 22 of 41 to Top 5

Potential Elimination of Score GapsPotential Elimination of Score Gaps

Unprecedented Achievement GainsUnprecedented Achievement Gains

Nooksack Valley School District

• 1,700 students.1,700 students.

• Three grade PreK-5 elementary schools.Three grade PreK-5 elementary schools.

• One grade 6-8 middle school.One grade 6-8 middle school.

• One grade 9-12 high school.One grade 9-12 high school.

• Grades PreK-5

• October 2005 Student Count 271

• Classes at each grade level 2

• Indian/Alaskan Native 3.3%

• Asian 1.1%

• Black 1.5%

• Hispanic 24.4%

• White 69.4%

• Free or Reduced Meals 54.4%

• Special Education 17.0%

• Transitional Bilingual 19.1%

• Migrant 8.1%

• Classroom Teachers 22

• Avg Years Teacher Experience 14.4

• Teachers with Master's Degree 54.5%

Nooksack Elementary (Oct 2006)

Nooksack Elementary

2003/2004: Seattle Pacific University: “From 2003/2004: Seattle Pacific University: “From Compliance to Commitment” - one of ten high-Compliance to Commitment” - one of ten high-achieving elementary schools in Washington state.achieving elementary schools in Washington state.

2004/2005: National Blue Ribbon School Award.2004/2005: National Blue Ribbon School Award.

2005/2006: Washington State Distinguished Principal 2005/2006: Washington State Distinguished Principal Award (Marion Evenson).Award (Marion Evenson).

3rd grade WASL reading: 3rd grade WASL reading: 73%73% 81%81%

3rd grade WASL math: 3rd grade WASL math: 66%66% 83%83%

4th grade WASL reading: 4th grade WASL reading: 94%94% 94%94% 98%98% 84%84%

4th grade WASL writing: 4th grade WASL writing: 88%88% 93%93% 95%95% 84%84%

4th grade WASL math: 4th grade WASL math: 88%88% 88%88% 91%91% 84%84%

5th grade WASL reading: 5th grade WASL reading: 92%92% 94%94%

5th grade WASL math: 5th grade WASL math: 83%83% 92%92%

5th grade WASL science: 5th grade WASL science: 41%41% 68%68% 80%80% 86%86%

2004 2005 2006 2007Nooksack Valley Elementary School

6th grade WASL reading: 6th grade WASL reading: 75%75% 86%86%

6rd grade WASL math: 6rd grade WASL math: 59%59% 63% 63%

7th grade WASL reading: 7th grade WASL reading: 67%67% 67%67% 70%70% 83%83%

7th grade WASL writing: 7th grade WASL writing: 68%68% 67%67% 82%82% 70%70%

7th grade WASL math: 7th grade WASL math: 59%59% 49%49% 65%65% 74%74%

8th grade WASL reading: 8th grade WASL reading: 73%73% 75% 75%

8th grade WASL math: 8th grade WASL math: 49%49% 56%56%

8th grade WASL science: 8th grade WASL science: 30%30% 32%32% 38%38% 58% 58%

2004 2005 2006 2007Nooksack Valley Middle School

3rd3rd/4th WASL reading: /4th WASL reading: 73%73% 84%84%

3rd3rd/4th WASL math: /4th WASL math: 66%66% 84%84%

4th4th/5th WASL reading: /5th WASL reading: 98% 98% 94%94%

4th4th/5th WASL math: /5th WASL math: 91%91% 92%92%

4th/4th/5th5th WASL reading: WASL reading: 94%94% 92% 92% 4th/4th/5th5th WASL math: WASL math: 88%88% 83%83%

Two Point in Time WASL DataTwo Point in Time WASL Data

2005 2006 2007

Student-involved assessment practices build student confidence, increase the achievement of all students and have the greatest impact on low-achieving students.

Student-involved assessment practices increase student motivation by increasing student success (or, at the very least, they reduce “lack of motivation” as a barrier to learning).

Students understand learning targets.

Formative assessments are aligned to the targets.

Frequent classroom-based summative assessments, reteaching, and retesting.

Students set goals and reflect on their learning.

Fair, standards-based grading practices are used at all grade levels and in all programs.

Students communicate about their own achievement and are involved in conferences with parents at all grade levels.

Key Element #1Establishing Clear Learning Targets:

Good assessment practices begin with students having a clear understanding of the specific learning targets they are expected to meet.

“Students can hit any target that is clear and that holds still for them.”

Rick Stiggins

The learning targets need to be clear to the teacher, students, and parents:

• What do students need to know/do?

• How well do they need to know/do it?

• How will you know they know?

• How will you get them there?

• What will you do when they fail?

Instruction should focus primarily on

Washington: Grade Level Expectations (GLEs).

Oregon: Content/Achievement/Performance Standards

Idaho: Content Standards

Passing the test(s) that your state uses for determining AYP must be seen as a valid goal by all teachers, students, and parents!

The ability of all students to pass the test(s) that your state uses for determining AYP must be seen as an achievable goal by all teachers, students, and parents!

Teachers have a moral obligation to prepare students to be successful on state tests.

Procedural directions are not the same thing as learning targets.

Understanding by Design (UbD) is a great model for developing clear learning targets.

• Essential Questions

• Enduring Understandings

• Vocabulary

• Activities

• Assessments (evidence of learning)

Probability (GLE 1.4.1)

Essential Questions:

• How do you determine the likelihood of an event?

• What is the difference between mathematical and

experimental probability?

• What mathematical notation is used to to express

probability?

• What makes a game fair or unfair?

Probability (GLE 1.4.1)

Enduring Understandings:

• Probability is the chance that an event will occur

out of all possible events.

• The actual outcome of an event may differ from its

mathematical probability.

• Probability can be expressed as a fraction or as a

number out of a total number.

• A game is fair if the outcome for all players has an

equal mathematical probability.

Probability (GLE 1.4.1)

Vocabulary:Certain More Likely/ProbableEqually Likely/Probable Less Likely/Probable Impossible Possible Outcomes

Mathematical Probability Actual OutcomeExperimental Probability Fair/Unfair Experimental Outcome

Activities and Assessments

To clarify the learning target use:

• Examples and non-examples

• Models

• Rubrics

• Scoring guides

• Test specification guides

Grade 1 and 2

Grade 2 and 3 morning calendar

Grade 5 work examples

Grade 4 behavior targets

Do your students know what they

need to know and do they know

whether or not they know it?

Key Element #2Engaging Students in Formative

Assessment Activities:

Throughout a unit of instruction students engage in practice and risk-taking activities and receive feedback that will help them move toward meeting the learning targets.

Formative Assessment is individualized assessment FOR learning.

Formative Assessment tells a student what he or she needs to do next to improve.

Every student needs to be able to answer the question, “What do I need to do next to improve my own work?”

Students need to be given multiple “next” opportunities to practice, experiment, and, ultimately, show growth.

Good feedback is less like a

grade and more like advice.

NON-SPECIFIC FEEDBACK • produces no changes in learning

SPECIFIC FEEDBACK • produces positive changes in learning

SPECIFIC FEEDBACK & STUDENT SELF-EVALUATION

• produces the most positive changes in learning

What the Research Says:

• minimum 7% achievement gain (Lysakowski & Walberg, 1981)

• maximum 41% achievement gain (Kumar, 1991)

• average 35% achievement gain (nine studies, 1981-1999)

It’s OK to mark, grade, or score daily work or homework for the purpose of informing students of work quality or accuracy, but those grades should seldom, if ever, be used to determine a student’s grade in that subject.

The teacher frequently conferences individually with students about their work.

The teacher provides opportunities for students to assess anonymous samples of work, their own work, and each other’s work.

The teacher creates an atmosphere of trust within the classroom where students feel free to take risks.

Are you just giving feedback?

Or are you taking time to “feed forward?”

Key Element #3Using High Quality Summative

Assessments:

The teacher uses a variety of high quality assessments that best measures the learning of their students.

Summative Assessment is assessment OF learning that has already occurred.

Grades are always summative!

Tell students, in advance, how they will be assessed, when they will be assessed, how the assessment will be graded, and what the consequences for failing to meet the learning targets will be.

Never give a summative assessment without advance warning; no pop quizzes or “gotcha” tests.

Use summative assessments that reflect the stated learning targets (no surprises).

Good, teacher-created tests are better than commercially-produced tests because they can focus more precisely on the learning targets.

For all tests use the smallest sample possible that covers all aspects of the learning target(s).

Separate knowledge/comprehension, application/analysis and synthesis/evaluation tests.

Creating Summative Assessments

Give students the results of summative assessments in a timely manner.

What the Research Says:

• feedback immediately after item: 7% achievement gain

• feedback immediately after test: 26% achievement gain

• feedback delayed after test: 21% achievement gain

Discuss the results of summative assessments with students so they can use the results to plan and guide their own learning.

Every test should also be used as a formative learning activity whenever possible.

Use summative assessments as opportunities for students to reflect on their own performance

The problem:My answer:The correct answer:What I did wrong:

What the Research Says:

• right/wrong answer: 3% loss in achievement

• correct answer: 9% achievement gain

• explanation: 20% achievement gain

Whenever possible, whenever any student fails to meet of any of the learning targets the teacher should provide opportunities for additional instruction and practice and should then reassess that student.

What do you teach that you

don’t want 100% of your

students to learn?

What the Research Says:

• when students repeat until correct: 20% achievement gain

Conducting frequent summative assessments:

• Lets students know how they’re doing.

• Identifies student misconceptions.

• Provides information for progress reports.

• Identifies students who need extra help.

• Minimizes the amount of content to reteach.

• Makes students accountable for learning.

Classroom assessments paint a more accurate picture of student achievement than district or state assessments:

• They occur more frequently.

• They cover a specific range of material.

• They are often more authentic.

• They use a greater variety of types of assessment.

• The assessor knows the students.

• They can be individualized.

Using Formative & Summative Assessments

Formative: coaching; as it happens; with training wheels.

Summative: after instruction & practice; on their own; independently & without help.

Be very careful about using formative assessments summatively.

Use summative assessments formatively whenever possible.

Key Element #4Involving Students in Goal-

Setting and Reflection:

Students set goals and reflect on their learning as it progresses and communicate their understanding to others.

“Self assessment by pupils, far from being a luxury, is in fact an essential component of formative assessment.”

Black & Wiliam, 1998

Students must be taught how to set realistic goals.

A goal without a plan is just a wish.

Students must be held accountable for making progress toward meeting their goals.

What the Research Says:

• minimum 18% achievement gain (Walberg, 1999)

• maximum 41% achievement gain (Wise & Okey, 1983)

• average 24.5% achievement gain (three studies, 1983-1999)

A key premise is that, for students to be able to improve, they must have the capacity to monitor the quality of their own work during actual production.

Royce Sadler, Australia, 1989

Portfolios of student work allow students to monitor their progress over time.

A portfolio without student reflections is just a scrapbook.

Ruth Sutton

Key Element #6Communicating About Student

Achievement:

Students are the primary users of assessment information and, as such, they regularly communicate about their achievement.

When students keep portfolios with self reflections they can see the quality of their work change over time.

Result? Success is within reach.

When students lead or participate in parent/teacher conferences they gain a greater sense of responsibility and pride in their accomplishments.

Result? Greater achievement

Involving students in conferences sends a powerful message to students that they are responsible for their own learning.

Student-Involved Conferences

Don’t have to be student led.

Parents want to hear from the teacher.

Don’t take more time than traditional conferences.

Can look different in different classrooms.

Need to be by direction, not by invitation.

Provide an opportunity to model communication.

Allow teachers to “publicly” recognize students.

May require some system changes.

Key Element #5Using Fair Grading Practices:

Grades are based on ample evidence that accurately reflects a student’s level of achievement in specific subjects, performances, products, or skills.

4 = 88% - 100% exceeding standard

5 = 75% - 87% meeting standard

6 = 62% - 74% not meeting standard

1 = 50% - 61% significantly below standard

The NVSD K-8 Grading Scale

What does a “C” mean?

A = 88 - 100% Exceeds Grade Level Standards

B = 75 - 87% Meets Grade Level Standards

D = 62 - 74% Doesn’t Meet GLS

F = 50 - 61% Significantly Below GLS

I = Incomplete Not Enough Evidence

Making A-F Grading Standards-Based

Fair vs Unfair Grading Scales

100

90

80

70

60

0

A

B

C

D

F

100

88

75

62

50

4

3

2

1

100

90

80

60

0

4

3

2

1

Fair

Unfair

4 = 88% - 100% 18-2014-16 11-12

5 = 75% - 87% 15-1712-13 9-10

6 = 62% - 74% 13-1410-11 8

1 = 50% - 61% 10-12 8-9 6-7

Using Points and Percents

20 points 16 points 12 points

10 11 12 13 14 15 16 17 18 19 20

8 9 10 11 12 13 14 15 16

6 7 8 9 10 11 12

4 5 6 7 8

75%

Using Points to Determine GradesUsing Points to Determine Grades

Consistently using just a few simple grading scales allows students to understand what constitutes meeting the standard.

J-curve Assessment Results

In an ideal assessment system the majority of students will achieve the learning target.

A Standard Curve on a Standards-Based Assessment

Standard curves may still naturally occur in an ideal assessment system, but the mean, median, and mode scores should all be at or above the grade level standard.

In a standards based system grades are criterion referenced and not norm referenced.

Student achievement is measured against a standard - students are not compared to each other.

Nooksack’s K-8 Grading Scale:

4: exceeding grade level standards

3: meeting grade level standards

2: not meeting grade level standards

1: significantly below grade level standards

Grades are motivating for some students.

A student’s grades belong to that student.

Students need to understand classroom grading practices.

Teach students the connection between effort and achievement.

A grade, by itself, cannot communicate the complexity of the learning that has occurred.

Grades, by themselves, give students LITTLE useful information and do LITTLE to improve student learning.

A grade should NOT reflect effort, improvement, extra credit, attitude, absences, or late or missing assignments.

(these should all be reported elsewhere)

Not everything needs to be graded.

(everything counts, but everything doesn’t need to go into the grade book)

Put grades into your gradebook in pencil - not in permanent ink.

DON’T grade pre-test, practice, risk-taking, or

formative tasks.

DON’T give group grades for cooperative work.

DON’T factor late work, effort, or improvement

into achievement grades.

DON’T give achievement grades for homework

or other work that you can’t be sure

was completed by the student alone.

Extra Credit

A student’s grade should not go up simply because that student has done extra work.

A student’s grade should go up if doing that extra work results in higher achievement.

Get rid of the zero score.

All grading is subjective.

All grades must be justifiable. (measurement theory says that you need at least

3 pieces of good evidence for triangulation)

There are no right grades - only justifiable grades

GOOD PRACTICES REGARDING GRADING

• Use the most-recent evidence.

• Use the most-comprehensive forms of evidence.

• Use evidence that reflects the most important learning goals.

• Use only selective, representative grades.

2 1 2 3 2 3 4 3

Average = 2

Most-Recent Evidence = 3

It’s OK to use averaging to calculate a student’s grade, but only when averaging gives a result that accurately reflects achievement.

Most computer grading programs convert scores to percents and then average those percents to arrive at a grade.

When using any grading scale, the highest possible score needs to be within reach of all students.

(the “achievable” 4)

When using any grading scale,

all the scores on the scale need

to be available to all students.

When using any grading scale, the divisions between grades need to be equal.

(a “fair” grading scale)

An adopted grading scale provides a consistent standard for determining grades, but ultimately it is the teacher who decides the grading standard for each assessment because it is the teacher who chooses how many questions to ask, what type of questions to include, the level of difficulty of the questions, and what constitutes “meeting the standard.”

(remember: all grading is subjective)

What standard do you set for your students?

…your airline pilot?

…your surgeon?

More importantly, what standard do your students set for themselves?

…your hairdresser?

The grading standards for each subject, performance, product, or skill should be consistent within and across grade levels.

Consistent standards can only be achieved through collaboration.

A student’s classroom grades should predict that student’s level of achievement on state and district assessments.

The grades a student receives can have a tremendous impact on that student’s life - especially in high school.

Good grading practices are important, but you don’t fatten a pig by weighing it!

Student-Involved Assessment:

Anything you do that helps students:

• Understand learning targets.

• Engage in self-assessment.

• Watch themselves grow.

• Talk about their growth.

• Plan the next steps in their learning.

Everyone Wins

• Students

• Teachers

• Parents

• Administrators

• More accurate assessments

• Stronger desire to learn

• Increased achievement

• Accountability for performance

Teaching is like trying to row a boat across a lake with one student at a time.

• Some kids will help you row.

• Some kids will make you do all the rowing.

• Some kids will try and jump out.

Student-involved assessment practices force students to grab the oars!