july 19-21, 2011 stonewall resort, roanoke, west virginia educator evaluation pilot
TRANSCRIPT
July 19-21, 2011Stonewall Resort, Roanoke, West Virginia
Educator Evaluation Pilot
Overview
• The “Big Picture”–Where are we headed?–Where have we been?–How will we get there?
• Two Day Agenda• Next Steps
VISION:West Virginia will have a
comprehensive and equitable evaluation system that clearly
articulates, measures, rewards, and develops educator
effectiveness
Where Have We Been?
Historical Perspective
2009 Standards Adopted
2011 Evaluation Task Force Worked
2012 Revised System Piloted
Importance of Leadership
• July 20-21 – Stonewall Resort• ALL Pilot schools leadership team
members– Principals and all assistant principals – Title I Directors and/or Assistant Superintendents– Teacher Leaders– Central office representative for non SIG pilot
schools
Clear Expectations
• Embrace the Opportunities:• Be “early adopter”• Engage in professional development• Work as part of a collaborative team• Provide input to policy makers
• Participate in the research study• Communicate, partner and problem-solve
Day 1 Agenda
• Expectations for Training• History Evaluation Creation • Conceptual Framework Overview• Self-Assessment • Observation• Student “Growth”
Day 2 Agenda
• SMART Goals and Collaboration• Student Learning Goals• Professional Conduct• Research• Performance Assessment
What are the Next Steps?
Teacher Evaluation Training Dates
• August 6-12 – Regional Training• Participants: all teachers, administrators
and transformation specialists• Fall and Spring Training• On site visits• Ongoing Technical Assistance
Historical Perspective
2009 Standards Adopted
2011 Evaluation Task Force Worked
2012 Revised System Piloted
2013 Begin Statewide Scale Up
Thank You!
Expectations for Professional Development
Expectations for Professional Development
• Learning Target– Understand how the new evaluation
system works to be able to implement the processes yourself and to be able to assist others in school
• Strategies to accomplish the goal– Hands-on activities– Multiple opportunities to master content– Collaboration with tablemates
Expectations for Professional Development
• Expectations for us– Well prepared– Clarity– Consideration of the audience
• Expectations for you– Active participation– Regular feedback– Focused attention
Teacher Evaluation Creation
Task Force Members’ Perspective
Evaluation Perceptions & Experiences
Table Tasks
• Describe most recent evaluation experience– What was the purpose? What feedback
was provided? What was your role in the process?
• What are the challenges of teacher evaluation (not including time)?
• What will you need to learn to make this PD beneficial?
A New Evaluation System:
The Rationale
A New Evaluation System:
The Rationale• Learning Target: Know
– the three drivers behind a revised evaluation system for principals and teachers
Converging Forces
New Teaching Standards
Weak Evaluation System
Federal Encouragement
Conceptual Framework Overview
Conceptual Framework Overview
• Learning Targets: Understand– the nexus between the WVPTS and the
Critical Standard Elements; – how levels of teacher performance are
defined with rubrics; – the tiered evaluation system with
progressions based on experience
Levels of PerformanceDistinguished Accomplished Emerging Unsatisfactory
Table Tasks
• Share one activity with distinguished performance, one activity with unsatisfactory performance
Levels of PerformanceDistinguished Accomplished Emerging Unsatisfactory
Distinguished performance describes professional teaching that engages students to be highly responsible for their own learning. Performing at this level involves contributing to the professional learning of others through teacher leadership.
Accomplished performance describes professional teaching that exhibits mastery of the work of teaching while improving practice and serving the professional community.
Emerging performance represents teaching that demonstrates knowledge and skills to implement essential elements albeit not always successfully at times.
Unsatisfactory performance describes teaching that does not convey sufficient understanding of concepts or the successful implementation of essential elements.
Self Assessment – Advance Progression
Self Assessment – Advance Progression
• Learning Targets:– Identify individual performance within
the established rubrics
Table Tasks
• Share an insight from doing this activity about individual performance
• Share reaction to critical standard elements
• What implications does this have for individual professional development
Table Tasks
• Discuss perception of how comfortable teachers with 6+ years of experience will be completing self assessment
• Discuss how this process could be used in relation to the PD plan for of the school
• Discuss how this process could be used in collaborative teams.
Observation
Observation
• Learning Targets:– Understand that observation is formative
and offers a window into instructional performance;
– Understand that observation, though limited, is informative about certain key aspects of instruction
Some specifics about observation
• Not the evaluation• Initial Progress-4, Intermediate-2,
Advanced if requested• Class period or minimum of 30 mins• One piece of a two part conversation• Will be supported by evidence and
conversation• Elements that contribute to the
research
Table Tasks
• What does observation tell you about the critical standard elements?
• What elements do you still need to know about to make a fair and accurate assessment about performance?
Evidence
Evidence
• Learning Targets:– Understand that the educator plays an
active role in demonstrating performance level by providing evidence;
– Identify and classify evidence
Table Tasks• Share the kinds of evidence that were
generated.• Were any common among the table?• Discuss whether you agree that
evidence “presented” was acceptable.• Even though none are required, which
ones would be considered essential?• Which ones should be brought to
conference with principal?
Evidence For Self Assessment
Table Tasks• Share the kinds of evidence that were
generated.• Were any common among the table?• Does the evidence convincingly support
the rating?• Discuss whether you agree that
evidence “presented” was acceptable.• Even though none are required, which
ones would be considered essential?
Student Growth 101
Juan D’BrotExecutive Director
Office of Assessment and Accountability
Student Growth 101
• Learning Targets:– Understand how the school-wide growth
measure is developed
The WV Growth Model:Changing Conversations
about Education
Juan D’BrotExecutive Director
Office of Assessment and Accountability
WV Growth Model
1. What questions?2. Purpose3. Status vs. growth4. What student growth data looks like5. What school growth data looks like
First…
A challenge!
Stop me for questions…
Asking the Right Questions…
• Until you’ve defined the question, you cannot examine the appropriate data.
• Why?– Different data answer different
questions – Different questions lead to different
conversations
Fact #1: …increased use of student assessment data as a “significant
factor” in (insert favorite accountability topic here…)
• But what student data? That leads us to start asking questions…
• But what questions?
• Whose questions?
Whose Questions?• Researchers
• Administrators
• Teachers
• Parents
• Policymakers
• Business Leaders
• Community Members
• Guardians
WHY?
To answer questions about the student
How?By not declaring a verdict. Instead, by changing the
conversation…
Components of the System should Align with the Purpose of the System• The revised teacher evaluation system is
intended to do what?– Change conversations– Provide support and direction– Identify areas of best practice and need
• The revised consideration of student learning (growth) is intended to do what?– Change conversations– Provide direction – Reframe evidence of student learning
Bigger Question:
How do we align the revision of both systems?
…Fact #2
Fact #2: Assessment Systems try to be Everything to Everyone
• Danger: answering too many questions with the same data– Differentiated Instruction– Student Proficiency– Class Performance– AYP
• Possible Solution: Can we come up with a common question across stakeholders?
The WV Growth Model: Our Purpose
• To provide an answer to a few common questions: 1. “How much did my student grow this
year?” 2. “What does this growth mean
compared to everyone else?”3. “Is it enough growth?”
What do these three questions have in common?
Information about students
Status vs. Growth
• Before growth, let’s discuss status
– Status: A snapshot measure of a single point in time – “Proficiency”
– Growth: Multiple snapshots across many points in time
Status vs. Growth A Balanced Approach
Low Growth High Growth
Quadrant 2 Is believed not to require
improvement because declining student growth is not recognized
under status model
Quadrant 3Does not require improvement
and is a potential site of interest for best practices
Quadrant 1Legitimately requires
improvement
Quadrant 4Is believed to require
improvement because high rates of growth are not recognized
under the current status model. May be a potential site of interest
for best practices.
High Performing
(Status)
Low Performing
(Status)
Questions - Revisited
1. “How much did my student grow this year?” (Time 2 – Time 1) – We can see this today
2. “What does this growth mean compared to everyone else?” (Normative Component)
3. “Is it enough growth?” (Criterion Component - CSOs)
How much did my student grow this year?
–1st Question » How much academic growth do individual students in
WV exhibit?» Scale Scores – Tell very different stories» Time 2 (2010 Scale Scores – Mastery = 550) – Time 1
(2009 Scale Scores – Mastery = 500)
2009 Scale Score 2010 Scale Score Net “Growth”
525 575 50
550 575 25
425 575 125
375 400 25
What does this growth mean compared to everyone else?–Pre-Growth Model
»This year and last year (only 2 years worth of comparison)
–Post-Growth Model»Contextual consideration of growth»Yields “Student Growth Percentiles” »Think height
What is a Student Growth Percentile?• Given (3 things):
– A student’s prior scale scores – Academically similar students – Current scale score
• A student’s current scale score represents a percentile of growth – a Student Growth Percentile
• Think of it as the probability of a student’s current achievement based on their past achievement:– Pr(Current Achievement|Past Achievement)
• Growth percentiles describe the probability/rarity of a student’s current achievement based upon their prior achievement.
Vertical Axis: Number of Students at a Particular Scale Score
Vertical Axis: Number of Students at a Particular Scale Score
Vertical Axis: Number of Students at a Particular Scale Score
Vertical Axis: Number of Students at a Particular Scale Score
What does this student’s growth mean compared to everyone else?
–2nd Question » What does the academic growth of an individual
student in WV really mean? » Scale Score Growth – Tells very different stories» Time 2 – Time 1: Based on other students with similar
academic histories. 2009 Scale
Score2010 Scale
ScoreNet “Growth” Growth
Percentile525 575 50 40th
550 575 25 20th
425 575 125 99th
375 400 25 50th
But is it enough growth?
–3rd Question » Is the academic growth of an individual student in WV
enough to make it to Mastery? What about staying at Mastery?
» This year’s Growth Percentile isn’t enough
» Where would a student be next year if s/he exhibited:• High Growth• Typical Growth• Low Growth
Our Growth Options• How much growth? – Starts with the student.
Can be aggregated to drive school conversations– Very Low Growth - Unsatisfactory– Lower Growth - Emerging– Typical Growth - Accomplished– Higher Growth – Distinguished
• Is it enough growth? – Can drive student learning conversations– Catching Up– Keeping Up– Falling Behind
How much growth?
Is it enough growth?
Novice
Partial Mastery
Mastery
Above Mastery
Novice Partial Mastery
Partial Mastery
Mastery
Novice
Partial Mastery
Mastery
Above Mastery
Novice
Partial Mastery
Mastery
Above Mastery
Novice Partial Mastery Partial Mastery Mastery
Partial Mastery
Partial Mastery Partial Mastery Partial Mastery
Novice
Partial Mastery
Mastery
Above Mastery
Novice
Partial Mastery
Mastery
Above Mastery
Novice Partial Mastery Partial Mastery Mastery
Partial Mastery
Partial Mastery Partial Mastery Partial Mastery
From Student to School Growth–For whom can we calculate growth?» Only students in grades 4 – 11, in tested subjects» Students must have at least 2 consecutive scores» We examine RLA and Math
–How do we aggregate growth to the school?
» Examine students by grade and by content for the state» Identify all of the students in the school» Take the median (middle) growth percentile of all students in
a school for each grade and content.
Let’s start with 5th gradeStudent Mathematics
Johnny 25th Suzy 35th
Kenny 75th Lori 85th
Juan 15th
Lisa 40th
Amelia 27th
Robert 40th
Lee 45th
Jan 62nd
Frank 51st
Mary 12th
5th Grade MEDIAN Growth 40th
How can you use these data? 1.Rank
ordering?2.Grouping?3.In conjunction
with other data?
Let’s start with 5th gradeStudent Mathematics Proficiency
Johnny 25th NoviceSuzy 35th Novice
Kenny 75th NoviceLori 85th Novice
Juan 15th Novice
Lisa 40th Novice
Amelia 27th Distinguished
Robert 40th Distinguished
Lee 45th Distinguished
Jan 62nd Distinguished
Frank 51st Distinguished
Mary 12th Distinguished
5th Grade MEDIAN Growth 40th
Distribution of Student Growth Percentiles by Scale ScoreGrade 5 Mathematics
Demographics
Aggregated Growth at the School Level
• Remember the 4 quadrants– High vs. Low Achievement– Based on a Scale Score– High vs. Typical vs. Low Growth– Based on Student Growth Percentiles
• An example of a school level report
Moving on to school
• Look at each grade distribution of SGPs• Take the median of a median
– 4th grade– 5th grade– 6th grade
Grade Percentile
4th 35th
5th 40th
6th 37th
School Growth for MATH 37th
How it fits in with the Evaluation System
• 5% of the total evaluation weight at the school level
• School level growth? – Lowest Growth (1-24th percentile)– Lower Growth (25th – 34th percentile)– Typical Growth (35th – 65th percentile)– Higher Growth (66th – 99th percentile)
Percentile Bands within the Evaluation System
Lowest Growth
Low Growth
Typical Growth
High Growth
Unsatisfactory
Emerging
Accomplished
Distinguished
Why the ranges?
• Empirical consideration of current data
• Proposed percentile bands for first pilot year
• Open for revision• Aligns with 4 point rubrics
Questions?
Student Learning in Context
Student Learning in Context
• Learning Targets:– Know the definition of student learning;– Identify examples and non-examples; – Evaluate the quality of potential
measures
Wrap Up & Feedback
A New Landscape
Some Parting Instructions
• Feedback Forms• Principals and Assistant Principals
– Part Two• Dine well and rest well!• See you tomorrow at 8:00 a.m.