the use of student assessments in teacher evaluations through the next decade national conference on...
TRANSCRIPT
The Use of Student Assessments in Teacher Evaluations Through the Next DecadeNational Conference on Student AssessmentJune 27, 2012Minneapolis
edc.org
Agenda
• Introduction: Julie Kochanek• Study profile: Julie Riordan
– Performance-based teacher evaluation systems in five states
• Study profile: Karen Shakman– Changing culture and building capacity: An exploration of district
strategies for implementation of teacher evaluation systems
• District profile: David Heistad– The use of value-added in Minneapolis public schools
• Q & A discussion: Julie Kochanek
04/19/23 2
edc.org
An Examination of Performance-based Teacher Evaluation Systems
Karen ShakmanJulie Riordan
Maria-Teresa SanchezKyle DeMeo CookRichard Fournier
Jessica Brett
04/19/23 3
edc.org 04/19/23 4
Rationale for Study
• Regional need• Recent research• Race to the Top• State policy context
edc.org
Selection Criteria
• Required for all practicing general educators• Operational statewide in 2010–11 school year• Multiple rating categories• Multiple measures of teacher effectiveness
04/19/23 5
edc.org
Research Questions
• What are the key characteristics of state-level performance-based teacher evaluation systems in the study states?
• How do state teacher evaluation measures, the teaching standards the evaluations are designed to measure, and rating categories differ across states that have implemented statewide systems?
04/19/23 6
edc.org
Approach
• Examined 50 SEA websites• Conducted additional online search• Reviewed evaluation guides/manuals, rubrics,
training materials, regulations, program reports• Contacted state directors
04/19/23 7
edc.org
States Included in Study
• Delaware (RTT Phase 1)
• Georgia (RTT Phase 2)
• North Carolina (RTT Phase 2)
• Tennessee (RTT Phase 1)
• Texas (did not apply)
04/19/23 8
edc.org
Evaluation Measures
• Classroom observations• Principal evaluations• Analysis of classroom artifacts• Analysis of teaching portfolios• Teacher self-reports of practice• Student ratings of teacher performance• Value-added (student growth) strategies
(Goe, Bell, and Little, 2008)
04/19/23 9
edc.org
Findings: RQ1 – Key Characteristics
04/19/23 10
edc.org
Findings: RQ 2 – Teaching Standards
04/19/23 11
edc.org
Findings: RQ 2- Rating Categories
04/19/23 12
edc.org 04/19/23 13
Future Directions
• Further investigation about specific nature of the measures in place, including how student data are used in the overall assessment of teachers • Identify the types of knowledge and skills that are
commonly observed and evaluated in performance-based systems • Investigate the fidelity of implementation of these
systems
edc.org
Changing Culture and Building Capacity:
Karen ShakmanNicole BreslowJulie KochanekJulie RiordanTom Haeford
04/19/23 14
An Exploration of District Strategies for Implementation of Teacher Evaluation Systems
edc.org
Context
• National attention on teacher evaluation
• Focus on districts
• Systems change as framework
04/19/23 15
edc.org
Research Questions
1. What challenges are districts facing as they design and implement new teacher evaluation systems?
1. What strategies have districts developed to address these challenges?
04/19/23 16
edc.org
Approach
• Conducted semi-structured interviews with 16 district representatives
• Employed an emergent code strategy to transcripts and interview notes to identify key themes
04/19/23 17
edc.org
Findings: 2 Major Challenges
1. Changing district and school level culture
1. Building district and school level capacity
04/19/23 18
edc.org
Findings: Strategies for Changing Culture
1. Focus on effective teaching
2. Engage stakeholders in the design process
3. Involve the teachers union
4. Align professional development
5. Establish differentiated and direct communication
strategies
04/19/23 19
edc.org
Findings: Strategies for Building Capacity
1. Invest in training evaluators and teachers
2. Establish district-level, cross-functional teams
3. Identify new roles and fill gaps in expertise
4. Sequence implementation strategies
04/19/23 20
edc.org
Remaining Challenges
• Redefined role for principals
• Use of student achievement measures
• Lack of alignment between local and state level reform efforts
04/19/23 21
edc.org 04/19/23 22
Future Directions
• Research–Implementation studies–Measurement studies–Impact studies
•Building a community of practice
edc.org 04/19/23 23
edc.org
History of Value-Added Use in Minneapolis Public Schools
• Evaluation of the Public School Academy (1992-94) Longitudinal Achievement Effects – Rob Meyer and U of W
• Teachers who Beat the Odds in 2nd Grade Reading (1997-99)• Quality Performance Awards (1995–2005) Multiple Measures with
School Awards • Teacher Advancement Program (TAP) School, Grade level and
individual teacher bonuses (2006-2010)• Teachers who Beat the Odds in Kindergarten Literacy (2006-2010)• Grade level value-added to all K-8 Principals (2010-11) – Bush
Foundation and VARC• Classroom level value-added to all K-8 Teachers (2012-13) – VARC
and Dr. Chris Moore
edc.org
Predictive Validity of the Minneapolis Public Schools Beginning of Kindergarten Assessment (BKA)
25
Measures (MPS Beginning of Kindergarten Assessment)N= 1674
9 months later (EKA Total Literacy)
1.75 years later (Gr. 1 Oral Reading)
2.75 years later (Gr. 2 MAP Reading)
3.75 years later (Gr. 3 MCA Reading)
Alphabetic Principle - letter names - letter sounds
.74
.74
.66
.71
.71
.60
.64
.64
.54
.58
.52
.50Phonological Awareness - initial sounds - rhyming
.70
.63.65
.60
.58
.53
.63
.57
.59
.63
.57
.58Vocabulary - picture naming .61 .47 .54 .52Listening Comprehension .63 .52 .65 .67
Concepts of Print .55 .49 .55 .57
Total Literacy .81 .74 .70 .65
edc.org
Value-added Kindergarten Teacher Effects “beat the odds teachers”
• Post-test reading score=– Pretest literacy score– + Free or reduced price lunch– + Racial/ethnic code (4 dummy codes)– + Gender – + English Language Learner status– + Special Education status– + Lives with single parent– + Teacher effects
edc.org
Follow-up with teachers who beat the odds
• Surveys, observations, video tape• These teachers can be leads in staff development,
mentors, etc.• What about inviting these teachers to work in
schools with the high concentrations of poverty and students of color?
• Videos at: http://rea.mpls.k12.mn.us/uploads/teachers_who_beat_the_odds.pdf
edc.org
Current Work with the Bush Foundation and Value-added Research Center (VARC)
1) Teacher level value-added rolled out K-8 in 2012-13 (Rubric Scores and Value-added)
2) Graduates from 14 institutions tracked into MPS system for feedback to first year teachers and the Institutions of Higher Education (IHEs)
3) Studies of the human capital teacher pipeline
04/19/23Presentation Title Appears Here 28
edc.org
For more information….
• [email protected]• [email protected] • [email protected]• [email protected]
04/19/23Presentation Title Appears Here 29