new york state’s teacher and principal evaluation system
DESCRIPTION
New York State’s Teacher and Principal Evaluation System. APPR Conference April 30 – May 1, 2012. 3 Initiatives. Conference Objectives. Understand in detail the component pieces of the APPR plan for teachers and principals (requirements and options). - PowerPoint PPT PresentationTRANSCRIPT
www.engageNY.org
New York State’sTeacher and Principal
Evaluation System
APPR ConferenceApril 30 – May 1, 2012
www.engageNY.org
3 Initiatives
2
www.engageNY.org
Conference Objectives• Understand in detail the component pieces of
the APPR plan for teachers and principals (requirements and options).
• Understand how to structure APPRs to support district/BOCES’ academic priorities and needs.
• Understand what makes a rigorous APPR plan that meets requirements of law and regulations.
• Understand how to use resources from NYSED to help districts/BOCES and constituencies shape sound APPR plans.
3
www.engageNY.org
Agenda Day 1 and 2Agenda Day 1:• Introduction to
Review Room• District Case
Study• State Growth,
Locally-Selected Measures, and Other Measures
• HEDI and Points
Agenda Day 2:• District Case
Study Re-Visited • Panel Discussion• Q & A with the
Commissioner
4
www.engageNY.org
Key Tools and Resources When Developing APPR Plans
• The Commissioner’s APPR form and instructions• All of the following resources related to APPR plans are located on EngageNY.org:
•Summary of regulations (AKA “the purple memo”)
•APPR Guidance •TLE Roadmaps•SLO Guidance, Exemplars, and Webinars•Materials from NTI trainings
5
www.engageNY.org
Workshop Model• Mini lesson• Application through case study• Codifying the learning through the APPR
form• Assessing the learning through HEDI• Sharing in the epiphanies• Q and A
6
www.engageNY.org
Expectations….
Your RoleMaking and/or influencing key
decisions in support of
APPR and its principles.
7
www.engageNY.org
Summary of APPR Components
(20% 25%)
(20% 15%)
(60%)
State-provided Growth/VA
Assessments and Measures
• Rubrics• Sources of evidence: observations, visits, surveys, etc
Subcomponents, Composite Scores, Ratings
Improvement Plans, Appeals, Training
Growth
Locally Selected
Measures
OtherMeasures
Scoring
Imple-mentation
Student Learning Objectives
8
www.engageNY.org 9
Teacher/Principal Evaluation Formula
www.engageNY.org 10
Let’s Look at Review Room
www.engageNY.org
NYSED APPR FormDuring the
conference, please remember to use the conference log-in ID
and password provided by NYSED
and not your official district log-in!
11
www.engageNY.org
NYSED APPR FormKeep in Mind: the
“Resource Tab” is where you will find helpful
items such as the State-approved list of 3rd party
assessments
These are the file types that
can be uploaded to
Review Room
12
www.engageNY.org
NYSED APPR Form
All pages include directions for
completing the page as well as
references to the specific sections of Guidance that
are most relevant.
13
www.engageNY.org
NYSED APPR Form
Some pages have dropdown boxes. Where there are dropdown boxes, you often will see a blank box next to it, like on this
page. Here you are asked to actually name the specific assessment. You would type that
into the blank box.
14
www.engageNY.org
NYSED APPR Form
Don’t get confused!
The dropdown boxes include
an abbreviated list of what is
above.
15
www.engageNY.org
Case study review
Applying the Learning
16
www.engageNY.org 17
Teacher/Principal Evaluation Formula
www.engageNY.org
• 60 Point Other Measures Overview•Regulations•Review Room Application
• HEDI Criteria for 60 Point Other Measures•Case Study Discussion•Review Room Application
Agenda
18
www.engageNY.org
Where can I find…..?
Where Can I Find Further Resources, Guidance, and Answers to My Questions Related to 60 Point Other Measures?•Section H of NYSED’s APPR Guidance•Some of Section I of NYSED’s APPR Guidance•Section J of NYSED’s APPR Guidance•Page 4 of the “Purple Memo”•Step 1 and 2 of the Teacher Road Map•Step 1 and 2 of the Principal Road Map
19
www.engageNY.org
60 Point Other MeasuresTeachers and Principals
Keep in Mind: Multiples measures
must be used in this subcomponent.
Keep in Mind:Measures, HEDI criteria, and the scoring bands for this subcomponent
must be locally-established through
negotiations.
20
www.engageNY.org
Teachers Principals
• NY State Teaching Standards: choice of rubric from State-approved list or variance, if approved by NYSED
Multiple Measures
• At least a majority (31) of the 60 points must be based on multiple classroom observations (at least 2) by principal or other trained administrator:
- At least one must be unannounced - May be conducted using video or
in-person
• Any remaining standards not addressed in classroom observation must be assessed at least once a year
• ISLLC 2008 Leadership Standards: choice of rubric from State-approved list or variance, if approved by NYSED
Multiple measures
• At least a majority (31) of the 60 points must be based on broad assessment of principal leadership and management actions by the supervisor based on the practice rubric:
-Must incorporate multiple visits by the supervisor, trained administrator, or a trained independent evaluator -At least one visit must be from a supervisor, and at least one visit must be unannounced
• Any remaining leadership standards not addressed through above requirements must be assessed at least once a year
60 Point Other Measures
21
www.engageNY.org
Teachers PrincipalsIn addition to classroom observations, remaining points (if any) must be based on:
1. Observation(s) by trained independent evaluators2. Observation(s) by trained in-school peer teachers3. Feedback from students and/or parents using a State-approved survey tool4. Structured review of lesson plans, student portfolios, and/or other teacher artifacts
In addition to broad leadership assessment, remaining points (if any) must be based on:
Results of one or more ambitious and measurable goal(s) set collaboratively with supervisors:
At least one goal must address the principal’s contribution to improving teacher effectiveness based on:
1. Improved retention of high performing teachers; 2. Correlation of student growth scores to teachers
granted vs. denied tenure; 3. Improvements in proficiency rating of the
principal on specific teacher effectiveness standards in the principal practice rubric.
Any other goals shall address quantifiable and verifiable improvements in academic results or the school’s learning environment.Goals must include at least two other sources of evidence:1.Structured feedback from teachers, students, and/or families using a State-approved tool (each constituency is one source);2.School visits by trained evaluators;3.Review of school documents, records, and/or State accountability processes (all documents are one source).
Other Measures: Remaining Points
22
www.engageNY.org 23
Working with 60 Point Other Measures
in Review Room
www.engageNY.org
Review Room Application: Teachers Keep in Mind:
the dropdown menu has the full list of State-approved
rubrics. There is also an option for “district variance” that can be selected if you are a
district who has already received a NYSED approved
variance.
24
www.engageNY.org
Review Room Application: Teachers
Keep in Mind: the number of points must add up to 60 for the Other Measures subcomponent. A
minimum of 31 points must be allocated to
classroom observations/broad
assessment of principal leadership and management
actions.
25
www.engageNY.org
Review Room Application: Teachers
Keep in Mind: survey tools must be
selected from NYSED’s State-
approved list, which will be available in
June.
26
www.engageNY.org
Review Room Application: Teachers
Keep in Mind: districts will need to
complete the form with specific information on
the observation requirements for probationary and tenured teachers.
Specifics regarding how many formal/long
versus informal/short must be noted.
27
www.engageNY.org
Review Room Application: Principals Keep in Mind: if any
points are assigned to goals, the first goal must be related to improving teacher
effectiveness. In the form, notice that
districts must also check the boxes to
identify which two (or more) of the options
listed will be utilized as part of assessing the
goals.
28
www.engageNY.org
Review Room Application: Principals
Keep in Mind: for principals, districts must enter
the specific information on the number of school visits that will be
conducted for principals in the building. Districts must complete this for probationary and tenured
principals.
29
www.engageNY.org
Review Room Application
1. Please take out the NYPSD case study and review the choices the district has negotiated for their 60 Point Other Measures subcomponent.
2. When you are ready, go into Review Room and find the Other Measures of Effectiveness – Teachers section.
3. Enter the choices the district has made for their Other Measures.
4. Do not worry about entering the HEDI criteria and scoring bands yet.
5. If you finish early, please move on to the Other Measures of Effectiveness – Principals section and complete the choices the district has made.
30
www.engageNY.org
HEDI Criteria for 60 Point Other
Measures
31
www.engageNY.org
Determining HEDI Criteria60 Point Other Measures
Standards for Rating
CategoriesOther Measures of Effectiveness(Teacher and Leader Standards)
Highly Effective
Overall performance and results exceed standards.
EffectiveOverall performance and results meet standards.
DevelopingOverall performance and results need improvement in order to meet standards.
IneffectiveOverall performance and results do not meet standards.
32
www.engageNY.org
Keep in Mind…
• HEDI for 60% must be negotiated: scoring bands and the process for assigning points.
• HEDI for 60% must assure it is possible to use all points (including 0) in the subcomponent and rating categories.
• Districts will need to determine how rubric scores translate into HEDI categories and within categories, into specific point awards.
• For example, if an educator earns a rubric score at the bottom of your Developing rubric range, then the educator should get HEDI points at the bottom of your developing point range.
33
www.engageNY.org
One Example of a HEDI Approach…for 60 Point Other Measures
Overall Rubric Score(Must be negotiated) Rating Category
0-60 point distribution by rating category (must be
negotiated)
1 - 1.8 Ineffective 0-491.9 - 2.8 Developing 50-562.9 - 3.6 Effective 57-583.7 - 4.0 Highly Effective 59-60
•The district negotiates procedures for conducting and scoring classroom observations and assessing other aspects of the rubric.
•The district negotiates the level of performance against the rubric for each HEDI category.
•Based on all the evidence gathered, a “rubric score” and its corresponding HEDI rating category is determined for each teacher/principal.
•The rubric score is then converted into a score on a scale of 0-60 according to the 60 point scoring bands negotiated by the district.
The chart below illustrates one potential result:
34
www.engageNY.org
APPR Form Requirements…for HEDI Criteria
Keep in Mind: Districts must
provide the locally-negotiated scoring
bands and then describe the level of
performance for each HEDI rating category.
Districts can also describe how scores from the rubric are converted into HEDI
and points.35
www.engageNY.org
Case DiscussionPlease take out the NYPSD case study and review the 60% Other Measures section. Then, discuss the following with your table groups:1.In this case, the district has agreed to prioritize certain key elements of their principal practice rubric in the 2012-13 school year, choosing some based on their relevance to district priorities and needs and weighting these proportionately more than others. Is this allowable? 2.If the district decided to prioritize certain key elements of the teacher practice rubric and use meetings that occur every other month to provide the opportunity for teachers and their principals to discuss evidence and feedback on performance on standards not directly assessed by classroom observation, would this be allowable?3.The district is considering eliminating some indicators on the rubric for certain non-tested teachers. Is this allowable?4.In this case, are enough observations conducted for teachers and principals? Are there any requirements missing?5.The district is considering awarding points to teachers who effectively mentor student teachers or new colleagues. Is this allowable?
36
www.engageNY.org
Review Room Application
1. When you are ready, go into Review Room and find the Other Measures of Effectiveness – Teachers section.
2. Enter the HEDI criteria and points.3. If you finish early, please move on to
the 60% Other Measures– Principals section and complete the HEDI criteria and points.
37
www.engageNY.org 38
Teacher/Principal Evaluation Formula
www.engageNY.org
• State Growth Model Overview•Definitions and characteristics
• Student Learning Objectives•State versus district decisions•Rules and options•Review Room Application•Case Study Discussion
• HEDI Criteria for SLOs•Case Study Discussion•Review Room Application
Agenda
39
www.engageNY.org
Where can I find…..?
Where Can I Find Further Resources, Guidance, and Answers to My Questions Related to State Growth?•Section D of NYSED’s APPR Guidance•Section F of NYSED’s APPR Guidance•Some of Section I of NYSED’s APPR Guidance•Page 1 and 3 of the “purple memo”•Step 3 of the Teacher Road Map•Step 3 of the Principal Road Map
40
www.engageNY.org
State Growth: Teachers and Principals
Keep in Mind: Most teachers will have SLOs for the 2012-13 school
year.
Keep in Mind: Most principals
will be covered by State-provided
growth measures for the 2012-13
school year
41
www.engageNY.org
Achievement and Gains
2011 2012
Proficiency
Achievement models tell you who is above and below the proficiency cut
•Two of five kids here scored above proficiency
Gain score models tell us some students received higher scale scores the following year
•Three students had higher scores, one didn’t change, and one had a lower score
Neither tells us enough to say whether student growth was unusually strong, weak or average.
www.engageNY.org
2011 2012
Proficiency
In a growth model, we look at how all students with similar scores in one year (or several years) do when compared to each other
In this example, we take one student from the previous slide and see how all students with that score in 2011 performed in 2012.
This tells us whether the change in scores between two years is average or above or below average.
Above Average
Below Average
Average
NYS Growth Model
www.engageNY.org
Growth Model: Student Growth Percentiles (SGP) Defined
Student 2010 Score 2011 Score SGP 5 600 650 98
6 650 650 42
7 675 650 20
8 700 650 3
Student 2010 Score 2011 Score SGP 1 625 600 10
2 625 625 40
3 625 650 70
4 625 675 95
44
www.engageNY.org
Median Student Growth Percentile: Defined
Student SGP 1 102 403 704 955 986 427 208 39 37
Student SGP 8 31 107 209 372 406 423 704 955 98
Order by
SGP
The MGP is the median. This is the
result that will describe a class or a school’s result.
45
www.engageNY.org
Growth Measures: “Similar” StudentsFor NYS Growth Measures, “similar” students will include:
•Up to 3 years of past State assessment history as available for each student (must have current and 1 prior to be included)
•In 2011-12, Poverty, SWD, ELL characteristics•For value-added model in 2012-13 and beyond, other student, classroom and/or school characteristics may be included
46
www.engageNY.org
Growth Measures: PrincipalsElementary and Middle School Principals:
•Median Student Growth Percentile of all the tested students in the school
•Not the average of all teacher results
High School Principals in 2012-13:•Measure is in development•Based on student growth in Regents exams passed compared to similar students
47
www.engageNY.org
Next Steps/Timeline for Growth Model
This Spring/Summer: •Approach to determining teacher and principal HEDI levels
•Training/communications materials and full technical documentation
•Create and provide 11-12 teacher and principal growth scores (July)
Fall 2012: •Provide online reporting to teachers, school, districts (Sept)
•Value-added measures for teachers and principals presented to Task Force and the Board of Regents
48
www.engageNY.org
Linking Students, Teachers and Schools• SED is collecting data now to connect students
and teachers to courses:•Data required by Federal law (not just for NYS statute)
•Teachers participate in verifying student rosters •Enrollment, assignment dates also collected to support over time different duration of teacher-student “linkage” if empirically proven
• Districts, principals, and teachers play a pivotal role in ensuring high quality inputs for the State-provided growth/value-added measures.
49
www.engageNY.org
Overview of Teacher and Principal Evaluation Regulations
(20% 25%)
(20% 15%)
(60%)
State-provided Growth/VA
Assessments and Measures
• Rubrics• Sources of evidence: observations, visits, surveys, etc
Subcomponents, Composite Scores, Ratings
Improvement Plans, Appeals, Training
Growth
Locally Selected
Measures
OtherMeasures
Scoring
Imple-mentation
Student Learning Objectives
50
www.engageNY.org
SLO Resources from NYSED
Please visit: http://engageny.org/resource/student-learning-objectives/
51
www.engageNY.org
What Does the District Determine?
52 52
www.engageNY.org
Assessment Options for SLOsReference Guide
Please see the “Assessment Options for SLOs: Reference Guide” for NYSED’s rules for assessment
options for teachers who have SLOs for State Growth
53
www.engageNY.org
Required SLOsReference Guide
54
Please see the “Required SLOs: Reference Guide” for NYSED’s rules
for teachers who have SLOs for State Growth
54
www.engageNY.org
Review Room Application
• Please take out the NYPSD case study and review the assessment choices Dr. Baery has made for SLOs for State Growth.
• Before we begin the practice session, let’s orient ourselves to the tool.
55
www.engageNY.org
Review Room Application
Keep in Mind: for teachers with State-provided
measures of student growth, districts
must check boxes that list assurances.
For other comparable
measures, SLOs, districts must list their decisions.
56
www.engageNY.org
Review Room Application
Keep in Mind: the dropdown menu has an
abbreviated version of the full list above
it. After selecting an option from the dropdown menu,
please write in the specific assessment
option.
57
www.engageNY.org
Review Room Application
1. Please go ahead now into Review Room to the State Growth – Teachers section.
2. Enter the assessment choices the district has made for SLOs.
3. Do not worry about entering the HEDI criteria yet.
4. If you finish early, please move on to the State Growth – Principals section and complete the assessment choices the district has made.
58
www.engageNY.org
Case Study Discussion
1. When filling in the district’s assessment choices for SLOs, were there any grades/subjects that you noticed had unallowable options? How did the form help with this?
2. If the district decided to let teachers create their own assessments for evaluation purposes, would this be allowable?
59
www.engageNY.org
HEDI Criteria for SLOs in
State Growth
60
www.engageNY.org
HEDI for SLOs in State Growth
Standards for Rating Categories Growth or Comparable Measures
Highly Effective
Results are well-above state average for similar students (or District goals if no state test).
EffectiveResults meet state average for similar students (or District goals if no state test).
DevelopingResults are below state average for similar students (or District goals if no state test).
IneffectiveResults are well-below state average for similar students (or District goals if no state test).
What are “district goals” if there is no state test for the grade/subject?
61
www.engageNY.org
HEDI Scoring Bands: Growth MeasuresState Value-added & Comparable Growth SLOs
2012-13 Growth SubcomponentScoring Bands
Where value-added measures
apply
Comparable Growth Measures:
SLOs
Highly Effective22-25 18-20
Effective 10-21 9-17Developing 3-9 3-8Ineffective 0-2 0-2
Remember that points are different for teachers in grades/subjects with value-added measures and those without:
62
www.engageNY.org
APPR Form Requirements…for HEDI criteria
63
www.engageNY.org
Setting HEDI Criteria – Three ExamplesAfter considering previous student performance, normative data, 3rd party data reports, district thresholds, district values/ priorities, districts have choices:
Choice One:Set specific growth expectations by grade/subject (for all or some
grades/subjects
Choice Two: Set generic growth expectations for students across grades/subjects
Choice Three: Set generic expectations for students meeting their individualized
growth expectations across grades and subject
64
www.engageNY.org
Example of Choice One: Setting Specific Growth Expectations by Grade/SubjectDistrict specifies that for grades 6 and 7 Science teachers, a State-approved 3rd party science assessment will be used as evidence of student learning for SLOs.
What Student Progress Meets District Expectations
Highly Effective 18-20 points
Growth exceeds 3rd party assessment benchmark
Effective9-17 points
Growth is equal to 3rd party assessment national benchmark for average growth compared to similar students. Specific points assigned based on place in range of “average”
Developing3-8 points
Growth below 3rd party assessment benchmark
Ineffective0-2 points
Growth significantly below 3rd party assessment benchmark
65
www.engageNY.org
APPR Form Requirements…for HEDI criteria
66
www.engageNY.org
Districts may decide there are certain levels of growth that meet/do not meet district expectations based on student’s baseline level of performance. In this example, multiple grades/subjects can utilize performance levels from 1-4 where 3 is on grade level/proficient like NYSED State tests. Districts will need to determine HEDI criteria.
What Student Progress Meets District
Expectations Performance Level
END: 1 END: 2 END: 3 END: 4
START: 1 NO YES YES YESSTART: 2 NO NO YES YESSTART: 3 NO NO YES YESSTART: 4 NO NO NO YES
Target is what % of students make their specific level of acceptable growth or better.
Example of Choice 2: Generic Student Growth Expectations Across Grades/Subjects
RatingPoints
Ineffective0-2 points
Developing3-8 points
Effective9-17 points
Highly Effective18-20 points
Percentage of students whose progress meets
expectations 0-29% 30-54% 55-79% 80%+
www.engageNY.org
APPR Form Requirements… for HEDI criteria
68
www.engageNY.org
Example of Choice 3: Generic Expectations for Student SLO Target Achievement
Highly Effective
18-20 points
Effective9-17
Developing3-8
Ineffective0-2
The work of the teacher results in exceptional student academic growth beyond expectations during the school year. 90%+ of students met or exceeded the Student Learning Objective.
The work of the teacher results in acceptable, measurable, and appropriate student academic growth.
80-89% of students met or exceeded the Student Learning Objective.
The work of the teacher results in student academic growth that does not meet the established standard and/or is not achieved with all populations taught by the teacher. 50-79% of students met or exceeded the Student Learning Objective.
The work of the teacher does not result in acceptable student academic growth.
Fewer than 50% of students met or exceed the Student Learning Objective.
69
www.engageNY.org
APPR Form Requirements…for HEDI criteria
70
www.engageNY.org
Case Study DiscussionPlease take out the NYPSD case study and review the HEDI criteria for State Growth that Dr. Baery has determined for SLOs for State Growth. Then, discuss the following with your table groups:1.In this case, the district has adopted the committee’s recommendations for generic growth expectations for all grades and subjects with the bar set at 80% for the percent of students who must meet their SLO targets in order for the teacher to receive an Effective rating. The district has also said that principals will set the targets with teachers. Is this all allowable?2.If the district decided to set expectations for some grades and subjects to ensure that all students will demonstrate at least one grade level growth if starting on grade level, and all students starting below grade level will demonstrate at least 2 years grade level growth, would this be allowable?3.If the district decided that for all SLOs for teachers, Highly Effective is well-above district goals; Effective equals district goals; Developing is below district goals; Ineffective is well-below district goals, would this be allowable? 71
www.engageNY.org
Review Room Application
1. When you are ready, go into Review Room and find the State Growth – Teachers section.
2. Enter the HEDI Criteria for SLOs.3. If you finish early, please move on to
the State Growth – Principals section and complete the HEDI Criteria for SLOs.
72
www.engageNY.org
END OF DAY 1
73
www.engageNY.org 74
Teacher/Principal Evaluation Formula
www.engageNY.org
• Locally-Selected Measures Overview•Regulations•Review Room Application•Case Study Discussion
• HEDI Criteria for Locally-Selected Measures•Case Study Discussion•Review Room Application
Agenda
75
www.engageNY.org
Where Can I Find…?
Where Can I Find Further Resources, Guidance, and Answers to My Questions Related to Locally-Selected Measures?
• Section E of NYSED’s APPR Guidance• Section F of NYSED’s APPR Guidance• Some of Section I of NYSED’s APPR
Guidance• Page 2 and 3 of the “Purple Memo”• Step 4 of the Teacher Road Map• Step 4 of the Principal Road Map
76
www.engageNY.org
Locally-Selected MeasuresTeachers and Principals
Keep in Mind: Locally-selected measures can
measure growth or achievement
Keep in Mind:Locally-selected measures count for 20 points (15 points with an
approved value-added measure)
77
www.engageNY.org
Locally-Selected Measures: Overview• Growth and local measures must be
different from one another• To ensure comparability, select the same
measure across all classrooms in the same grade/subject and/or for all principals in same or similar programs/buildings
• The State-approved list meets prescribed criteria for comparability and rigor; districts/BOCES who develop assessments will need to verify comparability and rigor
• Collective bargaining considerations
78
www.engageNY.org
RigorousRigorous means that the locally-selected measure is:
•Aligned to the NYS learning standards •To the extent practicable, the assessment must be valid and reliable as defined by the standards for Educational and Psychological Testing
79
www.engageNY.org
Comparable
80
Locally-comparable across classrooms means:•The same locally-selected measures of student achievement or growth are used across all classrooms in the same grade/subject in the district or BOCES. •A district may use more than one type of locally-selected measure for different groups of teachers within a grade/subject if the district/BOCES verifies comparability in accordance with the standards of Educational and Psychological testing. •For principals, the same locally-selected measure(s) must be used for all principals in the same or similar program or grade configuration in that school district or BOCES.
www.engageNY.org
Measures for Teachers Using State/Regents AssessmentsMeasures based on several options:
1. State assessments, Regents, examination, and/or Regent-equivalents. These include:
a. The change in percentage of a teacher’s students who achieve a specific level of performance as determined locally, on such assessments/examinations in the previous year.
b. Teacher-specific growth computed by NYSED based on % of the teacher’s students earning a State-determined level of growth. Methodology to translate such growth into State-established sub-component scoring ranges shall be determined locally.
c. Other teacher-specific growth or achievement measure using State, Regents, and/or department approved alternative examinations computed in a manner determined locally.
81
www.engageNY.org
Measures for Teachers Using All Other Options2. State-approved list of 3rd party assessments3. District, regional, or BOCES-developed assessment4. School-wide growth or achievement results based on:
a. State-provided school-wide growth score for all students taking State ELA or Math assessments in grades 4-8
b. Locally-computed measure based on State, State approved 3rd party, or a district, regional, or BOCES- developed assessment
5. SLOs used with any of the following (option is only for teachers without a State-approved Growth or Value-Added measure for Growth subcomponent):
– Any State, approved 3rd party, or district/ regional/ BOCES- developed assessment
82
www.engageNY.org
Measures for Principals of Elementary and/or Middle Schools1. Achievement levels on State assessments (% proficient or
advanced) in ELA and Math Grades 4-8.2. Growth or achievement for student subgroups (SWD, ELL) on
State assessments in ELA and Math Grades 4-8.3. Growth or achievement of students in ELA and Math (Grades
4-8) starting at specific performance levels (e.g., Level 1, Level 2) on State or other assessments.
4. SLOs used with any of the following (option is only for principals without a State-approved Growth or Value-Added measure for Growth subcomponent):
– Any State, approved 3rd party, or district/ regional/ BOCES- developed assessment that is rigorous and comparable across classrooms.
5. Student performance on any district-wide locally-selected assessments approved for use in teacher evaluations.
83
www.engageNY.org
Measures for Principals of High Schools1. Percent of cohort achieving specified scores on Regents exams,
AP, IB, or other Regents-equivalents.2. Graduation rates (4, 5, 6 years) and/or dropout rates.3. Graduation % with Regents Diploma with Advanced Designation
and/or with Advanced Designation with honors.4. Credit accumulation (e.g., 9th and 10th grade) or other strong
predictor of progress toward graduation.5. SLOs used with any of the following (option is only for
principals without a State-approved Growth or Value-Added measure for Growth subcomponent):
– Any State, approved 3rd party, or district/ regional/ BOCES- developed assessment that is rigorous and comparable across classrooms.
6. Student performance on any district-wide locally-selected assessments approved for use in teacher evaluations.
84
www.engageNY.org
Review Room Application
• Please take out the NYPSD case study and review the assessment choices the district has negotiated for their locally-selected measures.
• Before we begin the practice session, let’s just re-orient ourselves to Review Room together.
85
www.engageNY.org
Review Room Application
Keep in Mind: the full list of options are listed first, then
the dropdown boxes. For teachers in
courses with a State-provided growth or
value-added measure, SLOs are
not an option so they are not listed here.
86
www.engageNY.org
Review Room ApplicationKeep in Mind: the dropdown menu has an
abbreviated version of the full list above
it. After selecting an option from the dropdown menu,
please write in the specific assessment
option.
87
www.engageNY.org
Review Room Application
1. When you are ready, go into Review Room and find the Locally-Selected – Teachers section.
2. Enter the assessment choices the district has made for Locally-Selected.
3. Do not worry about entering the HEDI criteria yet.
4. If you finish early, please move on to the Locally-Selected – Principals section and complete the assessment choices the district has made.
88
www.engageNY.org
Case Study Discussion
Please take out the NYPSD case study and review the locally-selected measures that the district plans to use. Then, discuss the following with your table groups:1.The district is interested in the potential of using a national cosmetology assessment as a locally-selected measure, is this allowable?2.The district is interested in using SLOs for additional grades/subjects. Which grades/subjects are not allowed to use SLOs for locally-selected measures?
89
www.engageNY.org
HEDI Criteria for Locally-Selected
Measures
90
www.engageNY.org
Determining HEDI Criteria: Locally-Selected Measures
Standards for Rating
CategoriesLocally-selected Measures of
growth or achievement
Highly Effective
Results are well-above District or BOCES -adopted expectations for growth or achievement for grade/subject.
EffectiveResults meet District or BOCES-adopted expectations for growth or achievement for grade/subject.
DevelopingResults are below District or BOCES-adopted expectations for growth or achievement for grade/subject.
IneffectiveResults are well-below District or BOCES-adopted expectations for growth or achievement for grade/subject.
91
www.engageNY.org
HEDI for Locally-Selected Measures
• Must be collectively bargained.• Must describe a district-adopted level
of expectation for every grade/subject although can use generic expectations.
• Expectations can be based on either growth or achievement of students.
92
www.engageNY.org
Setting HEDI Criteria – Three ExamplesAfter considering previous student performance, normative data, 3rd party data reports, district thresholds, district values/ priorities, districts have choices:
Choice One: Set specific growth OR ACHIEVEMENT expectations by
grade/subject (for all or some grades/subjects)
Choice Two: Set generic growth OR ACHIEVEMENT expectations for students
across grades/subjects
Choice Three: Set generic expectations for students meeting their individualized
growth OR ACHIEVEMENT expectations across grades and subject
93
www.engageNY.org
Examples of Different Approaches to Setting District Expectations
GROWTH: change in student results
between two points in time
ACHIEVEMENT: student results at end of
yearLevel of growth over baseline (e.g., 20 percentage points growth)
Achievement level (e.g., score 85 out of 100, Level 3 out of 4)
Level of growth required given starting point to be on track
Achieve proficiency (or achieve advanced level)
Growth vs. a benchmark (State average growth, district average growth, vendor-provided benchmark)
Achievement versus a benchmark (State or district average achievement, vendor-provided benchmark)
Subgroup growth (lowest or highest achieving students; SWDs; ELLs)
Subgroup achievement
94
www.engageNY.org
HEDI Points…for Locally-Selected Measures
Locally-Selected Measures of Growth or Achievement:
Where state-provided value-added measure
applies for Growth
Where NO state-provided value-added measure
applies for Growth
Highly Effective14-15 18-20
Effective 8-13 9-17Developing 3-7 3-8Ineffective 0-2 0-2
Remember that points are different for teachers in grades/subjects with value-added measures and those without:
95
www.engageNY.org
APPR Form Requirements for HEDI criteria Keep in Mind:
Districts may decide to leave all HEDI criteria up to
the school level and/or to be
prescriptive with some
grades/subjects, but not with
others.
96
www.engageNY.org
Adjustment Factors
Keep in Mind: if any adjustments
or controls are going to be used, districts must not
only describe them in their APPR form submission, but
make a number of assurances
regarding their usage.
97
www.engageNY.org
Case Discussion
• Please take out the NYPSD case study. The district has not negotiated any adjustment factors for their locally-selected measures subcomponent for teachers.
• Are there any adjustment factors that would be allowable?
98
www.engageNY.org
Examples of Locally-Selected Measures from the Case: Teachers• The district has prioritized expository writing and STEM, and
has identified a critical need to focus academic interventions in these areas.
• When negotiating their APPR, the district and their collective bargaining units determined that the following measures would be used:
4th Grade Common Branch Teachers
7th Grade Science Teachers
School Librarians
% of 4th grade students earning at the proficient level (three) or better performance level on the 4th grade State Science assessment
% of 7th grade science students who demonstrate growth of at least one level higher than their baseline analytical writing sample, as measured by the district-developed 7th grade science writing assessment rubric.
School-wide growth based on State-provided school-wide growth scores for all students in the school taking the State ELA assessment in grades 4-8.
99
www.engageNY.org
Examples of Locally-Selected Measures from the Case: Principals
• The district has prioritized expository writing and STEM, and has identified a critical need to focus academic interventions in these areas.
• When negotiating their APPR, the district and their collective bargaining units determined that the following measures would be used:K-5 Principals 6-8 Principals 9-12 Principals
• % of students who achieve proficiency from those who scores a Level 1 or 2 on the prior year State assessment
• % of 4th grade science students who achieve proficient or higher on State science assessment
• Growth of ELL students on 6-8 ELA and Math State assessments. No students decrease if 3s or 4s, at least 40% increase one level if 3s or below, at least 75% increase if 1s.
• % of 8th grade science students who achieve proficient or higher on State science assessment
• % of students who score a 75 on ELA Regents and 80 on Math Regents increasing by at least 7%.
100
www.engageNY.org
Case Study DiscussionPlease take out the NYPSD case study and review the HEDI criteria for locally-selected measures that the district plans to use. Then, discuss the following with your table groups:1.In this case, the district and their collective bargaining units agreed that teachers/principals will use one generic HEDI criteria with all district-developed assessments across applicable grades/subjects within the district. Targets will be set based on what percentage of students make their specific level of acceptable growth or better. Also, the district will audit a random sampling of SLOs developed by teachers and principals in November to ensure rigor and comparability. Is this allowable?2.In this case, the district and their collective bargaining units agreed to use a local measure for principals based on the percentage of students in 4th and 8th grade Science who earn the proficient (level 3) or higher. Is this allowable?3.Is there any missing information that you notice from this section of the case? Is it allowable to submit an APPR form that is incomplete?
101
www.engageNY.org
Review Room Application
1. When you are ready, go into Review Room and find the Locally-Selected Measures – Teachers section.
2. Enter the HEDI Criteria for locally-selected measures.
3. If you finish early, please move on to the Locally-Selected Measures – Principals section and complete the HEDI Criteria for locally-selected measures.
102
www.engageNY.org
Putting it All Together:
Composite Scores
103
www.engageNY.org
HEDI Scoring Bands: putting it together
2012-13 where Value-Added growth measure applies
Growth or Comparable Measures
Locally-selected
Measures ofgrowth or
achievement
Other Measures
of Effectivene
ss(60 points)
OverallComposite Score
Highly Effective 22-25 14-15
Ranges determined locally
91-100Effective 10-21 8-13 75-90Developing 3-9 3-7 65-74Ineffective 0-2 0-2 0-64
2012-13 where there is no Value-Added measure
Growth or Comparabl
e Measures
Locally-selected
Measures ofgrowth or
achievement
Other Measures
of Effectivene
ss(60 points)
OverallComposite Score
Highly Effective 18-20 18-20
Ranges determined locally
91-100Effective 9-17 9-17 75-90Developing 3-8 3-8 65-74Ineffective 0-2 0-2 0-64
104
www.engageNY.org
Review Room Application
Keep in Mind: Districts must
enter the same scoring ranges for the 60 point other measures
here in the “Composite
Scoring” section of the form as they did in the
“Other Measures”
section.105
www.engageNY.org
Review Room Application
1. When you are ready, go into Review Room and find the Composite Scoring – Teachers section.
2. Enter the scoring bands for 60% Other.
3. If you finish early, please move on to the Composite Scoring – Principals section and enter the scoring bands for 60% Other.
106