esea flexibility peer panel notes - ed · esea flexibility peer panel notes state ... esea...

ESEA Flexibility

Peer Panel Notes

State Request: Nevada Date: March 29, 2012

ESEA FLEXIBI LITY – REQ UEST RE VIEW FORM U .S . DEPARTMENT OF E DUCATION

1

REVIEW AND EVALUATION OF REQUESTS

The U.S. Department of Education (Department) will use a review process that will include both external peer reviewers and staff reviewers to evaluate State educational agency (SEA) requests for this flexibility. This review process will help ensure that each request for this flexibility approved by the Department is consistent with the principles, which are designed to support State efforts to improve student academic achievement and increase the quality of instruction, and is both educationally and technically sound. Reviewers will evaluate whether and how each request for this flexibility will support a comprehensive and coherent set of improvements in the areas of standards and assessments, accountability, and teacher and principal effectiveness that will lead to improved student outcomes. Each SEA will have an opportunity, if necessary, to clarify its plans for peer and staff reviewers and to answer any questions reviewers may have during the on-site review. The peer reviewers will then provide comments to the Department. Taking those comments into consideration, the Secretary will make a decision regarding each SEA’s request for this flexibility. If an SEA’s request for this flexibility is not granted, reviewers and the Department will provide feedback to the SEA about the components of the SEA’s request that need additional development in order for the request to be approved. This document provides guidance for peer review panels as they evaluate each request during the on-site peer review portion of the review process. The document includes the specific information that a request must include and questions to guide reviewers as they evaluate each request. Questions that have numbers or letters represent required elements. The italicized questions reflect inquiries that reviewers will use to fully consider all aspects of an SEA’s plan for meeting each principle, but do not represent required elements. In addition to this guidance, reviewers will also use the document titled ESEA Flexibility, including the definitions and timelines, when reviewing each SEA’s request. As used in the request form and this guidance, the following terms have the definitions set forth in the document titled ESEA Flexibility: (1) college- and career-ready standards, (2) focus school, (3) high-quality assessment, (4) priority school, (5) reward school, (6) standards that are common to a significant number of States, (7) State network of institutions of higher education, (8) student growth, and (9) turnaround principles.


2

Review Guidance

Consultation

1. Did the SEA meaningfully engage and solicit input on its request from teachers and their representatives?

Is the engagement likely to lead to successful implementation of the SEA’s request due to the input and commitment of teachers and their representatives at the outset of the planning and implementation process?

Did the SEA indicate that it modified any aspect of its request based on input from teachers and their representatives?

Consultation Question 1 Panel Response Tally of Peer Responses: 3 Yes; 3 No

Response Component Peer Panel Response

Rationale

Although there was some evidence that feedback was solicited from teachers and their representatives, there is an overall lack of clarity on if, or how, such feedback resulted in significant modifications or improvements to the request.

Strengths

The Nevada Department of Education (NDE) provided multiple opportunities to teachers to provide feedback on the request, including a statewide survey (resulting in responses from more than 800 teachers), additional survey efforts in the most densely populated school district, and a designated representative at NDE (although no input was received this way) (p. 9).

Face to face meetings were held with Nevada State Education Association (NSEA) leaders.

Materials were prepared for NSEA leaders to use to support meaningful dialogue with their constituents.

Four teachers serve on a statewide Teachers and Leaders Council (TLC) to develop recommendations for the creation and implementation of a performance evaluation system.


3


Weaknesses, issues, lack of clarity

It is unclear how feedback from teachers and NSEA leaders resulted in modification of the request, which is a major concern.

The request references that the NDE refined the working Theory of Action upon which the proposal is built based upon the feedback from the TLC, however no evidence is included to support this.

Sufficient documentation is lacking to evidence regular meetings with NSEA leaders (i.e., dates and times, agendas, emails, meeting notes, etc). There is limited documentation that teachers or their representatives contributed meaningfully to Principle 2.

The state-wide survey was not sent directly to individual teachers or administrators, which could have led to an increased number of respondents.

Technical Assistance Suggestions

Provide additional documentation on engagement of teacher and teacher representatives, such as meeting agendas, email communication, meeting notes, etc.

Provide drafts of the Theory of Action to show modification based on the TLC’s feedback and suggestions.

Develop a plan to ensure meaningful, ongoing engagement with diverse communities including parents and advocacy organizations for students with disabilities and English Learners, teachers, and their representatives.


4

2. Did the SEA meaningfully engage and solicit input on its request from other diverse communities, such as students, parents, community-based organizations, civil rights organizations, organizations representing students with disabilities and English Learners, business organizations, and Indian tribes?

Is the engagement likely to lead to successful implementation of the SEA’s request due to the input and commitment of relevant stakeholders at the outset of the planning and implementation process?

Did the SEA indicate that it modified any aspect of its request based on stakeholder input?

Does the input represent feedback from a diverse mix of stakeholders representing various perspectives and interests, including stakeholders from high-need communities? Consultation Question 2 Panel Response Tally of Peer Responses: 4 Yes; 2 No


Rationale

NDE engaged and solicited input from some stakeholder groups that resulted in substantive modifications to the request. However, there was limited evidence that meaningful input was sought from parents and advocacy groups for students with disabilities and English Learner students, as well as representatives of Indian tribes.

Strengths

NDE created an Accountability System Redesign Committee and its subset, a Core Group, to serve as a “think tank” (p. 10).

NDE used feedback from two State associations that included educators of students with disabilities to emphasize an intervention approach grounded in Response to Intervention (RtI).

NDE lowered the minimum group n size from 25 to 10 in response to stakeholder feedback.


Outreach to organizations representing students with disabilities and English Learners was largely confined to a small group of organizations (Special Education Advisory Committee, Members of Clark and Washoe County-based Latino Chambers of Commerce, President of Nevada PTA) but included little direct input from parents of special education/English Learner students and advocacy groups.

Attachment 2, a resolution from Storey County School District’s Superintendent and Board of Trustees, states that the NDE had not submitted a plan for public discussion two weeks prior to the date of submission to the NDE, thus resulting in their opposition to the request on the grounds that the request was not made available to the public with sufficient time to comment. This is a cause for concern.

It is unclear how Indian tribes were meaningfully engaged and if their feedback led to modifications in the request.


5



Provide additional documentation on stakeholder engagement, such as meeting agendas, e-mail communication, drafts of request noting changes based on feedback, etc.

Develop a plan to ensure meaningful, ongoing engagement with diverse communities including parents and advocacy organizations for students with disabilities and English Learners, teachers, and representatives.


6

Principle 1: College- and Career-Ready Expectations for All Students

Note to Peers: Staff will review 1.A Adopt College-And Career-Ready Standards, Options A and B.

1.B Transition to college- and career-ready standards

1.B Part A: Is the SEA’s plan to transition to and implement college- and career-ready standards statewide in at least reading/language arts and

mathematics no later than the 20132014 school year realistic, of high quality? Note to Peers: See ESEA Flexibility Review Guidance for additional considerations related to the types of activities an SEA includes in its transition plan.

1.B Panel Response, Part A Tally of Peer Responses: 0 Yes; 6 No


Rationale

NDE’s plan to transition to and implement college- and career-ready standards in Mathematics does not comply with the timeline requirements of ESEA Flexibility (p. 23). Additionally, although the plan has many components, it lacks cohesion and clarity. Further, it is unclear how all the components referred to fit together and important elements for each component are missing, such as exact timeline, parties responsible, and resources needed.


7


Strengths

A comparative analysis of the CCSS with the existing Nevada State Standards was completed, with external validation from WestEd, and used to inform the transition plan (p. 22).

NDE is publishing the correlations between student scores on Nevada’s High School proficiency exam and scores on college readiness tests (SAT, ACT, Accuplacer, etc). Some panel members believe that, as a result, educators and other stakeholders will have a realistic picture of the gap between current student performance and college readiness standards.

A Nevada Common Core State Standards(CCSS) website was created to serve as a tool to disseminate information and share resources for teachers, administrators and parents.

Nevada’s two largest universities collaborated to create a system of graduate credit courses on implementation of the CCSS offered to teachers in the summer of 2011.

The literacy plan approach can be a strong foundation for the rollout of CCSS.

NDE joined three consortia (SMARTER Balanced Assessment Consortium – SBAC, National Center and State Collaborative - NCSC, and World-Class Instructional Design and Assessment - WIDA) and is working closely with them to ensure that it is ready for the transition to new assessments.


Full implementation of the CCSS in Mathematics will not occur until the 2014-2015 school year. This is not in adherence with ESEA Flexibility requirements (p. 23).

The request lacks most of the elements of a high-quality plan (e.g., benchmarks, key activities, timelines, persons responsible, and resources).

NDE is relying heavily on the consortia to provide the expertise needed to make the transition, rather than tailoring a plan to address specific issues in Nevada.

The professional development and training is not described in detail, nor is there a description of its delivery, length, breadth, or requirements for participants to demonstrate their knowledge and understanding.

There is no stated intention to provide teachers with tools and resources such as formative assessments, model curriculum units, etc. aligned to the CCSS.


8



Accelerate the timeline for full implementation of the CCSS in Mathematics by School Year (SY) 2013-2014.

Present a comprehensive and cohesive approach to transition to and implement college and career ready standards for all students.

Provide professional development to school leaders in order to purposely prepare them to lead the transition effort.

Develop instructional resources and formative assessments for teachers that align to the CCSS.

Ensure that all supports for schools and educators to implement the CCSS are in alignment and integrated with the supports and expectations Principle 2 and Principle 3.


9

Part B: Is the SEA’s plan likely to lead to all students, including English Learners, students with disabilities, and low-achieving students, gaining access to and learning content aligned with the college- and career-ready standards?

1.B Panel Response, Part B Tally of Peer Responses: 0 Yes; 6 No


Rationale

Without a robust strategy for implementation and a coherent, high-quality plan to guide it, the peers have no confidence that the NDE’s proposal will lead to all students gaining access to and learning content aligned with the college- and career-ready standards.

Strengths

NDE is working closely with the three consortia that it is a member of (SBAC, NCSC, WIDA) to ensure that the needs of students with disabilities and English Learners will be met.

All teacher preparation programs require one or more courses for undergraduate majors in teaching English Learners and one or more courses in working effectively with students who have disabilities.

In January 2012, NDE facilitated a WIDA standards workshop for district teams as the first step in implementing the rollout plan for these standards.


The strategy the NDE is relying on is weak and superficial. It depends on a multitude of “light-touch” strategies that have typically proven to be ineffective.

NDE is relying heavily on the consortia to provide the expertise needed to make the transition.

NDE did not provide a comprehensive and cohesive plan for students with disabilities and English Learners gaining access to and learning content aligned with college- and career-ready standards.

It is unclear when the English Learner Proficiency (ELP) standards (WIDA) will be implemented. There is little discussion or documentation as to how NDE will transition to implementation of standards for English Learners.

NDE indicates that it has been a challenge to meet the needs of students with disabilities and English Learners (p. 33), but does not provide specific details regarding how this challenge will be addressed.

Professional development related to the needs of students with disabilities is focused on special education teachers, not on all teachers who teacher students with disabilities.


10



Consider using methods and strategies that rely on teacher leadership for two purposes: 1) to generate instructional materials using “bottom up” strategies, drawing on the experiences of practitioners; and 2) to create buy-in and support for the new standards through collaboration and professional learning communities.

Provide a detailed plan for professional development for all teachers to support all students (including English Learners and students with disabilities) to learn content aligned with the CCSS. Clarify how professional development will be job-embedded, when and how it will be delivered, what checks for understanding will be given to ensure teachers’ understanding of and proper instruction of CCSS, and a timeline for rollout.


11

1.C Develop and Administer Annual, Statewide, Aligned, High-Quality Assessments that Measure Student Growth

1.C Did the SEA develop, or does it have a plan to develop, annual, statewide, high-quality assessments, and corresponding academic achievement

standards, that measure student growth and are aligned with the State’s college- and career-ready standards in reading/language arts and

mathematics, in at least grades 3-8 and at least once in high school, that will be piloted no later than the 20132014 school year and planned for

administration in all LEAs no later than the 20142015 school year, as demonstrated through one of the three options below? Does the plan include setting academic achievement standards?

Note to Peers: Staff will review Options A and C.

If the SEA selected Option B: If the SEA is neither participating in a State consortium under the RTTA competition nor has developed and administered high-quality assessments, did the SEA provide a realistic, high-quality plan describing activities that are likely to lead to the development of such

assessments, their piloting no later than the 20132014 school year, and their annual administration in all LEAs beginning no later than the

20142015 school year? Does the plan include setting academic achievement standards?

1.C, Option B Panel Response Not applicable because the SEA selected 1.C, Option A or Option C

Tally of Peer Responses: NA


Rationale

NA

Strengths

NA


NA


NA


12

Principle 1 Overall Review

Is the SEA’s plan for transitioning to and implementing college-and career-ready standards, and developing and administering annual, statewide, aligned high-quality assessments that measure student growth, comprehensive, coherent, and likely to increase the quality of instruction for students and improve student achievement? If not, what aspects are not addressed or need to be improved upon? Principle 1 Overall Review Panel Response Tally of Peer Responses: 0 Yes; 6 No


Rationale

Overall, this plan lacks clarity and cohesion. Important components, such as professional development for teachers and administrators, plans to develop and disseminate high-quality instructional materials aligned with the new standards, and details around implementation of CCSS standards for students with disabilities and English Learners, are all without sufficient detail and information surrounding timelines, responsibilities, resource plans, and potential obstacles. Further, the NDE’s timeline for implementation of CCSS in Mathematics does not comply with ESEA Flexibility requirements.

Strengths

NDE has taken steps towards implementation by beginning with the creation of a CCSS Implementation Steering Committee in the summer of 2010.

A report was commissioned to analyze similarities and differences between Nevada State Standards and the CCSS, an important starting place to determine a transition plan.

A Nevada CCSS website was created to support the dissemination of information related to the CCSS and to share resources with stakeholders such as parents and teachers.

NDE joined three consortia (SMARTER Balanced Assessment Consortium – SBAC, National Center and State Collaborative - NCSC, and World-Class Instructional Design and Assessment - WIDA) and is working closely with them to ensure that it is ready for the transition to new assessments.


The timeline for full implementation of the CCSS is not completed by the required deadline of the 2013-2014 school year; Mathematics CCSS will not be fully implemented until 2014-2015.

Overall, the plan for professional development for teachers lacks sufficient detail and clarity.

Although NDE adopted the WIDA standards for English Learners, an implementation plan for these standards was not described in detail.


13



Accelerate the timeline for full implementation of the CCSS in Mathematics by SY 2013-2014.

Ensure that all supports for schools and educators to implement the CCSS are in alignment and integrated with the supports and expectations Principle 2 and Principle 3.

Consider using methods and strategies that rely on teacher leadership for two purposes: 1) to generate instructional materials using “bottom up” strategies, drawing on the experiences of practitioners and 2) to create buy-in and support for the new standards through collaboration and professional learning communities.

Provide a detailed plan for professional development for all teachers to support all students (including English Learners and students with disabilities) to learn content aligned with the CCSS. Clarify how professional development will be job-embedded, when and how it will be delivered, what checks for understanding will be given to ensure teachers’ understanding of and proper instruction of CCSS, and a timeline for rollout.


14

Principle 2: State-Developed Differentiated Recognition, Accountability, and Support

2.A Develop and Implement a State-Based System of Differentiated Recognition, Accountability, and Support

2.A.i Did the SEA propose a differentiated recognition, accountability, and support system, and a high-quality plan to implement this system no later

than the 20122013 school year, that is likely to improve student achievement and school performance, close achievement gaps, and increase the quality of instruction for students? (note to Peers, please write to this question after completing 2.A.i.a and 2.A.i.b)

2.A.i Panel Response Tally of Peer Responses: 1 Yes; 5 No


Rationale

NDE’s proposed “Nevada School Performance Framework” (NSPF) of recognition, accountability, and supports is based on a theory of action that may hold promise in improving student achievement and school performance. However, some elements of a “high-quality plan” are absent from the proposal, making it difficult for peers to determine if implementation will likely lead to improved student achievement and school performance, closing achievement gaps, and an increase in the quality of instruction for all students. In addition, the index is normative instead of criterion-based.

Strengths

Some panel members believe that NDE’s 5 star accountability system, based on the Index scores, is a clear and publicly-transparent mechanism for differentiating school performance.

NDE’s underlying theory of change-- rewarding high-performing schools with greater autonomy and incentives, while providing more prescriptive interventions in low-performing schools-- is evidence-based and demonstrates institutional learning from past State efforts to intervene and support schools.

The consistency and alignment of school improvement planning processes (having one, uniform statewide planning tool, for example), helps reduce confusion and burdensome requirements for schools and districts.

NDE’s emphasis on building leadership capacity, especially in turnaround schools, is evidence-based, and notable.


15


NDE proposes to calculate the gaps for subgroups by: 1) subtracting the subgroups’ proficiency percentage from the statewide average proficiency percentage for all students; and 2) assigning index points based on the distribution of differences (percentiles). This practice compares one school to other schools rather than to an absolute achievement standard. As a result, this method does not give the schools or public a clear understanding of whether students are making progress towards proficiency. It will not encourage schools to use appropriate and effective practices, especially for subgroups.

While NDE has proposed some elements of a high-quality plan, the panel believes that the request could be strengthened by including all elements of a high-quality plan, particularly timelines, objectives, and parties/point people responsible.

Some panel members expressed concern about the exclusion of racial/ethnic sub-groups in the gap comparison.

Other members were concerned that there is an over-reliance on growth in the elementary and middle school level.

Other members were concerned about the absence of growth in the high school level.

The panel is concerned that NDE’s proposal to waive “rank and serve” requirements will result in perverse incentives (e.g., rapidly improving schools may lose resources to schools that have shown little or no improvement).


Create a simple, publicly-transparent implementation plan for NSPF supports, interventions, and recognitions that includes all described programs, their timelines for implementation, benchmarks for success, and parties/point people responsible with the state, district, and school.


16

a. Does the SEA’s accountability system provide differentiated recognition, accountability, and support for all LEAs in the State and for all Title I schools in those LEAs based on (1) student achievement in reading/language arts and mathematics, and other subjects at the State’s discretion, for all students and all subgroups of students identified in ESEA section 1111(b)(2)(C)(v)(II); (2) graduation rates for all students and all subgroups; and (3) school performance and progress over time, including the performance and progress of all subgroups?

2.A.i.a Panel Response Tally of Peer Responses: 0 Yes; 6 No


Rationale

The NSPF provides differentiated recognition, accountability, and supports based on: 1) student achievement for all students and for some subgroups; 2) graduation rates for all students and for some subgroups; and 3) school performance and progress over time for all students and for some subgroups. However, this index appears to be based on normative comparisons among schools instead of comparisons to a standard.

Strengths

The NSPF will reduce the minimum subgroup n size from 25 to 10 students for the three sub-group populations counted (English Learners, students with disabilities, and free and reduced price lunch (FRL)), thereby increasing accountability for these subgroups.

Some panel members believe it is a strength that the NSPF will include the performance of both current and former English Learners and both current and former students with Individualized Education Plans (IEPs) as part of its subgroup calculation in the “Gap” portion of its Index, thereby increasing accountability for English Learner and Special Education programming more generally.

In the NSPF for high schools, NDE will award a total of 30 points to schools based on graduation rates, with 15 points being awarded for overall graduation rates and 15 points being awarded for graduation rate gaps among English Learners, student with disabilities, and free and reduced price lunch subgroups. The panel found the focus on graduation rates to be a strength of the proposed system.


17



NDE proposes to calculate the gaps for subgroups by: 1) subtracting the subgroups’ proficiency percentage from the statewide average proficiency percentage for all students; and 2) assigning index points based on the distribution of differences (percentiles). This practice compares one school to other schools rather than to an absolute achievement standard. As a result, this method does not give the schools or public a clear understanding of whether students are making progress towards proficiency. It will not encourage schools to use appropriate and effective practices, especially for subgroups.

Some panel members are concerned about the potential overreliance on growth in the elementary and middle school levels and the absence of growth as a criteria in the high school, especially given high rates of student mobility and immigration (e.g., a student can immigrant or transfer at the high school level as well).

The NSPF does not include all ESEA subgroups in the gap portion of the index. Some panel members are concerned that all racial/ethnic subgroups are excluded from the index. They are concerned that this exclusion will lead to a de-emphasis on documented educational gaps such as the black-white achievement gap, the Hispanic-white achievement gap, etc.

Some panel members believe that assigning points to the gap portion separately for the three subgroups may permit the gap performance of one or two groups to compensate for the performance of other groups.

Some panel members are concerned that the subgroup gaps receive only 15% of the points in the index and as a result they are concerned that this could lead to a de-emphasis on gaps.

Some panel members believe it is a weakness that the NSPF will report the performance of both current and former English Learners and both current and former students with Individualized Education Plans (IEPs) as part of its subgroup calculation in the “Gap” portion of its Index, because of a concern that this would mask low performance of those in the actual subgroup.

While the “other indicator” may add important information to the overall index, without rigorous criteria it could mask school underperformance.


18



Consider the appropriate balance of weighing growth and performance.

Develop a stronger method for assessing growth at the high school level.

NDE should develop rigorous criteria for the “other indicator” section of the index.

As NDE considers future revisions in the index, engage experts in accountability for students with disabilities, English Learners, and racial/ethnic minorities to review the index to ensure that these subgroups are appropriately represented.

b. Does the SEA’s differentiated recognition, accountability, and support system create incentives and provide support that is likely to be

effective in closing achievement gaps for all subgroups of students? 2.A.i.b Panel Response Tally of Peer Responses: 4 Yes; 2 No


Rationale

NDE’s proposed system contains some promising interventions that may help close achievement gaps for all subgroups of students, especially for schools identified as focus schools. However, it is not clear to all members of the panel that the NDE’s system will create incentives and provide support that is likely to be effective in closing achievement gaps for all subgroups of students. The method for calculating the index does not give the schools or public a clear understanding of whether students are making progress towards proficiency.

Strengths

NDE will provide RtI programs and a grant-funded Literacy Initiative (available to some districts) that includes progress monitoring measures aligned to the CCSS and a Data-Based Decision-Making literacy team in each school.

The Nevada Comprehensive Audit Tool (NCCAT-S) process generates data to allow districts and schools to pinpoint particular areas of proficiency (special education math, for example) and create targeted interventions.

Coaching and mentoring uses Structured Instruction Observation Protocol Model (SIOP) which is a proven practice for English Learners.


19



The proposal identifies many promising supports that will be provided to teachers, but it is not clear how NDE will ensure that the tools and resources related to CCSS will be used in schools and classrooms.

As a result, this method does not give the schools or public a clear understanding of whether students are making progress towards proficiency, especially for subgroups.


Conduct a careful program evaluation and analysis of all programs proposed to increase achievement among subgroups.

Provide a clear, evidence-based rationale for continuing such interventions and additional detail as to how each program will explicitly improve specific subgroups’ performance. For example, select programs that have shown the most promise in providing targeted interventions and supports for English Learners.

a. Note to Peers: Staff will review 2.A.i.c

Note to Peers: Staff will review 2.A.ii Option A. ONLY FOR SEAs SELECTING OPTION B: If the SEA elects to include student achievement on assessments other than reading/language arts and mathematics in its differentiated recognition, accountability, and support system by selecting Option B, review and respond to peer review questions in section 2.A.ii. If the SEA does not include other assessments, go to section 2.B. 2.A.ii Did the SEA include student achievement on assessments in addition to reading/language arts and mathematics in its differentiated recognition,

accountability, and support system or to identify reward, priority, and focus schools?

a. Note to Peers: Staff will review 2.A.ii.a

b. Does the SEA’s weighting of the included assessments result in holding schools accountable for ensuring all students achieve the State’s college- and career-ready standards?

c. Note to Peers: Staff will review 2.A.ii.c


20

2.A.ii.b PANEL RESPONSE Not applicable because the SEA selected 2.A, Option A

Tally of Peer Responses; NA


Rationale NA

Strengths NA


NA


NA

2.B Set Ambitious but Achievable Annual Measurable Objectives

2.B Did the SEA describe the method it will use to set new ambitious but achievable annual measurable objectives (AMOs) in at least

reading/language arts and mathematics, for the State and all LEAs, schools, and subgroups, that provide meaningful goals and are used to guide support and improvement efforts through one of the three options below? Note to Peers: Staff will review Options A and B.

If the SEA selected Option C: Did the SEA describe another method that is educationally sound and results in ambitious but achievable AMOs for all LEAs, schools, and subgroups?

i. Did the SEA provide the new AMOs and the method used to set these AMOs?

ii. Did the SEA provide an educationally sound rationale for the pattern of academic progress reflected in the new AMOs?

iii. If the SEA set AMOs that differ by LEA, school, or subgroup, do the AMOs require LEAs, schools, and subgroups that are further behind to make greater rates of annual progress?

iv. Did the SEA attach a copy of the average statewide proficiency based on assessments administered in the 20102011 school year in reading/language arts and mathematics for the “all students” group and all subgroups? (Attachment 8)

Are these AMOs similarly ambitious to the AMOs that would result from using Option A or B above?


21

Are these AMOs ambitious but achievable given the State’s existing proficiency rates and any other relevant circumstances in the State?

Will these AMOs result in a significant number of children being on track to be college- and career-ready?

2.B, Option C (including Questions i–iv) Panel Response

Not applicable because the SEA selected 2.B, Option A or Option B Tally of Peer Responses: 0 yes, 6 No


Rationale

While NDE has developed an innovative index to measure school performance, the AMOs selected do not meet the requirements of ESEA Flexibility because they do not have proficiency targets.

Strengths

The minimum n size for subgroups has been reduced to 10 students and super-groups are used if subgroups are too small, ensuring greater subgroup accountability.


While NDE has targets for improvement in their index, the AMOs do not meet the requirements of ESEA Flexibility because they do not have proficiency targets.


AMOs that reflect the requirements of ESEA Flexibility need to be developed.


22

2.C Reward Schools

Note to Peers: Staff will review 2.C.ii. 2.C.i


Rationale

NDE describes clearly how three different types of reward schools will be identified: High Status, High Growth, and Exemplary (both High Status and High Growth). While not based directly on the NSPF index, the methods employ the same elements used in the index: proficiency percentages, Median School Growth, and subgroup gaps in Adequate School Growth. However, the method for identifying High Status high schools and middle schools is omitted – and no list has been provided. Without a method or a list it is not possible for the panel to discern the method that will be used.

Strengths NDE requires that the schools identified as High Status must be in the top 25 percent of all students proficient for the past two years (this avoids flips in the list).


The method for identifying High Status high schools and middle schools is omitted – and no list has been provided. Without a method or a list it is not possible to discern the method that will be used.


Presumably, NDE has conducted a dry run of these criteria using 2010-11 data. If not, this would be a good idea to gather an understanding of how many schools can meet these criteria.

Describe the method for identifying High Status middle schools and high schools.


23

2.C.iii Are the recognition and, if applicable rewards, proposed by the SEA for its highest-performing and high-progress schools likely to be considered meaningful by the schools?

Has the SEA consulted with LEAs and schools in designing its recognition and, where applicable, rewards?

2.C.iii PANEL RESPONSE Tally of Peer Responses: 6 Yes 0 No


Rationale

The recognitions and rewards proposed by the SEA for its highest-performing and high-progress schools are a robust mix of financial, professional, growth and public acknowledgment that are likely to be considered meaningful by the schools.

Strengths

Providing autonomy in areas such as budgeting, staffing, and program design as well as negotiated flexibility with use of allocations and with core instructional materials are meaningful rewards for successful school leaders and teachers.

A pay for performance system will allow reward schools to receive financial acknowledgement that can directly support the continuation of programming in the school to support improvement.

NDE proposes innovative mechanisms to maximize the human capital of outstanding teachers and administrations through face-to-face virtual communications, digital resources such as blogs and learning forums, teacher and principal-led networking, and other forums.


None.


Consider exploring models whereby high-performing leaders and teachers (based on the new performance evaluation system) receive financial incentives to work in low-performing schools and/or high-performers mentor low-performers


24

2.D Priority Schools

Note to Peers: Staff will review 2.D.i and 2.D.ii. 2.D.iii Are the interventions that the SEA described aligned with the turnaround principles and are they likely to result in dramatic, systemic change in

priority schools?

a. Do the SEA’s interventions include all of the following?

(i) providing strong leadership by: (1) reviewing the performance of the current principal; (2) either replacing the principal if such a change is necessary to ensure strong and effective leadership, or demonstrating to the SEA that the current principal has a track record in improving achievement and has the ability to lead the turnaround effort; and (3) providing the principal with operational flexibility in the areas of scheduling, staff, curriculum, and budget;

(ii) ensuring that teachers are effective and able to improve instruction by: (1) reviewing the quality of all staff and retaining only those who are determined to be effective and have the ability to be successful in the turnaround effort; (2) preventing ineffective teachers from transferring to these schools; and (3) providing job-embedded, ongoing professional development informed by the teacher evaluation and support systems and tied to teacher and student needs;

(iii) redesigning the school day, week, or year to include additional time for student learning and teacher collaboration;

(iv) strengthening the school’s instructional program based on student needs and ensuring that the instructional program is research-based, rigorous, and aligned with State academic content standards;

(v) using data to inform instruction and for continuous improvement, including by providing time for collaboration on the use of data;

(vi) establishing a school environment that improves school safety and discipline and addressing other non-academic factors that impact student achievement, such as students’ social, emotional, and health needs; and

(vii) providing ongoing mechanisms for family and community engagement?


25

2.D.iii.a (including questions (i)-(vii)) Panel Response Tally of Peer Responses: 2 Yes; 4 No


Rationale

The interventions that NDE describe align with some, but not all, of the turnaround principles. Some of the interventions described, especially in the area of human capital recruitment, selection, and capacity-building are not likely to result in dramatic, systemic change in priority schools (pp. 73-77).

Strengths

NDE’s focus on the “the right people for the right positions,” is a well thought-out delineation of the skill sets required of turnaround leaders, and its support of turnaround leadership through programs such as the UVA School Turnaround program are research-based approaches to school improvement.

NDE’s focus on measuring the effectiveness and reviewing the performance of school leaders and teachers, placing the most effective teachers in turnaround schools, and preventing ineffective teachers are priority schools are all clearly aligned with turnaround principles.

NDE’s requirement that priority schools must provide flexible work conditions designed to recruit/place/retain effective staff, promote the continuous use of data, and establish schedules and strategies that provide more learning time for staff are all clearly aligned with the turnaround principles.


While many key turnaround principles are listed, NDE does not provide sufficient detail on what specifically it will require districts to implement, especially in the area of extended learning time, teacher re-hiring/placement, leadership effectiveness, and flexible work conditions.

Other key turnaround principles such as family and community engagement, social-emotional supports, and using data to inform instruction are not sufficiently detailed.

NDE did not describe how it will ensure that the districts and the schools are implementing activities in all areas in a coherent, thoughtful and aligned manner.


Develop a high-quality plan to ensure that districts and schools are implementing activities in all areas in a coherent, thoughtful and aligned manner. Clearly describe, for each turnaround principle, what NDE will require both the district and the school to implement.

Build out the descriptions of the components of the turnaround principles (such as: data, extended learning time, social and emotional supports) in ways comparable to NDE’s description of human capital.


26

b. Are the identified interventions to be implemented in priority schools likely to — (i) increase the quality of instruction in priority schools;

(ii) improve the effectiveness of the leadership and the teaching in these schools; and

(iii) improve student achievement and, where applicable, graduation rates for all students, including English Learners, students with

disabilities, and the lowest-achieving students?

2.D.iii.b (including questions (i)-(iii)) Panel Response Tally of Peer Responses: 0 Yes; 6 No


Rationale

While some important high-quality, and research-based turnaround principles have been described, especially in the area of leadership development, recruitment, and placement, it is unclear how NDE will ensure that the district and school implement these principles with fidelity.

Strengths As described above, some of the programs described to train and recruit specialized school turnaround leaders,

hold promise for improving student achievement.


NDE has not sufficiently explained the roles and responsibility and authorities of the state, district, and school as they relate to priority schools.

The plan lacks sufficient detail as it pertains to interventions directly related to supporting English Learners and students with disabilities.


Describe, in specific detail, “who does what” for each turnaround principle in terms of roles and responsibilities of the district, state, and school levels.

Ensure that students with disabilities and English Learners and other low achieving students are clearly recognized in turnaround strategies that are appropriate to their language and learning needs (e.g., accommodations, universal design, and other supports).

c. Note to Peers: Staff will review 2.D.iii.c


27

2.D.iv Does the SEA’s proposed timeline ensure that LEAs that have one or more priority schools will implement meaningful interventions aligned

with the turnaround principles in each priority school no later than the 20142015 school year?

Does the SEA’s proposed timeline distribute priority schools’ implementation of meaningful interventions aligned with the turnaround principles in a balanced way, such that there is not a concentration of these schools in the later years of the timeline?

2.D.iv Panel Response Tally of Peer Responses: 6 Yes; 0 No


Rationale

NDE’s proposed timeline ensures that LEAs that have one or more priority schools will implement meaningful interventions aligned with the turnaround principles in each priority school no later than the

20142015 school year.

Strengths

This timeline allows NDE and local districts more time to learn from SIG implementation.

This timeline allows both NDE and districts to build capacity for implementation


There is not clarity as to whether there will be any pre-intervention activities in the first year.

There is a concern amongst the peers that a concentration of priority schools will begin implementation of interventions in the later years of the timeline, instead of distributing them in earlier years.


NDE should explain what it plans to do with the schools that it identifies prior to implementing interventions in those schools.

NDE should propose a plan for phased rollout of interventions in priority schools, so that there is not a concentration of these schools in later years of the timeline.


28

2.D.v Did the SEA provide criteria to determine when a school that is making significant progress in improving student achievement exits priority status?

a. Do the SEA’s criteria ensure that schools that exit priority status have made significant progress in improving student achievement?

Is the level of progress required by the criteria to exit priority status likely to result in sustained improvement in these schools?

2.D.v and 2.D.v.a PANEL RESPONSE Tally of Peer Responses: 1 Yes, 5 No


Rationale

NDE’s criteria for exiting priority status do not ensure that schools have made significant progress to improving student achievement, or that they will continue to exhibit sustained improvement.

Strengths NDE will rely on the framework for monitoring current SIG schools for analyzing capacity of priority schools to sustain improvement.


While, as discussed above, NDE describes using a framework to measure capacity, it is unclear how that framework will be used, which indicators will count towards exit, and whether the framework is directly linked to the original diagnostic/needs assessment of priority schools.

The panel is very concerned that there are not subgroup performance requirements for priority schools exit criteria.

The panel is very concerned that the graduation rate bar is set at 60 percent for exit from priority schools status.


Ensure that any measure of school capacity used at the end of three years directly and explicitly ties back to the original diagnostic/plan/needs assessment of the specific schools (SIG or not) completed at the beginning of the three years.

Provide specific detail as to what implementation, leading and lagging indicators will be used, and how, to determine the capacity of the school and district to sustain the improvement.

Include subgroup performance in exit criteria.

Increase graduation rate requirements for priority schools’ exit criteria.


29

2.E Focus Schools

Note to Peers: Staff will review 2.E.i 2.E.i Did the SEA describe its methodology for identifying a number of low-performing schools equal to at least 10 percent of the State’s Title I

schools as focus schools? If the SEA’s methodology is not based on the definition of focus schools in ESEA Flexibility (but is instead, e.g., based on school grades or ratings that take into account a number of factors), did the SEA also demonstrate that the list provided in Table 2 is consistent with the definition, per the Department’s “Demonstrating that an SEA’s Lists of Schools Meet ESEA Flexibility Definitions” guidance?

a. Note to Peers: Staff will review 2.E.i.a.

b. Is the SEA’s methodology for identifying focus schools educationally sound and likely to ensure that schools are accountable for the

performance of subgroups of students?

2.E.i.b Panel Response Tally of Peer Responses: 0 Yes, 6 No


Rationale

There is significant concern among the peers that, based on NDE’s methodology for identifying focus schools, there is the potential for the high performance of one subgroup to mask the low performance of another.

Strengths Cut scores for identifying the 10 percent of lowest-performing Title I schools will be applied to all schools.


There is the potential for the high performance of one subgroup to mask the low performance of another.

Achievement status is not included as a component of focus schools’ identification criteria.


Provide evidence that there is not a masking problem; if there is, provide a solution that ensures that schools with a low performing subgroup are included in this category.

2.E.ii Note to Peers: Staff will review 2.E.ii


30

2.E.iii Does the SEA’s process and timeline ensure that each LEA will identify the needs of its focus schools and their students and implement interventions in focus schools at the start of the 2012–2013 school year? Did the SEA provide examples of and justifications for the interventions the SEA will require its focus schools to implement? Are those interventions based on the needs of students and likely to improve the performance of low-performing students and reduce achievement gaps among subgroups, including English Learners and students with disabilities?

Has the SEA demonstrated that the interventions it has identified are effective at increasing student achievement in schools with similar characteristics, needs, and challenges as the schools the SEA has identified as focus schools?

Has the SEA identified interventions that are appropriate for different levels of schools (elementary, middle, high) and that address different types of school needs (e.g., all-students, targeted at the lowest-achieving students)?

2.E.iii Panel Response Tally of Peer Responses: 0 Yes; 6 N


Rationale

NDE has not complied with the requirement that it will begin to implement interventions in focus schools at the start of the 2012-2013 school year. The interventions proposed for focus schools include promising models but could be strengthened by more detail and more explicit description of NDE’s authority to ensure implementation (pp. 80-83).

Strengths

NDE proposes the implementation of technical assistance, focused professional development, and the Nevada Comprehensive Curriculum Audit Tool for Schools (NCCAT-S) system to target specific needs, especially in sub-groups, to support intervention.

NDE proposes requiring a district with one or more focus schools to be required to reserve up to 25 percent of its Title I, Part A funds on a sliding scale to support the implementation of the interventions. The panel applauds NDE’s efforts to provide districts and schools with flexibility in using these funds.


31



NDE proposed beginning interventions in focus schools in 2013-2014. This does not meet the requirements of ESEA Flexibility.

There is insufficient detail provided on how NDE plans to ensure that the district and school provide targeted supports and interventions, ensure instructional effectiveness, and other elements of school improvement (excellent leaders, effective teachers, social-emotional supports) for focus schools.

Focused professional development is recommended, but no specificity is provided as to what it will look like in practice, and how it will ensure that all educators have the knowledge and skills needed to successfully meet varied subgroup needs.

It is not clear how the index data will inform the interventions chosen to ensure that the interventions focus on the subgroups that led to the focus school identification.

While NDE notes that interventions for focus schools have been implemented in schools identified “In Need of Improvement” (INOI) Year 4 and beyond under the current accountability system and that such interventions have led to improvements (p. 83), there was insufficient detailed evidence how all of the support and interventions discussed will directly lead to improved performance among subgroups.


Move the timeline up one year to comply with federal requirements (secure statutory relief to prevent the delay).

Provide a detailed description of interventions and programs that districts with focus schools and focus schools must implement.

Describe the State’s, district’s, and school’s relative authorities, roles and responsibilities.

2.E.iv Did the SEA provide criteria to determine when a school that is making significant progress in improving student achievement and narrowing achievement gaps exits focus status?

a. Do the SEA’s criteria ensure that schools that exit focus status have made significant progress in improving student achievement and narrowing achievement gaps?

Is the level of progress required by the criteria to exit focus status likely to result in sustained improvement in these schools?


32

2.E.iv and 2.E.iv.a PANEL RESPONSE Tally of Peer Responses: 0 Yes; 6 N


Rationale NDE’s criteria for exiting focus school status do not ensure that schools have made significant progress to improving student achievement, or that they will continue to exhibit sustained improvement (p. 83).

Strengths None.


While NDE describes using a framework to measure capacity, it is unclear how that framework will be used, which indicators will count towards exit, and whether the framework is directly linked to the diagnostic/needs assessment that caused schools to be categorized as focus schools.

The panel is concerned that the exit criteria described are not sufficiently rigorous, or that they will ensure that the school has changed for sustained improvement over the long term.


Consider ways to increase the rigor or exit criteria to include leading, lagging, and implementation indicators that are directly aligned to the reasons the school became a focus school and reflect school’s needs assessment and data.

Consider inclusion of capacity and implementation indicators.

2.F Provide Incentives and Support for other Title I Schools

2.F.i Does the SEA’s differentiated recognition, accountability, and support system provide incentives and supports for other Title I schools that,

based on the SEA’s new AMOs and other measures, are not making progress in improving student achievement and narrowing achievement gaps?

2.F.i Panel Response Tally of Peer Responses: 0 Yes, 6 No


Rationale NDE does not address this question.

Strengths None.


33



NDE has not described specific interventions for other Title I Schools.


Develop a plan for how to support other Title I schools. In developing the plan, consider how to closely align, streamline, and coordinate State and district-level supports to schools, to ensure that Title I schools are not embarking on dramatically different program initiatives than are priority or focus schools (or other schools in the state). In addition, align the intervention efforts here with the work underway to support these same schools to implement the CCSS and assessments (Principle 1) and the new educator evaluation system (Principle 3).

Consider how to construct Title I programs to allow school maximal autonomy in exchange for maximal accountability.

Consider how to reduce burdensome compliance requirements associated with Title I.

2.F.ii Are those incentives and supports likely to improve student achievement, close achievement gaps, and increase the quality of instruction for all

students, including English Learners and students with disabilities?

2.F.ii Panel Response Tally of Peer Responses: 0 Yes, 6 No


Rationale NDE has not described incentives and supports for other Title I schools in its flexibility request (pp. 85-87).

Strengths None.


None.


In developing a plan for other Title I schools, consider interventions and strategies that are most likely to increase achievement for subgroups and close achievement gaps.


34

2.G Build SEA, LEA, and School Capacity to Improve Student Learning

2.G Is the SEA’s process for building SEA, LEA, and school capacity to improve student learning in all schools and, in particular, in low-

performing schools and schools with the largest achievement gaps, likely to succeed in improving such capacity?

i. Is the SEA’s process for ensuring timely and comprehensive monitoring of, and technical assistance for, LEA implementation of interventions in priority and focus schools likely to result in successful implementation of these interventions and in progress on leading indicators and student outcomes in these schools?

Did the SEA describe a process for the rigorous review and approval of any external providers used by the SEA and its LEAs to support the implementation of interventions in priority and focus schools that is likely to result in the identification of high-quality partners with experience and expertise applicable to the needs of the school, including specific subgroup needs?

ii. Is the SEA’s process for ensuring sufficient support for implementation of interventions in priority schools, focus schools, and other Title

I schools under the SEA’s differentiated recognition, accountability, and support system (including through leveraging funds the LEA was previously required to reserve under ESEA section 1116(b)(10), SIG funds, and other Federal funds, as permitted, along with State and local resources) likely to result in successful implementation of such interventions and improved student achievement?

iii. Is the SEA’s process for holding LEAs accountable for improving school and student performance, particularly for turning around their

priority schools, likely to improve LEA capacity to support school improvement?


35

2.G (including i, ii, and iii) Panel Response Tally of Peer Responses: 0 Yes; 6 No


Rationale The NDE has not outlined processes for building State, district, and school capacity that necessarily ensure the ability to improve student achievement in all schools (pp. 87-89).

Strengths

The consequences for schools that do not meet priority status (accidentally listed under section 2F) such as competency-based funding model; 2) restart and 3) school closure reflect measures that are urgent and potentially hold the possibility to dramatically improve student achievement.

NDE states its desire to create a streamlined, coordinated approach to building capacity across all levels. We applaud NDE’s efforts to streamline and coordinate in order to create a uniform message and set of criteria for schools.

NDE wisely recognized that new IT infrastructure and public reporting tools will help ensure the success of the NSPF system. We support allowing the state maximal flexibility in using federal funding to support these projects.


Given the re-organization of Nevada’s Board of Education, and the new appointment of the Superintendent, we are concerned that the State may not continue to invest sufficient resources to develop, implement, and sustain the new accountability system.

There is not a comprehensive, detailed plan for timely and comprehensive monitoring and reporting of LEA implementation efforts and outcomes.

There is insufficient evidence that the proposal has mechanisms that will hold LEAs accountable for the critical role they play in successfully intervening and turning around low-performing schools.

There is insufficient information provided as to how NDE plans to leverage and reallocate financial resources to ensure successful implementation of its accountability system.


Provide information on what percent of Federal funding the State plans to reserve and other, local funds would be available to build capacity.

Describe the implications of new political arrangements and authorities on the proposed waiver and accountability system.


36


Is the SEA’s plan for developing and implementing a system of differentiated recognition, accountability, and support likely to improve student achievement, close achievement gaps, and improve the quality of instruction for students? Do the components of the SEA’s plan fit together to create a coherent and comprehensive system that supports continuous improvement and is tailored to the needs of the State, its LEAs, its schools, and its students? If not, what aspects are not addressed or need to be improved upon? PRINCIPLE 2 OVERALL REVIEW PANEL RESPONSE Tally of Peer Responses: 0 Yes; 6 No


Rationale

There are significant compliance issues with Principle 2 of this request that do not meet the ESEA Flexibility requirements.

While NDE’s new accountability system contains many components that are based on sound theories of organizational change, or that reflect institutional learning and stakeholder feedback, the panel does not have sufficient evidence that the system will a) identify, target, and differentiate schools most accurately; or b) provide supports that will, in fact, result in increased student achievement.

Strengths

NDE should be commended for its efforts to focus on achievement gaps by reducing the minimum group size to 10.

The panel endorses NDE’s overall theory of organizational change—that high-performing schools and leaders should be rewarded and incentivized with greater flexibility, autonomy, performance rewards, and efforts to scale up and disseminate best practices.

The panel commends NDE’s efforts to coordinate initiatives across all levels of schools (i.e. everyone participated in the same planning process) and reduce competing, burdensome initiatives, and increase overall flexibility in use of funds.


37



While NDE has targets for improvement in its index, the AMOs do not meet the requirements of ESEA Flexibility because they do not have proficiency targets.

NDE proposed beginning interventions in focus schools in 2013-2014. This does not meet the requirements of ESEA Flexibility.

NDE has not described an aligned system of support for Title I schools.

As noted above, while there are many promising models proposed, NDE has not yet detailed the specific interventions required of priority and focus schools (for all areas related to turnaround principles and achievement gap reduction) as well as, critically, the state’s authority and role to mandate such principles.

NDE’s exit criteria are not sufficiently rigorous and do not include implementation and capacity benchmarks aligned with the key areas for growth originally diagnosed in priority and reward schools.


As noted above:

Create AMOs that are based on proficiency targets.

Ensure timelines are in compliance with ESEA Flexibility requirements.

Describe an aligned system of support for Title I schools.

Increase the rigor of exit criteria for priority and focus schools to include implementation and capacity benchmarks aligned with the key areas for growth originally diagnosed in priority and reward schools.

In light of new governance changes at the state, provide a more detailed description of inter-governmental authority to implement and ensure strong accountability actions in local districts and schools.


38

Principle 3: Supporting Effective Instruction and Leadership

3.A Develop and Adopt Guidelines for Local Teacher and Principal Evaluation and Support Systems

3.A.i Has the SEA developed and adopted guidelines consistent with Principle 3 through one of the two options below?

If the SEA selected Option A: If the SEA has not already developed and adopted all of the guidelines consistent with Principle 3:

i. Is the SEA’s plan for developing and adopting guidelines for local teacher and principal evaluation and support systems likely to result

in successful adoption of those guidelines by the end of the 2011–2012 school year?

3.A.i, Option A.i Panel Response Not applicable because the SEA selected 3.A, Option B

Tally of Peer Responses: 0 Yes; 6 No


Rationale The proposed timeline is not likely to result in successful adoption of guidelines for local teacher and principal evaluation and support systems by the end of the 2011-2012 school year.

Strengths A Teachers and Leaders Council (TLC) is in the process of crafting recommendations that explicate the guidelines established in State statute in AB 222 (p. 92).


The TLC responsible for creating regulatory recommendations will not present final guidelines to the State Board of Education for consideration until December 2012, making adoption by the end of the 2011-2012 school year impossible. Further, adoption of the guidelines will not likely occur until June 2013 (p. 93).

The preliminary recommendations of the TLC, as outlined in the “Systems Guidelines White Paper”, are incredibly broad in scope and provide only a general overview of what components should be included in an evaluation model. Specific performance criteria will not be established until March-May 2012 (Attachment 11A, p. 7).

Growth in student achievement is not specifically mandated in state statute or in the TLC’s early recommendations (p. 93; Attachment 10, p. 3).


The TLC should expedite its evaluation development timeline so that it meets the requirements of ESEA Flexibility for Principle 3.

Ensure that student growth data is used as a significant measure in performance evaluations.


39

ii. Does the SEA’s plan include sufficient involvement of teachers and principals in the development of these guidelines?

3.A.i, Option A.ii Panel Response Not applicable because the SEA selected 3.A, Option B

Tally of Peer Responses: 2 Yes; 4 No


Rationale Although the TLC itself is comprised, in part, of several teachers and principals, and although the TLC’s draft recommendations suggest the involvement of stakeholder groups, no specific plan for gathering and implementing stakeholder feedback as guidelines are developed has been outlined.

Strengths Four teachers and two principals serve on the TLC.

The TLC, in its initial “Systems Guidelines White Paper,” acknowledges the need and the intention to involve stakeholders in the process, beginning in February 2012 (Attachment 11A, p. 13).

The TLC has established a working “Communications Task Force” which will create recommendations in regard to creation of opportunities for interested stakeholders to engage in efforts to help shape the recommendations to be created by the TLC (p.95).

In developing its request, NDE demonstrated substantial outreach to teachers that included specific questions regarding Principle 3.


The specific methods for gathering and utilizing stakeholder feedback, and a plan for doing so, have not been determined (Attachment 11A, p. 9 and 13).

The teachers appointed to serve on the TLC are selected by the governor from a list of nominees appointed by the NSEA (p. 92). However, not all Nevada teachers are members of the NSEA and therefore the TLC might not accurately reflect the diverse opinions of Nevada’s teachers.


Propose a plan for a high-quality feedback loop at each stage of evaluation development and implementation to ensure that educators are involved in the creation of the educator evaluation system which will likely lead to greater stakeholder buy-in and support for implementation of the new system. Potential mechanisms for gathering feedback include a state-wide teacher and principal survey and/or focus groups and listening sessions in each local district.

iii. Note to Peers: Staff will review iii.


40

If the SEA selected Option B: If the SEA has developed and adopted all guidelines consistent with Principle 3:

i. Are the guidelines the SEA has adopted likely to lead to the development of evaluation and support systems that increase the quality of

instruction for students and improve student achievement? (See question 3.A.ii to review the adopted guidelines for consistency with Principle 3.)

3.A.i, Option B.i Panel Response

Not applicable because the SEA selected 3.A, Option A Tally of Peer Responses: NA


Rationale NA

Strengths NA Weaknesses, issues, lack of clarity NA Technical Assistance Suggestions NA

ii. Note to Peers: Staff will review ii.

iii. Did the SEA have sufficient involvement of teachers and principals in the development of these guidelines?

3.A.i, Option B.iii Panel Response



Rationale NA


ONLY FOR SEAs SELECTING OPTION B: If the SEA has adopted all guidelines for local teacher and principal evaluation and support systems by selecting Option B in section 3.A, review and respond to peer review question 3.A.ii below. 3.A.ii Are the SEA’s guidelines for teacher and principal evaluation and support systems consistent with Principle 3 — i.e., will they promote systems

that:

a. Will be used for continual improvement of instruction ?


41

Are the SEA’s guidelines likely to result in support for all teachers, including teachers who are specialists working with students with disabilities and English Learners and general classroom teachers with these students in their classrooms, that will enable them to improve their instructional practice?

3.A.ii.a Panel Response



Rationale NA


b. Meaningfully differentiate performance using at least three performance levels?

Does the SEA incorporate student growth into its performance-level definitions with sufficient weighting to ensure that performance levels will differentiate among teachers and principals who have made significantly different contributions to student growth or closing achievement gaps?

3.A.ii.b Panel Response



Rationale NA


c. Use multiple valid measures in determining performance levels, including as a significant factor data on student growth for all students

(including English Learners and students with disabilities), and other measures of professional practice (which may be gathered through multiple formats and sources, such as observations based on rigorous teacher performance standards, teacher portfolios, and student and parent surveys)?

(i) Does the SEA have a process for ensuring that all measures that are included in determining performance levels are valid measures,

meaning measures that are clearly related to increasing student academic achievement and school performance, and are implemented in a consistent and high-quality manner across schools within an LEA?


42

3.A.ii.c(i) Panel Response Not applicable because the SEA selected 3.A, Option A



Rationale NA


(ii) For grades and subjects in which assessments are required under ESEA section 1111(b)(3), does the SEA define a statewide approach

for measuring student growth on these assessments?

3.A.ii.c(ii) Panel Response Not applicable because the SEA selected 3.A, Option A



Rationale NA


(iii) For grades and subjects in which assessments are not required under ESEA section 1111(b)(3), does the SEA either specify the

measures of student growth that LEAs must use or select from or plan to provide guidance to LEAs on what measures of student growth are appropriate, and establish a system for ensuring that LEAs will use valid measures?

3.A.ii.c(iii) Panel Response Not applicable because the SEA selected 3.A, Option A



Rationale NA


d. Evaluate teachers and principals on a regular basis?


43

3.A.ii.d Panel Response Not applicable because the SEA selected 3.A, Option A



Rationale NA


e. Provide clear, timely, and useful feedback, including feedback that identifies needs and guides professional development?

Will the SEA’s guidelines ensure that evaluations occur with a frequency sufficient to ensure that feedback is provided in a timely manner to inform effective practice?

Are the SEA’s guidelines likely to result in differentiated professional development that meets the needs of teachers?

3.A.ii.e Panel Response Not applicable because the SEA selected 3.A, Option A



Rationale NA


f. Will be used to inform personnel decisions?

3.A.ii.f Panel Response



Rationale NA



44

3. B Ensure LEAs Implement Teacher and Principal Evaluation and Support Systems

3.B Is the SEA’s process for ensuring that each LEA develops, adopts, pilots, and implements, with the involvement of teachers and principals,

evaluation and support systems consistent with the SEA’s adopted guidelines likely to lead to high-quality local teacher and principal evaluation and support systems?

Does the SEA have a process for reviewing and approving an LEA’s teacher and principal evaluation and support systems to ensure that they are consistent with the SEA’s guidelines and will result in the successful implementation of such systems?

Does the SEA have a process for ensuring that an LEA develops, adopts, pilots, and implements its teacher and principal evaluation and support systems with the involvement of teachers and principals?

Did the SEA describe the process it will use to ensure that all measures used in an LEA’s evaluation and support systems are valid, meaning measures that are clearly related to increasing student academic achievement and school performance, and are implemented in a consistent and high-quality manner across schools within an LEA (i.e., process for ensuring inter-rater reliability)?

Does the SEA have a process for ensuring that teachers working with special populations of students, such as students with disabilities and English Learners, are included in the LEA’s teacher and principal evaluation and support systems?

Is the SEA’s plan likely to be successful in ensuring that LEAs meet the timeline requirements by either (1) piloting evaluation and support systems no later than

the 20132014 school year and implementing evaluation and support systems consistent with the requirements described above no later than the 20142015 school

year; or (2) implementing these systems no later than the 20132014 school year?

Do timelines reflect a clear understanding of what steps will be necessary and reflect a logical sequencing and spacing of the key steps necessary to implement evaluation and support systems consistent with the required timelines?

Is the SEA plan for providing adequate guidance and other technical assistance to LEAs in developing and implementing teacher and principal evaluation and support systems likely to lead to successful implementation?

Is the pilot broad enough to gain sufficient feedback from a variety of types of educators, schools, and classrooms to inform full implementation of the LEA’s evaluation and support systems?


45

3.B Panel Response Tally of Peer Responses: 0 Yes; 6 No


Rationale

Overall, the plan lacks specificity, details, and processes that are necessary to determine the NDE’s ability to ensure that each LEA successfully develops, adopts, pilots, and implements evaluation and support systems (pp. 97-100).

Strengths

The Nevada legislature passed an evaluation-related bill, AB 222, in June 2011, which requires the development and implementation of a four-tiered teacher and principal evaluation and support system.

As outlined by AB 222, a 15-member TLC has been convened, and is charged with making recommendations that explicate the guidelines established in the State statute.

The TLC released their initial framework in February 2012, and outlined a timeline and deliverables that will guide the process of creating such an evaluation system.

AB 222 requires the school board of each local school district adopt a policy for evaluations in accordance with the state law (Attachment 10, p. 8).


There is insufficient evidence of a high-quality plan for ensuring that each district develops, pilots, and implements an evaluation and support system.

The NDE refers to its role as one that will “provide oversight and implement general supervision responsibilities” (p. 98), however, it fails to describe in detail what this role will look like in practice.

Guidelines are set to be adopted by June 2013, and implementation of the system is to begin in September 2013, leaving little time to plan for rollout in a thoughtful and comprehensive way.

There is no required pilot program, by statute or in NDE’s plan.

Student growth data, specifically, is not mandated as a significant component by state statute or the TLC’s early recommendations.


46



Accelerate the timeline for adoption of State guidelines.

Use the state-wide growth model as a required measure of individual educator performance, when available.

Specify the guidance and technical assistance the State will provide to districts in order to ensure successful implementation.

Create a pilot program in order to gather sufficient feedback from a variety of educators, schools, and classrooms to meaningfully inform districts’ full implementation of the evaluation system.

Ensure that the evaluation system includes and appropriately attributes learning data for all students with disabilities.

Take advantage of NDE’s membership in WIDA and SIOP training in the creation of NDE’s student learning objectives and follow-up professional development for teachers who work with English Learners.


If the SEA indicated that it has not developed and adopted all guidelines for local teacher and principal evaluation and support systems consistent with Principle 3 by selecting Option A in section 3.A, is the SEA’s plan for the SEA’s and LEAs’ development and implementation of teacher and principal evaluation and support systems comprehensive, coherent, and likely to increase the quality of instruction for students and improve student achievement? If not, what aspects are not addressed or need to be improved upon? If the SEA indicated that is has adopted guidelines for local teacher and principal evaluation and support systems consistent with Principle 3 by selecting Option B in section 3.A, are the SEA’s guidelines and the SEA’s process for ensuring, as applicable, LEA development, adoption, piloting, and implementation of evaluation and support systems comprehensive, coherent, and likely to increase the quality of instruction for students and improve student achievement? If not, what aspects are not addressed or need to be improved upon?


47

Principle 3 Overall Review Panel Response Tally of Peer Responses: 0 Yes; 6 No


Rationale The current proposed timeline does not ensure adoption of regulations in adherence with the timeline required by ESEA Flexibility.

Strengths

The Nevada legislature passed an evaluation-related bill, AB 222, in June 2011, which require the development and implementation of a four-tiered teacher and principal evaluation and support system.

As outlined by AB 222, a 15-member TLC has been convened, and is charged with making recommendations that explicate the guidelines established in the State statute.


The TLC will not present final recommendations to the State Board of Education until December 2012, with adoption even later, therefore it will not meet the deadline established by ESEA Flexibility.

The use of student growth data has not yet been decided upon as a measure of teacher and principal performance.


Accelerate the timeline for adoption of state guidelines to ensure compliance with ESEA Flexibility requirements.

Use the state-wide growth model as a required measure of individual educator performance, when available.

Specify the guidance and technical assistance the State will provide to districts in order to ensure successful implementation.

Create a pilot program in order to gather sufficient feedback from a variety of educators, schools, and classrooms to meaningfully inform LEAs’ full implementation of the evaluation system.

Propose a plan for a high-quality feedback loop at each stage of evaluation development and implementation to ensure that educators are involved in the creation of the educator evaluation system which will likely lead to greater stakeholder buy-in and support for implementation of the new system. Potential mechanisms for gathering feedback include a state-wide teacher and principal survey and/or focus groups and listening sessions in each local district.


48

Overall Request Evaluation

Did the SEA provide a comprehensive and coherent approach for implementing the waivers and principles in its request for the flexibility? Overall, is implementation of the SEA’s approach likely to increase the quality of instruction for students and improve student achievement? If not, what aspects are not addressed or need to be improved upon? Overall Request Evaluation Panel Response


Rationale NDE has the potential to develop a comprehensive and cohesive approach; however, there are several critical weaknesses and compliance issues that need to be addressed for this to occur.

Strengths

The panel endorses NDE’s theory of action, which will incentivize positive behavior through a “loose-tight” framework that provides accountability in exchange for increased autonomy.

NDE’s accountability system of recognition, supports, and intervention shows alignment of thinking across different types of schools (pp. 46-48).

The 1-5 star rating system makes performance visible and NSPF accountability index is innovative and based on multiple measures.

A high-caliber set of incentives for reward schools provides empowerment and recognition, which can be motivating for all schools.


None of the principles as outlined in NDE’s flexibility request is fully in compliance with ESEA Flexibility requirements, specifically:

In Principle 1, the timeline for full implementation of the CCSS in Mathematics does not meet the requirements,

In Principle 2, there are no AMOs based on proficiency targets,

In Principle 2, the timeline for focus schools’ intervention does not meet the requirements,

In Principle 2, there is no discussion of interventions for other low-performing Title I schools,

In Principle 3, the timeline for adoption of educator evaluation system guidelines does not meet requirements, and

In Principle 3, there is no mandatory use of student growth data in educator evaluations.

While NDE has promising strategies for some principles, there is a lack of robust, high-quality plans for implementation for any of the principles, as well as for addressing the needs of subgroups.


49



Look at the three principles as a whole and craft a strategy that integrates the work that will be involved to accomplish all of the requirements of ESEA Flexibility.

Create a cohesive, coherent plan that includes all the elements of a high-quality plan (e.g., key milestones and activities, detailed timelines, party or parties responsible, evidence, resources, and significant obstacles) to ensure that each principle is in full compliance with the requirements of ESEA Flexibility.

esea flexibility peer panel notes - ed · esea flexibility peer panel notes state ... esea...

Documents