Selecting Demonstrable Improvement Indicators

September 18, 2015

Ira Schwartz Assistant Commissioner of Accountability

Shibu JosephAssociate, Education Research

Goals for Today’s Webinar

• Provide overview of Demonstrable Improvement Process.• Explain how to use the Demonstrable Improvement

Spreadsheet to Select Indicators.• Clarify Next Steps and How to Request Assistance.• Answer Questions, which you can send to

receivership@nysed.gov.

2

Demonstrable Improvement Recap

For more details, please read the PowerPoint presentation “Making Demonstrable Improvement: Request for Feedback” posted online at:

http://p12.nysed.gov/accountability/de/SchoolReceivership.html

Requirements for Demonstrable Improvement

• Persistently Struggling Schools must annually make Demonstrable Improvement or they will be placed in Independent Receivership.

• Struggling Schools must annually make Demonstrable Improvement or after a two-year period they will be placed in Independent Receivership.

• Commissioner shall consider: Performance on Metrics (Indicators) Years of Identification Superintendent’s Successful use of Receivership Powers to

Implement the School’s Plan

4

Guiding Principles

• One or more indicator(s) shall be established for each Metric specified in legislation.

• For each indicator, a school can make progress by achieving either a “universal goal” or a school specific progress target.

• The indicator goals and the school specific targets generally increase over the three-year period.

• Most indicators will be based on student performance; some indicators will be based on implementation of programs and/or processes.

• The State Education Department will select some of the school indicators, and the School Receiver in consultation with the Community Engagement Team shall select some.

• Selected indicators will be based primarily on where the school’s performance needs the most improvement.

• School Receiver may seek to have local measures approved by the Commissioner.

• The result of the process shall be a judgment that a) the school made Demonstrable Improvement, b) the school did not make Demonstrable Improvement unless there are shown to be extenuating or extraordinary circumstances, or c) the Commissioner shall review the totality of the school's record to make a determination as to whether or not the school made Demonstrable Improvement.

5

How Demonstrable Improvement is Determined

• Some indicators will be considered Level 1; some indicators will be considered Level 2; some indicators may be either Level 1 or 2, depending on circumstances.

• A minimum of five Level 1 and five Level 2 indicators will be selected for a school; a school with elementary-middle and secondary grades will have to select seven Level 1 and seven Level 2 indicators, indicators from both grade levels must be included.

• Both Level 1 and Level 2 indicators shall be weighted 50% in computing the Demonstrable Improvement Index.

• Each indicator within Level 1 and Level 2 shall be weighted equally.• The Demonstrable Improvement Index shall range from 0% to 100%.• If a school achieves an index of 67% or higher, the school has made

demonstrable improvement. If a school achieves below 40%, it has not, unless the school can demonstrate it would have achieved 67% of its goals absent extenuating or extraordinary circumstances.

• The Commissioner shall review the entirety of the record and after consulting with district and Community Engagement Team determine whether a school with an index of 40% or higher, but less than 67% shall be considered to have made Demonstrable Improvement.

6

Level 1 Indicators

• There are seven Level 1 indicators for elementary and middle schools and seven for high schools. A school with both elementary-middle and high school grades will have twelve indicators.

• If a school’s performance is below a Level 1 goal for an indicator, that indicator will be assigned to a school. If there are five or more indicators for which the school is below the 2015-16 goal then all of these will be used.

• If there are fewer than five (seven for schools with EM & HS Grades), then the associated Level 2 indicators will be used as Level 1 indicators.

• If the combined Level 1 indicators and associated Level 2 indicators are still fewer than five (seven for schools with EM & HS Grades), then the Level 1 indicators for which the school is above the 2015-16 goal can be used.

• An associated Level 2 indicator is a Level 1 indicator used for a specific accountability group (e.g., the percentage of students with disabilities in Grades 3-8 math performing at or above Level 2 is an associated Level 2 indicator for the Level 1 Grades 3-8 math indicator.)

• After year 1, if a school’s Level 1 indicator falls below the 2016-17 goal, then that indicator will be assigned as an additional Level 1 indicator to the school for 2016-17 and 2017-18.

7

The Level 1 Indicators

Elementary and Middle:

• Making Priority School Progress

• Percent of Students at or above Level 2 in ELA

• Percent of Students at or above Level 2 in math

• Mean Student Growth Percentile in ELA

• Mean Student Growth Percentile in math

• Percent of Students at or Above Level 3 in Science

• Serious Incidents (VADIR)

High School:

• Making Priority School Progress

• 4-year High School Graduation Rate

• 5-year High School Graduation• Percent of Students Graduating

with Regents Diploma with Advanced Designation

• Percent of 10th graders passing Math Regents

• Percent of 11th graders passing ELA Regents

• Serious Incidents (VADIR)

8

Level 2 Indicators

Level 2 Indicators Include:• Indicators for students subgroups (i.e., English language learners, low-

income students, racial/ethnic groups and students with disabilities).• Implementing a Community School Model, expanded learning time and

other key system initiatives.• School climate (e.g., attendance, suspensions).• Gaps between a student group and students who are not members of the

group (e.g., between students with disabilities and students without

disabilities).• Students passing courses.• High School Student Promotion Rates (promoted from grades 9,10 & 11).• College- and Career- Readiness.• Developmentally Appropriate Child Assessments: Pre-K to Third Grades.• Teachers Teaching out of Certification Area.• Teacher Turnover.• Post-graduation plans for students.• Local measures approved by the Commissioner.

9

Computing the Demonstrable Improvement Index: Example

10

Indicator Level Performance Goal Indicator Made Weighting

Made Priority School Progress Level 1 Did Not Make Progress Make Progress No 0%

Grades 3-8 math percent at or above Level 2 Level 1 28% 26% Yes 10%

Grades 3-8 ELA all students SGP Level 1 48% 46% Yes 10%

Grades 4 and 8 Science percent at or above Level 3 Level 1 35% 39% No 0%

Grades 3-8 Math SGP Level 1 45% 47% No 0%Implement Community School Model Level 2 First Year Implementation First Year

Implementation Yes 7.14%

Expanded Learning Time Level 2 Implement Program Program Implemented Yes 7.14%DTSDE Teacher Practices and Decision Making Level 2 Developing Developing Yes 7.14%

Percent of Newly Hired Teachers with State Provided Growth Ratings of Effective or Above

Level 2 80% 90% No 0%

Grades 3-8 ELA low-income SGP Level 2 52% 51% Yes 7.14%

Chronic Absenteeism Level 2 (Local Indicator)

Fewer than 29% of students chronically absent

30% Chronically Absent (For Example) No 0%

ELL students gaining one Level on NYSESLAT

Level 2 (Local Indicator) 41% 50% Yes 7.14%

Index Result 55.70%

Determining Demonstrable Improvement

• In this example, because the school’s Demonstrable Improvement Index is above 40%, but below 67%, the Commissioner reviews the entire performance of the school.

• After the review, the Commissioner determines the school has made Demonstrable Improvement.

• Note: Goals and progress targets become more rigorous in Years 2 and 3.

11

Additional Information About the Index

• Once selected, indicators remain in place for three years.• If a determination cannot be made in the current year about an

indicator, the weights for that year will be adjusted. • If a determination cannot be made in a future year about an

indicator, another indicator may be selected.• After year 1, if a school’s Level 1 indicator falls below the 2016-

17 goal, then that indicator will be assigned as an additional Level 1 indicator to the school for 2016-17 and 2017-18.

• After year 1, the Department will be reviewing the goals and progress targets for 2016-17 and 2017-18. If any are determined to be insufficiently or overly rigorous, the Department reserves the right to adjust them.

• If a school meets the criteria for removal from Priority Status, the district may petition for removal, even if the school does not make Demonstrable Improvement.

12

Selecting Demonstrable Improvement Indicators

Opening the Excel Spreadsheet File

NYSED has e-mailed to each Superintendent/EPO a Microsoft Excel spreadsheet file with the list of Demonstrable Improvement indicators available for selection for each of the Persistently Struggling and Struggling Schools in the district.

Please follow these steps to select the indicators:• Open the Microsoft Excel file with the Demonstrable Improvement indicators

data. • It is possible that the file will open with a notification “Security Warning:

Macros have been disabled. Enable Content.” If so, please click on “Enable Content” for the file to work properly.

A macro is a set of instructions that are programmed into the spreadsheet to automate tasks.

14

Opening the Excel Spreadsheet File

• It is also possible that the file will open with a notification “Protected View Office has detected a problem with this file. Editing it may harm your computer. Click for more details.” If so, click on the message and select the option “Edit Anyway.”

• It will now display the message “Security Warning: Macros have been disabled. Enable Content.” Please click on “Enable Content” for the file to work properly.

15

Step 1

Step 2

Step 3

Demonstrable Indicators File

The file has two tabs; school_worksheet and district_worksheet. Please review the data on the district_worksheet Tab first.

16

Demonstrable Indicators File: district_worksheet Tab

Even though the file will open to the “school_worksheet” Tab, please review the “district_worksheet” first to get a better understanding of which indicators the school(s) are below the 2015-16 goals.

• The tab “district_worksheet” is a database with indicator information for all the Persistently Struggling and Struggling Schools in the district. This tab also has the baseline for the school(s), goals and progress targets for 2015-16, 2016-17 and 2017-18.

• The “district_worksheet” tab has filters enabled for each of the columns; that means you can make data visible or hide data for the filters you choose.

• This spreadsheet will have all the indicators listed, even if the school did not have baseline data for the specific indicator. Therefore, use the filter to hide unwanted rows.

17

district_worksheet Tab

Here is a partial view of the district_worksheet Tab filtered by Status column to hide indicators with “NA/No Baseline Available” status.

18

Demonstrable Indicators File: school_worksheet Tab

19

1. The Excel file will open with the “school_worksheet” Tab visible. This is the tab from where indicators have to be selected for the respective schools.

2. The “school_worksheet” will open with no data for any school visible.

3. When the school is selected from the drop down menu the data gets populated for that school.

Pre-selected Level 1 Indicators and Selecting Level 2 as Level 1 Indicators

4. An elementary-middle (EM) or a high school (HS) must have a minimum of five Level 1 indicators; a school with both EM and HS grades must have a minimum of seven Level 1 indicators.

5. Level 1 indicators for which the school’s baseline is below the 2015-16 goal will be highlighted in yellow. These indicators have been pre-selected by the Department and cannot be changed.

6. An indicator counter is provided in cell C6 to display how many Level 1 and Level 2 as Level 1 indicators are selected.

7. If the school does not have the minimum required number of Level 1 indicators pre-selected, a message will be displayed stating “A school must have a minimum of 5 (7, if the school has EM and HS grades) Level 1 or Level 2 as Level 1 indicators.”

Demonstrable Indicators File: school_worksheet Tab

20

8. If this message is displayed, the school must select Level 2 as Level 1 indicators to meet the minimum requirement of five or seven indicators. Most of the Level 2 indicators will have a drop down in the “Level” column that will allow the indicator to be selected as a Level 2 as Level 1 indicator.

9. Once the minimum count of Level 1 or Level 2 as Level 1 indicators are reached, the message “A school must have a minimum of 5(7) Level 1 or Level 2 as Level 1 indicators” will disappear.

10. If the file does not display the message “A school must have a minimum of 5(7) Level 1 or Level 2 as Level 1 indicators,” then the school is not required to select any Level 2 as Level 1 indicators.

11. A Level 2 as Level 1 indicator will be weighted the same as a Level 1 indicator for calculating the Demonstrable Improvement index.

Demonstrable Indicators File: school_worksheet Tab

21

Selecting Level 2 Indicators

12. A school with EM or HS grades must have a minimum of five Level 2 indicators; a school with both EM and HS grades must have a minimum of seven Level 2 indicators.

13. Level 2 indicators for which the school’s baseline is below the 2015-16 goals will be available for selection.

14. Schools must select the minimum five (seven, if the school has EM and HS grades) Level 2 indicators.

15. If the school does not have the minimum required count of Level 2 indicators selected, a message will be displayed stating “A school must have a minimum of 5 (7, if the school has EM and HS grades) Level 2 indicators.”

16. Once the minimum count of five (seven) Level 2 indicators is reached, the message “A school must select a minimum of 5 (7) Level 2 indicators” will disappear.

17. Schools can select more than the minimum five (seven) Level 2 indicators.

18. An indicator counter is provided in cell C7 to display how many Level 2 indicators are selected.

19. Schools can select only one DTSDE Tenet Indicator. If more than one DTSDE Tenet Indicator is selected, a message will be displayed “Warning: Only one DTSDE Tenet Indicator can be selected.” Please deselect the additional DTSDE Tenet Indicator(s).

Demonstrable Indicators File: school_worksheet Tab

22

Schools with Insufficient Level 1, Level 2 as Level 1 and Level 2 Indicators Available for Selection

1. There are a few schools statewide with either:

a. Insufficient indicators available for selection.

b. Limited choice of indicators available for selection (e.g., majority of indicators available for selection are gap related or student growth related).

2. Districts must seek permission from the Department to add indicators for which their baselines are already above the 2015-16 goals.

3. The “above the goal” indicators selected must be spread across various performance areas.

Selecting the Level 1 Indicators: Example – School B

In the school_worksheet Tab, click on the school drop down menu and select a school. Let us select School B.

23

Selecting the Level 1 Indicators: Example – School B

1. The spreadsheet will be populated with Level 1 indicators pre-selected by the Department and Level 2 indicators available for selection.

2. The counter on cell C6 indicates only 4 Level 1 Indicators were pre-selected by the Department. The school needs to select one additional “Level 2 as Level 1” indicator to reach the minimum of 5 for this school.

3. Please note the message on cell D6.

24

Selecting the Level 1 Indicators: Example – School B

4. The school decided to select 3-8 ELA Hispanic Students Level 2 and above as the Level 2 as Level 1 indicator.

5. For the indicator, click on the Level 2 cell and select from the drop down Level 2 as Level 1. This will automatically highlight the cell and update the Indicator Selector to Yes. The counter on cell C6 has incremented to 5.

6. Please note that the message on cell D6 has disappeared. The message on cell D7 will remain till Level 2 indicators are selected.

25

Selecting the Level 2 Indicators: Example – School B

1. The school has to select 5 Level 2 indicators.

2. The school selects the 5 Level 2 indicators by selecting “Yes” from the Indicator Selected drop down menu. The counter on cell C7 has incremented to 5.

3. Please note that the message on cell D7 has disappeared.

4. Since the school has selected Chronic Absenteeism as one of the indicators for which goals/progress targets are not available, a message is displayed on cell I5.

5. After the minimum Level 1 and Level 2 indicators are selected, click on the Indicator Selected column, drop down the filter menu and uncheck the “No” filter. This will hide the unselected indicators.

26

Printing and Saving the File

1. For printing, click on File menu, select Print and click on Show Print Preview.

2. The selected Indicators will print on legal size paper.

3. If you see unselected indicators during Print Preview, repeat Step 5 on Slide 26.

4. The file may be saved with the respective school’s name.

27

Schools with Insufficient Indicators: Example – School C

1. School C has only one Level 1 indicator pre-selected by the Department and a limited choice of available indicators for selection.

2. With the permission of the Department, the school may select indicators with the status “Meets Goal/ Not Selected by SED” or “Meets Goal/Not Available for Selection.”

3. The Status column has a drop down filter menu to select indicators with the aforementioned status.

4. School C selected Indicator numbers 11,12, 15 and 33 for which the school is already above the 2015-16 goal. To make Demonstrable Improvement the school’s 2015-16 performance still needs to be at or above the 2015-16 goal or progress target.

28

TBD Indicator: Teacher Attendance

Teacher Attendance along with Chronic Absenteeism are “to be determined” Level 2 indicators the Department has not defined or set baselines, goals or progress targets. We use the Federal definition for Teacher Attendance given below.

During the regular school year, the average percentage of days that teachers are present when they would otherwise be expected to be teaching students in an assigned class. Teachers should not be considered present for days taken for sick leave and/or personal leave. Personal leave includes voluntary absences for reasons other than sick leave.

If a school selects Teacher Attendance as one of its indicators for Demonstrable Improvement, the district must be able to submit data using the above definition so that the Department can set the 2013-14 school year baseline. Goals and progress targets will then be set for the 2015-16, 2016-17 and 2017-18 school years.

29

TBD Indicators: Chronic Absenteeism

Schools in Receivership that elect to include tracking and monitoring chronic absenteeism as an Indicator can incorporate a three-year phase in plan as specified below:  

• Year 1: by December 1, 2015, the district will develop a comprehensive chronic absenteeism policy for each school using the indicator and an implementation plan aimed at reducing chronic absenteeism. Such policies and plans will include tracking individual student absences on a regular basis defined as preferably weekly, but no less than monthly; a process of identifying students who are missing 5% or more of the days that the student has been enrolled and school has been in session; prevention and intervention strategies, where appropriate, for students missing  5% or more of the days the student has been enrolled and school is in session; prevention and intervention strategies for students missing  10% or more of the days the student has been enrolled and school is in session; and a way to track overall chronic absenteeism rates. By no later than February 1, 2016, each school will begin to implement its chronic absenteeism plan.

• Year 2: Each school will review the results of the chronic absenteeism policy and implementation plan and will create a base year of chronic absenteeism data as well as adapt and amend the policy and plan to meet the individual needs of the school. This year should also be used to engage with and involve the community by sharing the chronic absenteeism process results from Year 1 and incorporating feedback into the plan for Year 2.  

• Year 3: Each school will reduce its rate of chronic absenteeism. Based on the data from Year 2, each school will reduce the number of students who are chronically absent by utilizing the policies and procedures. The school will also continue to engage with and involve the community in an effort to further reduce chronic absenteeism into future years.

• Districts that already have the ability to compute a chronic absenteeism rate may instead beginning in Year 1 or 2 submit an indicator based on reduction of chronic absenteeism. 

30

Local Indicator Templates

The Department has shared two templates (performance data and climate survey) with examples that districts can use as guides to submit Local Indicator proposal(s) for approval.

31

Local Indicator Templates

32

Next Steps

• Please discuss the selected Demonstrable Indicators with the school’s Community Engagement Team.

• Submit the file with the selected indicators to receivership@nysed.gov by Friday, October 2, 2015.

• If you need technical assistance with the Excel file please call (718) 722-4553 or send an e-mail to receivership@nysed.gov.

• Local Indicator proposal(s) must be submitted by Friday, October 2, 2015.

• We will make a recording of this webinar available at: http://p12.nysed.gov/accountability/de/SchoolReceivership.html

33