accountability areas of interest & 2019 performance ... · sb 09-163 (colorado educational...

Accountability Areas of Interest & 2019 Performance

Framework Target Setting

Ashley Piche, Accountability & Data Analysis UnitMarie Huchton, Accountability & Data Analysis Unit

Alan Dillon, School Quality & Support Division

October 20181

Agenda

2

I. Overview of Accountability in Colorado

I. 2019 Performance Framework Target Setting

I. Performance Frameworks Areas of Interest

Overview of Educational Accountability in Colorado

3

Accountability in Colorado

The Elementary and Secondary Education Act is the foundational federal statute that governs school and district accountability

• Every Student Succeeds Act of 2015 (ESSA)

4

SB 09-163 (Colorado Educational Accountability Act) is the foundation of our state accountability requirements

• One set of outcome data to determine overall district and school accountability -- the Performance Frameworks

• Identify those schools and districts with the greatest need, in order to direct resources and support and potential consequences

SB 09-163 (Colorado Educational Accountability Act) is the foundation of our state accountability requirements

✓ One set of outcome data✓ Identify those schools and districts with the greatest

need

5

Performance Frameworks | Primary Purpose

What does it mean to earn a rating of “Distinction” or “Performance”?

6

Performance Frameworks | Areas of Interest

✓ Achievement, Growth and Postsecondary & Workforce Readiness (PWR) Weighting on Performance Frameworks

✓ Addition of Growth-to-Standard (criterion-based growth) to Performance Frameworks

✓ School and District Ratings

✓ Achievement, Growth & PWR Targets on Performance Frameworks

What indicators of student performance do we value most at the school and district levels?

What expectations do we hold for schools and districts in regards to student performance?

Winter/Spring 2019

Rulemaking

Winter/Spring 2019

Rulemaking

November 2018 Target Setting

What expectations do we hold for schools and districts in regards to

student performance?

7


8








Winter/Spring 2019

Rulemaking

Winter/Spring 2019

Rulemaking


SBE Role in Target Setting

State statute requires that…

…the state board shall set, reaffirm, or revise, as appropriate,ambitious but attainable statewide targets for the measuresused to determine the levels of attainment of the performanceindicators [growth, achievement, growth-to-standard, PWR]for the coming academic year with the goal of raising the levelof academic performance in the public schools throughout thestate.

9

Performance Framework Target Setting

10

Approaching

Meets

Exceeds

Performance Indicator

Sub-Indicator

Performance Indicators Requiring Targets

11

* Evidence-Based Reading and Writing (EBRW)

Performance Framework Target Setting

Norm-Based Percentile Rank Methodology

Distribution of schools within each sub-indicator by

content area for the baseline year

12

Option 1 ● Sub-Indicator targets and target setting methodology remains the same

Option 2● Sub-indicator targets aligned with student performance

levels● Can be implemented in 2019 and/or 2020

Option 3 ● Sub-indicator targets aligned with ESSA targets ● Can be implemented in 2019 and/or 2020

‘Other’ Option ● SBE requests for additional options?

2019 Performance Framework Target Setting | Options

13

Proposed Options for 2019 Target Setting:

CMAS Scale Scores: School & district accountability targets vs. individual student

performance levels

Schools & Districts | Mean Scale Score

Students| Scale Score

Option 2 | School & District Achievement Targets & Student Performance Levels

2018 Student Scale Score Distribution

15

2018 School Mean Scale ScoreDistribution


16

These graphs show the distributionof the All Students sub-indicatorratings as calculated using thecurrent normative cut-scorescompared against hypotheticalratings calculated from the CMASperformance level cut scores of 725-750-775.

Using cut scores anchored aroundthe CMAS performance levelssignificantly increases the proportionof schools receiving Approachingratings and decreases theproportions earning Exceeds andMeets ratings.


17


Current TargetsTargets Aligned to Student

Performance Levels(725-750-775)

Performance 72.1% (N=917) 64.1% (N=819)

Improvement 19.6% (N=255) 24.8% (N=317)

Priority Improvement 6% (N=77) 8.1% (N=104)

Turnaround 2.3% (N=29) 3.0% (N=38)

Total Elementary & Middle Schools

Identified for Support106 142

2018 Preliminary E&M School Performance Framework Ratings

2019 Performance Framework Target Setting | Options

18

Option 1 ● Sub-Indicator targets and target setting methodology remains the same

Option 2● Sub-indicator targets aligned with student performance

levels● Can be implemented in 2019 and/or 2020

Option 3 ● Sub-indicator targets aligned with ESSA targets ● Can be implemented in 2019 and/or 2020

‘Other’ Option ● SBE requests for additional options?

Proposed Options for 2019 Target Setting:

What indicators of student performance do we value most at the

school and district levels?

19


20








Winter/Spring 2019

Rulemaking

Winter/Spring 2019

Rulemaking


21

Performance Frameworks | Performance Indicators

Statute and board rule specify the performance indicators that the department shall use to set the annual school and district plan type/accreditation ratings:

I. Student longitudinal academic growthII. Student academic achievementIII. Student academic growth to standards (re-introduced by

2019 or 2020)IV. Postsecondary and workforce readiness (PWR)

Board rule currently specifies that the Growth and PWR indicators are given the most weight



22


Taking recommendations from the Technical Advisory Panel for Longitudinal Growth’s (TAP) into consideration in 2016, the state board decided to adopt the following

Performance Indicator weighting for Performance Frameworks:

Elementary/Middle Schools❖ 60% Growth❖ 40% Achievement

High Schools and Districts❖ 40% Growth❖ 30% Postsecondary Workforce Readiness❖ 30% Achievement

Reintroducing Growth-to-Standard Measure

23

Performance Frameworks | Growth to Standard

• State Statute (HB18-1355) requires as part of the school and district performance frameworks, a measure of “student academic growth-to-standard, based on students’ progress toward meeting the state standards or, for students who meet grade-level expectations on the state standards, progress toward higher levels of achievement”

• With the assessment transition from CSAP/TCAP to CMAS and the new language in HB18-1355, we have the opportunity to revise our growth to standard definition and calculation methodology

• Historically, the growth to standard has been closely related to student achievement

24

Performance Frameworks | Growth to Standard

Growth to Standard | Considerations for on-going conversation with the Technical Advisory Panel for Longitudinal Growth (TAP)

• For a target to be attainable, what’s the necessary proportion of students who’ve been successful at moving up historically?

• For a target to be ambitious, should we be looking at the trajectories of high-growth exemplar schools?

Timeline: Our goal is to have a new growth to standard measure developed by Winter/Spring 2019.

25


Proposed State Board Action: Revisit weighting on performance frameworks with the re-introduction of growth to standard in 2019

Timing: 2019 Winter/Spring Rulemaking

Policy: • State statute requires as part of the school and district performance

frameworks:o Student longitudinal academic growtho Student academic achievemento Student academic growth to standardso Postsecondary and workforce readiness (PWR)

• State board rule requires that growth and PWR be given the most weight


26


27








Winter/Spring 2019

Rulemaking

Winter/Spring 2019

Rulemaking


School & District Rating Distributions

28

Performance Frameworks | School & District Ratings

29


• The state board has discretion over how many schools and districts are identified for each plan type or accreditation rating

• Rating cut points were set in 2010 to the following proportions:

School Plan Type2010

# %

Performance 1092 67.2%

Improvement 337 20.7%

Priority Improvement 130 8.0%

Turnaround 67 4.1%

District Accreditation

Rating2010

# %

Distinction 14 7.7%

Accreditation 97 53.6%



Turnaround 7 3.9%

30


• The state board reviewed the distribution of schools and districts in each category in 2016 and decided to maintain 2014 proportions of schools in each category for the 2016 frameworks:

School Plan Type2016

# %

Performance 1198 70.5%



Turnaround 55 3.2%

District Accreditation

Rating2016

# %

Distinction 27 14.8%

Accreditation 102 55.7%



Turnaround 1 0.5%

31


School & District Ratings Criteria & Distribution | Options to Consider

Option A: Adjust Cut-Points for School & District Ratings• Set cut-points for school and district ratings that result in the desired

distribution of schools

Distribution of 2018 Preliminary Elementary & Middle School Performance Plan Ratings

32



Option 1: Adjust Cut-Points for School & District Ratings• Set cut-points for school and district ratings that result in the desired

distribution of schools

Example Distribution of Elementary & Middle School Performance Plan Ratings

33



Option B: Set Additional Criteria for Performance and Distinction Ratings• Set criteria for assignment of Performance (school) and Distinction (district)

ratings in addition to percentage of overall points earned on frameworks

34



Timing: ● 2019 Winter/Spring Rulemaking● Flexibility on year of implementation

Policy: ● State statute specifies the number of school and district rating levels (4

school ratings, 5 district ratings)

What questions do you have?

35

Accountability Areas of Interest & 2019 Performance Framework Target Setting��Ashley Piche, Accountability & Data Analysis Unit�Marie Huchton, Accountability & Data Analysis Unit�Alan Dillon, School Quality & Support Division��October 2018��AgendaOverview of Educational Accountability in ColoradoAccountability in ColoradoSlide Number 5Slide Number 6What expectations do we hold for schools and districts in regards to student performance?Slide Number 8SBE Role in Target SettingPerformance Framework Target SettingPerformance Indicators Requiring TargetsPerformance Framework Target Setting2019 Performance Framework Target Setting | OptionsCMAS Scale Scores: School & district accountability targets vs. individual student performance levelsOption 2 | School & District Achievement Targets & Student Performance LevelsOption 2 | School & District Achievement Targets & Student Performance LevelsOption 2 | School & District Achievement Targets & Student Performance Levels2019 Performance Framework Target Setting | OptionsWhat indicators of student performance do we value most at the school and district levels?Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25What does it mean to earn a rating of “Distinction” or “Performance”?Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34What questions do you have?

accountability areas of interest & 2019 performance ... · sb 09-163 (colorado educational...

Documents