School Designators

Alaska Principal’s ConferenceOctober 2001

Section 14.03.123. School accountability requires:

“Beginning August 2002, and during each of the following 12-month periods, the department shall assign each public school in each district the performance designator of distinguished, successful, deficient, or in crisis based on multiple measures, including student achievement”

School Designator Committee

Representation from School Boards, Administrators, NEA, PTA, Business

Six meetings since February 2000

Consultants from the Center for Assessment Brian Gong Richard Hill

Design Assumptions

School Designator System (SDS) should: be educationally constructive, valid,

reliable, fair, operationally feasible, politically acceptable, and legally defensible

focus on student performance be able to accommodate addition of other

indicators in the future be well coordinated with other supports for

educational improvement, e.g., report cards may report indicators not included in SDS

Assumptions - continued

School Designator System (SDS) should: be objective, and not require intensive

resources (i.e., should not require broad-based school “inspections,” etc.)

Consider Status (most recent performance) and Growth over time in calculating school ratings

be amenable to “raising the bar” over time be robust enough to deal with

developments in the future

The SDS shall be based on a combination of:

Status (most recent performance)

Growth (the increased performance of students in successive grades/years, e.g., from Grade 3 in Year 1 to Grade 4 in Year 2).

IndicatorsThe School Designator System (SDS) shall base school designations upon:

student performance on both state benchmark and HSGQE assessments and commercial norm-referenced tests, and

dropout (for high schools).

Additional indicators may be reported but not used for determining school designations. Other indicators may be added to the designation system in the future.

Summary: Calculating a designation

Status Assign points based on student performance in

relation to state student performance standards (e.g., Advanced, Proficient, etc.)

Average across content areas and grades

Growth Assign points based on student year-to-year growth

in relation to “one year’s standard growth” Average across grade-pairs and content areas

Overall [Status + (2 times Growth)] / 3

Designation Compare Overall to cutpoints to be established by

state board

Example: Status Index

The Status Index represents the average performance of all students on the most recent set of assessments, across all content areas and gradesAn index is used because it is: directly linked to performance standards reflects the entire range of performance reliable

Example: Calculating a Status Index for Reading, Benchmark Grade 3

Assign each student a number of “index points” based on her/his performance.

Student Performance Level

Performance Level Score

Index Points

Advanced 4 180

Proficient 3 120

Below Proficient 2 60

Not Proficient 1 20

Did not participate

blank 0

Example: Calculating a Status Index

Add points and divide by number of students to calculate average. This is the index.

Student Performance Level

Number of

Students

Index Points

Subtotal

Advanced 3 180 540

Proficient 10 120 1200

Below Proficient 5 60 300

Not Proficient 1 20 20

Did not participate

1 0 0

TOTAL 20 2060

Index Score (Grade 3, Reading) 103.0

Example: Calculating Status Index for school

Calculate weighted average across all content areas, all grades. (Assume only Grade 3 for simplicity.)

Status Index for Content Area

Number of Students

Status Index

Reading - Grade 3 20 103.0

Math - Grade 3 20 97.8

Status Index Score, Grade 3 100.4

Growth performance

Based on the improvement made by a class of students from Year 1 to Year 2 sensitive to “where students started” uses classes of students; may not track

exactly same students

Represented by an index based on the ratio of the growth achieved by the school in relation to “one year’s standard growth”

Example: Calculating a Growth Index for Reading, Grades 3 to 41. Determine student’s performance in

grade 3 on the benchmark exam.2. Determine student’s performance in

grade 4 on the NRT (translated to scale score comparable to the benchmark scale).

3. Subtract Grade 3 from Grade 4 performance to determine how much student progressed.

Particular emphasis on helping lower-achieving students grow

Define “lower-achieving” as those below a certain standard score (e.g., standard score that defines lowest 25% of students statewide in 2001)Growth for these students gets double weight (student counted twice)

Example: Calculating Growth Index – continued

Performance Score, Grade 4, 2001

Performance Score, Grade 3,

2000

Growth Difference

449 346 103

393 293 100

420 321 99

365 266 99

306 201 105

306 201 105

Average 101.8

Example: Calculating a Growth Index for a schoolCombine growth across all content areas and grades. (Limited example).

Growth Area Number of

Students

Growth Index

Reading, Grade 3 to 4

25 110.0

Math, Grade 3 to 4 25 102.0

Reading, Grade 4 to 5

25 103.0

Math, Grade 4 to 5 25 92.0

Growth Index 101.8

Assigning an Overall School Designation

[Status + (2 * Growth)] / 3 = Overall ScoreExample

Use cutscores to be determined by state Board of Education to assign school designations

(Hypothetical only) 140 and higher = Distinguished 100 - 139 = Satisfactory, etc.

Status 100.4

Growth 101.4

Overall [100.4+ (2*101.4)]/3 = (100.4 + 202.8)/3 = 101.1

Reporting

Various disaggregations shall be reported to the school, e.g., school performance on Status and Growth by content area assessed.A “degree of certainty” (reliability or decision consistency) should be reported with each school designation. The degree of certainty should be taken into consideration when interpreting the school designation.

Implementation View – year available

Indicator Status

Growth

Notes

Primary Profile ’03 ’06 If growth to Gr. 3

Gr. 3 test ’02 ’03 Tracking in ’04

Gr. 4 test ’02 ’02* Tracking cohorts

Gr. 5 test ’02 ’03

Gr. 6 test ’02 ’03

Gr. 7 test ’02 ’03

Gr. 8 test ’02 ’03

Gr. 9 test ’02 ’03

Gr. 10 test ’02 ’03 ’02 after Aug.

Persistence (Dropout)

’02 ’02

Percent Graduating ’02 ’02

Post-grad. Success ’03 ’04 ’03 start at earliest

Local assessments ‘03 ‘04 ’03 start at earliest

Issues and ChallengesStatutory deadline- August 2002

Limitations of basic design Single indicator Status only for 2002 Scaffolds, support, intervention

Technical Issues Technical study not completed Problems associated with designating small

schools Problems associated with designating high

schools

Issues and Challenges -Continued

Timing Issues Impact of SB 133

HSGQE Standard Setting

Indicators Considered By Committee

NRT, benchmark exams, HSGQEgrowth, statusother student assessments (local by district, school)AttendanceStake holders’ involvementDrop out (high school and pre-high school)percent graduating seniorsTruancyTeacher certificationStaff developmentSchool safetyDiscipline data (suspensions, expulsions for drugs or weapons, violence) Transience ratePrincipal.staff tenure (number of years in building)Parent/teacher conference attendanceparent involvement (e.g., fill out teacher evaluation)

Indicators - continued

progress in relation to self-determined identify; ability to determine who you are (identify, mission)high school course offeringspupil/teacher ratio (over grades taught)bilingual education, ESL, immersionlocal indicatorsschool-business partnershipssurrogate parentingpoverty levellinguistic/cultural diversityparent’s/mother’s level of educationstake holders’ powerschool-community relations (e.g., school contribution to community economic viability)school’s curriculum focus in relation to content standards

Indicators - continued

student involvement in extra curricular programs (academic decathlon, sports)extensiveness of programshigh school accreditationpromotion/retention ratespercent of staff new to profession in schoolafter school programs (e.g., summer school)becoming teen parentstudents in alternative education programsparent satisfactionpreventative measureskindergarten developmental profilecourse completion rates for specific courses (e.g., algebra)college entrance exams (participation rates, scores)awards, honors, recognitions, achievements by school and its students (e.g., scholarships)

Indicators - continued

geographic isolation/community resourcesamount of environmental printBaldridge survey indicatorsphysical facilitiespost-secondary activityschool improvement plan

Questions, Comments, Concerns