m3 reviewing the slo-sso-final

1

Student Learning/Support

Objectives (SLO/SSO)

-Reviewing Module-

Student Learning/Support

Objectives

2

Module 3

Reviewing the SLO/SSO

3

ReviewGoal• Reviewing technically rigorous student learning/support

objectives (SLO/SSO) for use in guiding instruction (and/or

support services); while determining student mastery or growth

as part of an educator effectiveness system.

Objectives

• Participants will :

1. Conduct a multi-faceted, quality assurance review of the

student learning/support objectives for:

A. Completeness

B. Comprehensiveness (see Quick Start Training

Module*)

C. Coherency

Final Outcome

4

A refined SLO/SSO Form with:

• Rigorous, high-quality performance

measures designed to measure the

targeted content/professional

standards.

• Areas identified for further

improvement/consideration (as

documented within the Coherency

Rubric)

5

Helpful Resources

Participants should consult the following:

Training

• Handout #3: QA Checklist

Step 5: Quality Assurance of the SLO/SSO Form

Templates

• Template #3: Coherency Rubric

• Template #3a: Performance Measure Rubric

Resources• Help Desk

• Models: Scored Examples

Participants should consult the following:

Completion of the SLO Form

• Handout #3-Quality Assurance Checklist

Comprehensiveness of the Performance

Measures

• Template #3a-Performance Measure Rubric

Coherency of the SLO Design

• Template #3-Coherency Rubric

6

Helpful Resources (cont.)

Process Components

7

STEP #1 Goal

STEP #2 Standards

STEP #3 Blueprint

STEP #4 Form

STEP #5 QA

8

• Check the drafted SLO/SSO (including the

performance measures) for quality.

• Refine the measures and targets.

• Edit the text and prepare discussion points and

highlights for the principal/supervisor.

• Update the completed SLO/SSO form with

performance data.

REVIEW Phase

9

Review Phase Components

ReviewPreview

Checklist &

Rubrics

Completion: QA Checklist

Comprehensiveness: Performance Measure

Rubric

Coherency: Coherency

Rubric

10

Task Structure

QA 1. Completeness

Is the SLO/SSO Form completed correctly?

2. Comprehensiveness

Are the assessments of high technical quality?

3. Coherency

Are the SLO/SSO components aligned to each other?

11

STEP 5

Quality Assurance

12

Checklist & Rubric Preview

1. Apply the QA Checklist (Handout #3) to a completed

SLO or SSO Form.

A. What information is needed?

B. Who is the SLO/SSO focused on?

2. Preview the three (3) strands within the Performance

Measure Rubric (Template #3a) for each assessment

identified within the SLO/SSO.

A. What is the purpose of the assessment/performance

measure?

B. What standards does it purport to measure?

C. What technical data is provided/known?

13

Checklist & Rubric Preview

(cont.)

3. Preview the three (3) phases within the Coherency

Rubric for the SLO or SSO Form.

A. How well are the SLO/SSO components aligned to

each other?

B. How well do the identified assessments/

performance measures aligned to the SLO’s/SSO’s

stated goal?

14

Quality Assurance

Checklist

15

Quality Assurance Checklist

• The checklist is designed to verify each

element within the four (4) sections of the

SLO/SSO

• The checklist applies the business rules to

each element. The Help Desk document

provides examples for each element.

SLO/SSO Completeness


16

Element Definition

1.1 Content Area Name of the content area upon which the SLO is based.

1.2 Course Name of the specific course/subject upon which the SLO is based.

1.3 Grade LevelGrade levels for students included in the course/subject in Element

1.2.

1.4 Total Students

Aggregate number of students (estimated, across multiple sections)

for whom data will be collected.

1.5 Average Class SizeThe average number of students in a single session of the

course/subject identified in Element 1.2.

1.6 Class FrequencyThe frequency (within the given timeframe) of the course/subject

identified in Element 1.2.

1.7 Instructional

Setting

The location or setting where the course/subject instruction is

provided.

1.8 Instructional

IntervalThe time frame of the course/subject identified in Element 1.2.

SLO Section I: Context


(cont.)

17

Element Definition

1.1 Service AreaName of the primary service area (e.g., speech) upon which the

SSO is based.

1.2 Service Location Name of the location(s) services are provided.

1.3 Grade LevelGrade level(s) of students and/or educator-type services are

provided.

1.4 Total RecipientsAggregate number of students and/or educators for whom data will

be collected.

1.5 Average

Case SizeThe “average” number of recipients of the services identified in

Element 1.4.

1.6 Service Frequency

The typical frequency (within the identified service interval-

Element 1.8) services are provided to recipients identified in

Element 1.4.

1.7 Service SettingThe contextual setting (e.g., school library, student’s home) services

are provided.

1.8 Service Interval The typical time frame of the service model.

SSO Section I: Setting

18

SLO Section II: Goal

Element Definition

2.1 Goal Statement

A narrative that articulates a key concept upon which the SLO is based. The statement addresses What,

Why, and How.

2.2 Content Standards Targeted Content Standards, which are the foundation of performance measures, used to develop the SLO.

2.3 Instructional StrategyThe approach used to facilitate learning the key concept articulated in the Goal Statement and delineated

among the Targeted Content Standards.

SLO Section III: Objective

Element Definition

3.1 Starting Point

(Baseline)

The baseline data used for comparing student results at the end of the instructional interval.

3.2 Objectives

(Whole Class)

The expected level of achievement for the entire student learning objective (SLO) population (as defined in

Element 1.4).

3.3 Objectives

(Focused Students)

The expected level of achievement for a subset of the SLO population (as defined in Element 1.4).

3.4 End Point

(Combined)

At the end of the instructional interval, the aggregate performance classification as delineated by four,

empirical ranges (i.e., Unsatisfactory, Emerging, Effective, and Distinguished).


(cont.)

19

SSO Section II: Goal

Element Definition

2.1 Goal StatementA narrative that articulates a key concept upon which the SSO is based. The statement addresses What,

Why, and How.

2.2 Targeted Professional

Standards

Targeted Professional Standards outline the requirements an organization must fulfill to ensure that

products and services consistently meet customers' requirements. Content standards may also be

identified for those individuals providing instructional services.

2.3 Implementation StrategyThe approach used to attain the primary service goal articulated in the Goal Statement and delineated

among the Targeted Professional Standards.

SSO Section III: Objective

Element Definition

3.1 Starting Point (Baseline) The baseline data used for comparing client results at the end of the instructional interval.

3.2 Objectives (All Clients) The expected level of performance for the entire client population (as defined in Element 1.4).

3.3 Objectives (Focused

Clients)The expected level of performance for a subset of the client population (as defined in Element 1.4).

3.4 End Point (Combined)At the end of the service interval, the aggregate performance classification as delineated by four,

empirical ranges (i.e., Unsatisfactory, Emerging, Effective, and Distinguished).


(cont.)

20

SLO Section IV: Performance Measure

Element Definition

4.1 Name The name of each performance measure for which an objective is established in Element 3.2.

4.2 PurposeThe purpose statement for each performance measure that outlines: (a) What the assessment measures, (b) How

to use the scores, and (c) Why the assessment was developed.

4.3 Content StandardsThe Targeted Content Standards (the foundation of performance measures) used to develop SLOs. The content

standards are those aligned with each performance measure.

4.4 Performance TargetsUsing the scoring tools for each performance measure (as listed in Element 4.1), the expected level of

achievement for each student in the SLO population (as defined in Element 1.4).

4.5 Metric The metric by which the performance measure evaluates the performance target.

4.6 Administration

The administrative steps before, during, and after the assessment window, as well as the step-by-step

procedures during each phase of administration, including: (a) the requirements for completing the

performance measure, including accommodations, equipment, and materials, (b) standard time allotments to

complete the overall performance measure, and (c) standard scripts that educators read to give directions for

completing the performance measure.

4.7 Scoring ToolsScoring Keys: Objective Measures

Scoring Rubric: Subjective Measures

4.8 Results

The number of students participating in the performance measure

The number of students who met the target as stated in Element 4.4

The percentage of students who met the target as stated in Element 4.4

SLO RatingOne of four performance levels that the principal (or the evaluator) identifies after noting the actual

performance in respect to each objective stated in the SLO.

Notes and ExplanationSpace for the educator to note influences, factors, and other conditions associated with the SLO Rating, as well

as to reflect on a purposeful review of the data.


(cont.)

21

SSO Section IV: Performance Measure

Element Definition

4.1 Name The name of each performance measure for which an objective is established in Element 3.2.

4.2 PurposeThe purpose statement for each performance measure that outlines: (a) What the measure is evaluating, (b)

How to use the scores, and (c) Why the measure was developed.

4.3 Professional StandardsThe Professional Standards (the foundation of measures) used to develop SSOs. The professional standards are

those aligned with each identified measure.

4.4 Performance TargetsUsing the scoring tools for each performance measure (as listed in Element 4.1), the expected level of

attainment for each client in the SSO population (as defined in Element 1.4).

4.5 Metric The metric by which the performance measure evaluates the performance target.

4.6 Administration

The administrative steps before, during, and after the evaluation window, as well as the step-by-step

procedures during each phase of administration The requirements for completing the performance measure,

including accommodations, equipment, and materials The standard time to complete the overall evaluation.

4.7 Scoring ToolsScoring Keys: Objective Measures

Scoring Rubric: Subjective Measures; Data collection mechanisms

4.8 Results

• The number of clients participating in the performance measure.

• The number of clients who met the target as stated in Element 4.4.

• The percentage of clients who met the target as stated in Element 4.4

SSO ResultsOne of four performance levels that the supervisor identifies after noting the actual performance in respect to

each objective stated in the SSO.

Notes and ExplanationsSpace for the professional to note influences, factors, and other conditions associated with the SSO Rating, as

well as to reflect on a purposeful review of the data.


(cont.)

22

Procedural Steps

Step 1. Select the drafted SLO/SSO, including applicable

performance measures.

Step 2. Beginning with Section I, use the Handout #3-Quality

Assurance Checklist and evaluate each element.

Step 3. Identify any element with missing or incorrect

information (i.e., statement or data placed in the wrong

element of the SLO/SSO Form).

Step 4. Flag any element needing refinement or further

discussion with other educators/professionals and/or

the principal/supervisor.

Step 5. Repeat Steps 2 through 4 with the other sections on the

SLO/SSO Form.

23

Performance Measure

Rubric

24

Performance Measure Rubric

• The rubric is designed to examine the quality

characteristics of teacher-made performance

measures. The rubric is comprised of 18 technical

descriptors organized into three strands.

• The rubric’s purpose is to provide teachers with a

self-assessment tool that assists in building high

quality measures of student achievement.

SLO/SSO Comprehensiveness

25

• For vendor-developed assessments, examine the technical evidence to determine if the tool is designed to measure the Targeted Content/Professional Standards.

• For locally-developed assessments, follow the guidelines in the Quick Start training to create high-quality performance measures, and then apply Template 3a.

Performance Measure Rubric

26

Performance Measure Rubric (cont.)

Task ID Descriptor Rating

1.1The purpose of the performance measure is explicitly stated (who, what,

why).

1.2

The performance measure has targeted content standards representing a

range of knowledge and skills students are expected to know and

demonstrate.

1.3

The performance measure’s design is appropriate for the intended audience

and reflects challenging material needed to develop higher-order thinking

skills.

1.4

Specification tables articulate the number of items/tasks, item/task types,

passage readability, and other information about the performance measure -

OR- Blueprints are used to align items/tasks to targeted content standards.

1.5

Items/tasks are rigorous (designed to measure a range of cognitive

demands/higher-order thinking skills at developmentally appropriate levels)

and of sufficient quantities to measure the depth and breadth of the targeted

content standards.

Strand I: Design

27



2.1

Items/tasks and score keys are developed using standardized procedures, including

scoring rubrics for human-scored, open-ended questions (e.g., short constructed

response, writing prompts, performance tasks, etc.).

2.2

Item/tasks are created and reviewed in terms of: (a) alignment to the targeted content

standards, (b) content accuracy, (c) developmental appropriateness, (d) cognitive

demand, and (e) bias, sensitivity, and fairness.

2.3

Administrative guidelines are developed that contain the step-by-step procedures used to

administer the performance measure in a consistent manner, including scripts to orally

communicate directions to students, day and time constraints, and allowable

accommodations/adaptations.

2.4

Scoring guidelines are developed for human-scored items/tasks to promote score

consistency across items/tasks and among different scorers. These guidelines articulate

point values for each item/task used to combine results into an overall score.

2.5

Summary scores are reported using both raw score points and performance level.

Performance levels reflect the range of scores possible on the assessment and use terms

or symbols to denote each level.

2.6

The total time to administer the performance measure is developmentally appropriate

for the test-taker. Generally, this is 30 minutes or less for young students and up to 60

minutes per session for older students (high school).

Strand II: Build

28



3.1

The performance measures are reviewed in terms of design fidelity:

Items/tasks are distributed based upon the design properties found within the specification or

blueprint documents;

Item/task and form statistics are used to examine levels of difficulty, complexity, distractor

quality, and other properties; and,

Items/tasks and forms are rigorous and free of bias, sensitive, or unfair characteristics.

3.2

The performance measure was reviewed in terms of editorial soundness, while ensuring consistency

and accuracy of all documents (e.g., administration guide):

Identifies words, text, reading passages, and/or graphics that require copyright permission or

acknowledgements;

Applies Universal Design principles; and,

Ensures linguistic demands and readability are developmentally appropriate.

3.3

The performance measure was reviewed in terms of alignment characteristics:

Pattern consistency (within specifications and/or blueprints);

Targeted content standards match;

Cognitive demand; and.

Developmental appropriateness.

3.4Cut scores are established for each performance level. Performance level descriptors describe the

achievement continuum using content-based competencies for each assessed content area.

Strand III: Review

29



3.5

As part of the assessment cycle, post-administration analyses are conducted

to examine aspects such as items/tasks performance, scale functioning,

overall score distribution, rater drift, content alignment, etc.

3.6

The performance measure has score validity evidence that demonstrate item

responses were consistent with content specifications. Data suggest that the

scores represent the intended construct by using an adequate sample of

items/tasks within the targeted content standards. Other sources of validity

evidence such as the interrelationship of items/tasks and alignment

characteristics of the performance measure are collected.

3.7

Reliability coefficients are reported for the performance measure, which

includes estimating internal consistency. Standard errors are reported for

summary scores. When applicable, other reliability statistics such as

classification accuracy, rater reliability, etc. are calculated and reviewed.

Strand III: Review

Note: The indicators below are evaluated after students/clients

have taken the assessment (i.e., post-administration).

30

Procedural Steps

Step 1. Identify the performance measures being used within

the SLO/SSO.

Step 2. Examine the alignment characteristics of the

performance measure to those standards identified in

Section II.

Step 3. Determine the most applicable metric (e.g., growth)

based on the stated objectives in Section III.

Step 4. Evaluate the technical evidence provided by either the

vendor or the assessment’s developer.

Step 5. Repeat Steps 2 through 4 with the other performance

measures identified in Section IV.

31

Coherency Rubric

32

SLO/SSO Coherency

GOAL STATEMENT

SLO/SSO RATING

OBJECTIVES

PERFORMANCE MEASURES & TARGETS

ALL STUDENTS/ CLIENTS

FOCUSED

STUDENTS/CLIENTS

Coherency Rubric

33

• The Coherency Rubric, which examines the

alignment characteristics of each student

learning/support objective (SLO/SSO), serves as the

measurement tool to ensure that each SLO/SSO

meets the coherency criteria.

• The rubric evaluates each of the three (3) phases

used organize this training module. Each aspect of

the SLO/SSO should meet a specific descriptor in

the Coherency Rubric.

*Note* Template #3: Coherency Rubric is found in the Review module.

34

Phase I: Design


Meets

Criteria

Needs

Refinement

1.1

The goal statement articulates the “big idea”

under which targeted content/professional

standards are directly aligned. The statement is

concise and free of technical jargon.

1.2

Targeted content/professional standards have a

direct influence on student performance outcomes

and are viewed as “central” to the subject/service

area.

1.3

The course/subject (service) area associated with

the SLO/SSO is logically linked to the central

theme and targeted content/professional

standards.

Coherency Rubric

35

Phase I: Design


Meets

Criteria

Needs

Refinement

1.4

A blueprint or other design document illustrates

relationships among key components (i.e., Goal

Statement, Targeted Content/Professional

Standards, Objectives, Performance Measures,

and Overall Rating).

1.5

Performance measures are designed to evaluate

the targeted content/professional standards (as

demonstrated by the performance measure’s

alignment characteristics).

Coherency Rubric (cont.)

36

Phase II: Build


Meets

Criteria

Needs

Refinement

2.1

The goal statement represents a central concept

that is enduring, has leverage, and is foundational

to further, more complex learning outcomes.

2.2

The SLO/SSO is supported by a representative

sample of the educator’s/professional’s students,

with a sample size that is sufficient to make valid

inferences about student achievement and/or

outcomes.

2.3

Targeted content/professional standards are

selected using a valid and reliable approach that

is fair and unbiased.


37

Phase II: Build


Meets

Criteria

Needs

Refinement

2.4

Objectives are specific, criteria-focused, attainable

(yet challenging), and directly linked to the


2.5

Performance measures have benchmarks for two

or more points in time within a given school year

[Growth]. In addition or alternatively,

performance measures have a clear, date-specific

target for an on-demand demonstration of skill and

knowledge attainment [Mastery].

2.6

The overall rating is directly linked to a

performance continuum based on the percentage of

students meeting expectations across all

objectives.


38

Phase III: Review


Meets

Criteria

Needs

Refinem

ent

3.1

The SLO/SSO is based on performance

measures that are technically sound (i.e.,

reliable, valid, and fair) and fully align to

the targeted content standards.

3.2

The SLO/SSO form reviews mitigate

unintentional consequences and/or potential

threats to inferences made about meeting

performance expectations.

3.3

The SLO/SSO has data and/or evidence to

support the assignment of an overall teacher

rating (i.e., Unsatisfactory, Emerging,

Effective, and Distinguished).


39


Meets

Criteria

Needs

Refinement

3.4

The SLO/SSO form has been examined to

ensure that it is complete. Meaning, all

applicable elements within the SLO Form

(Template #2a or Template #2c) have been

addressed according to the prescribed business

rules.

3.5

The SLO/SSO from has been reviewed to

ensure it includes “comprehensive”

performance measures. Meaning, all

performance measures have been examined to

determine that they are appropriate for use in

the process AND are of high technical quality.

Needs

Refinement

Clarification

Coherency Rubric (cont.)Phase III: Review

40

Procedural Steps

Step 1. Select the drafted SLO/SSO, including applicable


Step 2. Beginning with Phase I, use the Coherency Rubric to

evaluate the alignment of the components to the overall

SLO/SSO design.

Step 3. Identify any component with weak alignment and/or

does not meet the rubric’s criteria.

Step 4. Flag any element needing refinement or further

discussion with other educators/professionals and/or

the principal/supervisor.

Step 5. Repeat Steps 2 thru 4 with the additional two sections

of the Coherency Rubric.

Reflection

• Quality Assurance• The SLO/SSO Form is complete.

• The SLO’s/SSO’s assessments are comprehensive.

• The SLO’s/SSO’s design is coherent.

Step 5

41

Summary

42

This SLO/SSO Review Phase:

• Applied a set of quality assurance criteria to

ensure that the student learning/support

objective, along with its applicable

performance measures, was complete,

comprehensive, and coherent (i.e., “high

quality”).

• Identified areas of improvement for

subsequent SLO/SSO refinement.

m3 reviewing the slo-sso-final

Education

reviewing module

learning objectives

slosso template

slosso formtemplatestemplate

drafted slosso

student learning objective

slosso review phase

learning activity