DDMs for School Counselors RTTT Final SummitApril 7, 2014Craig Waterman & Kate Ducharme

Agenda Overview of DDMs (30 minutes)

Introduction to DDMs Q&A with Craig Waterman

DDM Considerations for School Counselors – Group Planning Activity (40 minutes)

Selecting Measures (15 minutes) Closing and Next Steps

Massachusetts Department of Elementary and Secondary Education

2

The Educator Evaluation Framework

ExemplaryProficient

Needs ImprovementUnsatisfactory

HighModerate

Low

SummativePerformance Rating

Student Impact Rating

Everyone earns two ratings

3

Two Ratings – Intersection of Practice and Impact

Summativ

e Rating

Exemplary1-yr Self-Directed

Growth Plan2-yr Self-Directed Growth Plan

Proficient

Needs Improvement

Directed Growth Plan

Unsatisfactory Improvement Plan

Low Moderate High

Rating of Impact on Student Learning

Identifying DDMs – Key Questions Is the measure aligned to content?

Does it assess what the educators intend to teach and what’s most important for students to learn?

Is the measure informative? Do the results tell educators whether students are

making the desired progress, falling short, or excelling?

Do the results provide valuable information to schools and districts about their educators? See Technical Guide B for

more about these key questions: http://www.doe.mass.edu/edeval/ddm/TechnicalGuideB.pdf

Key Messages for Stakeholders

Difference Between Direct and Indirect Measures

Classroom-based educator provides direct instruction

to students.

Classroom-based Educator

Responsibilities

Students learn; acquire knowledge and skills

Measures of Student Learning

Classroom-based educators can directly measure the impact of their instruction on student learning.

Direct MeasuresDirect Measure

Difference Between Direct and Indirect Measures

For SISP who do not directly instruct students, there often is an an intermediary step that exists between SISP responsibilities and student learning, therefore making the SISP contribution indirectly tied to student learning.

SISP educator provides

specialized support services

to students.

SISP Responsibilities

Students can access general

education curriculum.

Indirect Measures

Students learn; acquire

knowledge and skills

Measures of Student Learning

Indirect MeasuresIndirect Measure

Do DDMs have to be identical? DDMs must be “comparable across schools,

grades, and subject matter district-wide.” (603 CMR 35.09(2)a)

Type 1: Comparable within a grade, subject, or course across schools within a district

Identical measures are recommended

Investigate Fairness for all students and educators

Example Elementar

y

5th Grade Teacher

Music Teacher

Another Elementar

y5th

Grade Teacher

Music Teacher

Type 2: Comparable across grade or subject level district-wide

Impact Ratings should have a consistent meaning across educators; therefore, DDMs should not have significantly different levels of rigor

See Investigating Fairness Implementation Brief: http://www.doe.mass.edu/edeval/ddm/Fairness.pdf

Massachusetts Department of Elementary and Secondary Education

10

Identify Commonalities One of the goals of DDMs is to support

common measures across a district Approach to selecting DDMs for SISP

educators could involve identifying commonalities across multiple roles For example, a district-wide DDM involving

collecting feedback from stakeholders on the quality, usefulness, and timeliness of communications

Massachusetts Department of Elementary and Secondary Education

11

DDMs for District “Singletons”

Districts should consider the following options to identify and develop DDMs for “singletons”: Work with neighboring districts to identify and/or

create DDMs. Group educators within the district who have

similar responsibilities, albeit different roles, to collaborate on DDMs.

Look to outside rating entities (such as state or national organization standards).

Use DDMs that can be easily interpreted (e.g., for a school counselor, using number of seniors who applied to college instead of number of students who are prepared to apply to college.)

DDM Implementation Plan – June 1, 2014

Resources: http://www.doe.mass.edu/edeval/ddm/ Implementation Briefs Technical Guide B Webinar Series Commissioner’s Memoranda Educator Evaluation Newsletter Technical Assistance and Networking Sessions Using Current Assessments in DDMs

(Curriculum Summit) Example Assessments Other ESE documents (Technical Guide A, Part

VII, Regulations)

13

Questions and Answers

Who scores the DDM? Outside Organizations

Commercial assessments Automated methods Paid raters (e.g., college students, retired

teachers) Teams of Teachers (e.g., all 5th grade

teachers) Team members rate each other’s students’

responses Multiple raters score each response

Individual Teachers Random auditing (rechecking) In

tern

al

E

xte

rnal

See Scoring and Parameter Setting Implementation Brief: http://www.doe.mass.edu/edeval/ddm/Scoring&PSetting.pdf

How do I determine high, moderate, or low growth?

Qualitative Quantitative

Previous Results

Variability

Fairness

Educator Review

Identification

Agreement

Ask educators how much growth is moderate on this assessment?

Does the DDM make meaningful distinctions? Do all students have an equal chance to demonstrate growth?See Scoring and

Parameter Setting Implementation Brief: http://www.doe.mass.edu/edeval/ddm/Scoring&PSetting.pdf

DDMs for School Counselors

Massachusetts Department of Elementary and Secondary Education

18

Identifying DDMs – Key Questions for Specialized Instructional Support Personnel

Is the measure aligned to content? For SISP educators (including school

counselors), content should reflect their job functions and responsibilities and should align to what they do to support students, educators, administration, and/or parents.

Instructions: With you table, identify key content (i.e., key job functions and responsibilities) of your role.

Massachusetts Department of Elementary and Secondary Education

19

Identifying DDMs – Key Questions for Specialized Instructional Support Personnel

What does success look like?What would you see if someone was successful

at these job functions and responsibilities?

Instructions: With your table, identify what success looks like for each of the job functions and responsibilities you identified.

Identifying DDMs – Key Questions for Specialized Instructional Support Personnel

What is measureable? Measurement is…

The systematic process to assign a number to an observation

Example: How tall is my friend?1. Place a measuring tape next to my friend and read the number next to the

line that is closest to the top of his or her head.2. Stand next to my friend and estimate how much taller or shorter they are

compared to myself.3. Place a ruler flat on my friend’s head and mark a line on a wall where the

ruler hits the wall. Then measure how far the line is off the ground.

Instructions: With your table, identify approaches could you use to measure the job responsibilities and functions you identified.

Massachusetts Department of Elementary and Secondary Education

21

Identifying DDMs – Key Questions for Specialized Instructional Support Personnel

Is the measure informative? Chosen measures should provide both

educators and districts with actionable information to inform practice and identify both areas of strength and areas where more supports are needed.

Key Messages for Stakeholders