Rater Training

November 18th, 2011

Learning Intention

We are learning to…understand the Fidelity ofImplementation (FOI) Observation Protocol for the

CLP.

Success Criteria

We will know we are successful

when we can…complete the FOI Protocol and input the data online to ensure consistency among all reliability

raters.

Fidelity of Implementation Observation Protocol (Reading K-8 & ELA K-12)

•Overview•Reading Observation guide and

protocol •Practice Video and Test Video•ELA Observation guide and protocol •Practice Video and Test Scenario.

Please write down any questions on your note cards.

We will answer questions after lunch and at the end of the day.

Fidelity of Implementation (FOI) is the extent towhich the actual classroom instruction looks likethe intended instruction.

• Reliability Raters ▫ Regional Coordinators of C & I▫ DIFI Supervisors▫ Literacy Specialists▫ RtI Specialists

• Raters▫ Literacy Coaches and ICLs

• A random sample of 30% of MPS teachers will be observed in DPI and MPS selected schools

.

•Observing Reading and ELA instruction in classrooms

•Collecting information on the Fidelity of Implementation protocols

• In classrooms during Reading and ELA in a random sample of classrooms

•The DPI sample included three schools per region during first walk window

• During regularly-scheduled Reading classroom blocks

• Observation Window #1: Sept. 16 – 30▫Entered by October 6

• Observation Window #2: Nov. 28 – Dec. 16▫Entered by December 16Observation Window #3: Feb. 27-16▫Entered by March 16Observation Window #4: May 14-June 1▫Entered by June 1

• During regularly-scheduled ELA classroom blocks

• Observation Window #1: January 16 – 31▫Entered by January 31

• Observation Window #2: April 2-5 & 16-27▫Entered by April 27

• NOT an individual teacher evaluation▫Observation only▫Data is confidential, but not anonymous

• District-wide collection of program-specific data

• Completed on paper and later entered online

•Ultimately, to help improve literacy teaching and learning in MPS

•To increase understanding of the CLP in action

•To help support teachers, students, and others involved in these efforts

•To provide data to DPI as well as to MPS leaders

•The Fidelity of Implementation Observation Protocol will help MPS:▫Conduct observations with inter-rater

reliability (yoked walks)▫Collect precise data▫Assess observations with validity▫Report observations with consistency▫Measure levels of implementation

around essential components of the CLP

• FOI checks are unannounced visits.

• Arrive to the school early, so you have enough time to enter the classroom a few minutes prior to the scheduled start time to ensure you begin the FOI check at the start of the class period.

•As you enter the classroom, please smile and make eye contact with the teacher when you introduce yourself. Explain that you are holding a Fidelity of Implementation check and will observe at various points during the lesson.

•Please make a copy of the “Milwaukee Public Schools Fidelity of Implementation General Information” sheet (located in your guide). Fill in the information at the top of the sheet. Please leave this sheet in the teacher’s mailbox at the culmination of your observation.

• Raters can access their assigned schools and teachers that require FOI checks by logging in to: http://www2.milwaukee.k12.wi.us/rti/reporting or http://www2.milwaukee.k12.wi.us/survey

• Be sure to click on Reading OR ELA accordingly

• Here you will find all the information you will need to plan for your FOI Reading and ELA checks (School, teacher, classroom, daily schedule, grade level).

•Use the Instructional Timeline to determine whether or not their instruction is on pace.

•The reading class is scheduled to start at 1:00. When you get to the classroom the students are working on a math lesson. At 1:10 they begin their reading lesson.

•10: No

•11: 6-10 minutes

•12: 1:10

•16: 1:15

• During Segment 1, the teacher is working with all students together in a whole group. She begins by asking the students to define the vocabulary words listed on the Focus Wall. Then, the teacher directs students’ attention to the punctuation marks on the board and tells the students how different punctuation marks alert the reader to use different expression when reading. Students practice with sentences that they make up. (this content is not listed on the focus wall or in the pacing guide).

•13: Whole Group

•14: Vocabulary (Aligned), Explicit Fluency Instruction (Not Aligned), Fluency Practice (Not Aligned)

•15: Yes

•The teacher remains in whole group instruction during the entire class period. Do you record anything on Sheet 3: Teacher-Led Small Group Observation Segment?

•Yes

•What do you do between Segment 2 (Teacher-Led Observation Segment) and Segment 3 (Whole Group Ending Observation Segment)?

•Remain in or return to the classroom to observe additional teacher-led small groups if they occur.

•You observe 3 different teacher-led small groups, one beginning at 9:30 led by the teacher, another beginning at 9:50 led by the special education teacher who teams in the classroom and a third at 10:05, led by the teacher. In each small group, the teacher follows the exact same lesson plan – using round robin reading and the same questions on the same story.

•18: 9:30•25: 9:35•26a: 3•26b: No

•A 2nd grade class begins at 9:00. You return to the class to observe Segment 2 at 9:25. The class is in the bathroom and remains there until 9:40.

•What are the next TWO questions you will answer?

•19: No

•Questions 20 and 28

•How do you know which stations are in use?

•How do you know if stations are aligned?

•The class was scheduled to go from 10:00-11:30. They began CLP instruction at 10:16 and taught CLP-related activities until 11:35.

•PLEASE ALSO ANSWER QUESTION 35

•28: 11:25 •34: 11:30•35: No

•Don’t worry about questions 1-4•You can use today’s date and segment

times•7th grade classroom•Scheduled time: 8:05-9:05•Not a bilingual classroom, in a traditional

school•Don’t answer questions 24, 26a or 26b•Alignment can also include district-adopted

materials

Now that we have completed working through the protocol, let’s see how you do on your own.

80% success rate to be a certified rater.

•Go to http://www2.milwaukee.k12.wi.us/rti/reporting to submit data for each observation.

•All data must be submitted online by the end of the day of the observation.

•Make sure to click submit!!

•Please give your literacy specialist a paper copy of Observation Protocols to keep on file for the academic year.

•Don’t worry about questions 1-4•You can use today’s date and segment

times•K5 classroom•Scheduled time: 8:00-9:30•Not a bilingual classroom, in a traditional

school•Don’t answer questions 24, 26a or 26b•Alignment can also include district-adopted

materials

•Many of the ELA questions are similar to the Reading Protocol

•Briefly review a few differences:▫Question 8▫Question 9▫Learning Intentions ▫Common Core State Standards▫Whole/Small/Whole Rating▫Learning Experiences

•Scenario A (Complete with a partner and share as a group)

•Video (Complete on your own and share as a group)

•Scenario B (rater reliability assessment)

•Go to http://www2.milwaukee.k12.wi.us/rti/reporting to submit data for each observation.

•All data must be submitted online by the end of the day of the observation.

•Make sure to click submit!!

•Please keep your paper copy of Observation Protocols for the academic year.