region2statesupportteam.wikispaces.comregion2statesupportteam.wikispaces.com/file/view/oip... ·...

OIP RESOURCE OVERVIEW – SEPTEMBER 2010 OIP STAGE KEY WEBSITES OIP TOOLS

Resources may be customized and used in more than one stage of the OIP. The chart to the right indicates when the resource can most appropriately be used.

P 1 2 3 4 Key websites associated with the tool and/or resource topic.

OIP tools used with resources.

RESOURCE 1 OIP Facilitator (Internal/External) Competency Tool

RESOURCE 2 OIP on a Page http://education.ohio.gov/GD/Templates/Pages/ODE/ODEDetail.aspx?page=585

RESOURCE 3 Chart of Responsibilities to Complete the OIP

RESOURCE 4 DF/BDF Flowchart and DF/BDF Tabs DF/BDFRESOURCE 5 PPT Slides for DLT/BLT Orientation

RESOURCE 6 Meeting Management Checklist

RESOURCE 7 Evaluating Meeting Effectiveness Checklist

RESOURCE 8 Six Conditions to Support Collaborative Teams Checklist

RESOURCE 9 Data Source Identification DF/BDF

RESOURCE 10 Expanded Facilitation Questions DF/BDFCCIP/IMM

RESOURCE 11 Process to Create a Relation Diagram

RESOURCE 12 Goal, Strategy, Action, Task and Indicator Definitions http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEPrimary.aspx?page=2&TopicRelationID=1186

CCIP/IMM

RESOURCE 13 Affirm Cause and Effect for Strategy Development

RESOURCE 14 Focused Plan Criteria Checklist CCIP/IMMRESOURCE 15 Stakeholder Feedback: Survey or Interview

RESOURCE 16 IMM Template IMMRESOURCE 17 Task Implementation Template

RESOURCE 18 Checklist of High Quality and Evidence-Based PD http://esb.ode.state.oh.us/PDF/Standards_ProfDev_sept07.pdfRESOURCE 19 PD Alignment Template IMM

RESOURCE 20 Selecting HQPD Providers

RESOURCE 21 Teacher-Based Teams (TBTs) – 5-Step Process

RESOURCE 22 Assessing Teacher-Based Team Effectiveness Checklist

RESOURCE 23 Sample Protocols

RESOURCE 24 District Assessment System and Inventory Template http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEPrimary.aspx?page=2&TopicRelationID=1143

DF/BDF

RESOURCE 25 “Look-For” Behaviors (Student and Adult) – Samples IMMRESOURCE 26 Procedures for Monitoring Student Performance IMMRESOURCE 27 Monitoring Processes for Adult Implementation IMMRESOURCE 28 Recording/Charting Monitoring Data IMMRESOURCE 29 Decision-Making Parameters for Plan Course IMM

CorrectionRESOURCE 30 OIP Plan Criteria (In process of development) CCIP/IMM

Legend: DF = Decision Framework; BDF = Building Decision Framework; CCIP = Comprehensive Continuous Improvement Plan; IMM = Implementation Management/Monitoring Tool

Resource 1 of 2

Resource 1: Ohio Improvement Process: Facilitator Competency Tool

Directions for completing the Facilitator Competency Tool:

1. Fill in the identifying information.2. Select as many purposes as applicable with a checkmark.3. Rate facilitator’s competencies with a checkmark:

Not Observed (situation did not call for the competency) Undeveloped (competencies unused and not applied when needed) Novice (competencies rarely used and not consistently applied when needed) Emergent (competencies used inconsistently with limited performance) Competent (skilled performance consistently utilizing competencies)

4. Provide evidence that illustrates how the rating was determined.

Identifying Information:

Facilitator: __________________Region____________ Date: _____________

Completed by: ________________________Position: ____________________

Purpose(s): Self-assessment Monitoring progress Training Other (please specify)_____________________________

Description:This Facilitator Competency Tool has been developed by the State Level Design Team in an effort to build consistency in the facilitation of the Ohio Improvement Process (OIP) as part of Ohio’s Statewide System of Support (SSOS). The intent of this tool is to assess competency and enhance performance in knowledge, skills and behavior for external and internal staff employed to facilitate the OIP. This tool is meant primarily to support continuous professional growth of OIP facilitators and is not meant to be used as an evaluation instrument.

A competency is a standardized requirement for an individual to properly perform a specific job. It encompasses a combination of knowledge, skills, and behavior used to improve performance. More generally, competence is the state or quality of being adequately or well qualified and having the ability to perform a specific role. All competencies are action oriented. Being competent requires one to be able to interpret the situation in context and to have a repertoire of possible actions. A facilitators’ competence grows through training, experience, and their willingness to learn and adapt.

The Competencies are organized into categories that are broad-based characteristics evident across all stages of the OIP. Look Fors are examples of observable behaviors that demonstrate the competencies. These examples are illustrative and not meant to be exhaustive.

Competent facilitators of the OIP can influence improvement in instructional practices that results in increased student performance. Use of the Facilitator Competency Tool may also inform recruitment, regional decision-making, professional development design and delivery, and quality assurance for the SSOS.

Resource 1: Ohio Improvement Process: Facilitator Competency Tool September 2010 OIP Resources

Category A: Knowledge of Research, Process and ToolsThe foundation for working with a district is the knowledge and understanding of the Ohio Improvement Process (OIP) as the process that operationalizes the Ohio Leadership Development Framework (OLDF). A competent facilitator has a working knowledge of related resources, including the OIP Facilitator’s Guide and Resources, Decision Framework (DF), Comprehensive Continuous Improvement Plan (CCIP), and the Implementation Management/Monitoring Tool (IM/M). Comprehension of the process, resources, tools, and the research that supports those means the facilitator can accurately articulate them to others and respond to questions relative to each, including the purpose and interrelationship of each.

Competencies Ratings and Evidence1. Know the current educational research related to the content

found in the OIP. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Articulate current research around areas directly related to the district/school plans with which they are working.

Evide

nce

b) Articulate a clear understanding of Ohio’s Accountability System.

2. Articulate accurately the foundation of the OIP and its connection to the OLDF Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Use OLDF language specific to the six areas of leadership as related to OIP.

Evide

nce

b) Identify the Superintendent, District Leadership Team (DLT), Building Leadership Team (BLT), and Teacher Based Team (TBT) as four key structures inherent in distributive leadership.

3. Explain the application and benefits of OIP related to continuous improvement. Not Observed Undeveloped Novice Emergent Competent

Look

For

s a) Explain the essential leadership practices and the enactment of those practices throughout the OIP.

Evide

nce

b) Explain the structure and benefits of the SSOS.

4. Articulate the OIP structure and Web-based tools such as the DF, IM/M, and CCIP as they relate to current educational research. Not Observed Undeveloped Novice Emergent Competent

Look

For

s a) Use accurately and appropriately the OIP Facilitator’s Guide and Resources.

Evide

nce

b) Use accurately the DF and IM/M as related to CCIP.

Category B: Facilitation and Customization Resource 1: Ohio Improvement Process: Facilitator Competency Tool September 2010 OIP Resources

Facilitation is the ability to help manage the progression of the discussion through all four stages of the OIP by serving as a critical friend and partner who asks thought-provoking questions and provides focused constructive feedback. A competent facilitator knows how to customize the OIP in a variety of educational settings based on the needs of the district.

Competencies Ratings and Evidence1. Tailor the OIP to meet the unique needs of individual districts. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Use a repertoire of techniques for engagement which includes tools/resources and activities at the appropriate time to accomplish the work.

Evide

nce

b) Demonstrate behavior that values and respects existing district efforts/initiatives and connects them to the OIP.

2. Recognize and respond appropriately to the underlying dynamics of the group, varying the training techniques to meet the unique needs, interests, and learning styles of adult learners.

Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Anticipate conflict and manage ambiguity, varying activities to maintain focus and achieve the desired outcomes: 1) Identify critical needs, 2) create a focused plan, 3) implement and monitor the plan, 4) evaluate the process and impact on the goals.

Evide

nce

b) Guide the group in negotiating consensus based on thoughtful discussion and consideration of evidence, using data to recognize patterns and trends.

3. Be both a critical friend and partner to guide conversation that promotes mutual trust and freedom from fear. Not Observed Undeveloped Novice Emergent Competent

Look

For

s a) Direct the course of each session by posing the “right question at the right time for the right purpose”.

Evide

nce

b) Provide focused constructive feedback and be responsive to the feedback received.

4. Demonstrate respect of the unique viewpoints, skills and experiences each participant brings to the group and create opportunities to benefit from the diversity found within the group.


Look

For

s

a) Demonstrate behavior that values and respects each individual’s talents, skills and knowledge.

Evide

nce

b) Engage all participants in discussion and consensus building in a respectful way that allows them to understand and confront their assumptions and challenge their current reality.

Category C: Organization and Management


Organization of the OIP requires that the facilitator know district structures, processes, and plans and has the ability to create a schedule and align/develop leadership structures consistent with Ohio leadership practices. Management of the process requires the facilitator to co-plan, meet agreed upon schedules, follow effective meeting management guidelines, and coordinate with district leadership to ensure effective implementation of the process.

Competencies Ratings and Evidence1. Acquire prior knowledge of the district, such as demographics,

performance, culture, organizational structure, leadership roles, as an integral part of preparation.


Look

For

s

a) Access existing resources to gather background information specific to the district.

Evide

nce

b) Gather additional information not accessible from existing data banks.

2. Prepare for meetings in advance, considering what needs to happen before, during and after each session. Not Observed Undeveloped Novice Emergent Competent

Look

a) In collaboration with the district, prepare materials and gather equipment needed for the session.

Evide

nce

b) Collaborate with the district to create working agendas.

3. Manage meetings effectively in a logical and sequential order to support learner understanding of the content and to complete tasks accurately and completely.


Look

For

s

a) Use the agenda to present information clearly and concisely to maintain focus on the overarching outcomes of improved teaching and learning.

Evide

nce

b) Model the ability to multitask and manage timelines in balancing the important OIP tasks with urgent needs of the leadership teams.

4. Communicate with clarity and regularity verbally and in writing. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Use verbal and written communication tools in an appropriate and timely manner.

Evide

nce

b) Actively listen and interpret nonverbal behaviors and respond appropriately.

Category D: Capacity BuildingCapacity building for the OIP is the development of a district’s core skills and capabilities in order to build the organization’s effectiveness and sustainability to improve student performance. It is the process of assisting the Leadership Teams – the District Leadership Team (DLT), Building Leadership Team (BLT) and Teacher Based Team (TBT) -- to identify and address issues and gain the


insights, knowledge and experience needed to solve problems and implement change. Competent facilitators build capacity by providing support, including coaching, training, specific technical assistance, and resource networking.

Competencies Ratings and Evidence1. Assist the district in creating distributive leadership and

collaborative structures to implement and sustain the OIP. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Guide leadership teams to learn, understand and practice the core skills of distributive leadership.

Evide

nce

b) Guide teams to identify and refine collaborative structures at the district, building and classroom levels in order to sustain the work of the plan.

2. Work with the district in establishing ongoing, two-way communication between and among the internal and external stakeholders.


Look

For

s

a) Facilitate the creation of the communication component of the district plan.

Evide

nce

b) Encourage and support the use of the communication component of the district plan.

3. Support efforts to align and intentionally use resources to achieve district goals. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Facilitate the district to audit all available resources (time, staff, programmatic resources, and money) to support the plan and needed HQPD.

Evide

nce

b) Monitor the transparency of resource alignment so all district resources are known and available to support the plan.

4. Develop knowledge, behavior and skills of the leadership teams in order to sustain the momentum of the OIP. Not Observed Undeveloped Novice Emergent Competent

a) Teach the fundamentals and model the process of OIP.

Evide

nce


Look

Fo

rs

b) Transition responsibilities from external to internal facilitator.

5. Assist the leadership teams in developing and using effective systems to measure the degree of implementation of the plan indicators.


Look

For

s

a) Assist the leadership teams in using monitoring results to guide data-based decisions that result in mid-course corrections between the projected and actual indicators.

Evide

nce

b) Support the leadership teams’ efforts to replicate data-based successes across the district/building.


Category E: Commitment and DispositionCommitment to continuous improvement requires fidelity in the execution of the process by initiating courageous conversations and by persisting in challenging traditional district thinking. A competent facilitator demonstrates through words and actions the moral imperative of maintaining a student-centered focus and ensures that improvement efforts are designed to support the success of every child. A competent facilitator also has a positive outlook, engages in professional growth opportunities that show personal commitment to continuous improvement, and displays the temperament needed to work collaboratively with a district, honoring and valuing the district’s history by connecting it to the OIP and essential leadership practices.

Competencies Ratings and Evidence1. Influence individuals and groups effectively to gain support, achieve

common goals and guide the group to desired outcomes. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Model the role of team player by encouraging enthusiasm, participation, creative thinking, trust, and honesty resulting in mutual respect.

Evid

ence

b) Exemplify a temperament conducive to an atmosphere of cooperation and collaboration

2. Accept responsibility for personal and professional development demonstrating a commitment to self reflection and continuous improvement.


Look

For

s a) Model a strong work ethic.

Evid

ence

b) Demonstrate a commitment to learning and a pattern of pro-active behavior that reflects continuous growth.

3. Demonstrate performance agility and the ability to work with a variety of people and situations while maintaining the fidelity of the OIP. Not Observed Undeveloped Novice Emergent Competent

Look

For

s a) Model perseverance and adaptability.

Evid

ence

b) Demonstrate the fortitude to provide a consistent message of shared leadership resulting in improved instructional practices and increased student outcomes.

4. Encourage meaningful interactions that are performance-based, data-related and nonjudgmental. Not Observed Undeveloped Novice Emergent Competent

Look

For

s

a) Demonstrate confidence and a belief in the group’s potential for success.

Evid

ence

b) Use an appropriate variety of facilitation tools and protocols that encourage meaningful interactions.


Resource 2: OIP on a Page

Resource 2: OIP on a Page September 2010 OIP Resources

District/Building Leadership Teams

State Diagnostic Teams (SDTs) work with selected high support districts

State Support Teams (SSTs) work with districts and schools in need of improvement

Educational Service Centers (ESCs) work with other districts requesting assistance

is involved?

Teams use data tools to identify critical needs

do these teams work in districts and schools?

District/Building Leadership Teams Regional Service Providers External Vendors Higher Education

is involved?

District/Building Leadership Teams State Diagnostic Teams State Support Teams Educational Service Centers

is involved?

District/Building Leadership Teams

State Diagnostic Teams

State Support Teams

Educational Service Centers

Regional Managers

Single Point of Contact

is involved?

Review data Gather evidence of implementation and impact

Provide technical assistance and targeted professional development

Leverage resources

Work with leadership to develop research based strategies and action steps focused on critical needs identified in stage 1.

How

Who

How

Who

How

How

Who




WhoSTAGE 1

STAGE 3

STAGE 2

Implement and Monitor the Focused Plan

Evaluate the Improvement Process

Identify Critical Needs of Districts and Schools

Develop a Focused Plan

Ohio Improvement Process

STAGE 4

Revised November 2008

District/Building Leadership TeamsTeacher Based Teams

Regional Service Providers

External Vendors

Higher Education

Resource 2: OIP on a Page

Preparing for OIP

Create Structures and Conditions

Orient Superintendent and Administrators, DLT, BLT to the OIP

Establish and implement collaborative structures and processes that support a culture of inquiry

STAGE 1

Identify Critical Needs Understand the structure and requirements of the DF/BDF including collection and organization of local data to complete the DF/BDF

DLT completes the DFBLT completes the BDF

Identify and affirm critical needs for content and expectations and conditions areas

STAGE 2

Develop a Focused Plan

Create a few SMART district goals (district)Review of District SMART goals (building)

Draft a limited number of research-based district strategies for each district goal and indicators for each strategy; buildings select appropriate district strategies for each goal and indicators for each strategyDLT enters into CCIP; BLT enters action steps into SIP

Develop district and building actions from cause information generated in Stage 1 and then align resources to strategies and actions

STAGE 3Implement and

Monitor the Focused Plan

Implement the plan systemically and systematically

Sustain and maintain collaborative structures and processes in support of a culture of inquiryDLT & BLT uses IMM tool

Monitor and analyze changes in student performance and adult implementation and make and report corrections to the plan

STAGE 4

Evaluate the Improvement Process

Evaluate the degree of plan implementation, impact on student achievement, and changes in educator practices

Provide annual and summative plan reports to the superintendent for informing the board of education and other stakeholders

Use monitoring and evaluation data to modify instructional practice and revise plan

Resource 2: OIP on a Page September 2010 OIP Resources

Resource 3: Chart of Responsibilities to Complete the OIPAdequate amounts of people and time need to be committed to the planning process in order to gain the long-term benefits of a plan that is “owned” by stakeholders. This chart provides an overview of the responsibilities for each group and/or individual by OIP Stage. All groups and individuals have the responsibility to communicate with each other across all levels and throughout all stages.

Group or Individual

Prepare for the OIP

Stage 1: Identify Critical Needs

Stage 2: Develop a Focused Plan

Stage 3: Implement and Monitor the Focused Plan

Stage 4: Evaluate the Improvement Process

District Leadership Team (DLT)

Orient the team to work as a collaborative workgroup.Develop/revise the mission.Revisit/develop policy.

Understand the structure and requirements of the DF.Facilitate the collection, organization and summarization of data.Complete the DF.Identify and affirm critical needs.

Create SMART goals.Draft/refine strategies and indicators.Solicit and consider stakeholder feedback.Produce actions.Align resources and budget.Consider public comments.

Maintain and sustain collaborative structures and processes.Implement plan.Monitor and analyze changes in student performance and adult implementation.Make course corrections.

Evaluate plan implementation.Evaluate impact and changes.Report annual and summative results.Modify instructional practice.Revise plan for next cycle.

Building Leadership Team (BLT)

Orient the team to work as a collaborative workgroup.Establish and implement collaborative structures.

Understand the structure and requirements of the building DF.Facilitate the collection, organization and summarization of data.Complete the building DF.

Produce actions aligned to district goals, strategies and indicators.Align resources to actions.

Maintain and sustain collaborative structures and processes.Implement plan.Monitor, analyze and report changes in performance.Make course corrections.

Evaluate plan implementation.Evaluate impact and changes.Report annual and summative results.Modify instructional practice.Revise plan for next cycle.

Teacher –Based team (TBT)

Prepare teams to work as collaborative workgroups and follow common protocols.

Understand DF data and prioritized critical needs.

Understand plan goals, strategies, indicators and actions.

Receive PD resources and support to implement relevant plan strategies and actions.Follow district and school protocols.Monitor and report changes in performance and practices.

Goal Workgroups

Understand the data and DF results.

Develop/critique goals, strategies and indicators.Develop actions.

Implement plan.

Stakeholders

Participate in forums.Critique goals and strategies.Participate in public hearings.

Learn about plan progress.Provide feedback to DLT/BLT.

Learn plan and process results at the end of each year and at the end of the plan cycleProvide feedback to DLT/BLT

Support Personnel

Arrange meeting logistics (including stakeholder forums).Distribute agendas/materials. Take meeting minutes.Create, format and edit documents.Maintain documentation.

Resource 3: Chart of Responsibilities to Complete the OIP September 2010 OIP Resources

Resource 4: Decision Framework (DF)/Building Decision Framework (BDF) FlowchartThe flowchart on this page shows the flow of the DF/BDF once Level I is completed. Once Level II is completed for each academic area, the results are then analyzed in Level IV. Data on the academic areas of science, social students and writing are also included in the DF/BDF and would follow the same analysis as Reading and Mathematics. Level III is completed independently and then analyzed in relation to Level IV.

Level I: Identified Issue – READING Level I: Identified Issue - MATHEMATICS

Identify the weakest grade level and/or grade-level band in reading. Identify the weakest grade level and/or grade-level band in mathematics.

Identify the subgroups of students and their performance in relationship to the identified reading grade level or band.

Identify subgroups of students in the district across all grade levels with significant gaps in reading.

Identify the subgroups of students and their performance in relationship to the identified mathematics grade level or band.

Identify subgroups of students in the district across all grade levels with significant gaps in mathematics.

Identify the magnitude of the problem (pervasive across the district or buildings that are extremely weak or strong in reading).

Identify the magnitude of the problem (pervasive across the district or buildings that are extremely weak or strong in mathematics).

Each identified content area would move through Level II separately, not concurrently. This would allow for the specific causes for each content area to be uncovered, explored, analyzed and evaluated based on the uniqueness of that area.

Level II: Instructional ManagementA. Curriculum, Assessment and Instructional PracticeB. Educator Quality C. HQPD

The components in Level III have an overall general/global effect on student academic performance across all content areas and can be viewed once. District personnel also may identify a specific issue that is unique to their district or community that needs attention to promote, facilitate or improve student performance.

Level IV components merge all the necessary resources to aggressively and intentionally implement the CCIP action plan to improve student academic performance.

Level III: Expectations and Conditions

A. LeadershipB. School ClimateC. Engagement: Parent/Family, Student, Community Involvement

Level IV: Resource Management

A. Teacher and PD AlignmentB. Focus on Student SuccessC. Aligning Spending to Strategic Priorities

Resource 4: Decision Framework (DF)/Building Decision Framework (BDF) Flowchart September 2010 OIP Resources

Resource 4: DF/BDF TabsHome: Introduction and basic directions of the toolLEVEL I Identified IssueData Proficiency by Grade: 3-year trend data for reading, mathematics, writing, science and social studies for all tested gradesData Proficiency by Building: 3-year trend data for reading, mathematics, writing, science and social studies for every tested building in the districtData Proficiency by Group: 3-year trend data for reading, mathematics, writing, science and social studies for all tested disaggregated groupsLevel I: Questions related to the data proficiency tabs described above

LEVEL I-II Instructional ManagementREADING Level I Subscale: Most recent year district level subscale scores for reading

• Reading Level II A Area 1: Curriculum questions related to reading subscale decisions• Reading Level II A Area 2: Assessment questions related to reading subscale decisions• Reading Level II A Area 3: Instructional practice questions related to reading subscale decisions

• Reading Level II C: Professional development questions related to reading issues

MATH Level I Subscale: Most recent year district level subscale scores for math• Math Level II A Area 1: Curriculum questions related to math subscale decisions• Math Level II A Area 2: Assessment questions related to math subscale decisions• Math Level II A Area 3: Instructional practice questions related to math subscale decisions

• Math Level II C: Professional development questions related to math issues

Level II B: Educator quality questions - Teacher turnover, administrator turnover, teacher qualifications

Level II Results: Summary report of the decisions and scoring for all Level I and II areas LEVEL III Expectations and ConditionsLevel IIIA: Leadership questions (follows OLAC framework)

Level IIIB: School Climate

• Level IIIB Area 1a School Climate: Student discipline occurrences by grade level; expulsions/out-of-school suspensions by grade level• Level IIIB Area 1b School Climate: Student discipline occurrences by grade span/building level; expulsions/out-of-school Suspensions by

grade span/building level• Level IIIB Area 1c: School Climate: Student discipline occurrences by student group; expulsions/out-of-school suspensions by student group• Level IIIB Area 2a Attendance: Student attendance by grade level; student attendance by student group• Level IIIB Area 2b Attendance: Student attendance by grade span/building level• Level IIIB Area 2c Graduation: District graduation; Graduation by student group; graduation by building• Level IIIB Areas 3, 4, and 5: Student mobility; multiple risk Factors; teacher and student perception

• Level III B Area 2c Dropout: Dropouts in the district; dropouts by student group; Dropouts by building

• Level III C Engagement: Parent/Family, Student, Community Involvement: parent and student participation and perception; communication; community involvement and support

Level III Results: Summary report of the decisions and scoring for all level III areas

Level IV Resource Management: Teacher and PD alignment; focus on student success; aligning spending to strategic priorities

PROFILE: Results from Level II and III and final priority determinations

Resource 4: Decision Framework (DF)/Building Decision Framework (BDF) Flowchart September 2010 OIP Resources

Resource 5: PPT Slides for DLT/BLT Orientation September 2010 OIP Resources

Resource 5: PPT Slides for DLT/BLT Orientation

Resource 5: PPT Slides for DLT/BLT Orientation September 2010 OIP Resources

Resource 6: Meeting Management ChecklistPLANNING THE MEETING

Think about the …

□ Who

□ Where

□ When

□ Why

□ How Many ______

Determine overall purpose of the meeting (e.g., inform, generate ideas, define goals, evaluate or give feedback, solve problems, plan or strategize, make decisions, gain commitment, get results).

Consider physical factors of the meeting site (depending on number of attendees).

Make site arrangements (choose space that will accommodate the group).

Plan the agenda according to priority of agenda items, length of time needed for each item and person responsible for each item.

Prepare and send out (or post) the agenda in advance of the meeting to ensure that everyone comes to the meeting prepared; clarify the expectations for the meeting; and structure the meeting’s content, process and time.

Assign roles for the meeting: facilitator, recorder and timekeeper.

STARTING THE MEETING

Facilitator should do the following:

Arrive early to set up room and arrange the seating based on the type of meeting (e.g., circular for face-to-face exchange, semicircle and away from the entrance for collaborative decision-making).

Begin and end the meeting on time.

Ask participants to introduce themselves and state expectations, if not already known.

Post the agenda and review it with team members.

Ask the leader to define the roles and outcomes for the meeting.

Establish (or reaffirm) the team’s operating principles/norms.

Make the group memory public, i.e., post ideas and decisions.

Review action items from prior meetings as needed.

Resource 6: Meeting Management Checklist September 2010 OIP Resources

It is everyone’s responsibility to do the following:

Keep the group on task.

Maintain productive working relations in the group.

STRUCTURING THE MEETING Use standard template to record minutes.

Use the wait list technique to keep the meeting moving. This technique is useful to save ideas, deal with hidden agendas and capture suggestions that are not related to the meeting but may be helpful later.

Focus on the same issue in the same way at the same time.

CLOSING THE MEETING Review agenda and confirm completion/status of agenda items.

Establish action items: who, what, when.

Review decisions and action items.

Review group memory.

Develop preliminary agenda for next meeting.

Evaluate the meeting’s effectiveness (e.g., plus/delta, stop/start/continue).

As soon as possible, distribute minutes that summarize the following: members present, what happened and how decisions and action items; next steps; next meeting date, time and place and possible agenda items for the next meeting.

Thank participants.

Resource 6: Meeting Management Checklist September 2010 OIP Resources

Resource 7: Evaluating Meeting Effectiveness ChecklistUse this checklist to evaluate the effectiveness of your meetings.

Alw

ays

Som

etim

es

Neve

r Meeting Members

1.Know what items are to be discussed prior to the meeting (e.g., agenda or meeting announcement sent in advance).

2.Arrive on time for the meeting. 3.Honor confidentiality as necessary. 4.Feel comfortable that they can say what they truly believe about an issue. 5.Ask clarifying questions when in doubt about an issue. 6.Have a chance to consider more than one solution to an issue. 7.Keep track of decisions in a public manner by posting them on chart paper. 8.Understand an issue thoroughly prior to taking a vote or reaching consensus. 9.Discuss “real” problems that are important to the members. 10.Stay on the topic and control excessive talking and interruptions.

11.Route off-task insights, problems and issues to a “wait list” for later consideration.

12.Discourage side conversations. 13.Set time limits for the meeting (and possibly for individual agenda items). 14.Encourage participation by everyone, even the quietest members. 15.Test to see whether they are ready before making a decision.

16.Identify which decision-making method to use for a particular decision (e.g., majority vote, consensus).

17.Summarize accomplishments from each meeting. 18.End meetings on time. 19.Have a facilitator guide the meeting process. 20.Identify a recorder to take minutes. 21.Review and refine the meeting’s agenda before beginning work.

22.Finalize plans on “who will do what by when” at the close of the meeting and record in the minutes.

23.Evaluate the meeting. 24.Take responsibility for and carry out decisions and commitments. 25.Develop a preliminary agenda for the next meeting.

Resource 7: Evaluating Meeting Effectiveness Checklist September 2010 OIP Resources

Resource 8: Six Conditions to Support Successful Implementation of Collaborative Teams

In order for districts and schools to effectively implement collaborative structures, i.e., DLT, BLT, TBTs, they must assess their readiness to support the structures by examining whether they have created the conditions for successful implementation. Once the DLT/BLT has identified the conditions that are not in place for successful implementation, they can then work toward establishing them. It is not imperative that all conditions for success are in place prior to implementation; however, there are basic conditions that, if not established, may thwart the process. These are:

The following resource describes the six conditions necessary to support successful implementation of collaborative teams and provides a way for teams to rate the extent to which these conditions exist in the district and/or building. It can be used in the following ways:

a) By a DLT or BLT to determine the overall status of these conditions in the district or building. b) By a DLT or BLT to identify specific tasks related to these conditions that must be implemented more effectively in order

to achieve success for the team.c) By a DLT or BLT to identify whether the conditions are in place for TBTs to be successful.d) By a TBT to give the BLT or DLT their perspective on whether these conditions have been supported by the BLT or DLT.e) By an OIP facilitator to discuss conditions for successful collaborative teams with the DLT or BLT.f) By a DLT, BLT or TBT as a baseline and progress measure for judging growth in supporting and implementing

collaborative teams.

Resource 8: Six Conditions to Support Successful Implementation of Collaborative Teams September 2010 OIP Resources

Conditions to Support Successful Implementation of Collaborative Teams Indentify the collaborative team that is using this resource: DLT BLT TBT

Identify the purpose for which this resource is being used:

Instructions: Rate each of the items below on a scale of 1 to 4. Tally the total score by Condition. Sort Conditions from highest to lowest total score. For any Condition with a score lower than 3, identify items or tasks which need to be done to move it to a higher score. The tasks within each Condition may need to be prioritized if many items receive a low rating. To what extent is this item true?

1 2 3 4

NOT AT ALL TRUE MINIMALLY TRUE MODERATELY TRUE VERY TRUE

CONDITION A: PREPARING EDUCATORS TO WORK COLLABORATIVELY BY DEEPENING THE CULTURE OF INQUIRY

STAGES OF TEAM DEVELOPMENT

1. We know where our team is in the stages of team development.2. We know how to move our team through the stages of development to become a high performance

team.

NORMS OF COLLABORATION

3. We know the seven norms of collaboration as tools for productive communication among group members.

4. We practice the seven norms of collaboration for productive communication.

PROFESSIONAL DEVELOPMENT

5. We have assessed our professional development needs to ensure there is common understanding of assessment literacy, research/evidence-based instruction, data analysis, high quality professional development, learning standards and curriculum.

6. Our team collects, charts and analyzes data, including data by subgroups.

TOTAL Score divided by 6

CONDITION B: FORMING OR REPURPOSING TEAMS TO IMPLEMENT AND MONITOR THE PLAN

TYPE 7. We have identified the name and types of teams we have in place to determine the extent to which

the district and schools have teams to accomplish the work of the district/school plans, e.g., vertical, data, instructional/content, project, grade level.

LEVEL8. We have identified all levels that actively participate on the identified teams, e.g., department,

grades, multiple grades.

PURPOSE9. We know the purpose of each identified team and have repurposed them (changed their work), if

necessary, or we have formed teams to implement and monitor the district/school plans.

BALANCE10. Teams identified to accomplish the work are balanced with other types of teams that may need to

operate within the system. TOTAL Score divided by 4


CONDITION C: CREATING SCHEDULES AND ROUTINES THAT SUPPORT COLLABORATIVE TEAMS

SCHEDULE

11. Time has been found for teams to meet, e.g., early release/early start, modified schedules during work day.

12. BLTs/TBTs discontinue practices no longer needed (e.g., spelling bees, showcases and conduct time audits to determine amount being spent and number of students affected) and use that time for collaboration.

13. Our teams have a regular calendar/time table that allows for teams to meet regularly, consistently with uninterrupted time, ideally weekly for TBTs and monthly for BLTs/DLT.

14. Our teams have a regular calendar/time table that allows for full participation of each member.

ROUTINES

15. Teams follow a general cycle for collaborative team work.16. Teams analyze pre-assessment data, identifying strengths and obstacles for target population

students and skill/sub-skill identified in the plan.17. Teams define behaviors for instructional strategies identified in the plan and process for examining

student work and students’ conditions for learning.18. Teams implement and monitor instructional practices and system supports that produce conditions for

learning, e.g., job-embedded PD.19. Teams analyze post-assessment data, determining level of progress, identifying strengths and

obstacles and make course corrections.TOTAL Score divided by 9

CONDITION D: MAKING MEETINGS PURPOSEFUL

MEETING MANAGEMENT

20. Our teams develop and distribute an agenda using a standard template that includes outcomes, topics related to outcomes, process, and summary of discussion from the previous meeting prior to each meeting.

21. Teams provide and distribute minutes/meeting notes that summarize the discussion, decisions and assignments after each meeting. Notes reflect priority assessments and response to common questions.

22. Teams maintain minutes/meeting notes in an accessible location. 23. Teams follow standard meeting operating procedures (norms, purpose, roles, decision-making

and conflict resolution processes, clear expectations for performance and accountability.

MEETING FOCUS24. Teams use results-oriented agenda. For example, an agenda that provides time to analyze data,

examine/study work or discuss implications of data, 25. Teams do not use collaboration time to discuss information that can be distributed in written form.

PROTOCOLS 26. Conversations during team meetings are focused on the work of the plan.27. Team use protocols to structure conversations.



CONDITION E: DEFINING ROLES AND RESPONSIBILITIES

CHAIR/CO-CHAIRS, FACILITATOR OR TEAM LEADER

28. The team has a person(s) who guides discussion. 29. The team has a person(s) who reminds the group of the team’s outcomes, purpose, topics and

process.30. The team has a person(s) who regulates group activities and contributions. 31. The team has a person(s) who monitors time and moves the team through a process/protocol.32. The team has a person(s) who coordinates the data and information necessary for team

functioning.33. The team has a person(s) who poses questions to the team and lets members work through them

in discussions.34. Monitoring behaviors and practices consistent with the expected changes identified by the

collaborative team.35. Teams are clear that the chair/co-chairs are not administrators, do not shoulder the responsibilities of the

entire team, evaluate team member’s performance and do not report or address peers who do not cooperate or fulfill their responsibilities.

ADMINISTRATOR (DISTRICT AND BUILDING)

36. The district and/or building administrator has prepared the teams on their roles and responsibilities.

37. The district and/or building administrator, as instructional leader, models the use and application of data to inform instructional effectiveness.

38. The district and/or building administrator ensures access to valid, reliable and useful data.39. The district and/or building administrator ensures data is shared across levels, i.e., DLT, BLTs, TBTs.40. The administrator is on an “equal footing or partnership” with all members of the team.41. The district and/or building administrator has adjusted his/her work day to accomplish the work of

instructional leader.

MEMBERS

42. Teams have membership that reflects all appropriate stakeholders.43. Members assume the responsibility of assessing student learning using formative assessment

data.44. Members organize and present data in ways that identify gaps and trends in student performance

and adult practices.45. Members make intentional decisions about teaching and learning, i.e., curriculum, instruction,

interventions, professional development.46. Members constantly monitor student progress in meeting performance targets using district,

building and classroom data.TOTAL Score divided by 19

CONDITION F: COMMUNICATING PLAN INDICATORS AND PROVIDING AVAILABLE DATA SHARING WORK 47. Teams share their work and provide feedback with other teams.

METHODS

48. Teams know how (methods) and have tools/templates for communicating data and information within and across teams, i.e., DLT, BLT, TBT.

49. Teams know how often (frequency) data and information is to be communicated within and across teams, i.e., DLT, BLT, TBT.

CONTENT50. Teams know the plan indicators.51. Teams know what data and information to communicate across teams, DLT, BLT, TBT.


TOTAL Score for All ConditionsCONDITION A: PREPARING EDUCATORS TO WORK COLLABORATIVELY BY DEEPENING THE CULTURE OF INQUIRY

CONDITION B: FORMING OR REPURPOSING TEAMS TO IMPLEMENT AND MONITOR THE PLANCONDITION C: CREATING SCHEDULES AND ROUTINES THAT SUPPORT COLLABORATIVE TEAMSCONDITION D: MAKING MEETINGS PURPOSEFULCONDITION E: DEFINING ROLES AND RESPONSIBILITIESCONDITION F: COMMUNICATING PLAN INDICATORS AND PROVIDING AVAILABLE DATA


Resource 9: Data Source Identification (Tables to Organize for Data Collection and Summary to Complete the DF)

This resource provides a list of possible data needed for informed, data-based decisions necessary to complete each level and area of the DF. The list includes ideas of where a district may find local data. It is not all inclusive but may serve to stimulate ideas for data sources. Bolded Data Sources are pre-populated in the DF by the ODE. Italicized documents named are state-level data that support completion of the DF but are not pre-populated in the DF.This tool has two facilitator uses: 1) to guide the discussion to determine what data the district has readily available that can answer the questions in the DF, and 2) to provide an organizer for summarizing data to support DF ratings/judgments. Few districts will have data for every question in the DF. If there is no data, the facilitator needs to help the district make a choice to either 1) identify data that needs to be collected for future decisions, 2) collect data in the very near future to respond to questions in the DF, 3) include data collection in the district plan as an action, and/or 4) choose to disregard the question at this time. Whichever choice is made, the district/facilitator will want to make note of it.

Data Source Document Name Data Source Document Name1. Alignment Studies 27. Improvement Plans – District and

SchoolIntegrated Monitoring Process and Continuous Improvement Tool (IMPACT)Early Childhood Education Profile

2. Assessment Protocols 28. KRAL (P) KRA-L3. Assessment Results (District) 29. LPDC Procedures4. Assessment Results (State) State Required Assessments by District, Building, Subgroups;

Performance Index Data and Subscale DataGet It, Got It, Go (GGG)Ages and Stages Questionnaires: Social Emotional (ASQ:SE)Early Childhood Outcomes Summary Form (ECO)

30. Meeting Agendas and Minutes(e.g., Board – District – Data Teams)

5. Attendance 31. Mobility6. Audit Reports 32. Monitoring Procedures and Resources7. Bargaining Agreement 33. Partnerships8. Budget Analysis and Resource Allocations 34. PBIS Data9. Budget Process 35. PD Attendance10. Budgets including fund sources 36. PD Evaluation11. Building Attendance 37. PD-Process, Content and Context12. Building Schedules 38. Perception Surveys

(Parents, Students, Community)ECE and PSE programs required to administer survey once every two years

13. Classroom Observations by Administrators and Coaches

Early Language and Literacy Classroom Observations (ELLCO)IMPACT ObservationsEarly Language and Literacy Reflection Tool (ELLRT)

39. Performance Targets for Buildings

14. Coaching/Models for Teacher Support 40. Planning Meeting Agendas/Minutes15. Communications – written, Web-based 41. Policies and Procedures 16. Community Committee Members 42. Program Evaluation Criteria-Results17. Community Partners, Education

Foundations and Organizations43. Program Evaluation Data Strong Beginnings, Smooth Transitions,

Continuous Learning: A Ready School Resource Guide for Elementary School Leadership Self Assessment Tools

18. Curriculum Alignment Results Curriculum Based AssessmentCurriculum Alignment Tool

44. Purchasing Procedures

Resource 9: Data Source Identification (Tables to Organize for Data Collection and Summary to Complete the DF) September 2010 OIP Resources

Data Source Document Name Data Source Document Name19. Discipline Occurrences, suspensions and expulsions 45. Recruitment Plan and Results20. Documents (e.g., Human Resources) 46. Reports21. Drop Out 47. Schedules – Building22. Family activities: type, number, frequency 48. State Performance Plan (SPP)23. Family activity participation 49. Student Council/Group Input24. Graduation 50. Surveys of Enacted Curriculum25. Health Health and Developmental Screenings and Referrals 51. Teacher Leader – Names and Roles26. Highly Qualified Teachers (HQT) % of core academic areas not taught by HQTs, taught by

certified/licensed teachers or taught by temporary/long-term substitutesPraxis II and Praxis III scores on content area and PLT testsTeacher Credentials of Preschool Teachers

52. Union Records

26. Human Resource Records 53. Value-Added

Level I: Identified Issus: Overall and Subgroup Student Performance Data (For each content area)

Data Sourcesby Number Data Summary

1. Grade level and/or grade-level band(s) where overall performance is low. 4 28

2. Student achievement by district and building for subgroups (students with disabilities, economically disadvantaged students, LEP, minority students) scoring below the standard compared to all students.

4 28

3. Grade levels showing improvement over the trend (3 year) period to include comparison of higher performing to low performing grade levels, comparison of subgroups performing well in relationship to AYP.

4 28

Level I: Student Performance Subscale Performance Data1. Reading: Vocabulary, Reading Process, Information Text,

Literary Text4 28

2. Math: Numbers/Number Sense, Measurement, etc. 4 28

Level II A: Instructional Management: Curriculum, Assessment and Instructional Practice


Area 1: Curriculum1. Use of a standards-based curriculum aligned to the Ohio Academic Content

Standards, standards-based curriculum.2. Alignment of textbooks and instructional materials to all applicable areas of the

Ohio Academic Content Standards and district curriculum materials3. Use of curricular materials that stress higher order thinking skills Area 2: Assessment1. Formative and summative assessments aligned to Ohio Academic Content

Standards


Level II A: Instructional Management: Curriculum, Assessment and Instructional Practice


2. Required use of formative/short-cycle assessments aligned with Ohio Academic Content Standards to help make learning goals clear to students, engage students in self-reflection and assessment and provide descriptive feedback to students

3. Opportunities for teachers to work together to develop common classroom formative assessments aligned with Ohio Academic Content Standards.

4. Aligned formative/short-cycle assessments with district curriculum/instructional materials that are available and used across the district.

5. Accessibility to, frequency of teacher use and monitoring of teacher use of formative/short-cycle assessments including the use of scoring protocols.

Area 3: Instructional Practice1. Benchmarks and grade-level indicators used to guide students' progress

toward meeting Ohio Academic Content Standards2. Learning goals and activities at all grade levels aligned to Ohio Academic

Content Standards3. Learning goals communicated to students, staff and parents/families4. Use by all teachers at all grade levels of instructional practices that are

research- and evidence-based (Listed in DF)5. District monitoring of use of instructional strategies6. Access to the district’s core instructional program for all students7. Academic/behavioral supports 8. Opportunities for teachers to collaboratively plan and deliver instruction in

core content areas9. Teacher evaluation reflects use of instructional best practices.

Level II B: Instructional Management: Educator Quality Data Sourcesby Number Data Summary

Area 1: Teacher TurnoverArea 2: Administrator Turnover1. Number of full-time classroom teachers and administrators employed by

position and assigned to each building2. Number of teachers and administrators who have changed positions (grade

level, building, assignment) in the last three years, by year3. Number of teachers and administrators who retired, moved, resigned, or left

due to reduction in force4. Number of teachers and administrators who are new to the district


Area 3: Educator Qualifications1. Consider teacher competencies and skills in making decisions about

assignments/transfers/reassignments2. Percentage of non-HQTs in underperforming buildings compared to (or less

than) the percentage of non-HQTs district-wide.25

3. Processes to departmentalize or compartmentalize teacher assignments to take advantage of individual teacher expertise

4. Hiring process that requires teachers to demonstrate skills and competencies beyond Praxis II or III scores

Level II C: Instructional Management: HQPD Data Sourcesby Number Data Summary

Area 1: Aligned to Identified Problem Area Area 2: Promote Shared Work1. HQPD in identified weaknesses in the content area of concern2. PD meets Ohio Standards for HQPD3. LPDC approval process for individual PD plans (IPDP) based on the

districts/building’s identified weaknesses in the content area of concern.4. HQPD provided based on research-based instructional strategies aligned to

identified weaknesses in the content area of concern 5. Participation rates in HQPD to meet identified weaknesses in the content

area of concern6. Identification of gaps/weaknesses in the PD offerings7. Opportunities for educators to share their work and collaboratively develop

new instructional strategies to address areas of concernArea 3: Application/Impact of Knowledge/Skills Learned

1. Follow-up teacher observations/evaluations to verify application of knowledge/skills learned through HQPD

2. Processes for monitoring expected behavior change intended to result from participation in HQPD

3. Use short cycle assessments and other data to provide evidence that participation in HQPD contributed to improved student performance.

Level III A: Expectations and Conditions: Leadership Data Sourcesby Number Data Summary

Area 1: Data and the Decision-Making Process1. Procedures implemented require effective use of data at all levels of the

system to drive improvement in instructional practice, assess the impact on student achievement and make decisions about teaching and learning.



2. Use of aggregated and disaggregated student achievement data to establish district goals and measurable strategies for instruction and achievement.

3. Performance targets for each building and grade level set and communicated4. Assistance for administrators in monitoring staff use of data to inform

instructional decisions.5. Training, support, and guidance in the effective use of data for building

leadership teams (BLTs).Area 2: Focused Goal Setting Process1. District plans have goals based on data2. District has a mission that guides the collaborative development of district

goals 3. SIPs aligned to district plan4. Internal accountability system holds all levels accountable for results5. Monitor progress of the district improvement plan Area 3: Instruction and the Learning Process1. Support implementation of standards-based instruction aligned to district

curriculum, plan goals for instruction and achievement on a district-wide basis.2. District curriculum is the curriculum used in all schools.3. Priorities identified among the district's instructional goals and objectives.4. Delivery of instruction on a district-wide basis based on research-based

practices, engaged students, culturally responsive practices and ongoing assessment and progress monitoring to inform instruction.

5. Processes that accurately monitor the district's instructional program.6. Monitor student achievement of school instructional practices designed to

provide full access and opportunity to all students to meet district goals.7. Implementation and ongoing evaluation of prevention/intervention strategies

as a part of its instructional program8. Frequent use of collaboratively developed common formative classroom

assessments to gauge student progress and guide instructional planning.9. Assist administrators in fulfilling instructional leader responsibilities.10. Assist administrators and BLTs to effectively monitor student progress.Area 4: Community Engagement Process1. Internal and external community members involved in development and

support of district goals2. Communicate clear expectations with regard to district goals3. Opportunities for meaningful input and feedback from internal and external

community members with regard to district goals4. Partnerships focused on district goals



5. Training/support needed by internal and external community members to enable them to participate in activities aligned to goals

Area 5: Resource Management Process1. Recommendations to the superintendent regarding financial and capital

management aligned to goals2. Data used to inform the budget process and allocate district resources to

support district goals3. Appropriate time, training and resources allocated equitably to support the

effective use of data to improve planning and instruction.4. Recommendations to the superintendent regarding scheduling that is aligned

to goals5. Support and allocate resources to schools to meet district goals6. Screen, interview and select staff based on district goals7. Supervisory systems to ensure progress toward meeting district goals8. Extensive job-embedded professional development aligned with district goals9. Eliminate initiatives not aligned with or ineffective in meeting district goalsArea 6: Board Development and Governance Process1. Partner with board members to adopt and review progress toward goals2. Data and reports communicated by the superintendent to the board of

education3. High expectations for district and school performance4. Community members articulation of district goals5. Professional development for board members aligned with district priorities

Level III B: Expectations and Conditions: School Climate Data Sourcesby Number Data Summary

Area 1: Student Discipline Occurrences 14

1. Three year trend data on discipline occurrences by grade, building, subgroup 14

2. Three year trend data on suspensions/expulsions by grade, building, subgroup

14

Area 2: Student Attendance1. Three year trend data on attendance by grade, building and subgroup. 5Area 3: Student Mobility1. Mobility data identifies students at risk and set priorities for improvement2. Use data to monitor student mobility (number and percentage of students

who moved within the district during the school year) by building, grade level, gender and subgroup

3. Comparison of the performance of mobile and non-mobile studentsResource 9: Data Source Identification (Tables to Organize for Data Collection and Summary to Complete the DF) September 2010 OIP Resources

Level III B: Expectations and Conditions: School Climate Data Sourcesby Number Data Summary

4. Buildings by level affected by district initiated moves (e.g., building closures, building openings and population reassignments)

5. Comparison of performance of students from stable v. less-stable buildingsArea 4: Multiple Risk Factors1. Number and percentage of students by building, gender and sub-group with

multiple risk factors (i.e., high levels of discipline occurrences, absences and mobility rates)

4 19

2. Evaluation of affect of multiple risk factors on student performance 4 5 19

3. Feedback mechanisms for students, families and personnel regarding school climate and use data to monitor and improve environment

4. Support for schools in identifying effective ways of improving student engagement in school activities and a sense of belonging

Level III C: Expectations and Conditions: Engagement: Parent/Family, Student and Community Involvement


Area 1: Parent Participation and Perception1. Parent activities designed to share information of their child's academic

progress and performance2. Parents beliefs (e.g., about district academic focus, provision of safe

environment, sensitivity to academic and behavioral needs, ability to provide a challenging curriculum for all students)

Area 2: Communication1. Communicate with families to increase their knowledge and skills to support

their children’s learning at home and their academic progress at school2. Build families’ knowledge and skills to support their children’s learning at

home and their child’s academic progress at school3. System of two-way communication for families, using clear language4. Communicate in the primary language of the parent and community5. Family activities reflect the needs and characteristics of students’ families6. Professional development for administrators, teachers and other staff on how

to build strong partnerships with familiesArea 3: Student Participation and Perception1. Student beliefs - considering gender and disaggregated groups - (district

focus student outcomes, safe environment, sensitivity to academic and behavioral needs, challenging curriculum).

Area 4: Community Involvement and Support


1. Parent and community agencies/partners/businesses work with the district to support improved student outcomes and maximize the use of available resources in improving student performance

2. Engage pre-schools in PD, transition services and curriculum planning3. Involve community partners in policy development & planning

Level IV: Resource Management Data Sourcesby Number Data Summary

Area 1: Teacher and PD Alignment1. Recruiting strategies get the highest quality staff possible2. Support and in-house instructional guidance and assistance3. Faculty assignments meet student needs in cost-effective ways4. Teachers make key decisionsArea 2: Focus on Student Success

1. Evaluations of initiatives ensure they meet instructional and content needs of all students

2. Student instructional time is spent learning mathematics and language arts3. Contract year increases teacher development and planning time4. Focus district and building meetings on learning and instruction5. Schedule keeps non-instructional activities to a minimumArea 3: Aligning Spending to Strategic Priorities

1. Use district assessment data to guide initial and follow-up decisions related to performance and alignment

2. Membership of DLT includes district treasurer3. Budgeting and related processes ensure effective spending4. Multiple purchasing options reduce inefficiencies5. Budget decision-making process directly supports the district’s plan6. Use data to make intentional budget decisions that continue support for

successful strategies and remove support for unsuccessful strategies7. Pool funds from multiple sources to support academic programming,

especially for at risk populations and subgroups8. Resolution of financial determinations brought by the State Auditor


Resource 10: Expanded Facilitation QuestionsThe following expanded facilitation questions are provided to assist facilitators in probing deeper in all stages of the OIP. The additional questions are designed to deepen investigation of root causes for discrepancies in student achievement and deepen the investigation of factors stressing student achievement.

Decision Framework (DF) questions are bulleted with an apple. Providing the bulleted DF questions helps the facilitator put the expanded questions in proper context.

Additional questions have been added to address:

General considerations that supplement those in the DF.

Students with Disabilities. These questions were developed based on the trends identified in the Universal Gap Study (2009) and the experience of practitioners focused on improving the performance of students with disabilities.

Early Child Programs (which include an emphasis on preschool through third grade programs). Facilitators can use these questions as a way to promote a preschool through 12th grade focus and discuss critical subgroups during the OIP process. While the data and prompts in the DF may not focus specifically on the early childhood programs in the district, these should be considered in the conversations that occur when completing the decision framework.

Student Physical, Social and Emotional Health and School Climate. Broadening conversations around proficiency, instructional practices and teacher preparedness to include discussion about what is limiting a subpopulation’s ability to learn from a physical, social or environmental standpoint, allows districts to address learning comprehensively and develop plans to effectively achieve annual targets. By working with families and communities to address these learning and behavioral support issues during and outside the school day, districts create environments where students are able to focus, absorb and apply knowledge gained through their rigorous education to coursework and life.

Facilitators will need to be selective in the use of the additional questions, basing decisions on the data and concerns raised by the DLT/BLT, including those that arise in planning, monitoring and evaluation. As always, the expertise of facilitators in posing questions, acting as a critical friend and guiding data based investigations by district and building teams cannot be underestimated. The questions are by no means all inclusive, they are meant to be “food for additional thought”.

LEVEL I: PROFICIENCY

GENERAL EXPANDED QUESTIONS

Consider that the data portrayed show only percent (%) proficient. For those student groups not meeting the state standard for AYP, look for other signs of significant progress that indicate they are likely/not likely to be proficient by 2013-14.

Did a significant percentage of a student group move from “below basic” to “basic” over the last three years? If you average the scale scores for the various groups, is the student group in question progressing at a faster rate over the last three

than the others student groups or faster than the “all” group category? If that trend continues, would you expect that most of the students will be proficient by 2013-14?

Do value-added data reflect positive progress even though the percent of proficient students may not meet state standards? Is this progress sufficient to ensure students will graduate with a solid education?

Are other assessment data used to gauge student progress against grade level standards and/or their response to intervention for the purpose of modifying instructional practice (see Assessment Questions)?

EXPANDED QUESTIONS FOCUSED ON STUDENTS WITH DISABILITIESResource 10: Expanded Facilitation Questions September 2010 OIP Resources

What structures are in place to ensure the district does not have a disproportionate representation of students with disabilities from racial, ethnic, and socioeconomic groups?

Using your SPP indicator data, does your district meet or exceed the established targets for disproportionate representation of students by race/ethnicity as students with disabilities or in specific disability categories?

What is the representation of specific racial/ethnic groups compared to: the total number of SWD in the district? the total number and percentage of students in the racial/ethnic group in the overall school population?

How does your district’s percentage of SWD compare to the national average of 12%?How does your district’s percentage of SWD compare to those districts comparable in size and demographics?

EXPANDED QUESTIONS FOCUSED ON EARLY CHILDHOOD

KRA-L DataGet It, Got It, GoWhat is the mean score in the areas of Picture Naming, Rhyming, and Alliteration during the Fall and Spring for all students and by ages 3, 4

and 5? What percentage of children made progress from the Fall to Spring administration in all three assessment areas?

Ages and Stages Questionnaire: Social Emotional What percent of children are at risk for social and emotional concerns?

Early Childhood Outcome Summary Form DataWhat are the percentages of children ages 3 to 5 that demonstrate improved positive social emotional skills, acquisition and use of

knowledge and skills, and use of appropriate behaviors to meet their needs?

KRA-L DataHow do KRA-L data vary across subgroups of children (compare data according to major disaggregated groups to determine if a single

group or several groups are coming to school significantly behind their peers)? How many (number, percentage) students’ scores count in more than one subgroup? In reviewing disaggregated student data, are there achievement gaps among the groups and what is happening to the

gap(s) over time? Are they widening, staying the same or closing?How do the KRA-L scores of children participating in formal early education or care experiences compare to the scores of children

who had no prior formal education or care experiences?

LEVEL II: CONTENT AREA

CURRICULUM

Has the district leadership team (DLT) ensured the use of an aligned, standards-based curriculum on a district-wide basis?To what degree does the DLT:

1. Align the [applicable Content Area] curriculum with all applicable areas of the Ohio Academic Content Standards and Early Learning Content Standards?

2. Ensure that [applicable Content Area] textbooks and instructional materials are aligned to all applicable areas of the Ohio Academic Content Standards and Early Learning Content Standards?

3. Ensure that [applicable Content Area] textbooks used are aligned to district curriculum materials?4. Ensure that curricular materials are used to facilitate the use of higher order thinking skills (i.e., 21st Century Skills)

on a routine basis?


Is there uniformity across the district regarding textbook/materials that are used by all students at specific grade levels/subject areas?

Do all teachers have copies of the core materials? If a student group is not performing well (e.g., English language learners), are there materials that teachers can use as part of

the routine instruction to improve success? Are additional high quality materials, aligned to the curriculum, available to help students succeed (reading sources, universally

designed software, on-line tutoring, etc.)? Are curricular materials challenging to the students but differentiated to match the students’ instructional needs?

Resource 10: Expanded Facilitation Questions September 2010 OIP Resources

EXPANDED QUESTIONS FOCUSED ON STUDENTS WITH DISABILITIES Describe how the IEP team determines the level of services and accommodations that allow students to progress in the general

education curriculum and participate in co-curricular and extracurricular activities.

ASSESSMENT

Has the DLT ensured the district-wide use of effective assessment practices aligned with standards-based curriculum and instructional materials to continuously monitor student progress and make instructional decisions?

To what degree does the DLT:

Ensure that formative and summative assessments are aligned with the Ohio Academic Standards and Early Learning Content Standards?

Require the use of formative/short-cycle assessment aligned with the Ohio Academic Content Standards and Early Learning Content Standards and to help make learning goals clear to students?

Require the use of formative/short-cycle assessment aligned with Ohio Academic Content Standards and Early Learning Standards to engage students in self-reflection and self-assessment?

Require the use of formative/short-cycle assessment aligned with Ohio Academic Content Standards as a way to provide descriptive feedback to students?

Facilitate opportunities for teachers to work together to develop common classroom formative assessments aligned with Ohio Academic Content Standards and Early Learning Content Standards?

Alignment of formative/short-cycle assessment with the district curriculum/instructional materials. Availability of formative/short-cycle assessment across the district. Assess the level of cognitive demand of formative/short-cycle assessment across the district. Assess the frequency of teacher use of formative/short-cycle assessment. Assess teacher accessibility of formative/short-cycle assessment. Assess the use of consistent scoring protocols for formative/short-cycle assessment. Assess the degree to which the district monitors teacher use of formative/short-cycle assessment.


Do data collected from other assessments (e.g., DIBELS, diagnostics, standardized, common formative assessments) validate or challenge the results of the state assessments?

Do any of these assessments allow the district/building to dig deeper into a subset of the content area (i.e., reading comprehension, fluency, vocabulary, etc.)?

What trend data does the district use to identify the early warning signs for students at risk? How does the district use that data to inform instructional approaches and interventions and to connect students with academic/behavior supports and community services?

EXPANDED QUESTIONS FOCUSED ON STUDENTS WITH DISABILITIES

Is progress monitoring or other strategies used to gauge students’ response to interventions to improve access to grade-level content and to reduce inappropriate referral to special education?


Does the preschool program have an adopted curriculum that includes a curriculum based assessment aligned with the early learning content standards?

How is the district reviewing and using the information from the Ages and Stages Questionnaires-Social Emotional and the results from the Health and Development Screenings to support preschool children’s development?

How is the preschool program using information from curriculum based assessments, Get it, Got it, Go and the Early Childhood Outcome Summary Form to adjust instruction to meet the learner’s needs?

Does the district review universal screening data for incoming kindergarten students to Identify children who need additional assessments and connect families to support services where needed? Does the district review universal screening data for incoming kindergarten students to ensure that instruction is differentiated to meet all

learners’ needs?

INSTRUCTIONAL PRACTICE


Has the DLT ensured the district-wide use of effective standards-based instructional practices in teaching all students? To what degree does the DLT:Alignment with Standards: Ensure that the benchmarks and grade-level indicators are used to guide students' progress toward meeting the Ohio

Academic Content Standards and Early Learning Content Standards? Require the learning goals and activities at all grade levels be intentionally aligned to Ohio Academic Content Standards and

Early Learning Content Standards?Quality/Rigor of Instructional Practice Require teachers at all grades to use instructional practices that are research and evidence based? Require that learning tasks at all grade levels teach students to use higher order thinking skills? Take action to increase the district-wide use of high-yield instructional strategies such as the following:

o Writing across the curriculumo [Applicable Content Area] in all content areaso Identifying similarities and differenceso Summarizing and note takingo Reinforcing effort and providing recognitiono Homework and practiceo Nonlinguistic representationso Setting objectives and providing feedbacko Generating and testing hypotheseso Using cues, questions, and advance organizers

Monitoring Degree of Use of Effective Strategies Monitor the district-wide use of high-yield instructional strategiesStudent Engagement Support teachers in ensuring that all students are actively engaged in learning activities and tasks? Require full access to the district's core instructional program for all students Require the provision of additional academic/behavioral supports for students who need them to meet the learning goals

aligned with the Ohio Academic Content Standards and Early Learning Content Standards? Require the learning goals at all levels be clearly communicated to all students, staff, and parents/families?Shared Responsibility Provide opportunities for teachers to collaboratively plan and deliver instruction in core content areas to all students? Ensure that teacher evaluations at all levels reflect the use of instructional "best practices?" Discourage teachers from using counterproductive teaching strategies?


Is there agreement at the district level on what constitutes high-quality teaching and learning?Have teachers been trained in differentiated instruction and do they regularly employ it in their classrooms?

o Does the district expect more challenging opportunities to be provided to all students who are typically succeeding well in class?o Does the district expect more challenging opportunities to be provided to high-able learners?o What evidence do you have of systemic and systematic early intervening services in place for academic and behavioral supports to

ensure students are learning?o Does the district expect instruction to be modified, as needed, to meet the instructional needs of struggling learners?

To what degree are you providing direct and systemic instruction when needed? If a student group is not performing well, is the district sure that these students have access to, and are participating in, regular classroom

instruction and learning?Is there an expectation that students will receive what they need to maximize learning: targeted interventions, acceleration, small group

instruction, re-teaching, etc.Are there built in opportunities for additional high quality instruction and practice for students who are not achieving at grade level?Do all layers of the district including teachers, administrators, and students share the responsibility for student improvement?Are instructional responses to cultural and/or language differences a possible partial cause of student underachievement?Are teachers aware of cultural differences among the student body?


Is assistive technology available to meet student needs at all grade levels/buildings?


What percentage of the available instructional time are intervention specialists engaged in: providing direct, specialized instruction; providing homework support; or modifying instructional materials and assignments? Is the use of available instructional time resulting in student learning that is closing achievement gaps?

To what degree are educators using universal design for learning, assistive technology, and other research based instructional techniques and supports to meet the needs of students with the most intense educational needs?

Has the district insured that students are provided with high quality instruction and intervention prior to being referred for evaluation for a suspected disability?

Using district SPP data and EMIS data, determine to what degree you are ensuring that SWD are served in the least restrictive learning environment with supports and specialized instruction needed to be successful.

Describe the district/school service delivery model. Is it consistent across the district? What structures are in place to ensure that the service delivery model changes as the needs of SWD change?

Describe the involvement of general education teachers in the education of SWD and the development of the IEP. Are you including students with disabilities who can be successful in the general education classroom as well as providing small

group instruction for students who need more support? Are intervention specialists who provide direct academic instruction in content areas and serve as the teacher of record highly

qualified in those area(s)? What structures are in place to guide teams making LRE determinations?


Do the early childhood learning environment and instructional strategies reflect a balance between child directed and adult directed experiences? Is there opportunity in the daily schedule for large group, small group, and individual experiences?

Do early childhood teachers employ developmentally appropriate, evidence based instructional strategies to meet the diverse needs of learners?

How do early childhood teachers design instructional experiences that demonstrate their understanding of children’s learning processes and support children’s social emotional and academic needs?

EXPANDED QUESTIONS FOCUSED ON STUDENT PHYSICAL, SOCIAL AND EMOTIONAL HEALTH AND SCHOOL CLIMATE

Are teachers and support staff trained to understand and recognize signs of physical, social and emotional barriers to student learning? Does the district engage the community and families to help deal with these issues in during and outside the school day?

o Are student tired?o Are student hungry?o Do students struggle from uncontrolled asthma? o Are students struggle from uncontrolled ADHD? ADD?o Can student see the board/computer? Are vision and hearing screenings available in the district? Are linkage

protocols in place to align students with interventions/services?o Are teachers trained to recognize sign and symptoms of depression or childhood trauma? Are linkage protocols

in place to align students with interventions/services?o Are teachers rained to recognize sign and symptoms of depression or childhood trauma? Are linkage protocols

in place to align students with interventions/services?o Are teachers trained to recognize sign and symptoms of substance/ drug use and abuse? Are linkage protocols

in place to align students with interventions/services? Does the district have partnerships with local businesses and organizations to help teachers create curriculum, instruction and learning

opportunities that engage students in relevant workforce application? Do students and their families know and have access to community-based learning opportunities? Are teachers trained to recognize when students are disengaged from course work because of a physical, social or emotional stressor? Are teachers and support staff trained to understand and recognize signs of physical, social and emotional barriers to student learning? Does the district engage the community and families to help deal with these issues in during and outside the school day?Are school administrators, teachers and support staff trained to understand how school climate and culture impact student connectedness

and ability to learn? Does this training include strategies on how school staff can contribute to a positive school climate?Does district implement School Connectedness Strategies designed to increase protective factors for students and families?

Students that feel connected to school perform better academically and are less likely to engage in risky behavior. Parents that feel connected to school are more willing to be involved in school activities and engage in regular communication with their child’s teachers.

Do classroom teachers or school principals make introductory calls home to all parents of student at the beginning of the year?Do classroom teachers stand in the hallway between classes and address students by name?


Is a student recognition program in place that celebrates “Strong Character” not just academic achievements? Is this program communicated, implemented and celebrated district wide?

Are students physical, social and mental health needs assessed and addressed when developing behavioral supports?Are school administrators and teachers trained to recognize the physical, social and mental health stressors that impact learning? Are

protocols in place to address issues if they are identified? Does the district have linkage protocols in place to connect students and families with providers in the community to address needs before, during or after the school day?

Is there an expectation that parents/families will be contacted early and regularly to discuss students academic and behavioral needs and interventions.

PROFESSIONAL DEVELOPMENT

HQPD Aligned to Identified Problem Area – To what degree has the district leadership team (DLT) ensured the provision of high quality PD to clearly address the weakness(es) in the identified problem area of [applicable Content Area]: Ensure the provision of HQPD across the district to meet identified weaknesses in the content area of concern? Ensure that the PD provided met Ohio Standards for high quality professional development (HQPD)? Review and take action to ensure that the local PD committee (LPDC) approves individual PD plans (IPDP) based on the

district's/building's identified weaknesses in the content area of concern? Ensure that the HQPD provided is based on research-based instructional strategies aligned to identified weaknesses in the

content area of concern? Ensure that a critical mass of teachers within each low-performing building participate in the HQPD to meet identified

weaknesses in the content area of concern? Ensured that the district identified any gaps/weaknesses in its PD offerings?

HQPD to Promote Shared Work – Has the DLT ensured the implementation of HQPD to promote shared work across the district/buildings to address the weakness (es) in the area of [applicable Content Area]? To what degree does the DLT: Provide opportunities for educators to share their work and collaboratively develop new instructional strategies to address

areas of weakness? Provide HQPD to advance educators' content knowledge? Provide HQPD to advance educators' ability to use research-based instructional strategies to improve student success?

Application/Impact of Knowledge/Skills Learned through HQPD – Has the DLT established a process to verify application of knowledge/skills learned through HQPD? To what degree does the DLT: Set an expectation that teachers apply knowledge/skills learned through HQPD? Conduct follow-up teacher observations/evaluations to verify application of knowledge/skills learned through HQPD? Establish processes for monitoring expected behavior change intended to result from participation in HQPD? Use short cycle assessment to provide evidence that participation in HQPD contributed to improved student performance? Review data to determine the amount of improvement that can be attributed to participation in HQPD (focused PD, follow-up

activities, training, and support) and make necessary adjustments


Does the district have a policy on PD that is aligned with the Ohio Standards for Professional Development and, if so, does the district operate in accordance with these standards?

Are all teaching staff, including instructional coaches and mentors included in professional development on core content? Are administrators included in professional development on core content? Are administrators, teachers and staff trained on the district behavioral support and student intervention system? Are teachers provided with opportunities to learn from each other? Have administrators and teachers been trained in effective data collection tools and use of data to make educational decisions? Is there a commitment to full implementation of any evidence-based practices learned through PD? Is there a process in place for principals, teacher team, and/or BLTs to frequently observe/monitor classroom instruction to ensure that the

“learnings” from PD are being consistently implemented at a high level?


Do administrators, teachers, and staff receive professional development on creating a safe and supportive school climate Are all administrators, teachers and staff trained on effective strategies for engaging families and community partners in student learning. Are school administrators, teachers and support staff trained to understand and recognize signs of physical, social and emotional barriers to

student learning?


LEVEL II: EDUCATOR QUALITY

TEACHER TURNOVER Has the district leadership team (DLT) identified the rate of teacher turnover across the district and its impact on student

achievement?

EXPANDED QUESTIONS FOCUSED ON STUDENT HEALTH, WELLNESS AND SCHOOL CLIMATE

Does the district support an employee wellness program? Has the district completed a staff survey to understand what employees perceive as the benefits and challenges to working in the

district?

ADMINISTRATOR TURNOVER Has the district leadership team (DLT) identified the rate of principal turnover across the district and its impact on student

achievement?


Does the district support an employee wellness program? Has the district completed a staff survey to understand the what employees perceive as the benefits and challenges to working in

the district ?

EDUCATOR QUALIFICATIONS Has the DLT ensured that all students receive instruction from teachers with demonstrated subject matter competence?

To what degree does the DLT? Require each building to consider teacher competencies and skills in making decisions about

assignments/transfers/reassignments? Require teachers to demonstrate skills and competencies beyond Praxis II or III scores during the hiring process? Consider teacher demonstration skills and competencies beyond Praxis II or III scores in making decisions about

transfers or reassignments? Ensure that the percentage of non-HQT teachers in underperforming buildings is comparable to (or less than) the

percentage of non-HQT teachers district-wide? Implement on a district-wide basis processes to departmentalize or compartmentalize teacher assignments to take

advantage of individual teacher expertise? Support buildings to departmentalize or compartmentalize teacher assignments to take advantage of individual teacher

expertise?

LEVEL III: LEADERSHIP

DATA AND DECISION MAKING Has the district leadership team (DLT) facilitated the effective use of data at all levels of the system to drive improvement in

instructional practice? Assess the impact on student achievement. To what degree does the DLT: Establish and implement procedures and norms requiring the effective use of data at all levels of the system to drive

improvement in instructional practice, assess the impact on student achievement, and make decisions about teaching and learning?

Model the effective use of data as an ongoing strategy to improve student performance? Require the use of current aggregated and disaggregated student achievement data to establish district goals and measurable

strategies for instruction and achievement? Based on analysis and interpretation, set performance targets for each building and grade level, planning for the success of all

children and designed to close achievement and expectation gaps for every subgroup? Assist administrators in monitoring staff use of data to inform instructional decisions? Provide training, support, and guidance in the effective use of data for building-level teams (DLT)?


To what degree has the DLT consider the use of student health, behavior, school climate , family and community engagement data to develop strategies and action steps that that support district academic goals?

Is there a culture of trust and openness so that mistakes are not viewed as failures?Resource 10: Expanded Facilitation Questions September 2010 OIP Resources

Are there collaborative teams/structures in place for the purpose of improving achievement across grades, subjects, buildings? Is there an expectation for common meeting time for teachers and administrators to collaboratively used data in decision making to

inform their work at the classroom/building level?


To what degree are administrators and/or other designated trained in identifying and monitoring the instructional practices and learning supports that meet the needs of learners with intense educational needs?


To what degree does the DLT consider the use of student health data, student health behavior data, school climate and family and community engagement data to establish effective learning support strategies and action steps to support instruction and achievement?

Are there training opportunities for school administrators and teaching staff to learn to effectively link academic indicators with health, behavior and school climate data to benchmark and measure impact of learning and behavioral support initiatives?

FOCUSED GOAL SETTING

Has the DLT ensured the development and ongoing monitoring of a focused district improvement plan built around district goals for instruction and achievement? To what extent does the DLT: Support the development and ongoing monitoring of a single district improvement plan (CIP) that focuses on a limited number

of district goals? Facilitate the implementation of the district improvement plan with a limited number of district goals that are based on current

aggregated and disaggregated student achievement data? Commit to continuous improvement toward meeting the district goals? Convey to all schools the district's vision and mission for guiding the collaborative development of district goals, and

communicate performance targets to all buildings? Ensure that all schools have a focused SIP clearly aligned to and designed to meet the district's CCIP? Implement an internal accountability system that holds adults at all levels accountable for results? Monitoring the progress of the district improvement plan and, based of current data, make necessary adjustments?


Has the DLT ensured that goals identified in the early childhood continuous improvement process align with the district CIP?

INSTRUCTION & LEARNING Has the DLT ensured the implementation of high-quality standards-based instruction aligned with district goals for instruction and

achievement? To what extent does the DLT: Support the implementation of high-quality standards-based instruction aligned with the district's curriculum and goals for

instruction and achievement on a district-wide basis to satisfy Ohio Revised Code requirements incuding health and physical education?

Assure that the district curriculum is the curriculum used in all schools? Convey clear priorities among the district's instructional goals and objectives? Ensure the delivery of high-quality instruction on a district-wide basis that is based on research-based practices, engages

students, incorporates culturally responsive practices, and relies on ongoing assessment and progress monitoring to inform instruction?

Implement the processes that accurately monitors the district's instructional program? Monitor student achievement, ensuring that school instructional practices are designed to provide full access and opportunity

to all students/students groups to meet district goals? Ensure the systemic implementation and ongoing evaluation of prevention/intervention strategies as a part of its instructional

program to ensure that all students meet performance targets? Require the frequent use of collaboratively developed common formative classroom assessments to gauge student progress

and guide instructional planning toward meeting district goals? Assist administrators in fulfilling instructional leader responsibilities? Assist administrators and building leadership teams (DLTs) to effectively monitor the progress of all students in their building

toward meeting district goals?


Does the district have a policy on prevention/intervention that:


o Addresses adult actions necessary when children are not meeting grade-level indicators?o Ensures the use of a consistent prevention/intervention approach on a district-wide basis? o Reduces the inappropriate identification of children as SWD?o Reduces the disproportionate representation of children from racial/ethnic groups as SWD?o Improves access to challenging curriculum and instruction for all learners?

Does the district have in place practices (e.g., accelerated learning strategies, differentiated instruction, before/after/summer experiences, etc.) to systematically accelerate the learning of students who are below grade level to ensure that they graduate at or above grade level?

Has the district eliminated barriers and improved access to accelerated content and placement, including dual credit options, for high-able learners?

Has the district ensured equal access to accelerated content and placement for all students, particularly those from traditionally underrepresented groups?

What policies/practices are in place to minimize the loss of instructional time? Subs, Pull out services, discipline, classroom management, testing interruptions, transitions, organization of school day


Do you have a district philosophy supporting the practice of a universal education approach as opposed to a regular education/special education dichotomy?


Does the district’s focused plan include student health, wellness and school climate strategies and actions steps to support achievement of district goals? Are these strategies and action substantiates with data and linked to academic indicators?

COMMUNITY ENGAGEMENT Has the DLT engaged parents, families, community members, and stakeholders in support of district goals for instruction and

achievement? To what extent does the DLT: Collaborate effectively with internal and external community members in the development and support of district goals? Communicate clear expectations with regard to district goals? Offer opportunities for meaningful input and feedback from internal and external community members with regard to district

goals? Develop partnerships focused on district goals? Provide training/support needed by internal and external community members to enable them to meaningfully participate in

activities aligned with district goals?


To what extent does the DLT:o Use community resource mapping to identify external partners who can help achieve district goals?o Use protocols to standardize how the district works with outside agencies, vendors and providers to achieve district goals. o Utilize their Family and Civic Engagement Team to link school with community based resources and services to support student

learning?o Inventory, align and coordinate partnerships activities to district goals?o Engage families and communities in developing the district vision, policies and goals?


To what extent does the DLT support children as they transition into Kindergarten from district operated preschool, community operated preschool/childcare, and home settings?


Does the district’s family and civic engagement team have families that are representative of their student body (ie. pre-school – 12, socio-economic and culturally diverse)?

RESOURCE MANAGEMENT Has the DLT facilitated the intentional use of district resources, including time as well as staff, programmatic, and monetary

resources to support district goals for instruction and achievement? To what extent does the DLT: Assess and make recommendations to the superintendent regarding financial and capital management aligns to district

goals for instruction and achievement?Resource 10: Expanded Facilitation Questions September 2010 OIP Resources

Use data to inform the budget process and allocate district resources to support district goals? Allocate equitable and appropriate time, training, and resources to support the effective use of data to improve focused

planning and instruction on a district-wide basis? Assess and make recommendations to the superintendent regarding scheduling aligned to district goals? Support and allocate resources to schools to meet district goals? Screen, interview and select staff based on district goals? Establish and implement supervisory systems the ensure progress toward meeting district goals? Provide for extensive job-embedded professional development aligned with district goals? Identify initiatives not aligned with or ineffective in meeting district goals that should be eliminated?


To what extent does the DLT assess and make recommendations to the superintendent regarding financial and capital investments to support district goals for student/staff wellness initiatives that promote student achievement?

BOARD DEVELOPMENT & GOVERNANCE Has the DLT worked in partnership with board members to adopt and continually review progress toward meeting district goals for

instruction and achievement? To what extent does the DLT: Keep the purpose of ensuring the success of every student central to all decisions? Support the superintendent's work in partnering with board members to adopt and continually review progress toward

meeting district goals? Provide data and reports to the superintendent to inform the board as a part of policy governance? Maintain high expectations for district and school performance? Continually promote high expectations so that all internal and external community members can articulate district goals? Support the provision of professional development aligned with district priorities for board members?


Utilize their family and civic engagement to provide recommendations to the board on students’ physical, social and emotional health, family and community engagement needs and activities.

Adopt and continually review progress toward meeting district-school climate and physical, social and emotional health and family and community engagement goals.

LEVEL III: SCHOOL CLIMATEDISCIPLINE

Has the District evaluated and taken action to minimize the impact of student discipline on student performance?


Is there a proactive and positive behavior policy in place to minimize discipline referrals and increase access to instruction?

NOTE: disaggregate discipline referral data – who is referred for what – what are the differences across student subgroups? If there is a drop in discipline occurrences, are data reviewed to ensure that the drop is due to intentional changes at district/building/grade

levels, and NOT due to:o Increased drop-outs at the high schoolo Decreases in student attendanceo Decreases in student enrollments


How are families and communities engaged with district and school in developing discipline, safety and school climate policies and plans? Are systems in place at the building and district levels to document the physical, social, emotional health or school climate stressors

impacting truancy? Discipline? Absenteeism? Drop out? Is this data used throughout the improvement process to develop strategies and action to keep students in the classroom, engaged, focused and achieving?

EXPULSIONS


For each student group in the District, determine the three-year trend data for expulsion (EXPL)/out-of-school suspension (OSS) by Grade / Group / Building level.


When analyzing the SPP data for SWD, what is the level of concern around the following data:o Rates of suspension/expulsiono How does the rate of suspension/expulsion for SWD compare to that of students without disabilities?

What systematic and targeted practices are in place to address the behavioral needs of SWD?o Describe efforts by the district to address behaviors of students with disabilities that impede learning, and efforts to

reduce the reoccurrence of student behavior that has resulted in the student removal from the classroom.

ATTENDANCE Has the District evaluated the impact of student attendance and taken action to minimize the negative effect of

student absences on student performance?

GRADUATION Level of Concern - Based on your data, identify the following areas of concern related to graduation.1. Is the trend moving in the right direction (i.e., is the number of graduates increasing)?2. Is the number of graduates acceptable for the District?


If there is an increase in the graduation rate, are data reviewed to ensure that the increase is due to more students graduating, rather than a decrease in the number of students available to graduate (i.e., more high school students going to community schools).


When analyzing the SPP data for SWD, what is the level of concern for graduation and dropout rates? How does the percent of SWD graduating from high school and percent of SWD dropping out of high school compare to percent of

students without disabilities?

DROPOUTS Level of Concern - Based on your data, identify the following areas of concern related to dropouts.1. Is the trend moving in the right direction (i.e., is the number of dropouts increasing)?2. Is the number of dropouts acceptable for the District?


When analyzing the SPP data for SWD, what is the level of concern for graduation and dropout rates? How does the percent of SWD graduating from high school and percent of SWD dropping out of high school compare to percent of

students without disabilities?

Other Questions Student Mobility – Has the district leadership team (DLT) evaluated the impact of student mobility on student performance? To

what degree does the DLT? Examine mobility data to identify students at risk and set priorities for improvement? Use data to monitor student mobility (number and percentage of students who moved within the district during the school year)

by building, grade/grade level/gender, subgroup? Compare the performance of mobile and non-mobile students? Identify the buildings by level that were affected by district-initiated moves (e.g., building closures, building openings,

population reassignments)? Compare the performance of students from stable v. less-stable buildings?


Multiple Risk Factors – Based on your DLT's review of your district's risk profile data, indicate your level of concern with regard to the impact of multiple risk factors on student performance.

Teacher and Student Perception – Has the DLT considered the impact of multiple risk factors on student performance? To what degree does the DLT? Conduct periodic surveys of teacher and student perception? Compare the teacher perception data to the student perception data to identify similarities and differences? Compare the teacher perception data to student performance? Compare student perception data to student performance?


Risk Profiles are available from the ODE website (for more information go to: [email protected])

LEVEL III: ENGAGEMENT

PARENT PARTICIPATION AND PERCEPTION Has the district leadership team (DLT) evaluated the level of parent participation and its impact on student performance? Using the best available data from records of participation, parent surveys, parent focus groups, etc., determine the district's effectiveness

in: Engaging parents in activities designed to share information of their child's academic progress and performance (e.g., parent-

teacher conferences, automated progress reports/updates, automated home notices/hotline)? Ensuring all parents - considering gender and disaggregated groups - generally believe that the district focuses on positive

outcomes for all students? Ensuring all parents - considering gender and disaggregated groups - generally believe that the district provides a safe

environment for learning? Ensuring all parents - considering gender and disaggregated groups - generally believe that the district demonstrates

sensitivity to the needs of children with academic or behavioral difficulty? Ensuring all parents - considering gender and disaggregated groups - generally believe that the district provides a challenging

curriculum to help students meet their maximum potential?


Do parents believe the district/school curriculum is challenging and is adequately preparing their child for post-secondary opportunities?


Do district and building improvement plans have family engagement strategies and actions steps that: Provide a welcoming school climate and celebrates the culture and diversity of their families Enables families to check on their child’s progress, access homework assignments and homework help on a regular and consistent

basis Engage families in activities designed to share information of their child's physical, social, emotional wellness (e.g., parent-teacher

conferences, health prevention programs/ health awareness campaigns, school linked health interventions and service opportunities)?

Provides families multiple opportunities and a variety of ways to volunteer that matches their families skills and interests. Help families access community resources to support their child’s academic, social, emotional and health needs? Helo families access community resources to support their families basic needs? Engage families in district and building leadership policy making and improvement planning?

Are family engagement strategies and activities developed collaboratively with Title I, 21st Century, IDEA, Gear Up and other program staff responsible for family engagement activities. Are family engagement strategies and action steps identified to support instructional goals?

Are family engagement strategies and action steps identified in CCIP related to school climate and student physical, social and emotional health strategies?

COMMUNICATION Has the district leadership team (DLT) fostered effective communication with parents for the purpose of improving student

performance?


Using the best available data from records of participation, parent surveys, parent focus groups, etc., determine the district's effectiveness in: Informing families' knowledge and skills in supporting their children's learning at home and their child's academic progress at

school? Building families' knowledge and skills in supporting their children's learning at home and their child's academic progress at

school? Implementing a system of two-way communication for families, using language that is clear and that fosters family

participation? Communication is in the primary language of the parent? Provide for a range of meaningful family activities that reflect the specific needs and characteristics of their student's families? Providing ongoing professional development for administrators, teachers, and other staff on how to build strong partnerships

with families?


Determine the district's effectiveness in: Informing families’ knowledge about the connections between good health and academic achievement? Building families' knowledge and skills to support their children's physical, social and emotional health at home in an effort to support

academic progress at school? Communicating to families sent in multiple formats, timely and consistent?

Does the district provide multiple and variety of opportunities for families to give input on policies, programs and education services. Does the district have a communication policy and procedures that ensure families have quick access to their child’s teacher or

administrator? How does the district ensure all teachers are following the policy and procedures?

STUDENT PARTICIPATION & PERCEPTION Has the district leadership team (DLT) evaluated the level of student participation and support and its impact on student

performance? Using the best available data from records of participation, parent surveys, parent focus groups, etc., determine the district's

effectiveness in: Ensuring all students - considering gender and disaggregated groups - generally believe that the district focuses on positive

outcomes for all students? Ensuring all students - considering gender and disaggregated groups - generally believe that the district provides a safe

environment for learning? Ensuring all students - considering gender and disaggregated groups - generally believe that the district demonstrates

sensitivity to their needs when they experience academic or behavioral difficulty? Ensuring all students - considering gender and disaggregated groups - generally believe that the district provides challenging

curriculum to help them meet their maximum potential?


Do students generally believe the daily classroom work is challenging and relevant? Do students generally believe that homework is value added and extends their learning? Do parents believe the district/school curriculum is challenging and is adequately preparing their child for post-secondary opportunities? Is the first language and culture of children valued? In what ways?


Have you examined parent/family, community and student perception data to identify differences based on subgroup populations?

COMMUNITY INVOLVEMENT & SUPPORT Has the district leadership team (DLT) evaluated the level of community participation and support and its impact on student

performance? Using the best available data from records of participation, parent surveys, parent focus groups, etc., determine the district's

effectiveness in: Engaging community agencies to actively work with the district in support of improved student outcomes? Engage pre-schools to actively participate with the district in professional development, provision of transition services, and

curricular planning? Informing the community on student performance in the primary language(s) of the community?


Working with parent organizations (e.g., PTA/PTO), community organizations, and local businesses to focus on the academic success of students?

Involving, as appropriate, community partners in policy development and the school planning process? Welcoming community partners into the district and schools? Working with community partners to maximize the use of available resources in improving student performance?


Describe community efforts to connect students with community supports such as: community mental health, post-secondary transition, interpreter services, Help Me Grow

How does the district communicate to the community the positives, challenges and needs? Is this communication timely, in multiple formats and consistent?

How does the district provide support and contribute to the community (e.g. opening up their building for community activities, community service, adult learning opportunities)?

Engage in community planning and align community resources to students and their families needs (e.g. participate in Family and Children First Councils, United Way, Early Childhood planning teams?

Does the DLT Work with parent organizations (e.g., PTA/PTO), community organizations, and local businesses to focus on school climate and student/staff health (health is defined as physical, social and emotional wellness)?

LEVEL IV: RESOURCE MANAGEMENT

TEACHER & PD ALIGNMENT Has the DLT ensured that staffing assignments and professional development are aligned to needs? To what degree does the

DLT: Use aggressive recruiting strategies to get the highest quality staff possible? Support and enhance classroom teachers' performance with high quality, in-house instructional guidance and assistance? Assign and group faculty flexibly to meet student needs in cost-effective ways? Use teachers in making key decisions? Ensure district PD meets Ohio standards for Professional Development (PDP)?

FOCUS ON STUDENT SUCCESS Has the DLT ensured a district-wide focus on student success in core content areas? To what degree does the DLT:

Consider only research-based initiatives and evaluate them thoroughly before implementation to ensure they meet instructional and content needs of all students?

Give priority to instructional time students spend learning mathematics and language arts? Extend the contract year to increase teacher development and planning time? Focus district and building meetings on learning and instruction Keep non-instructional activities (e.g., walk-a-thons, assemblies, non-academic field trips, athletic events during school) to an

absolute minimum?

SPENDING ALIGNMENT

Has the DLT ensured that spending is aligned with the district's focused plan and strategic priorities, clearly reflecting immediate and future needs? To what degree does the DLT:

Use district performance assessments to generate data for guiding initial and follow-up decisions related to performance and alignment?

Include the district treasurer when it meets? Establish budgeting and related processes to ensure effective spending? Encourage cost avoidance through multiple purchasing options and reduction of inefficiencies? Make intentional budgeting decisions to directly support the district's plan? Using data, make intentional budget decisions to continue support for successful strategies and remove support for

unsuccessful strategies? Pool funds from multiple sources to support academic programming, especially for at risk populations and subgroups? Take appropriate and immediate action to resolve financial determinations brought to its attention by the Auditor of the State of

Ohio Department of Education?



Use district health assessments (school climate and student health) to generate data for guiding initial and follow-up decisions related to performance and resource alignment?

Using data, make intentional budget decisions to support student/staff physical, social and emotional health initiatives (from health promotion and prevention to intervention and services) that promote and support student achievement.

How does the district collaborate with community organizations and businesses to access resources to support students’ academic, physical, social and emotional health needs?


Resource 11: Process to Create a Relation DiagramRelation diagrams are used to study the relationships between and among causes. The interrelationship diagram is used when a complex issue is studied with multiple causes. Analysis of the diagram is reviewed to assure the causes are the most beneficial to the aim of the process. The process to create the diagram is:

1. Each cause should be written on large index cards, large post-it notes, or printed out in large type on card stock. Place the causes in a circle pattern on a large sheet of paper.

2. Study the relationship between each of the causes and ask the following questions:a) Does a relationship exist? If so, draw a line connecting the two causes. Move to the next pair of causes

and ask the question. Continue clockwise around the circle until all of the causes have been studied. Make sure this question is asked for every possible pairing of causes.

b) Between causes where a relationship exists, which is the cause and which is the effect? Decide which cause affects the other the most. Put an arrow on the end of the line that points to the affected cause. Move to the next pair of causes and ask the same questions. Continue clockwise around the circle until all of the causes have been studied.

3. Count the number of arrows stemming from each cause. 4. Rank the causes, from highest to lowest according to the number of arrows coming from each cause.5. Study the results and compare them. This becomes the basis for identifying the expectations and conditions

priority area. a) Identify the items with most incoming and outgoing arrows. b) Review other items with lesser arrows but may be viewed as critical based on knowledge and

experience of the DLT.c) By consensus, confirm the most critical items on the diagram.d) Once the group has come to consensus on critical items double box the items.

At the conclusion of examining the relationships, the DLT should have a clear idea about the expectations and conditions priority area, based on the total number of arrows and discussion among the members. If there is more than one priority area that is identified, the DLT will need to prioritize and select the one area that will be addressed in the district plan. It is possible that the problem without the most arrows is selected. This may be due to a variety of factors such as the area chosen is a pressing community problem; the area will have a greater systemic impact on student performance (higher leverage) or by addressing the area first there will be a positive impact on the other area. Causes that do not relate to the area selected will need to be temporarily set aside for Stage 2 consideration (strategies and actions).

Resource 11: Process to Create a Relation Diagram September 2010 OIP Resources

Resource 12: Grain Size Visual: Goal, Strategy, Action, Task and Indicator Definitions September 2010 OIP Resources

Resource 12: Grain Size Visual: Goal, Strategy, Action, Task and Indicator Definitions

Multi-year Goal Indicator – Goal indicators are the gauges by which a goal is determined to be met.

Adult Implementation and Student Performance Indicators – Strategy indicators are the gauges by which a strategy is determined to be met in terms of student performance and adult practices.

A baseline measure establishes a starting point.Progress measures assess short-term measures of change.

Annual Goal Target – Goal targets are the gauges by which an annual goal is determined to be met or to be achieved.

Resource 13: Affirm Cause and Effect for Strategy Development

Before meeting with the DLT to affirm causes and effect among the data as a basis for developing strategies, the facilitator will need to meet with the DLT chair/cochairs to build an agenda, prepare packets and make meeting arrangements. Packets should include the list of district priority problems and the board approved goals. The meeting space will need to allow for significant wall space. Someone also will need to enlarge and make two to three copies of the district priority problems (one set per goal).

The superintendent/chair/co-chairs may want to expand to a larger audience of district- and building-level leadership representatives at this time. It’s best to gather a team of people who have the right skills, knowledge and experience of the problem to collectively identify all the reasons why the problem may be occurring. Another reason for expanding the DLT at this time is that since the work forms the basis for developing strategies, and all district staff and schools will be expected to use these in developing their plans, then their understanding and involvement will help to build ownership and commitment to the final plan.

This part of the process is not simple and not easy and will require strong facilitation. The Meeting Management Checklist (Resource 6) can be used as an aide in preparing for the meeting. The agenda should be sent at least a week before the meeting. Request a copy be emailed to you. A sample agenda for this DLT meeting follows. Within each agenda item, time should be built in for questions.

SAMPLE AGENDADistrict Leadership Team Meeting

Date: Time: Location: Attendees:

8:30 – 8:45 a.m. Meeting Purpose, Ground Rules Review and Assignments

Superintendent/Chair/Co-chairs

8:45 – 9:00 a.m. Benefits of Using a Graphic Organizer to Determine Cause and Effect OIP Facilitators

9:00 – 10:15 a.m. Create a Tree Diagram for Each Goal DLT Members Goal Workgroups

10:15 – 10:45 a.m. Review Tree Diagrams DLT Members

10:45 – 11:00 a.m. Next Steps Superintendent

Summary of Discussion/Decisions

Resource 13: Affirm Cause and Effect for Strategy Development September 2010 OIP Resources

During this meeting, the facilitator will need to introduce the concept of cause and effect. Without an understanding of the factors that contribute to each of the goals, it is difficult to identify strategies to address them. A cause-and-effect analysis generates and sorts hypotheses about the possible causes of problems by asking the DLT to correlate high priority problems identified in Stage 1 to the goals generated in Stage 2. This analysis forms the basis for identifying the strategies that the district will use to respond to the goals/problems. The cause and effect process should take no more than a few hours. Following are talking points that follow the above agenda.

AGENDA TOPIC FACILITATOR TALKING POINTS HANDOUTS

Meeting Purpose, Ground Rules Review and Assignments

Purpose of the meeting is to use the Stage 1 district priority problems to identify primary and secondary causes of the problems (which become the goals generated in Stage 2) as the basis for developing strategies (and potentially actions).

Review the agreed upon ground rules that were identified at the first DLT meeting.

Determine who will fulfill assignments for this meeting, e.g. timekeeper, process observer and information seeker.

District Priority Needs/Problems (generated in Stage 1)

Board Approved Goals (generated in Stage 2)

Benefits of Using a Graphic Organizer to Determine Cause and Effect

Explain the benefits of using a graphic organizer:

o Illustrates major branches reflecting categories of causes to stimulate and broaden thinking about potential or real causes and facilitates further examination of individual causes

o Because all ideas can find a place on the diagram, a cause-and-effect analysis helps to generate consensus about causes.

o It can help to focus attention on where a problem is occurring and to allow for constructive use of data.

o Data lends credibility to possible hypotheses. The absence of data encourages districts to revert back to opinion as the primary rationale.

Create a Tree Diagram for Each Goal

The tree diagram provides a logical structure to help move from large, broad issues to small, specific items. It can help arrange potential causes and effects and be used to document why a problem exists.

Ask the DLT members to self-select into work teams for each goal. Ask that each workgroup have an equal number of members. Next, give each group the following tasks for their assigned goal:

1. Write the goal in the form of a question (problem statement) on the extreme left side of a wide piece of paper, or on a landscape page of a Tree Diagram.

Note: Framing the goal as a “why” question is important because each cause should answer the question.

2. Identify the categories/areas from the list of district

Blank Tree Diagram

Sample Tree Diagram



priority problems, e.g., curriculum, instruction, climate and leadership and subdivide or branch off the first-level tree item.

Note: There is no one right way to do this. Six boxes may be required or only four. Make the tree fit the problem and categories that emerged from the data.

3. Using the district priority problems, sort the primary causes onto the tree. Moving from left to right in the tree, the items get more and more specific. Evaluate and review each major subheading or path to ensure that each level has a direct cause-and-effect relationship with the one before. Analyze and confirm that fulfilling all the levels and areas will ensure the problem is solved.

Note: Using large sticky notes with each priority problem written on it or cutting the priority problems into strips so they can be moved is the best technique to use for this. There will be significant movement of causes to the categories and the members physically moving the causes forces discussion about the relationship among causes. If a cause relates to more than one category, it can be duplicated and placed under both categories.

4. Build a detailed picture of why the problem is happening as a team. This may require identification of secondary causes. When possible data should be identified to support these causes. This may require revisiting the data from Stage 1. It may be that additional monitoring and data collection is required to establish a solid connection between the causes and the effects of each problem, or the team may have to rely on common sense and perception. When additional data are needed, they should be identified and kept as information for when the actions are developed.

Note: Remember that people are natural problem solvers and may have spent years doing creative work to get over routine problems. Indeed, these practices may be so ingrained they’re considered part of the job. It is the job of the facilitator and DLT chair(s) to flush this out and solve the cause of the problem by asking “why, why, why” for each reason (see More On Facilitation below), and what data support team members’ hypothesis. The question “why” should be asked five times for each reason (i.e. Five Whys).

5. If the team identifies more than four categories, they may wish to prioritize them. This information may be helpful as the DLT moves into developing the few (no more than four) focused strategies.



Review Tree Diagrams

Have each team present their tree diagram to the whole group, asking for feedback and comment.

Next Steps

The chair/co-chairs will need to share how these causes will form the basis for the strategies that will be developed. Creating a diagram leads to a clearer understanding of the goal (problem) and what’s causing it so that investment of resources can be directed toward the right strategy/solution. Jumping to a solution without first looking at all possible causes can be a costly mistake. Remember, a problem well stated is a problem half solved.

Note: If time allows, the group can proceed with developing the strategies at this time.

The chair/co-chairs should summarize the next steps to include:

o Date of next meetingo Draft agenda for next meeting

After the meeting, the tree diagrams should be transferred onto an EXCEL spreadsheet or other software, e.g., Inspiration, so that the causes and effect results can be transmitted to the DLT.

More on FacilitationUse the “Five Whys” method to help drill down and identify the cause of a problem. The question “why” is asked five (or more) times. The “Five Whys” cause people to use higher order thinking skills, cut through layers of bureaucracy to find the true meaning and cause people to challenge their current situation or problem. For example,

1. Why are our DLT meetings unproductive?

Answer: We spend too much time talking and sharing stories about things that happen in the school.

2. Why do we spend too much time talking about personal things and sharing stories about things that happen in the schools?

Answer: We don’t have a focus for our meetings.

3. Why don’t we have a focus for our meetings?

Answer: We aren’t organized with an agenda.

4. Why aren’t we organized with an agenda?

Answer: We don’t have a process for developing an agenda.

5. Why don’t we have a process for developing an agenda?

Answer: We haven’t taken the time to look at our data to assess our needs


Goal 1 The number of students scoring proficient or higher at each grade level as assessed by the reading OAT/OGT will increase 20% by 2010.Strategies Developed Based on Tree Template

1. A Implement a district-wide system to effectively and efficiently monitor instructional practices for consistent implementation of ELA instruction.2. A Implement a standards-based curriculum and evidence-based instructional practices in ELA.

Curriculum

PD is not aligned to student achievement goals and less than 15% of PD is focused on reading.

District does not have a rigorous Pk-12 curriculum.

Curriculum is not aligned to standards.

Teachers use a variety of ELA materials to support curriculum resulting in inconsistency.

Principals do not regularly observe classrooms for curriculum implementation.

Instruction

PD is not aligned to achievement goals and less than 15% of PD is focused on reading.

There is a lack of coordination of design, delivery, evaluation and monitoring of PD in reading.

Most (85%) instruction is whole group.

Differentiated instruction is not applied consistently.

PD is inconsistently offered and not supported by follow-up and embedded activities.

No data is available to show the relationship between PD and change in teacher practices.

Use of effective instruction is not consistent.

Limited use of higher order questioning and advanced organizers.Less than 25% of teachers use common planning time 1X/wk. Purpose of meetings is unknown.

Data are not used efficiently to monitor program implementation and drive ELA instruction for all students.


Why are reading scores

declining?

Assessment

The district does not monitor the use of short-term cycle assessment in grades 4-12

Not all grades have a short-cycle assessment available to them to measure reading progress.

65% of teachers report that they do not know how to administer and interpret reading short-cycle or quarterly assessments in order to improve instruction.

65% of teachers report that assessment results are not immediately accessible to them in order to guide instruction.

Short-cycle assessment results are not used to engage students in self-reflection or self-assessment.


Resource 14: Focused Plan Criteria ChecklistComment

SMART GoalsSpecific

1. Are action word(s) used to describe what the district is going to do?2. Does the goal describe what the district wants to accomplish?3. Is the goal clear about what the district intends to do for all students and designated student populations?4. Does the goal specify when the district is expecting to see change?

Measurable5. Does the goal identify the annual target indicator of what the district will see annually? 6. Does the goal identify the multi-year indicator of what the district will see when it reaches the goal?7. Does the goal ensure there are data that will be collected to demonstrate appreciable change (in quantity or

quality) that can be calculated?

Attainable and Achievable8. Does the goal stretch the district to improve yet is possible to attain?9. Does the goal target the population(s) that data show is in greatest need?

Realistic and Relevant10. Is implementation of the goal possible within the capabilities of the district?11. Is the goal consistent with the mission?12. Is the bar set high enough for significant achievement?13. Is the goal the most important and significant aspiration of the district to improve student learning?14. Does the goal contradict other goals?

Timely15. Does the goal have an end point?16. Is the time frame for accomplishment of the goal realistic?

Strategies and ActionsIndividual Strategy Review

17. Is the strategy focused? Does it address the core of the goal?18. Is the strategy achievable, feasible, reasonable and practical (i.e., time, skill, knowledge, and culture can

support them)?19. Is the strategy the right grain size (i.e., not so broad that it is a goal but not so narrow that it is an action or task)?20. Does the strategy have a sufficient research base with identified sources?21. Does the strategy respond to the prioritized data needs and cause and effect analysis?22. Will the strategy be understood by stakeholders (i.e., clear, jargon-free language, able to stand on its own

without additional explanation)?23. If the strategy is implemented with fidelity, is it likely to achieve the goal?

Individual Action Review24. Is the action/intervention backed by evidence of effectiveness?25. Does each action identify the monitoring evidence/data sources that will be used to document implementation?26. Does each action identify who is responsible for implementation, the timeline for implementation of the actions

and the resources needed to execute the actions?27. Given the goal of improving student performance, do the benefits of each action outweigh the costs, i.e.,

Resource 14: Focused Plan Criteria Checklist September 2010 OIP Resources

Comment

amount of time, number of people, money, necessary materials, supplies, technology?

Total Strategy and Action Review28. Are there a reasonable number of strategies for the goal (two to four) and actions for the strategy (no more than

10)?29. If applicable, will the strategies and actions respond to the needs of the student populations for which the goal

is directed? Are they culturally relevant and responsive?30. Will the strategies improve education in the district as a whole? Do the actions reach a critical mass of targeted

school staff, students and/or facilities?31. Can the strategies be applied in multiple settings (e.g., elementary schools, secondary schools or district

departments)?32. Do the strategies and actions complement, not contradict, one another?33. Will the set of actions allow the district to accomplish its goals and strategies and enable the district to meet the

plan indicators?34. Are the actions, taken as a whole, coherent and aligned?

Indicators35. Are data available for the indicators?36. Are the costs (e.g., time, money and people) of getting the data acceptable?37. Are the data reliable and accurate?38. Are the data valid and based on reality? 39. Can the data be collected on a regular basis?40. Are the data understandable and meaningful?41. Can the data be easily communicated?42. Are these the right measures, i.e., practical, relevant and realistic?43. Will the indicators measure the effective implementation of the strategy?44. If the numbers are small, are they of sufficient scope?45. Are the indicators designed to address the needs of the stakeholders?46. Are the data accessible to all who need or desire it?47. Is there clearly assigned responsibility for collecting, organizing, analyzing and interpreting the data?48. Will the data provide an early warning of any potential problems?49. Is the indicator capable of being monitored to provide statistically verifiable and reproducible data that

show changes over time?50. Are the indicators amenable to combination with other indicators to produce more general information

about district changes?51. Will the indicators and data contribute to the fulfillment of reporting obligations under federal and state

requirements?

Resource 14: Focused Plan Criteria Checklist September 2010 OIP Resources

Resource 15: Stakeholder Feedback Survey on Goals, Strategies and IndicatorsYou and/or your organization have been identified as a critical stakeholder (parents, community and business representatives) for the District. We are in the process of revising our continuous improvement plan (CIP) for the school year. Your input on the district’s CIP would be appreciated. Please take a few minutes to provide feedback to us about the attached set of goals, strategies and indicators. Send your completed survey to ________________________________ by ___________________ so your comments and suggestions can be incorporated into the district plan. If you have any questions, please contact . Thank you!

Are the goals appropriate for our district?

Ye

s

No Comments

Are the strategies clear?

Are the measures/indicators appropriate?

Is the layout and design (format) of the document easy to read and follow?

Overall: Are the goals, strategies and indicators clear, concise, attainable, realistic, etc.

General Comments

Resource 15: Stakeholder Feedback September 2010 OIP Resources

Note: Adapted from materials provided by the Tecumseh Local School District


Interview on Goals, Strategies and IndicatorsAnother method to gather stakeholder input on the goals, strategies and indicators is for each member of the DLT to interview five people who represent their constituent group. For example, the parent representative would interview five parents, the superintendent might interview five local board members or five other top-level administrative staff, and the teacher leader would interview five other teacher leaders. When leadership team members select their five interviewees, they should keep the following criteria in mind:

1. Are the five persons selected by individual DLT members representative of the constituent group? For example, did the building administrative representative select individuals from different schools and from the various levels of administration (e.g., principals, assistant principals)?

2. Do the five persons selected include individuals who are opinion leaders? Are persons included those who set the tone for how others will respond? They may be formal or informal leaders and persons who have conferred or inferred position. These are individuals who either through their wisdom, knowledge or strength of personality place themselves in an influential position.

3. Do the five persons selected include individuals who are experts in the goal area?4. Do the five persons selected include individuals who may have diverse opinions or perspectives?

Looking at the goal through different lenses (political, economical, social, etc.) will these individuals offer insights that may have been missed?

Option A: Questions for the interviews should include qualitative and quantitative questions. For example, a qualitative question might be: “Given the mission (why the district exists), how will the draft goals, strategies and indicators accomplish the mission?” This might be followed up with a quantitative question such as, “On a scale of five to one with five being high, how well will the goals, strategies and indicators if implemented with fidelity, result in accomplishment of the mission?”

Option B: Another option for the interview process would be to show the person being interviewed a one- to two-page sheet with the goals, draft strategies and indicators, and proceed to follow a structured protocol, like the one below.

STATEMENT-AGREEMENT INTERVIEWEE COMMENTS

The goals target the greatest problems faced by the district.

Agree Disagree

The goals reflect my concerns.

Agree Disagree

The strategies, if fully, implemented, have a great likelihood to achieve the goal.

Agree Disagree

The strategies are reasonable and practical.

Agree Disagree

These are the right indicators that will tell me if the goals and strategies are successful.

Agree Disagree

I understand the goals, strategies and indicators.

Agree Disagree


Resource 16: District IMM (Implementation Management/Monitoring) Tool –TEMPLATE

SMART GOALS

Goal 1: Student Performance Content Area Goal 2: Student Performance Content Area __________ Goal 3: Expectations and Conditions ____________ _ Goal 4: Operational/Cross-content____________

GOAL 1:

STRATEGIES, INDICATORS AND PROGRESS MEASURES

STRATEGY 1A:

BASELINE MEASURE PROGRESS MEASURE PROGRESS MEASURE PROGRESS MEASURE PROGRESS MEASURE

ADULT IMPLEMENTATION INDICATOR Measure

Description

(DATE)PROJECTE

D

ACTUAL RESULTS

(DATE)PROJECTE

D

ACTUAL RESULTS

(DATE)PROJECTE

D

ACTUAL RESULTS

(DATE)PROJECTE

D

ACTUAL RESULTS

STUDENT PERFORMANCE INDICATOR

Implementation Details

ACTION STEPSMonitoringEvidence/DataSources

Person(s)Responsible/Group(s)

Implementation Timeline Resources, e.g., Budget, Material, Supplies, Technology

June-Aug.

Sept.-Nov.

Dec.-Feb.

March-May Future

1.a.1.

1.a.2.

1.a.3.

1.a.4.

1.a.5.

1.a.6.

1.a.7.

1.a.8.

1.a.9

1.a.10

Resource 16: District IMM (Implementation Management/Monitoring) Tool –TEMPLATE September 2010 OIP Resources

SCHOOL IMM (Implementation Management/Monitoring) Tool – TEMPLATE

SMART GOALS

Goal 1: Student Performance Content Area Goal 2: Student Performance Content Area __________ Goal 3: Expectations and Conditions ____________ _ Goal 4: Operational/Cross-content____________

GOAL 1:

STRATEGIES, INDICATORS AND PROGRESS MEASURES

STRATEGY 1A:

BASELINE MEASURE

PROGRESS MEASURE PROGRESS MEASURE PROGRESS MEASURE PROGRESS MEASURE

(Date) (DATE)PROJECTED DATA

ACTUAL DATA (DATE)

PROJECTED DATA

ACTUAL DATA (DATE)

PROJECTED DATA

ACTUAL DATA

(DATE)PROJECTED

DATA

ACTUAL DATA

ADULT IMPLEMENTATION INDICATOR District School DIST. SCH. DIST. SCH. DIST. SCH. DIST. SCH.

STUDENT PERFORMANCE INDICATOR

Implementation Details

ACTION STEPSMonitoring Evidence/DataSources

Person(s) Responsible/ Group(s)

Implementation TimelineResources, e.g., Budget, Material, Supplies, TechnologyAu

g.Se

pt.

Oct

.No

v.De

c.Ja

n.Fe

b.M

arch

April

May

June

Ju

lyFu

ture

1.a.1.

1.a.2.

1.a.3.

1.a.4.

1.a.5.

1.a.6.

1.a.7.

1.a.8.

1.a.9.

1.a.10.

Resource 16: District IMM (Implementation Management/Monitoring) Tool –TEMPLATE September 2010 OIP Resources

Resource 17: Task Implementation TemplatePurpose: This template will help define the tasks that district and building level collaborative teams will undertake to implement actions. Directions: Complete one form for each action. The district may wish to provide parameters which DLTs/BLTs will need to follow in creating its tasks, e.g., amount of in-school time that can be devoted to collaborative team time or professional development and budget limits, such as amount of pay for substitutes, hourly rates for out-of-school time, consultant daily fees, etc.District: School:

DISTRICT GOAL: DISTRICT STRATEGY: ADULT IMPLEMENTATION INDICATOR:

STUDENT PERFORMANCE INDICATOR:

ACTION The WHAT: ACTION MANAGER:

What is your plan to implement this action?

TASKS: List the tasks in sequential order. The HOW?We will …

Timeline

The WHEN?

Person(s) Responsible/

Involved

BY WHOM?

Budget

What will it COST to be successful? Districts may wish to add a column that identifies fund source.

Start Date Completion Date

Salaries/ Stipends

Fringe Benefits

Materials/Supplies

Other Services

Capital Outlay TOTAL

Resource 17: Task Implementation Template September 2010 OIP Resources

Resource 18: Checklist of High-Quality and Evidence-Based PDWhen planning and implementing PD for continuous improvement, the following evidence-based characteristics will ensure that PD is high quality, aligned with your improvement plan and state standards for PD, and likely to improve student achievement and teacher practice.

Use this list of characteristics as a checklist to guide your choice and use of PD services. Consider using the implementation/monitoring section to monitor the PD implementation.

PD service: ___________________________________________________ Date: ___________

Person(s) assessing service: ________________________________________________________________________

Characteristics of High-Quality and Evidence-Based PD for Continuous Improvement

Planning Implementation/Monitoring Notes/Suggested Changes

Yes Somewhat No Yes Some

what No

1. Aligns with district goals, strategies and district/school actions. [1]*

2. Focuses on the content students need to know in relation to goals, strategies and indicators (e.g., Ohio Academic Standards, key curriculum concepts, assessments). [2, 5, 6]*

3. Improves teacher content knowledge in areas identified in the plan. [2,5, 6]*

4. Advances teacher use of effective instructional strategies as delineated in the plan strategies, actions and indicators. [2, 5, 6]*

5. Provides sufficient opportunities and support for building efficacy and mastery of new content knowledge and instructional strategies contained in the plan goals, strategies and indicators. [1, 3, 4, 6]*

6. Involves active learning by participants (e.g., hands-on learning and inquiry-based learning). [4]*

7. Involves participants working in collaborative groups. [1, 3]*

8. Brings together educators who are already associated in some manner (i.e., collaborative teams or similar grades, subjects, issues or leadership roles). [3]*

Resource 18: Checklist of High-Quality and Evidence-Based PD September 2010 OIP Resources

Characteristics of High-Quality and Evidence-Based PD for Continuous Improvement

Planning Implementation/Monitoring Notes/Suggested Changes

Yes Somewhat No Yes Some

what No

9. Customizes to match the participants’ needs. [2, 4]*

10. Embedded within the school day or school year. [1]*

11. Long-term on-going contact and initial and follow-up varied opportunities for learning. [1, 4]*

12. Monitors and evaluates for effectiveness as described in indicators (i.e., Five Levels of PD Evaluation). [2, 5]*

13. Multiple sources of data are analyzed and documented in order to guide present and future decision making. [2]*

14. Actively supported by leadership (i.e., DLT/BLT). [1]*

15. Documented basis in evidence - scientific research or effective practice. [2, 6]*

16. Serves as a model of high standards for staff development (i.e., Ohio Standards for PD, National Staff Development Council Standards). [1-6]*

*Aligns with designated standard(s) from the Ohio Standards for PD which are available online at esb.ode.state.oh.us/PDF/Standards_ProfDev_sept07.pdf


References

1. Bolam, R., McMahon, A., Stoll, L., Thomas, S., & Wallace, M. (2005). Creating and sustaining professional learning communities. Research Report Number 637. General Teaching Council for England. London, England: Department for Education and Skills.

2. Darling-Hammond, L., Wei, R.C., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Oxford, OH: National Staff Development Council and the School Redesign Network at Stanford University.

3. Garet, M. S., Porter, A. C., Desimone, L., Birman, B., & Yoon, K. (2001). What makes professional development effective? American Educational Research Journal, 38(4), 915–945. Retrieved March 13, 2008, from www.aztla.asu.edu/ProfDev1.pdf

4. Guskey, T. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin Press.

5. Killion, J. (2002). Assessing impact: Evaluating staff development. Oxford, OH: National Staff Development Council.

6. Mid-continent Research for Education and Learning. (2000). Principles in action: Stories of award-winning professional development [video]. Aurora, CO: Author.

7. National Staff Development Council. (2001). Standards for staff development. Oxford, OH: Author. Retrieved March 13, 2008, from http://www.nsdc.org/standards/index.cfm

8. Parsad, B., Lewis, L., & Farris, E. (2001). Teacher preparation and professional development: 2000 (NCES Publication No. 2001-088). Washington, DC: National Center for Education Statistics. Retrieved March 13, 2008, from http://nces.ed.gov/pubs2001/2001088.pdf.

9. Porter, A., Garet, M., Desimone, L., Yoon, K., & Birman, B. (2000). Does professional development change teaching practice? Results from a three-year study. Washington, DC: U.S. Department of Education. Retrieved March 13, 2008, from http://www.ed.gov/rschstat/eval/teaching/epdp/report.pdf

10. Steiner, L. (2004). Designing effective professional development experiences: What do we know? Naperville, IL: Learning Point Associates.

11. Vescio, V., Ross, D, Adams, A. (2008). A review of research on the impact of professional learning communities on teacher practice and student learning. Teaching and Teacher Education 24 (1), 80-91.

12. Yoon, K. S., Duncan T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. L. (2007). Reviewing the evidence on how teacher professional development affects student achievement. (Issues & Answers Report, REL 2007–No. 033). Washington, DC: National Center for Education Evaluation and Regional Assistance, Regional Education Laboratory Southwest. Retrieved March 13, 2008, from http://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL_2007033.pdf


Five Levels of PD Evaluation Evaluation Level What Questions Are Addressed? How Will Information Be

Gathered?What is Measured or Assessed?

How Will Information Be Used?

1. Participants’ Reactions

Did they like it? Was their time well spent? Did the material make sense? Will it be useful? Were the refreshments fresh and tasty? Was the room the right temperature?

Questionnaires administered at the end of the session

Initial satisfaction with the experience

To improve program design and delivery

2. Participants’ Learning

Did participants acquire the intended knowledge and skills?

Paper-and-pencil instruments Simulations Demonstrations Participant reflections (oral and/or written) Participant portfolios

New knowledge and skills of participants

To improve program content, format, and organization

3. Organization Support and Change

Was implementation advocated, facilitated and supported? Was the support public and overt? Were the problems addressed quickly and efficiently? Were sufficient resources made available? Were successes recognized and shared? What was the impact on the organization? Did it affect the organization’s climate and procedures?

District and school records Minutes from follow-up meetings Questionnaires Structured interviews with participants and district or school administrators Participant portfolios

The organization’s advocacy, support, accommodation, facilitation and recognition

To document and improve organization support To inform future change efforts

4. Participants’ Use of New Knowledge and Skills

Did participants effectively apply the new knowledge and skills?

Questionnaires Structured interviews with participants and their supervisors Participant reflections (oral and/or written) Participant portfolios Direct observation

Degree and quality of implementation

To document and improve the implementation of program content

5. Student Learning Outcomes

What was the impact on students? Did it affect student performance or achievement? Did it influence students’ physical or emotional well-being? Are students more confident as learners? Is student attendance improving? Are student dropout rates decreasing?

Student records School records Questionnaires Structured interviews with students, parents, teachers and/or administrators Participant portfolios

Student learning outcomes: Cognitive (Performance and Achievement) Affective (Attitudes and Dispositions) Psychomotor (Skills and Behaviors)

To focus and improve all aspects of program design, implementation and follow-up To demonstrate the overall importance

Adapted by Learning Point Associates from a handout by Thomas R. Guskey, NCREL’s Annual Meeting, 2002.


Resource 19: PD Alignment TemplatePurpose: Define and align PD in the district and school plans.

Directions: For each level of the organization, answer each of the questions. Add or expand rows as necessary to include all district and school professional development contained in the plans. When finished, review the entire chart and respond to the summary questions. The DLT may wish to send the form to each building to complete and then consolidate the information on one form. Complete a separate form for each content area, i.e., reading, math.

PD Strategy/Actions (from district and school plans)

Strategy Indicator(s):

Organizational Level

Specific Content Based on Data

PD Outcomes Expectations for Application

Process Collaborative Structure

Timeframe Evaluation Resources

What is the specific content focus from the plan’s actions and/or tasks?

What are staff expected to know and be able to do, both short-term and long-term?

How is the PD linked to daily practice?

What is the set of connected and coherent tasks to accomplish the actions?

How will teams discuss and reflect on new learning and its application?

What flexible schedule/ calendar will be used to allow for professional learning in and out of the workday?

How will the PD use multiple evaluation measures and how well the results feed into the monitoring system? (Refer to Resource 18)

What resources are needed for this to occur, including internal and external expertise?

District

School A

School B

School C

School D

SUMMARYQUESTIONS

To what degree are the content and expected outlines aligned to achieve a common result? To what degree are the tasks and timelines logically sequenced to achieve the expected outcomes? To what degree will the experiences, when taken as a whole, improve student performance and change adult practice? Is there a cost-benefit to the overall plan - weighing the total expected costs against the total expected benefits? What is the relationship of the above PD to any other PD being offered at the district or building level?

Resource 19: PD Alignment Template September 2010 OIP Resources

Resource 20: Selecting HQPD Providers

The selection of the right internal and/or external provider is an important step in ensuring that the district and schools receives HQPD.

A critical review of providers requires objective analysis. For this reason, the following recommendations and procedures are offered as guidance. First, the standards for HQPD are listed. Next is a procedure for selecting an internal/external provider. And last are criteria for selecting a provider.

A. What are the standards for HQPD (Ohio Standards for PD)?

1. HQPD is a purposeful, structured and continuous process that occurs over time.

2. HQPD is informed by multiple sources of data.

3. HQPD is collaborative.

4. HQPD includes varied learning experiences that accommodate individual educators’ knowledge and skills.

5. HQPD is evaluated by its short- and long-term impact on professional practice and achievement of all students.

6. HQPD results in the acquisition, enhancement or refinement of skills and knowledge.

B. What process should be used to select a provider?

District and schools may begin the selection process by conducting an examination of the potential pool of providers. In addition, districts and schools will want to be clear about the PD outcomes because it makes sense to direct a search toward those providers that address particular knowledge and skill areas. Districts also will want to consider the need for differentiation among schools and/or grade levels.

Once a final pool of internal and/or external providers has been identified, send each provider the criteria and PD characteristics and ask them to respond with a proposal. The DLT/BLT would be involved in the review and selection of a person(s) or an organization. Realistically, a strategy manager or strategy sponsorship team may be responsible for the initial review and reduce the "possible" options to a reasonable number. The chart below should only be completed for the best possible options.

C. What criteria should be used to select a provider?

A converging body of evidence is available to guide the selection of providers, particularly as it relates to the need for providers to have strong content knowledge and instructional skills, to customize learning to the audience and to adhere to adult learning styles (Yoon, K. S., Duncan T., Lee. S. W.-Y., Scarloss, B., & Shapley, K. (2007). The next page describes the criteria for selecting a provider.

Resource 20: Selecting HQPD Providers September 2010 OIP Resources

A Guide to Selecting PD Providers:A Critical Elements Analysis

A key assumption is that district- and school-wide PD (a) is based on the data analysis in Stage 1 of the OIP that identifies the most critical needs, (b) focuses on strategies/actions identified in the district and schools plans, and (c) uses formative and summative data will be used to monitor and revise professional learning. The following critical elements analysis focuses on those skills and strategies essential for HQPD.

General Review Process

1. Scope of Review

To begin, identify the outcomes and grade levels that the PD is targeting. Often the outcomes will be across grade levels. If not, outcomes should be listed by grade level.

2. Type of Review

To gain an adequate pool of potential providers, the following steps are recommended:

a) Identify current or former staff who may be qualified and experienced,

b) Contact your SST and ESC for possible providers,

c) Contact relevant organizations that may be able to provide PD,

d) Survey other districts and professional associations to identify potential providers, and

e) Contact relevant department in institutions of higher education to identify potential providers.

Once a pool is identified, the individual or organization should be asked to submit a

proposal or scope of work based on the scoring criteria. The criteria along with the focus and outcomes should be provided to the potential provider in advance.

3. Documenting Evidence

On the review form there is space to document an assessment of the specific information that the potential provider is supposed to submit, e.g., written proposal of work including time and budget; resume or evidence of successful experiences; video tapes; etc.

4. Scoring Criteria

The criterion for scoring each element is listed below. When evaluating each element, place the respective circle that represents your rating in the qualifications rating box.


Use the following criteria for each critical element:

= Provider is a very strong candidate and exceeds criterion.

= Provider has the necessary background and meets the criterion.

= Provider does not satisfy criterion.


Criterion Provider Name Qualifications RatingEvidence

Summarize evidence of sufficient and insufficient instructional quality in the focus area of the plan

Criterion RatingFocus/Content Area from District and School Plans:

_________________

Evide

nce

of O

vera

ll Ef

fecti

vene

ss

Evide

nce

of

Effe

ctive

ness

in F

ocus

Ar

ea(s

)

Appr

opria

te C

rede

ntial

s

Reas

onab

le Co

sts

Stro

ng

Cand

idate

Adeq

uate

Ca

ndida

te

Unsa

tisfa

ctory

Ca

ndida

te

1) Presents a strong, clearly written design/scope of work (including short-and long-term impact on professional practice and achievement) that has been customized to the districts’/ schools’ data and plans (Ohio PD Standards 1, 2 and 5).

a)

b)

c)

d)

2) Provides sufficient opportunities (initial and follow-up) to incorporate new knowledge and skills into practice (Ohio PD Standards 1, 2 and 4).

a)

b)

c)d)

3) Involves participants working in collaborative groups, balanced with opportunities that accommodate individual educator’s knowledge and skills (Ohio PD Standards 1, 3 and 4)

a)

b)

c)d)

4) Employs a variety of research-based teaching techniques that results in the acquisition, enhancement or refinement of skills and knowledge (Ohio PD Standard 2 and 4).

a)

b)

c)

d)

5) Demonstrates that it has had experience with similar districts and schools.

a)b)c)d)


Resource 21: Implementing Teacher-Based TeamsShared accountability and distributed leadership also extend to the work of teacher-based teams (TBTs) at the school level. In Ohio's Leadership Development Framework, the Ohio Leadership Advisory Council references data teams, which are teacher-based grade level/content area teams, and their work as a natural extension of the work of the district leadership team (DLT) and building leadership teams (BLT). (OLAC Module 3: Developing Shared Accountability - The Why, Who, How, and What of Teams).

Collaboration within and across teams is the highest level of functioning on a continuum of how information, knowledge and working together operate in any organization. The goal in Ohio is for collaborative teams at the district, building and classroom levels to operate at a high level of effectiveness with the ability to do the following:

o Implement procedures for the effective use of data to assess the impact on student learning and to make decisions about teaching and learning;

o Organize and present data in ways that identify gaps and trends in student performance and require intentional decisions regarding curriculum, instruction, assessment, interventions and professional development; and

o Use building, course and classroom data to constantly monitor progress in meeting indicators for the building and at each grade level that are designed to close achievement gaps and contribute to the success of all children.

CONDITIONS FOR GETTING STARTED WITH TEACHER-BASED TEAMS (TBTs)

In order for TBTs to effectively implement the Ohio 5-Step TBT Process for improving student learning and changing teacher practice, the DLT/BLT must assess their readiness to support them by examining whether they have created the conditions for successful implementation. These conditions are described in the Threaded Practices Section of this Guide and are detailed in Resource 8, Six Conditions (see below) to Support Collaborative Teams Checklist which can be used to assess support of TBTs . Once the DLT/BLT has identified the conditions that are not in place for successful implementation, they can then work toward putting them in place. It is not imperative that all conditions for success are in place prior to implementation; however, there are basic conditions that, if not in place, may thwart the process. Following are activities that OIP facilitator’s can use to work with BLTs as appropriate.

CONDITION A. PREPARING TEACHERS TO WORK COLLABORATIVELY BY DEEPENING THE CULTURE OF INQUIRY INTO THE CLASSROOM

CONDITION B. FORMING OR REPURPOSING BUILDING TEAMS TO IMPLEMENT THE 5-STEP PROCESS (SEE NEXT SECTION)CONDITION C. CREATING SCHEDULES AND ROUTINES THAT SUPPORT COLLABORATIVE TEAMS

CONDITION D. MAKING TEACHER-BASED TEAM MEETINGS PURPOSEFUL

CONDITION E. DEFINING ROLES AND RESPONSIBILITIES

CONDITION F. COMMUNICATING PLAN INDICATORS AND PROVIDING AVAILABLE DATA TO ALL TEACHERS

Facilitator Activity to Assess Support of TBTs

Have the DLT/BLT complete Resource 8 to assess their readiness. It is advisable to have each member do their own scoring. Once each member of the DLT/BLT has completed the Resource and obtained item and section total scores, ask them to study each of the items and have a brief discussion about each, particularly determining what evidence they have to respond to any of the items for which they rated a 3, 4, or 5. Reach consensus on item and total scores. Be sure to challenge the DLT by asking questions such as: Do you do this consistently? Do you do this with some buildings or are you just beginning to do this? Change collective scores as appropriate based on the discussion. Once all statements are discussed, have the DLT/BLT prioritize those that are rated a 1 or 2. For those with the highest priority, the OIP facilitator will need to help the DLT/BLT prepare a list of tasks to move them to a higher level. Following are activities the OIP facilitator can use to assist the DLT/BLT in further understanding how to do that.

Resource 21: Implementing Teacher-Based Teams September 2010 OIP Resources

http://www.ohioleadership.org/pd/mod_intro.php?mod_id=2

http://www.ohioleadership.org/pd/mod_intro.php?mod_id=2

CONDITION A: PREPARING TEACHERS TO WORK COLLABORATIVELY BY DEEPENING THE CULTURE OF INQUIRY INTO THE CLASSROOM

Robert Garmston and Bruce Wellman in The Adaptive School: A Sourcebook for Developing Collaborative Groups (1999) describes seven norms of collaboration as the tools for productive communication between group members. These are referenced in Resource 8 and are listed below:

Pausing – Pausing before responding or asking a question allows time for thinking and enhances dialogue, discussion and decision-making

Paraphrasing – Using a paraphrase starter that is comfortable and following the starter with a paraphrase assists members of the group to hear and understand each other as they formulate decisions.

Probing for specificity – Using gentle open- ended probes or inquiries such as ”please say more” or “I’m curious about” increases the clarity and precision of the group’s thinking

Putting ideas on the table – Ideas are the heart of a meaningful dialogue. Label the intention of your comments, e.g., “Here is one idea” or “One thought I have is”

Paying attention to self and others – Meaningful dialogue is facilitated when each team member is conscious of self and of others and is aware of not only what she/he is saying but how it is said and how others are responding.

Presuming positive intentions – Assuming that others’ intentions are positive promotes and facilitates meaningful dialogue and eliminates unintentional put-downs. Using positive presuppositions in your speech is one manifestation of this norm.

Pursuing a balance between advocacy and inquiry – Pursuing and maintaining a balance between advocating a position and inquiring about one’s own and other’s positions assists the group in becoming a learning organization.

Facilitator Activity for TBT Condition A

The purpose of this activity is to agree on what makes a team effective by discussing personal experiences and observing a team in action. In small groups, discuss your prior experiences working in a team and identify the attitudes, behaviors and relationships that made the team successful or not successful. Create a Venn diagram.Individually consider the following questions while viewing the video clip and write your comments:

1) What elements did you observe that made the collaboration effective?2) How would you describe the attitudes, behaviors and relationships among the team? What stage(s) of team

development did you observe?In small groups, discuss how your small group responses correlate to what you observed. Make a combined chart of the knowledge, skills and dispositions necessary for teachers to work collaborative


CONDITION B: FORMING OR REPURPOSING BUILDING TEAMS TO IMPLEMENT THE 5-STEP PROCESS

TBTs are most often comprised of groups of educators who teach the same grade or the same content area. For example, an elementary school may have teacher-based teams at each grade level. Intervention specialists supporting the needs of students with disabilities are regular members of these teams. In middle schools and high schools, TBTs may center on specific content areas within existing department structures. TBTs may also be arranged vertically across grade levels or across disciplines to provide continuity of focus in instruction, curriculum, and assessment. The following chart provides a description of common types of teams found in schools, including membership and purpose.

Once the DLT/BLT has formed TBTs or identified the teams operating in the buildings so they function as a TBT, they will need to discuss the purpose of the team - that is, to follow the Ohio 5-Step TBT Process for the purpose of improving student learning and changing teacher practice.

TYPE OF STRUCTURE MEMBERSHIP PURPOSE

Vertical Team

Group of educators from various grade levels in a given discipline. In addition to teachers, vertical teams also may include counselors, principals, special education or elective teachers (art, music, physical education) or other school officials

Develops and implements a vertically aligned program aimed at helping more students acquire the academic skills necessary for success. Typically one of two types:a) Vertical advanced placement (AP) teaming or AP teaming, which seeks

to prepare students for advanced, or AP courses.b) Transitional Vertical Team that works together to develop a curriculum

that provides a seamless transition from grade to grade. This method focuses on sequencing, or linking elementary curriculum with middle level curriculum, and middle level curriculum with high school curriculum.

Horizontal TeamGroup of educators from the same grade level or a given discipline

Develops and implements a horizontally aligned program aimed at helping more students acquire the academic skills necessary for success.

Inquiry Team

Group of diverse, cross-role teachers, non-classroom-based instructional staff (e.g., coaches, counselors, librarians) and administrators

Examines the performance problems of selected or targeted populations of students with the goal of identifying instructional “change strategies” that will improve their performance by using performance data and other information to diagnose and monitor student learning and to make recommendations for changes in any aspect of the school to help close the achievement gap and improve student outcomes

Data TeamGroup of grade or content level teachers and administrators

Supports assessment literacy by collaboratively analyzing data from multiple data sources, primarily common formative assessments, identifies strengths of learning and obstacles to student learning and determines instructional research-based instructional strategies that will best address student and learning objectives. Analyzes and monitors the effectiveness of the instructional strategies selected and implemented by examining student performance data that was measured using common assessments.

Grade Level/Department

Team

Group of teachers within a grade level or teachers who teach the same subject or related subjects and are in the same department

Plans and discusses lesson delivery (based on curriculum-embedded assessment data) for an adopted program in a content area (e.g., use of regularly scheduled meetings focused on lesson delivery).

Instructional or Content Team

Group of building administrator(s), content area coaches or

Plans for effective implementation of a model or program and maintains the quality/integrity of the program;

Makes decisions about the best practices for

TYPE OF STRUCTURE MEMBERSHIP PURPOSE

coordinators (e.g., literacy, math, technology), classroom teachers and school specialists

content area instruction in their school based on a common understanding of literacy theory and research;

Develops efficient schedules for collecting, submitting and analyzing assessment data related to the content;

Coordinates training and professional development for classroom teachers;

Communicates about the implementation of the model or program and the progress of the students.

Project Team

Group of staff within one functional organization or members from many different functional organizations. A cross-functional team has members from multiple organizations.

The project team is assigned to work on the deliverables of a project, with each member of the team helping achieve the project objectives. The team is collectively responsible for: Understanding the work to be completed; Planning the assigned activities in more detail if

needed; Completing assigned work within the budget,

timeline and quality expectations; and Proactively communicating status, and managing

expectations.Quality

Improvement Team Group of multi-skilled staff Group charged with the responsibility of improving a production process or designing a new one.

Facilitator Activity for TBT Condition B

The purpose of this activity is to study the district/building teams and be clear about current expectations. In small groups, identify all the teams/groups/task forces/study groups that exist in the buildings. What are they called and what do they do? How is what they do different from what is expected of Ohio TBTs? How many teams is it realistic to have in a building? What will you do as an OIP Facilitator to move the DLT/BLT in forming or repurposing these teams to implement the Ohio TBT process? How would you move one or more teams from where they are to where they need to go?

CONDITION C: CREATING SCHEDULES AND ROUTINES THAT SUPPORT TBTS

There are numerous ways for time and structures to be developed so that TBTs are able to meet frequently. These can be divided into three general ways in which time has been successfully used in a variety of schools. Districts and buildings may also want to consider discontinuing practices no longer needed (e.g., spelling bees, showcases and time audits to determine amount being spent and number of students affected) and then use that time for collaboration. It is also wise to involve teachers in creating schedules - teachers are creative!

1) Early Release/Late Start Early outs (or late starts) scheduled weekly or every other week.

When using early outs or late starts, make adjustments to the schedule by increasing instructional time to the other days of the week. Do not shorten instructional time experienced by the students. Be sure to inform the public about how time is used and the rationale for changing the schedule, including the benefits to students.

At high school level develop an early-release schedule that avoids cutting the same period each time students are released early, i.e. one week Periods 1, 3, 5, 7 and next week Periods 2, 4, 6, 8.

Build in nine additional days (based on 37 weeks of school) with a very focused agenda by adding 1/2 hour one morning per week, one hour after school per week and one common planning time per week.


“Are teachers and administrators willing to accept the fact that they are part of the problem? God didn't create self-contained classrooms, 50-minute periods, and subjects taught in isolation. We did—because we find working alone safer than and preferable to working together. (Roland Barth, 1991, pp. 126–127)

When district or school-based staff development sessions are scheduled, arrange for them to begin late or end early with the saved time being used for collaboration.

Schedule a late arrival or early dismissal day. This time can occur once per week, once per month, or once per grading period. Typically, the school day is lengthened and the additional minutes are "banked" to provide release time.

Stay late after school once per month, but make it enjoyable by bringing snacks, flowers, music or other pleasant "atmosphere" items.

2) Modifying Schedules During Student Days Block scheduling provides more possibility of flexibility. Merge classes for assemblies/films/common activities with fewer adults supervising. Schedule common planning time for learning teams or other appropriate groups in which the master schedule is

built to provide teachers of the same grade level, course or subject to meet once weekly in place of that day’s individual planning time.

All teachers at each grade level have one common planning time per week. Schedule back-to-back sessions where teachers are released on a scheduled basis. Interdisciplinary teams set time aside for PD and/or collaboration. Make student lunch hour longer and build in collaborative teacher time. Have two classes team to release one teacher (e.g., two fourth grades, a third grade and a fifth grade). Use other adults (e.g., principals, assistant principals, counselors, social workers, volunteers, psychologists, and

supervisors) to help cover classes, ensuring to follow local policies on who can supervise students. Treat collaboration time as the equivalent of school committee time especially if you are operating a pilot

program. 3) Support Resources to Generate Teacher Time Rotate several substitute teachers through the building and/or use substitute teachers to provide lessons for large

groups of students with paraprofessionals providing small group/individual support. Shared classes can be orchestrated, in which one team takes an entire grade level or course as the other

members have collaboration time. There can also be group activities, in which large groups of students come together and are supervised by those who are not part of the collaborative team, so the team can meet. This may involve support personnel such as counselors, social workers, special teachers.

Link with colleges/universities to free up time. Donate “faculty meetings” or other required staff meetings for collaborative team meetings. Find "volunteer" substitutes -- retired teachers, members of social or civic organizations, teacher trainees from

local universities. Reduce other work to have time to meet -- for example, use peer editing in place of teacher editing of written

assignments have students correct each others' work or create self-correcting materials.

Once a DLT/BLT decides how to capture time given its collective bargaining agreement parameters, it will need to determine how often teams will be expected to meet. Although the amount of time will vary, it is advisable for teams to meet regularly (weekly) for at least 45 minutes. Thus, if a BLT provides teachers with daily planning time, one of those days should be devoted to the TBT. Two resources to assist the DLT/BLTs with scheduling common planning time are Elementary School Scheduling: Enhancing Instruction for Student Achievement by Robert Lynn Canady and Michael D. Rettig (2008) published by Eye on Education, Inc. (www. schoolschedulingassociates.com ) and www. asctimetables.com .

Facilitator Activity for TBT Condition C


http://www.asctimetables.com/

http://www.schoolschedulingassociates.com/

The purpose of this activity is to help the DLT/BLT identify ways to redesign time in order to implement TBTs. In small groups, discuss the current calendar and identify opportunities to schedule time for TBTs to meet. This may look different at the elementary, middle and high school levels. Use the examples above to stimulate ideas. Write a newsletter article describing how the schedule will look. Consider including in the article: Why TBTs and why was this time chosen? What will teachers do …and not do…during this time? What benefits and opportunities will be created as a result of changing the schedule? What will the change cost (if any)? How will TBTs be evaluated? What will the effect/impact be on students?

Once that is done, identify the steps that need to be taken to revise and communicate the schedule. Consider how teachers, parents and students will be informed.

CONDITION D: MAKING TEACHER-BASED TEAM MEETINGS PURPOSEFUL

Often, when BLTs are beginning to create collaborative structures, teachers focus initially on planning special events such as a fall festival, arranging schedules such as parent-teacher conferences, and/or other comfortable tasks for which teachers traditionally have had responsibility. It is difficult to get teams to have efficient processes that guide their time together. The right agenda template saves time, helps teams develop, and increases productivity. Robert Garmston and Debbie Welch discuss in the Journal of Staff Development how at the International School Bangkok, they worked on developing a culture of inquiry by analyzing 16 hours of meeting agendas to determine the percentage of time spent on administrivia versus teaching and learning topics. Analysis of the agendas revealed that more than 60% of groups’ time was spent on administrative issues, such as calendars, short-term scheduling, arranging special events, and discussing policies. To move toward a learning community in which teachers engaged in instructional conversations, they refocused staff members’ energy during the time they spent together. Beginning with the end in mind, they stated the purpose of group time through a clearly outlined agenda that addressed expected outcomes. The results-oriented agendas provided open time to analyze test data, examine student work, score common assessments, discuss effective lessons, and talk with other grade levels. They found that by improving the meeting agenda, groups clarified their outcomes and purposes, reduced meeting time, and increased time spent on student learning issues. A well-constructed agenda includes common elements.

An agenda for a TBT might look like the following: (Note: The italicized commentary provides further instructions for using the template).

AGENDA

Date: Time: (Include start and end times)

Location: Name of Team:

Attendees: (designate recorder, timekeeper, chair/co-chairs, data organizer))

Internal Facilitator: Chair/Co-chairs:

Timekeeper: Recorder: Data Organizer

Meeting Outcome(s):(An agenda with several topics may have more than one outcome. Review outcomes at the beginning of the meeting to “start with the end in mind.” Good outcomes describe a product/result, not a process.)

Strategy/Indicators: (Identify the strategy and indicator that are the focus of the work.)


Topic/Purpose/Process:(Topics need to be directly related to meeting outcome(s). Which topic makes sense to discuss first? What issues need to be addressed at the beginning because information is required that will help a group get from the starting point to its outcome? Label the purpose of each topic such as understand; inform; recommend; or decide. Keep information items to a minimum or eliminate them. Most information can be communicated more efficiently in other ways, such as through e-mail. Describe any processes, including the amount of time, the group will use such as a protocol, making a decision using a consensus process, analyzing data using a tree diagram.)Summary of Discussion/Decisions:(At the close of the meeting, identify who is to do what by when and describe decisions that have been reached. This helps set the agenda for the next meeting and ensures that progress will be made.)

Attach TBT Data Charts/Analysis (See Step 1 of the TBT Process)

The other important way that the DLT/BLT can help teams to have purposeful meetings it to provide and expect the use of protocols to structure conversations. As defined in the OIP Facilitator’s Guide, a protocol consists of agreed upon guidelines for an in-depth, insightful conversation about teaching and learning. Resource 26 provides several examples of protocols that can be used for a variety of purposes.

Facilitator Activity for TBT Condition D

The purpose of this activity is to help the DLT/BLT learn what makes a purposeful TBT meeting. In small groups, create an agenda for the first TBT meeting. Think about what you would need to provide, in addition to the agenda, to support the agenda. Also, consider the role of the principal or other building administrators in this first

meeting. What has occurred before the meeting to prepare the team for the first meeting?

CONDITION E: DEFINING ROLES AND RESPONSIBILITIES

TBTs, as with all teams, need someone who can chair/co-chair the work. This assignment may rotate among team members or be assigned to a member for a set period of time, e.g., semester so leadership is shared regardless of role. The primary responsibility of the chair/co-chair is to manage the process and keep the group on track. Whether the individual(s) are called chairs/co-chairs, team facilitator(s) or team leader(s), he/she is responsible for:

Guiding discussion; Reminding the group of the team’s outcomes, purpose, topics and process; Regulating group activities and contributions; Creating and distributing agendas and minutes to other team members and the BLT; Monitoring agenda time; Helping move the team through the process, e.g., use of a tuning protocol; Meeting with the BLT to coordinate the data and information that is necessary for team functioning; Posing questions to the teams and letting members work through them in discussions.

It is also important to identify what the chair/co-chairs is not. They are not administrators. They do not shoulder the responsibilities of the entire TBT. They do not evaluate TBT member’s performance. They do report or address peers who do not cooperate or fulfill their responsibilities.

It is also important to recognize the importance of the principal’s responsibility in leading the data work. Fullan (2008) found that the impact of principals promoting and participating directly with teachers in the formal and informal learning of the use of data to influence appropriate instructional activities, was more than twice as powerful as any other leadership dimension. In an unpublished work, Philbin (2008) found that strong instructional leadership around the use


and application of data to inform instructional effectiveness is tied to both higher levels of collective efficacy and greater student achievement gains.

It is advisable that as many employees as possible become a member of a collaborative team since everyone in the district and building is responsible for improving student achievement. Typically, membership includes general education teachers from every grade level in every discipline, and may also include counselors, administrators, special education teachers, elective teachers (e.g., art, music and physical education), non-classroom-based instructional staff (e.g., coaches, counselors, librarians) and paraprofessionals. The TBT’s responsibilities may include the following:

Assessing student learning using formative assessment data; Organizing and presenting data in ways that identify gaps and trends in student performance; Making intentional decisions about teaching and learning, i.e., curriculum, instruction,

interventions and professional development based on the team discussion; Contributing knowledge, skills and experience to the team; Constantly monitoring student progress in meeting performance targets using building, course and

classroom data; and Adhering to agreed upon norms, routines and processes established by the team. Norms may

include but are not limited to participating honestly, respectfully and constructively; engaging fully in the process; being punctual; and coming prepared to the meeting.

Within the team, specific roles will need to be assigned, i.e., recorder, timekeeper, data organizer each of which can be rotated as frequently as the team desires. The recorder’s responsibilities are to take minutes during the meeting on the template provided. The timekeeper follows the timeframes on the agenda and informs the team of timeframes during dialogue. The role of data organizer may be filled by a member of the team or a support person in the building. This individual will need data submitted by a designated time and he/she will organize the data in clear, simple, forms in order to distribute the data to the entire TBT prior to a meeting. The DLT/BLT will need to communicate the roles and responsibilities to the chair/co-chairs and members of the team perhaps through an orientation to TBTs.

Facilitator Activity for TBT Condition E

The purpose of this activity is to ensure the DLT/BLT has a way to communicate the desire to implement TBTs and their role in supporting and maintaining them. In small groups, design an orientation for teachers to familiarize them with the TBT process and how the DLT/BLT will support them in the implementation of the process. Think about the “who, what, when, where and why”. With a partner group, present your orientation, allowing the other group to ask questions as if they are the audience for the orientation. Debrief the experience, identifying anticipated and unanticipated questions.

CONDITION F: COMMUNICATING PLAN INDICATORS AND PROVIDING AVAILABLE DATA TO ALL TEACHERS

The DLT has likely reviewed its assessment data and other data that can be used by teacher-based teams. Resource 24 in the OIP Facilitator’s Guide provides an organizer to conduct an inventory of available assessment data and identify gaps. In addition, the DLT and BLT through the completion of the Decision Framework (DF) have identified readily available data, such as attendance, behavior, perception, etc. When the DLT created its plan strategies and indicators, it had to begin thinking about the data it would collect to monitor them. Using this information, the district should identify what data can be used/collected by the TBTs and how that data should be collected, charted and reported

to the DLT/BLT.

Schools typically suffer from the DRIP syndrome (Data Rich/Information Poor). The results-oriented TBTs welcome data from the DLT/BLT and turn data into useful and relevant information for staff. Teachers have never suffered from a lack of data, even working in isolation. However, data becomes a catalyst for improved teacher practice if the TBT has a basis of


TBT

“The measure of success is not whether or not you have a tough problem to deal with, but whether it’s the same problem you had last year.” John Foster Dulles

comparison. When TBTs develop/use common formative assessments throughout the school year, each teacher can identify how his/her students performed on each skill compared with other students. Individual teachers on the team can call on their colleagues to help them reflect on areas of concern. Each teacher has access to the ideas, materials, techniques, and talents of the entire team.

Possible examples of data are:Assessment Data Other Data Including Adult Action Data

State Assessments Leadership PracticesFormative Assessments Levels 2, 3, 4 of the DFCommonly Developed Formative Assessments Observations of instructional strategiesValue Added Growth PD Implementation

Although it is not the intent of this Guide to provide districts/buildings with a “how to” for developing formative assessments, it is advisable that districts have these assessments. Following are general guidelines on their development:

1. Select a topic to assess with a common formative assessment considering those topics identified in the district/school plan.

2. Locate the standards that match the selected topic by grade level/course, defining the important concepts and focal skills/sub-skills.

3. Write questions/items that are matched to the important concepts and focal skills/sub-skills in order to focus instruction and assessment.

4. Design in collaborative teams the pre-and post-assessments, including selected (forced choice) and constructed response items.

5. Create the scoring instruments (e.g., rubrics, multiple choice) ensuring the questions/items will provide data specific to the student performance indicators.

6. Administer and score pre-assessments.7. Analyze results in collaborative teams (see Resources 10 and 23).8. Use results to differentiate classroom instruction and provide appropriate interventions.9. Administer and score post-assessments.10. Analyze results in collaborative teams (see Resources 10 and 23) and revise, if needed.

Facilitator Activity for TBT Condition F

The purpose of this activity is to help the DLT/BLT identify available and needed data for successful implementation of TBTs. Create a T-chart with the first column reading “Available Data Relative to Plan Indicators”. In this column identify the data that your district and/or building have that teacher-based teams can begin to use as they work through the Ohio 5-Step TBT Process. Label the second column “Data We Want to Collect”. In this column, list the data that will need to be collected in order for TBTs to assess level of implementation of the strategy/actions.


Throughout the remainder of this document, an example is used to illustrate each step of the process. In order to give the reader some context, the following example background information is provided.

IMPLEMENTING TEACHER-BASED TEAMS: THE OHIO 5-STEP TBT PROCESS FOR IMPROVING STUDENT LEARNING AND CHANGING TEACHER PRACTICE

The Ohio 5-Step TBT Process offers a way of changing instructional practice and improving student learning that is organized and manageable. These five steps include protocols and techniques for structuring the conversations and overcoming resistance to collaboration through a culture of inquiry. Although a team may begin with limited data, the real objective is to engage teachers in tough conversations about how their teaching practices impact student learning. These steps, although somewhat linear, are much like learning a dance. At first, the steps

may appear stiff and the dancer may feel clumsy or awkward and even view the dance as an unnatural act, but as individual dancers become more accomplished, the dance troupe becomes more fluid and natural, like a work of art, adapting the steps to the rhythm of the music (team). Successful schools recognize that the process is designed to circle back on itself, and that those changes that do not take root the first time around are not cause for abandoning the work. The process is recursive in that data collection and analysis become part of the culture in the ongoing pursuit of improved student achievement.


EXAMPLE BACKGROUND: Each step of the process is described on subsequent pages and includes an example of how one 5th grade TBT at Tactful Local Elementary C followed each step. The following background information is provided on the conditions that were set up by the DLT/BLT.

CONDITION A: PREPARING TEACHERS TO WORK COLLABORATIVELY BY DEEPENING THE CULTURE OF INQUIRY INTO THE CLASSROOM

The district provided PD for all teachers delivered at the building level on how to function as a team and the plan strategy on summarizing. This was offered on an early release day with a follow-up after school and modeling of the strategy by the instructional coach in all 5th grade classrooms.

CONDITION B: FORMING OR REPURPOSING BUILDING TEAMS TO IMPLEMENT THE 5 STEP PROCESS

Elementary C has had grade level teams for many years; however, the focus had primarily been on operational procedures, such as how the team would supervise recess, prepare for parent-teacher conferences and conduct seasonal events. The school had separate voluntary committees to oversee different facets of the school's operation, such as discipline and technology. Each grade level team was repurposed to implement the Ohio 5-Step TBT Process during one of two common planning periods.

CONDITION C: CREATING SCHEDULES AND ROUTINES THAT SUPPORT TBTS

Elementary C teachers have daily individual planning time and common planning time twice a week using a combination of time built into the master schedule in place of one day’s individual planning time and 45-minute early dismissal on Wednesday. The school day is lengthened 15 minutes on Monday, Tuesday, and Thursday which providers 45 “banked” minutes to provide for the Wednesday common planning time. The 5th grade team is required to use their time on Tuesday to implement the Ohio 5-Step TBT Process. The common planning time on Wednesday is devoted to other more traditional team work.

CONDITION D: MAKING TBT MEETINGS PURPOSEFUL

The 5th grade team uses the meeting agenda template provided by the DLT and was instructed by the BLT during its first meeting on how to manage the meeting time. The following agenda is an example of the agenda that will be used in the Ohio 5-Step TBT Process described in the next section. When the agenda was distributed by the team leader, it included a request for the team to bring student papers, the scoring guide developed by the team, student papers arranged from most proficient to least proficient and ideas about why some students are proficient and why other students are not proficient.

The agenda on the following page has been provided to illustrate how one team in Tactful Local Elementary C is implementing the Ohio 5-Step TBT Process. All examples are double-boxed to set them apart from a description of the process. Please note that this agenda covers four (4) forty-five (45) minute meetings and should be sent ahead. Blank forms are provided at the end of this document.



AGENDADate: Tuesday, March 6, 13, 20 & 27, 2010 Time: 2:30 – 3:15 PM (Include start and end times)

Location: Ms. Smith’s Classroom Name of Team: 5th GradeAttendees:

Internal Facilitator: Violet Jones (BLT member) Chair/Co-chairs: Mary Smith/John Green

Timekeeper: Sarah Brown Recorder: Emily White Data Organizer: Cathy Black

Meeting Outcome(s): o Chart data based on most recent formative assessment data and observation datao Identify specific students who require interventiono Agree on specific interventions that will be provided

Strategy/Indicator(s): STRATEGY : Implement district-identified, research-based instructional strategies for reading comprehension

ADULT IMPLEMENTATION INDICATOR 100% of teachers will consistently implement identified (see plan) instructional strategies for reading comprehension. STUDENT PERFORMANCE INDICATOR 100% of students will be at benchmark on district required formative assessments.

Topic Purpose Process/Timeline1. Teachers assessment results Understand which students are above/below

expectations (benchmarks)Individual presentation – 5 min. each (25 min. total)

2. Observation results Learn how the BLT views our classes BLT presentation – 15 min.3. Strengths/obstacles Discover possible strengths/obstacles to learning based

on an analysis of student workProtocol & T-Chart– 110 min.

4. Correlation of Data and Strengths/Obstacles

Compare assessment/observation data in relation to strengths/obstacles

Protocol (continued) – 40 minutes

5. Classroom practice changes Decide the changes which will made for specific students who are not at benchmark

Consensus– 10 min.

Summary of Discussion/Decisions: (Attach TBT Data Charts/Analysis)

DEFINING ROLES AND RESPONSIBILITIES

The five members on the 5th grade TBT have each assumed a role and have agreed to keep the role for one quarter. Roles are then rotated so everyone over the course of the year has filled each.

COMMUNICATING PLAN INDICATORS AND PROVIDING AVAILABLE DATA TO ALL TEACHERS

The DLT communicated the plan strategy and indicator to the BLT who then shared it with the entire faculty at a faculty meeting. The BLT collects data on the Adult Implementation Indicator by conducting weekly observations using an agreed upon set of “look fors” which has been communicated to all teachers. The “look-fors” observed are: A. Provides reason for learning technique and explains the techniqueB. Introduces note-taking technique /graphic organizer as way to collect and remember important information by

modeling/demonstrating its use C. Models use of technique with input from studentsD. Guides student practice using note-taking form or graphic organizer for students to complete E. Provides additional focused guided practice for students who need more time and practice F. Provides time for students to practice techniques independentlyG. Models writing a summary using the information collected H. Checks for understanding using questioning and specific feedback as students use technique independently

TBT 5-STEP PROCESS FOR IMPROVING STUDENT LEARNING AND CHANGING TEACHER PRACTICESTEP 1: COLLECT AND CHART DATA

In addition to the data that the DLT/BLT provides, it is likely that teachers will also have common formative assessment data. If this is not available at the initial formation of TBTs, most teachers will come to realize that they will need to generate this data. In his work, Accountability for Learning (2004), Doug Reeves states that “schools with the greatest improvements in student achievement consistently used common assessments.”

Early in the section on roles and responsibilities, a potential role of data organizer was described. One of the responsibilities is to present the data in simple graphs or charts. Once these are prepared, they can be displayed in halls, classrooms, teachers’ lounges, data walls.

During this step, it is important for teachers to come prepared for the meeting. The TBT would have received an agenda (see above example) and brought their data with them. The BLT would have provided a summary of the observation data. By focusing on the area identified in the district/school plan, it allows the TBT to focus on a single area as a starting place.

There are many software programs that can produce graphs or chart, including Microsoft Word and Excel. The main intent of any plot or graph is to convey complicated or detailed information in a straightforward, understandable manner so that question of educational importance can be understood, discussed and resolved by the TBT. In Data Wise (2005), several suggestions are given for creating good data displays for TBTs. The following may be considered:

TIPS FOR CREATING EFFECTIVE DATA DISPLAYS


EXAMPLE STEP 1

The five 5th grade teachers studied the Ohio Academic Content Standards, grade level indicators and the district curriculum guide, and then created a rubric to pre-assess summarizing. On the basis of the shared knowledge generated by this review, the TBT agreed on the critical elements that they will make sure each student achieves.

Next, the team turned its attention to developing a common formative assessment to monitor each student's mastery of the essential elements. Team members discussed the most authentic and valid ways to assess student mastery. They set the standard for each skill or concept that each student must achieve to be at benchmark. They agreed on the criteria by which they would judge the quality of student work, and they practiced applying those criteria until they could do so consistently. In order to judge a student at benchmark, he/she must have a score of 35 with no element at a 0. Finally, they decided to administer the assessment during the week of March 13 and use this data as the pre-assessment. Each teacher was requested to submit the data to Cathy Black, the data organizer, prior to March 20 so she could prepare the charts and graphs.


EXAMPLE STEP 1 CONTINUEDSTUDENT PERFORMANCE ON PRE-ASSESSMENT

Building Name/Grade: Elementary C, 5 th Grade Content Area: Reading Strategy Focus: Comprehension

Teacher Name

# of Students

Who Took Assessment

# of Students

At Benchmar

k

# of Students

Approaching

Benchmark

# of Students Far Below

Benchmark

Students At/Exceeding Benchmark

Students Approaching Benchmark

Students Far Below Benchmark

Mary Smith 14 4 2 8 Mark, John, Abbey, Jan

Dana, Dave

Gail, Bob, Tom, Sam, Tess, Dan,

Rick, EdJohn Green 28 16 6 6

Sarah Brown 26 18 4 4

Emily White 25 21 2 2

Cathy Black 29 24 2 3

Total 122 83 16 23

Steps for entering data: •

a. Enter teacher names b. Enter # of students who participated in assessment c. Enter # of students who are at benchmark or higher d. Enter # of students who are approaching benchmarke. Enter # of students who are far below benchmarkf. Add to obtain totals

Data Chair/Co-chair will ask the following questions: • What is the total percentage of [Grade 5] students who are proficient and higher? (68%) • What is the total percentage of [Grade 5] students who are not proficient? (32%)

PRELIMINARY ADULT OBSERVATION DATA FROM BLT

“Look Fors”….5th Grade Teacher Summary

1-Not At All

2-To A Slight Extent

3-To An

Acceptable

Extent

4-To A Great Extent

A. Provides reason for learning technique and explains the technique 0B. Introduces note-taking technique /graphic organizer as way to collect and

remember important information by modeling/demonstrating its use 0

C. Models use of technique with input from students 0D. Guides student practice using note-taking form or graphic organizer for students

to complete 0

E. Provides additional focused guided practice for students who need more time and practice

0

F. Provides time for students to practice techniques independently 0G. Models writing a summary using the information collected 0H. Checks for understanding using questioning and specific feedback as students use

technique independently 0

STEP 2.ANALYZE STUDENT WORK SPECIFIC TO THE DATA

“Teachers learn to draw connections between their instructional practice and student learning through the deliberate analysis of data,” says Richard Elmore, Harvard Graduate School of Education Professor, “and this sense of efficacy in teaching is central to internal accountability at the school level.” What strengths and challenges do you see in this data? What are your priorities?

Resource XX provides a common protocol that can be used to analyze student work. The 5th grade TBT modified the protocol and followed these steps taken from Stage 1 of the protocol.

A. Assignment and Expectations (10 minutes) – The presenting teacher briefly describes the assignment that resulted in the student work to be presented, then describes his or her expectations in terms of qualities that would be present in the work if it fully met the summary elements using the rubric below. Each teacher is allotted 2 min.

SUMMARIZATION RUBRICSUMMARYELEMENTS

DEVELOPING(0 points)

NOTABLE(5 points)

EXEMPLARY(10 points)

Summary contains a main idea or thesis statement, written in students’ own

words

Summary does not include a main idea or

thesis sentence

Main idea is included, but is poorly written or not written in

student’s own words

Main idea is not only well written, but written in the

student’s own words

Summary shows evidence that the student used note-

taking form or graphic organizer

Student work does not show evidence of using a note-taking form or

graphic organizer

Student work provides a note-taking form or graphic

organizer, but is incomplete or inaccurate

Student work provides a note-taking form or graphic organizer that is complete, accurate and used in the

final written summary

Summary provides details that support the main idea

Details are not included in summary

Details are included in summary but do not support

main idea

Details are provided and support main idea or

thesis

Summary is brief and contains relevant

information

Summary is too long and contains

irrelevant information

Summary is either brief or contains relevant

information, but not both

Summary is brief and contains relevant

information to support the main idea

Complete sentences are used and summary is free of

grammatical errors

Summary is not written in complete sentences

and contains grammatical errors

Complete sentences are used or summary is free of

grammatical errors, but not both

Complete sentences are used and summary is free

of grammatical errors

Total Possible Points 50 Points

B. Assessment of Selected Student Work Samples From Each Category (80 minutes) – Participants look at each example of student work together and assess the work according to the degree to which it meets the rubric. Using the chart in Step 1,


EXAMPLE STEP 2

Each teacher has examined the results of the common formative assessment (rubric) for his or her students prior to the meeting. The TBT analyzed how all 5th grade students performed in Step 1. Each teacher has brought four samples (two at benchmark, two below) of student work to collectively examine against the summarization rubric. Each teacher was asked to bring copies of the samples. The TBT used a protocol to analyze student work (see Resource 26c). They seek to answer the questions using common formative assessments "Are students learning what they need to learn?" and "Who needs additional time and support to learn?" rather than relying solely on summative assessments that ask "Which students learned what was intended and which students did not?"

Collaborative conversations call on team members to make public what has traditionally been private—curriculum, instruction, materials, pacing, questions, concerns, and results. These discussions give every teacher someone to turn to and talk to, and they are explicitly structured through protocols to improve the classroom practice of teachers—individually and collectively.

they check the score of each student to determine if the score for each student matches the score the individual teacher gave to the student, i.e. below benchmark, approaching benchmark, at or exceeding benchmark. Each of the 5 teachers has brought four samples for a total of 20 samples. Each sample is allotted 4 minutes. .

C. Pattern Analysis (20 minutes) – Once all the student work has been discussed and category verified, participants look for any patterns they may discern, including those related to the gender, language, cultural background, disability or program affiliation of the students, plus any other factors that seem relevant. Here the presenter’s knowledge of the students takes precedence, but questions raised by other participants may prove crucial to discernment. (Note: If the group work does not verify the individual teacher work, this process may need to be repeated with all students and/or each teacher may need to review his/her individual results and make changes to the category as appropriate.

D. Strengths and Obstacles (10 minutes) – Based on the discussion above, the TBT identifies strengths and obstacles. The TBT considers consistent /inconsistent skills, misconceptions about concepts/skills, common errors, issues related to subgroups (gender, limited English proficient, disabilities), patterns of failure to apply certain skills, These are listed on a T-chart (see Example Step 2)

STEP 3. ESTABLISH SHARED EXPECTATIONS FOR IMPLEMENTING SPECIFIC EFFECTIVE CHANGES IN THE CLASSROOM (E.G., INSTRUCTIONAL, BEHAVIORAL) THAT HAVE THE GREATEST LIKELIHOOD TO IMPROVE STUDENT PERFORMANCE

The transition from Step 2, analyzing student work specific to the data to Step 3 requires the team to identify a learner-centered problem, or problem of understanding common to many students as evidenced in student performance. These are identified in the T-chart above. The TBT must investigate the manner in which they are currently teaching these areas or responding to students, and re-cast the problem as a problem of practice. However, before they are able to articulate the problem of practice, the TBT must come to an understanding of what effective instruction or behavior for the identified problem would look like.

What do we want each student to learn, or how do we want them to behave? How will we know when each student has learned or acquired it (post-assessment)? How will we respond when a student experiences difficulty in learning or behaving?

This may be the point at which teachers become aware of the incongruity between their commitment to ensure learning for all students and their lack of a coordinated strategy to respond when some students do not learn. The staff addresses this discrepancy by choosing something to change that will ensure that struggling students receive additional time and support. This response to students who experience difficulty must be:

Timely. The TBT identifies students who need additional time and support. Will the TBT target students who are far below as well as approaching benchmark?

Based on intervention rather than remediation. The teacher provides students with help as soon as they experience difficulty rather than relying on summer school, retention, and remedial classes.

Directive. Instead of inviting students to seek additional help, the TBT requires students to devote extra time and receive additional assistance until they have mastered the necessary concepts or acquired the necessary skills.

Student Owned. The student assumes responsibility for the learning as opposed to the teacher being responsible.

The 5th grade TBT continued to follow the protocol to determine their shared expectations for implementing specific changes in their classrooms.


Example Step 2

STRENGTHS OBSTACLES

Use of one type of graphic organizer Students not using own words

Students use complete sentences Lack of variety of graphic organizers

Relevant information is included Details are weak

Frequent grammatical errors

E. Clarification of Student Learning Needs (15 minutes) – In this step, the TBT examines the obstacles to infer what each student needs to know or be able to do in order to progress to a higher level, i.e., move from “Approaching” to “At Benchmark.” In the example, the obstacles are:

Students not using own words Lack of variety of graphic organizers Details are weak Frequent grammatical errors

F. Discussion of Differentiation Strategies (30 minutes) – The chair/co-chair invites team members to discuss and suggest changes that might help each of the students to advance in the quality of their work. These may be ones that could be undertaken by the teachers within the context of a whole class. They also might include suggestions related to supplementary curricular resources or student services. One way to structure the discussion is through brainstorming all possible changes. It is advisable to consider what teachers who have most students “At Benchmark” or “Above Benchmark” are doing to cause a high degree of success. These may be replicable. It is important to select only changes that teachers can make in the classroom. Once changes are listed, each can be discussed in terms of the impact on student learning.

Using the generated list, have team members collaborate on choosing one or two that they all agree to implement during the next few weeks. This can be done using any consensus process, such as “Fist to Five”. The discussion summary, including the agreed upon changes, should be recorded on the meeting notes.

G. Closure and Debriefing (5-10 minutes) – The chair/co-chair asks the teachers to make any final comments regarding the process and asks all TBT members to comment on the protocol itself, including any ways in which it has provided insight into concerns related to student performance.

Learning from team members, colleagues, consultants, etc. how to implement the specific effective changes in the classroom through job-embedded professional development may be needed. Learning through job-embedded professional development can take many forms, depending on what needs to be learned and practiced, and the degree to which all teachers know how to implement the agreed upon changes. Among the options are:

Modeling: Have teachers on the TBT who know and are currently using the changes in their classrooms model the changes for the entire team. If there is no one on the team, an instructional coach, other building-level teacher or district level person may be able to model the desired changes

Coaching: Often schools/districts employ instructional coaches whose job responsibilities include coaching teachers in using effective practices. Coaching most often takes place in the teachers’ classrooms.


EXAMPLE STEP 3

Step 2 resulted in the team identifying strengths and obstacles in student learning. Based on this discussion, the TBT begins to discuss how they can build on the strengths and address the obstacles.

The entire team gains new insights into what is working and what is not, and members discuss changes that they can implement in their classrooms to raise student achievement. The classroom changes the teachers agreed to are: 1) to teach all students at least three types of graphic organizers, including the most appropriate time to use them, 2) allowing them opportunity for guided practice, and 3) agreeing to build vocabulary so students are able to provide more details when writing a summary and to use their own words.

The intervention selected by the 5th grade TBT at Tactful Local Elementary C required each student who was identified as below or approaching benchmark to receive a progress report every week on Wednesday. Before the progress reports begin, the 5th grade teacher for each student talks with the student individually to discuss how they will work together, thus creating ownership on the part of the student to acquire the concepts or learn the behaviors. The teacher also notifies the student's parents of the plan. Any student who continues to fall short of expectations at the end of three weeks, despite the interventions, is required, rather than invited, to attend tutoring sessions. The teacher makes weekly checks on the student's progress. If tutoring fails to bring about improvement within the next six weeks, the student is assigned to a daily guided study hall with 10 or fewer students. After the three weeks, parents attend a meeting at the school at which the student, parents, and classroom teacher sign a contract clarifying what each party will do to help the student meet the expectations.

Workshop: If the building finds that many teachers within or across grade levels require the information, it may offer building-based training.

Demonstration: Having another teacher model the change in the teacher’s classroom, with the teacher observing, followed by discussion between the two.

Co-teaching: Pairing a teacher who has exemplified use of the change with another teacher in a co-teaching situation often gives the inexperienced teacher the confidence needed to try the change on his/her own.

Whichever approach is used, it should follow the Ohio Professional Development Standards, as overviewed below:

Specific to the changes agreed upon which are aligned to goals, strategies and district/school actions. Improved teacher content knowledge and use of effective instructional strategies. Sufficient opportunities and support for building efficacy and mastery of the desired change. Active learning by participants working in TBTs. Embedded within the school day. Long-term on-going contact and initial and follow-up varied opportunities for learning.

STEP 4. IMPLEMENT CHANGES CONSISTENTLY ACROSS ALL CLASSROOMS (GRADE LEVEL/DEPARTMENT/CONTENT)

Each member of the TBT must agree to the consistent implementation of the changes that have been prioritized and learned in Step 3. As indicated in this Guide, implementation is the most complex part of the process, primarily because it requires changes in adult behaviors and practices that are part of the unique culture of every district and every building. In implementing the changes identified by the TBTs, the approach will either be whole school or specific to a group of students or targeted to an individual student. The three-tier model described in the Ohio Integrated Systems Model for Academic and Behavior Supports (OISM) illustrated below uses this same approach. In this approach, the three tiers of intervention expect the use of scientifically-based research and increasing intensity based on data collected through monitoring of student performance.


Behavioral Academic

STEP 5. COLLECT, CHART AND ANALYZE POST-DATA

AFTER the classroom changes have taken place according to the timeframe set out by the TBT, the TBT will re-administer the assessment to determine progress. The TBT will want to choose the best way to display the data, e.g.

Bar graphs are good for showing rankings, information for multiple groups and trends across time. Line graphs are useful for showing trends over time and comparing trends for different groups across time. Scatter diagrams are useful for showing the relationship between two sets of scores. Pie charts are good at showing parts to a whole or percentages of a whole. Tables are good for showing actual numbers or percentages for multiple variables.

Steps for entering data using chart generated in Step 1:

a. Enter # of students who participated in post-assessment b. Enter # of students who are at benchmark or higher on post-assessmentc. Enter # of students who are approaching benchmark on post-assessmentd. Enter # of students who are far below benchmark on post-assessmente. Add to obtain totalsf. Identify students in each category on the post-assessment

Data Chair/Co-chair will ask the following questions: • What is the total percentage of [Grade 5] students who are proficient and higher? • What is the total percentage of [grade 5] students who are not proficient?


EXAMPLE STEP 5STUDENT PERFORMANCE ON POST-ASSESSMENT

Building Name/Grade: Elementary C, 5 th Grade Content Area: Reading Strategy Focus: Comprehension

Teacher Name

# of Students

Who Took Assessment

# of Students

At Benchmark

# of Students

Approaching Benchmark

# of Students Far Below

Benchmark

Students At/Exceeding Benchmark

Students Approaching Benchmark

Students Far Below Benchmark

Mar Apr Mar Apr Mar Apr Mar Apr March April March April March April

Mary Smith 14 14 4 8 2 3 8 3

Mark, John,

Abbey, Jan

Mark, John,

Abbey, Jan. Dana, Dave, Ed,

Tess

Dana, Dave

Gail, Tom, Dan

Gail, Bob, Tom, Sam, Tess, Dan,

Rick, Ed

Bob, Sam, Rick

John Green 28 16 6 6

Sarah Brown 26 18 4 4

Emily White 25 21 2 2

Cathy Black 29 24 2 3

Total 122 83 16 23

Note: Students with IEPs are underlined. Students with multiple-risk factors are in bold.

OBSERVATION DATA FROM BLT

“Look Fors”….5th Grade Teacher Summary1-Not At All 2-To A Slight

Extent3-To An

Acceptable Extent

4-To A Great Extent

Number of Observations: March – 20, April – 20 (once each week) Mar Apr Mar Apr Mar Apr Mar Apr

A. Provides reason for learning technique and explains the technique 2 0 2 1 1 2 0 2B. Introduces note-taking technique /graphic organizer as way to collect and remember important information by modeling/demonstrating its use

2 0 2 0 1 2 0 3

C. Models use of technique with input from students 2 0 1 1 2 2 0 2D. Guides student practice using note-taking form or graphic organizer for students to complete

2 0 2 0 1 2 0 3


2 0 1 1 2 2 0 2

F. Provides time for students to practice techniques independently 2 0 1 0 2 2 0 3G. Models writing a summary using the information collected 2 0 1 1 2 2 0 2H. Checks for understanding using questioning and specific feedback as students use technique independently

2 0 1 1 2 2 0 2

One of the most important conversations the TBT will need to have is about individual students and their level of progress, or lack thereof. The TBT may choose to examine the actual student work from the pre-and post tests of those who are “Approaching” or “Far Below Benchmark.” As for those students who made progress but are not at benchmark -- e.g., moved up a level -- the intervention may need to be provided for a longer period of time or more often. For those students who did not make progress, additional or alternative supports may need to be provided, e.g., team teaching, development of instructional lesson packets that can be reinforced at home, re-teaching, Saturday school.

A more difficult conversation relates to the BLT observation data. While most teachers are now implementing at an acceptable or above level, one teacher is not at an acceptable level in some areas. Since information and data is transparent, the name of the teacher is known. The TBT must delicately address this issue, offering support and encouragement so the teacher will feel comfortable and open to suggestions for improvement.

Once the data has been analyzed, the DLT/BLT will want to discuss successes, challenges and ultimately make decisions based on the results. The following protocol is adapted from the Resource 26 protocol for generalizing successes and challenges. 1) Preparing a Case (10 minutes) – Each TBT member is asked to reflect on and write (in the form of notes) a short description

of one area where he or she is finding success or making progress in practice and a challenge. The case should include specific details concerning his or her own involvement in it – what he or she did that may have contributed to its success. It also should account for other factors that may underlie the success, including any favorable conditions present. This step may be done in advance of the TBT meeting.

2) Sharing (5 minutes) – In the small groups, the first person shares orally his or her description of successful practice and challenge while the others take notes.

3) Analysis and Discussion (5 minutes) – The group reflects on the successes and challenges. Participants offer their own insights. They discuss specifically what they think the team member may have done to contribute to success, and they also name what they take to be other factors involved. The team member is encouraged to participate and is prodded through questioning.

4) Repeating the Pattern – Repeat Steps 2 and 3 for each member of the TBT.5) Compilation (5 minutes) – The group then compiles on chart paper a list of specific, successful behaviors and obstacles that

seem characteristic of the cases presented.


SUCCESSFUL BEHAVIORS – WHAT WE LEARNED OBSTACLES – WHAT WE LEARNED

6) Interpret Results (XX minutes) - The final step in the TBT process is to interpret the results and determine what will be replicated. The team may consider:

Are results being compared with expectations? Are results being compared to previous data—e.g. last year, prior quarters? Are results being compared among subgroups—e.g. ethnic groups, disabilities or gender?

The TBT may wish to pose the obstacle they are addressing by writing it in the form of a statement, along with the way in which the team sought to address the obstacle.

The obstacle is ______. The change we made to respond to the obstacle is _________.

We know this because we collected _________ (identify key data) and the data reflected the following characteristics __________ (e.g. change in level or trend, differences from expectations).

We are _________ (moderately? cautiously? highly?) confident in this conclusion because ________.

We will replicate the following ______________________________ for _______________________.Options for the team to consider are:

a)Continue implementing changes and monitor for sustained improvementb)Provide more or targeted professional developmentc) Abandon the changed)Consider an alternative change (implement and monitor)e)Adapt the change (implement and monitor)

TBTs judge their effectiveness on the basis of results. Working together to improve student achievement becomes the routine work of everyone in the school. The cycle begins again.


Follows the 5-Step TBT Process Periodically assesses the

effectiveness of the TBT

SHARED ACCOUNTABILITY OF THE DLT-BLT-TBT IN MONITORING PROGRESS

As shown in the illustration, the DLT identifies specific data and tools to collect information from the BLTs/TBTs as evidence of monitoring progress. The BLT monitors the TBTs, and the

TBTs report data to the BLT. The BLT reports progress to the DLT. The following provides more detail about how these relationships work and provide an example to illustrate the relationship.

TBTS TO/FROM BLTS

As illustrated in Step 1 above, the 5th grade TBT provided data using a template provided by the DLT. Each TBT would provide the same information for their respective grade level to the BLT. The BLT then combines this information into a school chart that would look similar to the following.

Building Name Elementary C Student Performance Content Area: Reading Strategy Focus: Comprehension/Summarizing

Grade Level

# of Teachers

# Students Who Took Assessment # of Students At Benchmark # of Students Approaching

Benchmark# of Students Far Below

Benchmark

Oct. Jan. Mar. May Oct. Jan. Mar. May Jan. Mar. May Oct. Jan. Mar. May

K 4 60 35 5

1st Gr. 3 60 25 10

2nd Gr. 3 65 30 15

3rd Gr. 3 66 15 30

4th Gr. 4 95 60 20 15

5th Gr. 5 122 83 16 23

TOTAL 22 468 248 122 98

The BLT has also conducted observations of all classrooms during the first quarter and generated the following table that provides results for all grade levels.


Collects, charts and analyzes data from BLTs

Uses data analysis to generalize successes across the district

Revises the district plan as appropriate

Makes recommendations to BLTs

Reports TBT data using district tools/templates including aggregated TBT data for the building

Requests support for implementation with fidelity

Establishes, implements, monitors and evaluates goals, strategies, indicators and actions

Identifies specific data and tools to collect information from the BLTs/TBTs Creates, implements,

monitors and evaluates actions aligned to district goals, strategies and indicators

Provides job-embedded PD to teachers on the TBT process

Monitors teacher-based teams/classrooms

Reports BLT/TBT/classroom monitoring data/results

Reports challenges regarding implementation of the district plan

Identifies support needed for implementation with fidelity

Follows the 5-Step TBT Process Periodically assesses the

effectiveness of the TBT

Look Fors…. 1 (Not At All) 2 (To A Slight Extent) 3 (To An Acceptable Extent)

4 (To A Great Extent)

Teacher: K 1 2 3 4 5 K 1 2 3 4 5 K 1 2 3 4 5 K 1 2 3 4 5

A. Provides reason for learning technique and explains the technique

2 1 2 0 5 2 1 1 3 1 1 2 2 1 1 3 1 1 2 2 0 1 1 0

B. Introduces note-taking technique /graphic organizer as way to collect and remember important information by modeling/demonstrating its use

3 1 1 0 5 2 2 1 3 1 1 2 1 1 2 3 1 1 1 2 0 1 1 0

C. Models use of technique with input from students 2 1 1 0 5 2 2 1 4 1 1 1 2 1 1 3 1 2 1 2 0 1 1 0

D. Guides student practice using note-taking form or graphic organizer for students to complete

2 1 1 0 5 2 3 1 3 1 1 2 0 1 2 3 1 1 2 2 0 1 1 0


3 1 1 0 5 2 2 1 3 1 1 1 1 1 2 3 1 2 1 2 0 1 1 0

F. Provides time for students to practice techniques independently 2 1 1 0 5 2 2 1 4 1 1 1 2 1 1 3 1 2 1 2 0 1 1

G. Models writing a summary using the information collected 2 1 1 0 5 2 3 1 3 1 1 1 0 1 2 3 1 2 2 2 0 1 1

H. Checks for understanding using questioning and specific feedback as students use technique independently

3 1 1 0 5 2 2 1 3 1 1 1 1 1 2 3 1 2 1 2 0 1 1

The BLT may also request copies of documents such as TBT meeting minutes and lesson plans. Resource 35 provides guidance on document reviews, but in general the BLT would review meeting minutes to ascertain whether the TBT is accountable for the following:

Meeting according to schedule; Including all team members; and Following the Ohio 5-Step TBT Process to include analyzing student work and data from (common) formative

assessments; implementing agreed upon changes consistently across all classrooms; using protocols to guide meeting discussions; and discussing levels of student progress and activities to respond accordingly to the progress, or lack thereof.

The BLT may also consider examining lesson plans to ascertain whether the TBT is: Using the district/school lesson structure, e.g., review, anticipatory set/class opener, whole class - direct

instruction/modeling, guided practice, small group learning/independent work, closure/student reflection; Implementing district and/or school formative assessments at the appropriate time;


Consistently adhering to the pacing and scheduling of academic program delivery, e.g., reading program, mathematics program; and

Connecting what happens from one grade to another.

Using this information, the BLT has the responsibility to analyze this information and provide a report to the DLT (see below). In addition, the BLT has the responsibility to communicate results to the TBTs and school-as-a-whole.

BLTS TO/FROM DLTS

There are many options for BLTs to report data to the DLT. Whichever format is used, the report should contain the following and be submitted at least by semester but preferably by quarter. It is desirable to have these reports submitted electronically, so data can be added over time to show progress.

School name Date/Monitoring Period Plan focus area/goal/strategy Changes identified/to be implemented (during the monitoring period) Adult implementation indicator(s) progress by grade level (pre and post results) Student performance indicator(s) progress by grade level (number of students assessed by proficiency level (pre and

post results) Summary narrative of findings with explanation of irregularities or variation in the data and how the BLT knew they

were/were not implemented Summary of changes made – successes, obstacles, failures Building plan action steps that need to be revised, if any

Copies of TBT agendas and minutes from meetings during this monitoring period may be attached to the report to illustrate a point. The DLT may also want to know:

How student achievement is progressing for the school as a whole. What support, if any, is requested of the DLT.

Resource 28 provides a template and sample for recording monitoring data, both for student performance and adult implementation indicators. These forms can easily be adapted for use in the following ways:

a) Collaborative teams can complete the forms on a specific schedule, e.g., monthly, and submit them to the BLT. The BLT can create charts/graphs that show differences/similarities between classrooms/departments or aggregate data to show total numbers/percents for the building.


b) BLTs can complete the forms on a quarterly basis and submit them to the DLT. The DLT can then create graphs that show differences/similarities between buildings, grade levels, etc. The DLT can aggregate data to show total numbers/percents for the entire district. The following illustrates how Tactful Local created a chart of all 5th grade classrooms in the district and how the data was graphed.

A DLT may also choose, as does Norfolk Public Schools in Virginia using a web-based form, to request an annual performance report from each school. This year-end performance report may include:

Identifying information (name, student and teacher disaggregated and total population enrollment, staff attendance, etc.);

State assessment results (proficiency levels by grade/subject across multiple years); District assessment results (proficiency levels by grade/subject across multiple years); Attendance rates; Promotion/retention rates; Teacher preparation/qualification data (licensed, certified, endorsements, professional development); Learning environment data (teacher, parent and student survey results, discipline incidents); Parent/business/community engagement data (volunteers, parent contacts regarding students, professional

development for parents, outreach efforts); Plan indicator progress data (pre/post over multiple years); Narrative (to provide opportunity for other school-specific information.

Another alternative is for the DLT to conduct observation sweeps of all classrooms in all buildings (may be targeted to a specific content area or grade level) for a specific purpose. For example, the DLT may want to determine or verify the use of graphic organizers. This data would be collected in a short time frame, e.g., one week, and then aggregated by grade level and school.


School# Grade 5 Students Who Took

Assessment# of Grade 5 Students At

Benchmark# of Grade 5 Students

Approaching Benchmark# of Grade 5 Students Far

Below Benchmark

Oct. Jan. Mar. May Oct. Jan. Mar. May Oct. Jan. Mar. May Oct. Jan. Mar. May

Ele. A 330 125 125 80

Ele. B. 408 275 85 48

Ele. C. 468 248 122 98

TOTAL 1206 648 332 226

Tactful Local 5th Grade Quarterly Assessment Results

0%

10%

20%

30%

40%

50%

60%

70%

80%

At Benchmark Approaching Benchmark Below Benchmark

Ele. A

Ele. B

Ele. C

There are 3 elementary schools in Tactful Local, the largest of which is Elementary C. Overall 54% of the 5th grade students are at benchmark, 28% approaching benchmark and 19% far below benchmark. Of the 3 schools, Elementary A is performing the lowest and is the smallest in size. Elementary B has the best overall scores. In disaggregating the data, the DLT found that a disproportionate number of students with disabilities were placed at Elementary C in self-contained classrooms.

The observation data could be compared to the assessment data to determine if there is any correlation. This information would be provided back to the BLT for action.

A DLT may also conduct academic reviews, either as a result of examining BLT reports, as verification of BLT report data, or to garner a clearer picture of BLT needs. In Norfolk Public Schools, the DLT identifies specific schools by tiers. Tier 1 schools receive a full academic review which entails bi-monthly DLT visits. Tier 2 schools receive monthly visits by the DLT. Because this is a large system, not all schools are visited each year. Schools in Tier 1 and 2 may also have all areas reviewed or a subset of areas. Norfolk Public Schools chooses academic content (reading, math, science) for review. A DLT should choose areas identified in the plan as requiring review. During the review, the DLT meets with the BLT, allowing the school to provide an overview and discuss logistics. The DLT, using a pre-identified focus question, visits classrooms to observe students, teachers and/or student work. The following steps to develop a focused question are adapted from Norfolk Public Schools:

a. Review current data from the BLT report to determine current trends;

b. Review the building improvement plan to determine actions relative to the data;

c. Develop focus question with a powerful, narrow focus;

d. Develop feedback form with “look fors” specific to the focus.

e. Determine if it is a good question. It is if: You don’t already know the answer to it. The wording is specific and clear. It will guide information/data

collection. It relates to something that can be

altered or improved. It is either too broad or too narrow. It can pass the so what test (assures it’s

worth doing).

After the observation, the DLT discusses its findings by answering four questions:

1) What evidence did you find to support the focus question?2) What evidence was not available to support the focus question?3) What would you suggest to student or adult learning related to the focus question?4) Are there any other suggestion related to the question?

The DLT then presents their summary responses of the four questions to the BLT. Observed teachers are not invited to the feedback session. After the presentation, the DLT asks the BLT to respond to the following questions. Responses to the questions should be placed on chart paper so there is group memory of the responses. A written report summarizing the question responses should be developed and kept on record.

1) What have you learned from the feedback (strengths/opportunities)?2) What action will you take based on the feedback (brainstorming/reflection)?3) What additional support or resources do you need from the DLT?4) How will you share this information with the TBTs and continue the reflection/dialogue process?5) What changes will you make to the way in which you are monitoring TBTs and/or your building plan?


EXAMPLE

a. The Elementary C BLT report shows that 32% of students were below benchmark in the area of summarizing as measured by the building rubric.

b. The building plan action steps call for teachers working and instructional coaching in the area of summarizing.

c. In what ways are students being provided instruction on how to summarize?

d. Are students engaged in any of the following: working in small groups or pairs practicing use of

technique including main ideas, important details in graphic

organizers sharing notes/graphic organizer information with class using collected information to complete summary frame

or writing summary of content

Stage 3: Implement and monitor instructional practices and system supports that produce conditions for learning,

Stage 1: Analyze pre-assessment data, identifying strengths and obstacles (target population students and focal skill/sub-skill)

Stage 2: Define behaviors for instructional strategies identified in the plan and process for examining student work and students’ conditions for learning.

CONCLUSION

The process for TBTs is rigorous. As indicated previously, TBTs are NOT book clubs, study groups, or traditional grade-level/department team meetings. The process is aligned with the OIP as illustrated in the figure below taken from Resource 11. The figure illustrates the collaborative process that supports implementation of the OIP at the classroom level. The district expects that collaborative teams/TBTs will be used as the primary means to focus on improved teaching and learning and for accomplishing the work in the CCIP and SIPs.


Stage 4: Analyze post-assessment data, identifying strengths and obstacles and make course corrections

Some important lessons learned about TBTs…

This work is not for the faint of heart—it is a very different way of learning for teachers. As with any significant change, it often results in resistance during early stages of adoption.

Provide adequate time for each TBT to meet. Given time, teams can go deeper into improving instruction and evidencing results. Much of the time at the beginning is spent understanding and practicing this new way of learning together. As time passes, teams begin getting some exciting results.

To quote Quincy Gray, Curriculum Instructional Services at Miami Trace Elementary, “I cannot put into words the value that collaborative planning time has had for our teacher-based teams. We do still continue to work on the process, but many improvements have been made just from last year to this year.”

Do not assume that if you provide time and a structure for TBTs that they will make progress. Professional development in effective meeting management, decision making strategies, and assessment literacy is essential to building the skills teachers need for effective and efficient collaborative work. The use of protocols is critical.

An important foundation for TBT success lies in the ability of teachers to collect and organize classroom data. Many examples and on-going support in data collection, organization, and analysis is critical to the process.

REFERENCES

Barth, Roland (1991). Restructuring schools: Some questions for teachers and principals. Phi Delta Kappan, 73(2), 123–128

Bolam, R., McMahon, A., Stoll, L., Thomas, S., & Wallace, M. (2005). Creating and sustaining professional learning communities. Research Report Number 637. General Teaching Council for England. London, England: Department for Education and Skills.

Boudett, Kathryn Parker, City, Elizabeth, and Murnane, Richard. (2006) Data Wise: A Step-by Step Guide to Using Assessment Results to Improve Teaching and Learning, Harvard Education Press

DuFour, R. (2004). What is a “Professional Learning Community”? [electronic version]. Educational Leadership, 61(8), p.6-11.

Filbin, J. F., (2008) Examining the effects of changes in pedagogical precision, principal data use, and student achievement on collective efficacy. Unpublished dissertation Univ. of Denver

Fullan, M. (2008). What’s worth fighting for in the principalship? New York: Teacher College Press.

Garmston, Robert J. and Welch, Debbie, Results-Oriented Agendas: Transforming Meetings into Valuable Collaborative Events, Journal of Staff Development, Vol. 28, No.2, Spring 2007 (retrieved November 14, 2009 www. nsdc.org /news/getDocument.cfm?articleID=1447 )

Garmston, Robert J. and Wellman, Bruce (1999), The Adaptive School: A Sourcebook for Developing Collaborative Groups.

Hord, Shirley, (1997). Professional Learning Communities: Communities of Continuous Inquiry and Improvement

Leithwood, K., & Jantzi, D., (2008). Linking leadership to student learning: The contributions of leader efficacy. Educational administration quarterly. 44(4) 496-528.

Louis, K., & Marks, H. (1998). Does professional learning community affect the classroom teachers’ work and student experience in restructured schools? American Journal of Education, 106(4), 532–575.

McDonald, Joseph, The Power of Protocols: An Educator’s Guide to Better Practice

Reeves, Douglas, (2007). The Daily Disciplines of Leadership: How to Improve Student Achievement, Staff Motivation, and Personal Organization

Vescio, V., Ross, D, Adams, A. (2006 in press). A review of research on the impact of professional learning communities on teacher practice and student learning. Teaching and Teacher Education. Paper presented at the NSRF Research Forum.


http://www.nsdc.org/news/getDocument.cfm?articleID=1447

Resource 22: Assessing Teacher-Based Team Effectiveness ChecklistPurpose: Use this checklist to assess the effectiveness of a TBT for formative evaluation or by the DLT/BLT to evaluate effectiveness of TBTs.Instructions: Rate each of the items below on a scale of 1 to 4. To what extent is this item true?

1 2 3 4

NOT AT ALL TRUE MINIMALLY TRUE MODERATELY TRUE VERY TRUE

1) DATA

□ Our team collects, charts and analyzes data, including data by subgroups.

□ All teachers have ready access to data and information consistent with the district/building plan indicators.

□ Student performance data (formative assessment) is gathered as an instructional strategy and used to determine effectiveness.

□ Teacher implementation of the change/instructional strategy is monitored.

□ Data are well-organized and easily maintained, e.g., technology, notebooks, support staff.

□ Total Score divided by 5 = _____

2) COMMUNICATION

□ Everyone on the team has multiple opportunities for open, relevant, and timely sharing of information within and across grade levels, departments and whole faculty.

□ We receive or conduct ongoing analysis about student needs and findings regarding effective practices, monitoring data and information relative to our plan indicators.

□ Our communication allows for continuous feedback from/to the TBT/BLTs/DLT/

□ Total score divided by 3 = _____

3) ATMOSPHERE AND RELATIONSHIPS

□ We provide an environment where everyone is open and friendly with each other in order to maintain positive, trusting working relationships.

□ Processes allow individuals on the team to relate to each other in a safe and meaningful manner in order to explore and work together.

□ Everyone on the team trusts each other.

□ Everyone on the team has a sense of belonging and there is empathy among members.

□ Teams have clear rules of conduct in times of conflict that everyone has agreed to.


4) CLEAR ROLES AND RESPONSIBILITIES

□ Everyone on the team knows their roles and responsibilities of team members.

□ Everyone on the team is committed to a common purpose, performance and common working approach.

Resource 22: Assessing Teacher-Based Team Effectiveness Checklist September 2010 OIP Resources

□ Everyone on the team holds themselves and each other mutually accountable for student success or failure.


5) DECISION-MAKING

□ We have agreed to and communicated decision-making methods and parameters for team functioning.

□ We know how to build consensus.


6) EFFECTIVE AND APPROPRIATE LEADERSHIP

□ Support is available to teams “just in time.”

□ Administrator’s model collaboration and an inquiry-based attitude.

□ There is a monitoring process in place to consistently collect data and information on the expected changes identified in the plans by each team?

□ Administrators’ are present or available and have clearly defined roles and take actions to support team decisions.

□ Administrators protect the collaborative structure time.


7) PARTICIPATION, PERFORMANCE AND FAIR WORK DISTRIBUTION

□ Our team includes representatives from special education, gifted education and/or LEP.

□ All team members participate adequately, i.e., present, on-task and engaged.

□ Underperformers are dealt with immediately.

□ Our team receives regular, sufficient performance feedback.

□ Work is divided fairly and equitably among the team.

□ Teachers are assigned to teams based on best fit and availability.

□ Members appropriately use knowledge, skills and experience of individual members.

□ Total score divided by 7= _____

8) ATTENTION TO PROCESS

□ Our team follows routines and use templates that ensure well-run meetings, e.g. start and end on time, agendas, norms, minutes.

□ Our team knows and follows procedures for reporting and documentation.

□ Our team has a dedicated schedule that allows for them to meet regularly, consistently with uninterrupted time, ideally weekly, minimally monthly.

□ Our team uses time effectively.Resource 22: Assessing Teacher-Based Team Effectiveness Checklist September 2010 OIP Resources


Resource 22: Assessing Teacher-Based Team Effectiveness Checklist September 2010 OIP Resources

Resource 23: Sample ProtocolsThis resource offers protocol options for a district and/or building to use with their collaborative structures. Each protocol is for a different purpose as described in the OIP Guide. Protocols create a structure to ask challenging questions and allow for active listening. They make the work transparent. It is important to remember that the point is not to do the protocol well, but to have an in-depth, insightful conversation about teaching and learning.

a) Protocol for Looking at Data and Teacher Work – Adult Implementation of Focused Plan Strategies

b) Protocol for Looking at Effects on Student Progress – Collaborative Assessment Conference

c) Protocol for Exploring Student Work – Analysis of Student Work

d) Tuning Protocol

e) Protocol for Developing “Look Fors”

f) Protocol for Analyzing Success

For other protocols and information about protocols, contact:

Colorado Critical Friends Group: www.coloradocfg.org

National School Reform Faculty, www.nsrfharmony.org

Coalition of Essential Schools, www.essentialschools.org

Looking at Student Work, www.lasw.org

Bay Area Coalition for Equitable Schools, www.bayces.org

Resource 23: Sample Protocols September 2010 OIP Resources

http://www.bayces.org/

http://www.lsw.org/

http://www.essentialschools.org/

http://www.nsrfharmony.org/

http://www.coloradocfg.org/

Protocol for Looking at Data and Teacher Work – Adult Implementation of Focused Plan StrategiesThis protocol is a tool to guide groups of teachers to look at what data reveals about teacher practices and student response. Time: 50 –60 minutes. There are six steps to this process.

1. Getting started (10 minutes)a. Presenter briefly explains the data that the group will be examining but does not explain what was

hoped for or expected to be seen. Presenter avoids any statements about whether or not the work or data is good or poor. Then the presenter moves out of the group to listen and observe.

b. Data is broadly defined but needs to focus on what the teacher did as a way of implementing the focused plan strategy(ies).

c. Examples include student work, test scores, attendance charts, parent teacher conference logs, student survey results, video tapes, lesson plans, etc.

2. Describing the work (10 minutes)a. Facilitator asks, “What do you see?” b. Group describes what they see in the data, avoiding judgments about quality or interpretations.c. If judgments do arise, the facilitator asks the person to describe, not interpret.d. Option: record observations on chart paper.

3. Interpreting the work (10 minutes)a. The facilitator asks for interpretation of the data. For instance, what do we make of this work? How

are you making sense of this data? What patterns do you see? What accounts for those patterns?b. During this period, the group tries to make sense of what the data tells them. The group should try to

find as many different interpretations as possible and evaluate them against the evidence.c. From the evidence gathered, the group infers what is happening. For instance, they might discuss

what students seem to be doing in response to the teacher and what the teacher does/does not understand about implementation of the practice. Think broadly and creatively.

d. As the group listens to each other’s interpretations, they ask questions to better help understand each other’s perspectives.

4. Implications for classroom practice (10 minutes)a. The facilitator asks, “What are the implications of this work for teaching and assessment? For

professional development? For the allocation of resources?”5. Group discusses the classroom implications, including:

a. What steps could the teacher/school/district take next?b. What would make the teaching strategy(ies) more effective?c. What other information would they like to see in the student work/response? d. What kind of assignments or assessments could provide this information?e. What do we learn about equity within our classroom/building/district?f. What does this conversation make you think of in terms of the group members’ own practice? About

teaching and learning in general or as it relates to the strategy?6. Debrief


Protocol for Looking at Effects on Student Progress - Collaborative Assessment ConferenceThis protocol has several purposes which include: a) Enhance teachers’ perceptions of student work by sharpening the teachers’ observation skills; b) Encourage a balance in perception, looking for strengths and needs; and c) Encourage conversation among teachers about what the work shows and how they can act individually and collectively on what it shows in order to benefit students. Time for this protocol ranges from 45 to 90 minutes and can involve up to 30 teachers; however, each must have access to the student work (hard copy or visual display) under study. The steps of this protocol are:

1) Presenting – The group facilitator begins by reviewing the protocol and asking the presenting teacher what he or

she has brought to the group. The teacher then presents the student work (may be one assignment or work from different assignments, from one or several students), offering only minimal context. The student work could be videotapes, art, essays, test, projects, etc. Participants read silently or otherwise examine the work, marking on their copies any sections that hold particular significance for them.

2) Describing – The group facilitator asks the group, “What do you see? What evidence do you see of his/her thinking, learning and /or understanding?” Group members respond by describing components or aspects of the work without making judgments of quality.

3) Raising Questions – The facilitator asks, “What questions does this work raise for you? What skills is the student working on? How did the student try to fulfill the assignment? What questions did the student seem to be

answering?” Participants respond with questions they have about the student, the work, the assignment, the classroom, etc. During this time the presenting teacher listens and makes notes, but does not respond.

4) Speculating – The group facilitator asks, “What do you think this student is working on?” Respondents say what they think the student was attempting to learn, accomplish, practice or improve. The group facilitator presses for evidence to support these speculations.

5) Responding – The group facilitator invites the presenting teacher to speak: “After hearing all this, what are your thoughts?” This is the presenter’s chance to respond to questions raised, to offer additional context, to share his or her own thoughts about the student’s work and to respond to any other questions participants may have regarding the student, the context, the assignment and so forth. At this time the presenter might also share any surprises or unexpected

feedback heard during the earlier steps.6) Reflecting and Discussing – The group facilitator invites open discussion, asking participants to reflect on the

experience of the protocol in light of their own larger experiences in teaching and learning. Everyone discusses what the next instructional steps are. He or she also may ask participants to share what they found particularly helpful or difficult while participating in the activity and also how they might use the protocol in their own work with colleagues and/or students. “What new understandings have I come to? What new ideas do I have for my students? Based on what I saw today, what strategies might I use?”

7) Variations – If the group is experienced, smaller groups can be facilitated enabling simultaneous conferences to occur. For a large group, a fishbowl can be used, with 6-10 volunteers, with an experienced group facilitator and presenter, and all others observing. Step 6 can then be open to observers and participants.

Adapted from The Power of Protocols: An Educator’s Guide to Better Practice, Second Edition (2007), J. P. McDonald, N. Mohr, A. Dichter, E. C. McDonald, pg. 80-83


When listening to colleagues’ thinking: Listen without judging. Tune in to differences in

perspectives. Use controversy as an

opportunity to explore and hear from others.

Focus on understanding where different interpretations come from.

Protocol for Exploring Student Work – Analysis of Student WorkThe purpose of this protocol is to shift a teacher’s focus from instructional moves and student behaviors to analysis of learning outcomes and the development of differentiation strategies. As a result of thoughtful engagement with other protocol participants, teacher gain new insights into gaps in student learning as well as a better understanding of their own learning needs with regard to content and instructional practices.

The protocol can be used with any set of content and performance standards and can provide enhanced opportunities for teachers at any stage of their careers (novice to experienced/veteran). This protocol uses a two-stage protocol, where the first stage serves as necessary preparation for the second stage. Each stage takes about an hour and can be done on different days or on the same day. Chart paper and markers are needed for recording ideas. There are five steps in each stage.

Stage 1 – May be done by one grade level. Stage 2 – May be done across grade levels.

1) Assignment and Expectations – The presenting teacher briefly describes the assignment that resulted in the student work to be presented, then describes his or her expectations in terms of qualities that would be present in the work if it fully met standards. The presentation may refer to a district- or teacher-developed rubric. The facilitator records the expected qualities on chart paper. (10 minutes)

6) Introduction – The group facilitator from Stage 1 of the protocol briefly describes the steps accomplished in that Stage – noting the source and size of the original sample, the qualities of work that the participants associated with meeting standards and the pattern hunches they made. The presenting teacher adds any additional details that he or she wishes. (3 minutes)

2) Reactions and Adjustments – This is an optional step. Participants (including the presenter) react to the list of expected qualities and adjust for clarity, specificity and relation to standards or rubrics with which they are familiar. The point is not to force a substitution of the colleagues’ expectations for the teacher’s but to encourage common expectations. (10 minutes)

7) Description of Student Performance Across Categories – Starting with the work sample that meets the standard, participates generate a concise list of descriptors relative to the standard. The group facilitator charts these descriptors where all can see. The descriptors should be concrete based on evident features. For the work samples below standard, the descriptors should refer not to just shortcomings but also to strengths on which the student might build. During this step, the facilitator or presenting teacher may make reference to the pattern analysis of Stage 1. In what ways does each student fit or not fit any of the pattern hunches? Do other patterns emerge here that did not surface in the earlier Stage? (10 minutes)

3) Assessment of Work Samples – Participants (including the presenter) look at each example of student work together and assess the work according to the degree to which it meets the standard. They assign it to one of four categories: far below standard, approaching standard, meeting standard or exceeding standard. Using a graphic organizer, the facilitator records the first names of each student in the appropriate category. (20-30 minutes)

8) Clarification of Student Learning Needs – In this step, the participants go beyond evident features of the work samples to infer what each student needs to know or be able to do in order to progress to a higher level of understanding or performance. The presenting teacher can add additional information about the student’s past performance on similar tasks, individual challenges and strengths, learning preferences, and so on. The inferences are recorded (10-15 minutes)

4) Pattern Analysis – Once all the student work has been discussed and sorted by category, participants look for any patterns they may discern in the sorting, including those related to the gender, language, cultural background, disability or program affiliation of the students, plus any other factors that seem relevant. Here the presenter’s knowledge of the students takes precedence, but questions raised by other participants may prove crucial to discernment. If the work all comes from a single class- and particularly if the work is a whole class set – it will be useful to calculate the percentages of students in each category. These “pattern hunches” are charted for reference during Stage 2 (10 minutes)

9) Discussion of Differentiation Strategies – The group facilitator invites participants to discuss and suggest teaching strategies that might help each of the four students to advance in the quality of their work. These may be ones that could be undertaken by the teacher within the context of a whole class. They also might include suggestions related to supplementary curricular resources or student services. The group facilitator reminds participants of the patterns of hunches identified earlier and asks how any of the strategies suggested might beneficially interrupt patterns. The presenting teacher makes notes in whatever format may be most useful for future planning purposes. (20 minutes)


5) Selection of Representative Student Work From Each Category – The presenting teacher selects a student work sample from each category for further analysis and discussion. These are the work samples that move to Stage 2. Note that for use in Stage 2, copies should be made of the samples, with the identities of the students masked.

10) Closure and Debriefing – The group facilitator asks the presenting teacher to make any final comments regarding the process and asks all participants to comment on the protocol itself, including any ways in which it has provided insight into concerns related to their own students’ performance. (5-10 minutes)

Adapted from The Power of Protocols: An Educator’s Guide to Better Practice, Second Edition (2007), J. P. McDonald, N. Mohr, A. Dichter, E. C. McDonald, pg. 88-89


Tuning Protocol

This protocol was developed by David Allen and Joe McDonald at the Coalition of Essential Schools primarily for use in looking closely at student exhibitions. However, the process is very adaptable for a variety of OIP uses, for example when strategy managers are presenting monitoring data and information.

In the outline below unless otherwise noted, time allotments indicated are the suggested minimum for each task.

I. Introduction [10 minutes]. Facilitator briefly introduces protocol goals, norms and agenda. Participants briefly introduce themselves.

II. Teacher Presentation [20 minutes]. Presenter (e.g., strategy manager) describes the context for the work and presents evidence of the student work (data and information, may include samples of the work).

III. Clarifying Questions [15 minutes maximum]. Members ask questions to better understand what has been presented.

IV. Pause to Reflect on Warm and Cool Feedback [2-3 minutes maximum]. Presenter and members make note of "warm," positive feedback and “cool," more critical feedback.

V. Warm and Cool Feedback [15 minutes]. Members, among themselves, share responses to the evidence and its context; presenter is silent. The facilitator may lend focus by reminding participants of an area of emphasis.

VI. Reflection/ Response [15 minutes]. Presenter reflects on and responds to comments or questions. Members are silent. The facilitator may clarify or lend focus.

VII. Debrief [10 minutes]. Beginning with the presenter ("How did the protocol experience compare with what you expected?"), the group discusses any frustrations, misunderstandings or positive reactions members have experienced. More general discussion of the tuning protocol may develop.

Guidelines for Facilitators

1. Be assertive about keeping time. A protocol that doesn't allow for all the components will do a disservice to the presenter, the work presented and the participants' understanding of the process. Don't let one participant monopolize.

2. Be protective of presenters. By making the work more public, presenters are exposing themselves to kinds of critiques they may not be used to. Inappropriate comments or questions should be recast or withdrawn. Try to determine just how "tough" your presenter wants the feedback to be.

3. Be provocative of substantive discourse. Many presenters may be used to blanket praise. Without thoughtful but probing "cool" questions and comments, they won't benefit from the tuning protocol experience. Presenters often say they'd have liked more cool feedback


Protocol for Developing “Look Fors”This protocol is a procedure to guide groups to develop adult implementation “look-fors” specific to the district plan indicators. Its’ focus is on developing the “WHAT” or the content in the “look fors”, not the HOW or the process.

1. Review the connection of task to OIP.

a) Identify the strategy from the district plan.

b) Identify the specific indicator for which data needs to be collected in order to measure progress.

c) Describe how the indicator relates to the first two components of the overall monitoring system.

o What to Monitor - may relate to strategy and/or actions

o What to Gather - identification of “look fors”

What do you expect to see across the district in specific observable behavioral terms? Would you want to observe adult behaviors, student behaviors or both? What type segments of practice are you going to “look for” - routine, content specific, or segments enacted on the spot?

Note that the DLT will want to identify a few critical “look fors.” BLTs may expand beyond these few depending on the building plan.

d) Emphasize the necessity of developing “look fors” that reflect the criteria or features of a good “look for”:

o Explicit about what is to be seeno Evidence-basedo Transparent to everyoneo Focusedo Manageableo Specific to district plan strategy/actiono Meets Ohio grade level expectations, when applicable

e)Share cautions:

o Too much is too mucho Less is moreo Process may take a year or more to internalize; think big, start small

2. Describe and define the indicator content.

a) Describe the terms that need to be defined in the indicators, starting first with the content. May need to revisit why the indicator was selected based on the data from the DF and research.

b) Determine if there is an agreed upon definition of the content.

This may be done by creating a two column T-chart of “What it is” and “What it isn’t”. Brainstorming responses and then asking:

o Is it clearly understood by all?o What does it look like across the district?o Are there differences among grade levels or content areas?


3. Reach consensus on the few things that the DLT wants to see as evidence that the strategy content is being implemented (observable behaviors).

a) What will the DLT expect to see from adults?

b) What will the DLT expect to see from students?

c) What will the DLT expect to see in the classroom environment?

d) Check to see that “look fors” meet criteria.

4. Describe and define other monitoring system components.

a) When to Gather and Submit Data - degree, amount or frequency for implementation, monitoring, reporting.

b) Who Will Provide and Gather Data - Who will be reviewing or observing this indicator? Who will provide the initial data and who will summarize the data, e.g., data teams?

c) How to Gather Data - What is the format and methods for collecting the data, quantitative and qualitative (observations, document review, examining student work, etc.)?

d) How to Record Data – What is the collecting/reporting format? How will it record quantitative as well as qualitative data? What is to be reported to the BLT? To the DLT? To teacher-based teams?

e) How to Manage Monitoring Data - How will the data be rolled up and back? Determine: if paper/pencil or technology will be used; how data will be stored/archived and used for comparison over time to demonstrate improvement; how to know the reason(s) for improvement or lack thereof. Who is responsible for managing the data?

f)How to Communicate Monitoring Expectations & Results – Determine: how the DLT will communicate: a) common definitions, b) teacher and/or student behaviors and/or c) environmental expectations for implementation; and how the information will flow up and down the system.

5. Determine next steps.

a) Develop “look fors” for all plan indicators and identify those that may be crosscutting.

b) Forewarn the DLT that adjustments may need to be made to the IM/M, as baseline data was probably not available when the plan was originally developed.

c) Collect baseline data using “look fors”.

d) Revisit task list for possible modification.

e) Consider what support is needed to make use of data in instructional decision-making.

f)Check against professional development identified in plan. What training might be needed or modified?

g) (Other)

.


Protocol for Analyzing Success This protocol’s purpose is to engage colleagues in collaborative analysis of cases from practice in order to understand the circumstances and actions that make them successful ones, and then to apply this understanding to future practice. The DLT would work in small groups of 3 to 6 individuals. Chart paper or LCDs should be available. The size of the small groups will determine the length of the activity, with variations of 1 to 2 hours. There are eight steps to using this protocol.

7) Preparing a Case – Each participant is asked to reflect on and write (in the form of notes) a short case describing one area where he or she is finding success or making progress in practice. The case should include specific details concerning his or her own involvement in it – what he or she did that may have contributed to its success. It also should account for other factors that may underlie the success, including any favorable conditions present. This step may be done in advance of the DLT meeting. (10 minutes)

8) Sharing – In the small groups, the first person shares orally his or her case of successful practice while the others

take notes. (5 minutes)

9) Analysis and Discussion – The group reflects on the success. Participants offer their own insights into what made this case of practice successful. They discuss specifically what they think the presenter may have done to contribute to success, and they also name what they take to be other factors involved. The presenter is encouraged to participate and is prodded through questioning. (5 minutes)

10) Repeating the Pattern – Repeat steps 2 and 3 for each member of the group.

11) Compilation – The group then compiles on chart paper a list of specific, successful behaviors and underlying principles that seem characteristic of the cases presented. (5 minutes)

12) Reporting Out – If there are multiple small groups, the groups report out in some way, for example, by means of posting lists around the room and “gallery walking” to read all the lists. (5 minutes)

13) Discussion – The facilitator prompts a general discussion with the questions. “Do the lists have elements in common? Do any contain behaviors or underlying principles that surprised you?” (10 minutes)

14) Debriefing – Still in the large group, the facilitator asks, “How might we apply what we have learned in this protocol to other parts of our work? How might BLTs/collaborative teams/students use this protocol or a variation of it to reflect on their work?”

Variation

The OIP Facilitator may ask that everyone focus on the same theme, e.g., a successful staff meeting, a successful peer observation or a successful project design. Small groups compile a composite list and report out. The OIP Facilitator then leads a general discussion about common successful practices. Overall questions are: What have we learned? How are we learning? Where has it worked/not worked and Why? What supports are we providing that contributed to success?Adapted from The Power of Protocols: An Educator’s Guide to Better Practice, Second Edition (2007), J. P. McDonald, N. Mohr, A. Dichter, E. C. McDonald, pg. 6


TIP: If DLT/BLT members are asked to prepare their own cases in advance, the OIP facilitator should give them a few minutes to review their notes. This will ensure that people are focused and provide those who did not actually prepare a chance to develop their thinking.

TIP: When participants make generalized statements, the OIP Facilitator should remind them to say what they did themselves, since this is helpful in demystifying success.

Resource 24: DISTRICT ASSESSMENT SYSTEM (types, definitions and purposes/uses)Type of Assessment Definition General Purpose/Use Example(s)

A) Initial/

Diagnostic

Assessments that identify students’ strengths and weaknesses, what students already know about a topic and/or gaps or misconceptions.

Check student progress toward meeting standardsDetermine what a student does and does not know about a topic or his/her understanding of fundamental concepts and skillsUsed to evaluate, adjust and strengthen (inform) instruction Determine whether further diagnosis is neededIdentify students who may be at risk and in need of interventionMake up the initial phase of assessment for learning

Ohio Diagnostic Assessments: reading (K–2), writing K-3), mathematics (K-2) have four components:

screening measures observational measures diagnostic measures short screening measures

Kindergarten Readiness Assessment – Literacy (KRA-L)

B) Formative/

Interim

Assessments used by teachers and students during instruction that provide feedback to adjust ongoing teaching and learning to improve students’ achievement of intended instructional outcomes.

Adjust teaching and learning (inform instruction)Differentiate instruction or lesson level/concept levelDetermine students’ knowledge and skills, including learning gaps as they progress through a unit of study Occur during the course of a unit of study

Pre-post assessmentsProgress monitoring assessmentsObservationStudent conferencesPerformance TasksRubricsStudent self-assessments

Sub-set: Collaboratively Developed Formative Assessments

Teacher generated assessments that are collaboratively designed by grade level or department teams that are administered to students by each teacher periodically throughout the year.

Adjust teaching and learningDifferentiate instruction or lesson level/concept level

RubricsPre-/post-assessments

C) Summative Periodic assessments given to determine at a particular point in time what students know and do not know relative to content standards. Summative assessments are too far down the learning path to provide information at the classroom level and to make instructional adjustments and interventions during the learning process.

Provide grades or marks against an expected standardEvaluate effectiveness of programs or alignment of curriculumMeasure student achievementDemonstrate evidence of continuous improvement at the state and local levelProvide data for Ohio’s accountability systemOccur at the end of a unit of study to determine the level of understanding students have achieved

State assessments, i.e., OAT/OGTDistrict assessments, i.e. end-of-term or semester examsGrade level achievement testsEnd-of-course examsEnd-of-chapter quizzes

Resource 24: DISTRICT ASSESSMENT SYSTEM (types, definitions and purposes/uses) September 2010 OIP Resources

DISTRICT ASSESSMENT SYSTEM INVENTORY TEMPLATE

Directions: List all the assessments administered to students during a school year. Content Area (complete by content area)

Type and Name of Assessment

GRADE LEVEL FREQUENCY OF ADMINISTRATION PROTOCOL AND USE*

P K 1 2 3 4 5 6 7 8 9 10 11 12 A B Q M W O (Please specify) R OINITIAL/ DIAGNOSTIC

FORMATIVE/INTERIM

Subset: Collaboratively Developed Formative Assessments

SUMMATIVE

Codes for Frequency of Administration: A = annual, B = biannual (e.g., fall/spring), Q = quarterly, M = monthly, W = weekly, O = other (please specify)Codes for Protocol and Use: R = required, O = optional (i.e., discretion of building or teacher). Describe protocol and use.

*Questions for Determining Use: What is the format of the results/data? Who is the audience for the data? How much time is used to administer the assessment? How important is the data to improving student achievement and changing classroom instruction?


Guidelines for Designing Collaboratively Developed Formative Assessments

11. Select a topic to assess with a common formative assessment considering those topics identified in the district/school plan.

12. Locate the standards that match the selected topic by grade level/course, defining the important concepts and focal skills/sub-skills.

13. Write questions/items that are matched to the important concepts and focal skills/sub-skills in order to focus instruction and assessment.

14. Design in collaborative teams the pre-and post-assessments, including selected (forced choice) and constructed response items.

15. Create the scoring instruments (e.g., rubrics, multiple choice) ensuring the questions/items will provide data specific to the student performance indicators.

16. Administer and score pre-assessments.

17. Analyze results in collaborative teams (see Resource 27).

18. Use results to differentiate classroom instruction and provide appropriate interventions.

19. Administer and score post-assessments.

20. Analyze results in collaborative teams (see Resource 27) and revise, if needed.


Resource 25: “Look Fors” Behaviors (Student and Adult) - SamplesThe following resource offers three options for describing behaviors associated with a variety of district indicators. Each uses a different format to explain student and adult behaviors associated with an indicator. The examples provided are not complete documents but serve to provide illustrations of formats. Information on how to locate the full document is provided.

Content “Look Fors”

These "Look Fors" were developed by Dr. James V. Foran, Director of Secondary School Development, and the High School Improvement Program Technical Assistance Group. Retrieved on January 25, 2009. from: www.msde.state.md.us/testing/administratorlookfors.html

Differentiated Classroom – Formative Assessment “Look Fors”

These “Look Fors” are excerpted from an instrument developed by Carol Ann Tomlinson and Jessica Hockett for Learning Sciences International. The complete Look-For can be found at www.learningsciences.net

Curriculum – Academic Rigor “Look Fors”

These “Look Fors” are from an instrument used by the Norfolk Public School s in Norfolk, Virginia, Leadership and Capacity Development, July 2007.

Resource 25: “Look Fors” Behaviors (Student and Adult) - Samples September 2010 OIP Resources

http://www.msde.state.md.us/testing/administratorlookfors.html

"Look Fors" in Algebra and Geometry(Developed by Dr. James V. Foran, Director of Secondary School Development, and the High School Improvement Program Technical Assistance Group)

Rather than random lesson planning in algebra or geometry or simply following the table of contents of a textbook…

Teachers plan with the core learning goals, expectations, indicators and assessment limits in mind. Assessment Limits are extremely important because they prescribe the non-negotiable topics of each concept that must be covered to ensure that students have been taught the material that will be tested. Teachers are invited to go as far beyond the assessment limits as time, their level of expertise and the ability levels of the students they are teaching will allow.

Rather than simply doing problems in isolation…Teachers involve students in real-world applications of mathematical concepts.

Rather than having students sit and listen…Teachers actively engage students in mathematics through reading, writing and oral communication.

Rather than students simply finding answers on their own…Teachers foster more communication of mathematics by encouraging students to explain mathematical concepts to each other.

Rather than using technology for simple calculations…Teachers encourage students to use technology for exploration and insight.

Rather than simply telling and explaining…Teachers ask more questions to draw out high level thinking, such as "why," "explain," "justify" and "elaborate."

Rather than simply emphasizing correct answers…Teachers engage students in mathematical concepts through investigations and discovery learning.

Rather than simply emphasizing rote multi-step manipulations…Teachers relate skills and symbol manipulations to functions, tables and graphs.

Rather than simply memorizing procedures…Students frequently ask themselves appropriate questions and reasons to solve problems appropriate to the content of the course.

Rather than simply talking about mathematics in isolation…Teachers help students make connections with previous knowledge in mathematics, as well as between mathematics and other disciplines.

GenericRather than being simply a disseminator of knowledge…

Teachers are facilitators of student learning and guide students in forming their own questions and setting, monitoring and evaluating their own learning goals and processes.

Rather than relying solely on written tests…Teachers assess student performance through multiple forms including tests, written and oral reports, projects, peer reviews and portfolios.

Rather than assessing simply to assign grades…Teachers use assessments to guide instruction and evaluate students.

Rather than keeping grading criteria secret…Students are graded based on criteria that are clear to students through the use of rubrics.

Resource 25: “Look Fors” Behaviors (Student and Adult) - Samples September 2010 OIP Resources

“Look-Fors” in an Effectively Differentiated Classroom(An excerpt from an instrument developed by Carol Ann Romlinson and Jessica Hockett for Learning Sciences International)

CATEGORIES, “LOOK-FORS” and RUBRICS

“Look For” 3: Formative AssessmentCategory: The teacher employs formative assessment as the primary tool for informing differentiated

instruction.

Indicators (Teacher Behavior): The teacher formatively assesses students’ readiness, interest and learning profile needs and uses the results to inform

adjustments to content, processes and products The teacher integrates formative assessment as an important aspect of classroom life.

Evidence: The teacher employs a variety of formative assessment techniques that glean information most critical to making adjustments

for student need. The teacher pre-assesses students’ readiness, interest and learning profile needs relative to the learning goals prior to a unit

of study. The teacher uses ongoing assessment to gauge students’ progress during a unit. The teacher uses data gathered through informal or formal formative assessment to make decisions about how/ when to use

instructional arrangements. The teacher uses data gathered through informal or formal formative assessment to adjust content, processes and products. The teacher explicitly communicates the purpose of formative assessment to students.

Rubric: Formative AssessmentAdvanced - 4

a. The teacher plans and administers pre-assessment well in advance of a unit of study and ongoing assessment diligently throughout a unit.

b. The teacher uses formative assessment results to make low-prep and high-prep adjustments to curriculum and instruction.c. The teacher designs formative assessments that require students to demonstrate their understanding, knowledge and/or skill

in multiple modes.d. Students look forward to formative assessment opportunities due to their numerous positive experiences with how their

teacher uses the results to make adjustments for their individual needs.Proficient - 3

a. The teacher administers formative assessments before and during a unit of study.b. The teacher uses formative assessment results to make low-prep adjustments to curriculum and instruction.c. The teacher uses formative assessments that allow students to demonstrate their understanding, knowledge

and/or skill in modes other than writing.d. Students understand how and why the teacher uses formative assessment.

Basic - 2a) The teacher periodically administers formative assessments during a unit (e.g., quiz, exit card).b) The teacher uses assessment results to determine student progress, but does not use results to

inform instructional adjustments.c) The teacher uses formative assessments that limit student response to one mode of expression

(e.g., written).d) Students view a formative assessment as a way to earn grades.

Below – Basic - 1a. The teacher rarely, if ever, uses formative assessment.b. The teacher uses formative assessment primarily to fill a grade book rather than to inform instruction.c. The teacher chooses assessments that severely inhibit students’ capacity to fairly demonstrate what they have

learned (i.e., due to poor design, due to misalignment with curricular goals).d. The teacher fails to give a rationale for formative assessment or to help students distinguish between the

purposes of formative and summative assessments.Academic Rigor “Look Fors”

Resource 25: “Look Fors” Behaviors (Student and Adult) - SamplesSeptember 2010 OIP Resources

(From an instrument developed in the Norfolk Public Schools, Norfolk, Virginia)

Definition: Academic rigor is teaching and learning which leads a student to a deeper understanding of a topic, concept, issue and/or a specific discipline. More complex content, more-depth and quality products, more opportunities for individually challenging instruction, a greater focus on higher-level thinking and reasoning skills, and increased student responsibility will help achieve deeper understanding. Such teaching and learning must occur in a setting that is characterized by higher expectations, support and flexibility.

INDICATOR TEACHER BEHAVIORS STUDENT BEHAVIORS

1. Thorough and challenging learning experiences

Plans instructional tasks that reflect an analysis of student performance data and adjusts plans based on student needs

Uses observations of students at work, their exhibitions and portfolios as sources of assessment

Provides many and varied opportunities for student to demonstrate understanding at advanced levels

Provides opportunities for students to deepen their understanding of topics and issues by exploring their own beliefs and those of others

Designs and selects tasks requiring more complex thinking, reasoning and problem-solving to build powerful literacy

Works on homework assignments that are modified based on demonstrated ability

Uses higher grade-level materials and resources in the study of a given topic/concept

Works on a more advanced curriculum unit

Learns new content based on pre-testing to determine mastery

Selects more challenging assignments when presented with a range of options

Works on projects/assignments requiring extended time

2. Strong base of knowledge and understanding

Remains current in the knowledge of his/her discipline and instructional best practices

Collaborates in grade level, departmental and vertical teams to ensure horizontal and vertical content articulation

Primarily functions as a facilitator of learning while not abandoning the responsibility to provide direct instruction

Provides differentiated instruction to accommodate a variety of interest, abilities, learning styles and pacing preferences and/or needs

Integrates reading and writing strategies in all content areas to build powerful literacy

Incorporates the use of rubrics to assess the performance of students

Creates questions that focuses instruction on essential understanding and relates academic problem solving and realistic outcomes

Provides opportunities for students to deepen their understanding of topics and issues by

Identifies concepts and relationships among various disciplinary and interdisciplinary topics

Bases opinions and analysis on sound principles of logic and reasoning

Uses authentic academic research methods to draw conclusions

Demonstrates an ability to function effectively in a variety of groups based on learning styles, interests and/or instructional need

Responds to and asks question requiring reasoning and critical thinking

Demonstrates a depth of understanding of various perspectives on specific topics and issues; produces clear and insightful written, oral, visual and kinesthetic products

Masters the vocabulary of the discipline under study

Reads and understands content material provided from a variety of sources



exploring their own beliefs and those of others

3. Ability to use information and skills in other settings

Designs learning tasks and experiences that are “real world”, meaningful and that meet the education needs, interests and learning styles of ALL students

Utilizes technology to inform, enhance and extend instruction

Provides learning experiences that require students to apply and extend major concepts, as opposed to sole focus on memorization of facts

Evaluates student background knowledge prior to engaging students in content based instruction

Uses simulations, games, projects and graphic organizers to illustrate application of learning in each lesson

Incorporates significant external influences such as guest speakers, internships and community-based activities to support student achievement

Designs and uses problem-based learning

Readily identifies connections of learning to “real world”

Independently utilizes and constructs graphic organizers to clarify relevancy of content

Poses questions about relationship of content/skill/process to a career, current issue, problem

Seeks additional resources to find out more about a topic

Develops alternative solutions/strategies for a variety of “real world’ concepts

Utilizes technology to inform, enhance and extend instruction

Participates in simulations, interviews, internships and community-based activities to extend their background knowledge

4. Continuing curiosity and interest in further learning

Allows student responses to enrich lessons, to shift instructional strategies and alter content

Engages students in the design and planning of their learning experiences

Encourages student interest by asking thoughtful, open-ended question

Encourages students to ask questions of each other

Provides enrichment opportunities

Presents various points of view on a topic

Allows students to study self-selected topics for credit

Participates in capacity building opportunities offered through local, state and national professional organizations

Displays a questioning attitude and seeks information to answer his/her own questions

Readily engages in discussions about class material with other students

Studies independently to learn new information

Participates in class discussions on student generated topics

Presents material which he/she has worked on in class

Presents information to class on topic of interest

Suggests alternatives to long-term projects based on personal interest related to the content

5. A sense of self direction

Models and presents opportunities for metacognition (thinking about thinking) critical and/or creative thinking and problem-solving

Consistently involves students in investigative techniques unique to a variety of disciplines

Employs a variety of instructional activities that

Studies on own to learn new material

Uses a planner/calendar to organize independent study projects

Becomes intensely engaged in a self-selected investigation



incorporate varied learning styles and differentiated instruction (e.g., flexible grouping, multiple intelligences and tiered assignments)

Encourages, suggests or provides independent extension activities outside of class

Creates conditions that encourage students to interact, respond, participate, ask questions and volunteer comments throughout the lesson

Provides multiple and varied resources within the classroom

Encourages students to keep records of their own progress

Identifies what is not known about a topic and develops a plan of research

Exhibits self-discipline, strong study and organizational skills

Independently sets academic goals

6. An internalized ethic of quality work

Communicates the objectives of the course to students and parents

Respects the dignity of colleagues and works cooperatively with colleagues in the interest of fostering student development

Responsible for taking adequate steps to ensure that assessment of students if valid, open, fair and congruent with course objectives

Communicates high expectations

Respects diverse talents and ways of learning

Strives to attain the standard of excellence established by the teacher and/or self

Develops a portfolio of work projects reflecting growth

Incorporates feedback from knowledgeable sources/resources to improve the quality of his or her work

Recognizes when clearer criteria will improve his or her work

Uses project specific criteria to improve his or her work


Resource 26: Procedures for Monitoring Student PerformanceMonitoring student performance based on CCIP and/or SIP indicators is one of the steps in systematic problem-solving. Monitoring involves the frequent and repeated collection and analysis of student performance data. Monitoring student performance is not to be used only when time permits. Data is collected on a regular basis during the course of plan implementation. Progress monitoring provides a standardized and empirical method for evaluating the effectiveness of whether the strategies, actions and tasks implemented are improving student learning. In order to begin on-going data collection and analysis, the following must be in place and followed:

1) Have a well-defined set of knowledge and skills (academic indicators) that a student is expected to learn.

These knowledge and skills identify the focus of ongoing measurement. The knowledge and skills must be defined in terms that are concrete, observable, specific and measurable. In most cases, the target knowledge and skills should be ones that need to increase in frequency.

2) Establish measurement method(s).

The selected knowledge and skills must allow for the frequent and repeated collection of student performance data. The method(s) of measurement should be time and cost efficient, and should be sensitive to small changes in student performance over relatively short periods of time. Standardized measurement procedures should be used.

3) Describe student’s current level of functioning.

Information regarding a student's current level of functioning is helpful in setting an appropriate indicator of improvement and provides a baseline against which subsequent performance can be compared.

4) Have a well-defined strategy indicator.

It is impossible to evaluate progress without a yardstick against which to compare it. A strategy indicator provides such a yardstick by clearly describing the expected outcome of actions in terms of improved student performance. The strategy indicator should specify the behavior, the conditions under which this behavior will be exhibited and the criterion for satisfactory performance.

5) Implement actions (and tasks that put the action into motion) that will result in improved student performance.

No set of actions (and tasks) works all of the time or with every student. Plans should be developed with an expectation that they will be altered if data indicates a need to do so. At the building level, plans should clearly delineate materials and procedures to be used, as well as roles and responsibilities for collaborative team members. This may include having classroom plans in addition to a building plan.

6) Collect data regularly and visually represent the data, such as on a graph or chart.

Resource 26: Procedures for Monitoring Student Performance September 2010 OIP Resources

Data must be collected in short intervals, e.g., once per week, at the building level in order to provide sufficient information for trend analysis within a month (based on the monitoring schedule the building has set) or across a quarter (based on the monitoring schedule the district has set).

Monitoring procedures generate a large quantity of data that must be systematically recorded and analyzed. A visual representation of data such as a graph or chart provides a visual depiction of expected (as described on the graph by the indicator line) and actual student performance.

7) Make course correction(s).

Course correction should allow for the systematic interpretation of performance trends with regard to progress toward the identified strategy. The process for course correction should be established prior to implementation of the actions/tasks. Course correction should include a rule for raising the strategy indicator if student progress exceeds expectations. Conversely, there also should be a rule which prompts the alteration of the actions/tasks if insufficient progress is demonstrated.

If an action must be modified, only one instructional factor at a time should be altered. If more than one factor at a time is modified, and if student performance subsequently changes, the collaborative team will not be able to determine which factor was responsible for the change in student performance. Small changes or refinements in plans should usually be attempted before major alterations in the plan are made. At the same time, however, the modifications must be ones that are judged to be sufficiently substantial to result in improved student performance.

Whenever a modification in the plan is made, this modification should be noted on the graph. A vertical line should be drawn on the graph at the point in time when the plan was changed. The collaborative team meeting notes should record the specific procedures and materials that are modified. This will assist in determining what actions/tasks should be generalized across a building/grade level/district.

Resource 26: Procedures for Monitoring Student Performance September 2010 OIP Resources

Resource 27: Monitoring Processes for Adult Implementation The intent of these processes is to institutionalize observation processes that provide evidence of the degree of adult implementation of identified research-based strategies in the district and school plans. The data collected through these processes, along with the student performance data from the common formative assessments, will be the basis of facilitated conversations by collaborative teams that will lead to changes in practice.

The following resource offers six options for a district. It is possible for a district to select multiple processes. Each uses a different format for collecting data and information and reporting information.

Also included is a description of the common elements of classroom observation and limitations to them.

Classroom Observation Overview

District Walkthrough Protocol (Generic)

Learning Rounds Model

Supervisor Observation and Teacher Self-Report Using a Common Rubric Based On Instructional Design Questions

Observation Checklist Protocol

Selected Document Review

Focused Academic Review

Resource 27: Monitoring Processes for Adult Implementation September 2010 OIP Resources

CLASSROOM OBSERVATION OVERVIEWSystematic classroom observation is a quantitative method of measuring classroom behaviors from direct observations that specify both the events and behaviors that are to be observed and how they are to be recorded. Generally, the data that is collected from this procedure focuses on the frequency and duration with which specific behaviors or types of behavior occurred in the classroom. The following elements are common to most observational systems:

A purpose for the observation, Operational definitions of all the observed behaviors, Training procedures for observers, A specific observational focus, A setting, A unit of time, An observation schedule, A method to record the data, and A method to process and analyze data.

Although there are several types of observational procedures or techniques that have been used to examine effective teaching (e.g., charts, rating scales, checklists and narrative descriptions), the most widely used procedure or research method has been systematic classroom observation based on interactive coding systems. These interactive coding systems allow the observer to record nearly everything that students and teachers do during a given time interval. These interaction systems are very objective and typically do not require the observer to make any high inferences or judgments about the behaviors they observe in the classroom. In other words, these low-inference observational systems provide specific and identifiable behaviors that observers can easily code.

There have also been several criticisms and cautions related to the use of structured observation techniques:

(1) Theoretical Concerns: Selection of events or behaviors may not be clear to anyone except the observer or instrument developer; teaching behaviors are often viewed independent of the curricular context with which the techniques are associated, i.e. observers generally focus on isolated behaviors without concern for the preceding and subsequent behaviors that provide the context and meaning of the behavior; and it is difficult to record complex instructional behaviors.

(2) Methodological Concerns: Observer effects may occur because teachers and students are aware that their behaviors are being observed; reliability and validity of observational systems; amount of time and number of observations to obtain reliable and valid measures; and generic observations (not specific to the plan focus)

(3) Pragmatic Concerns: Extensive training and time; access to schools and classrooms to conduct observations; and misuse of classroom observation data.

The previously mentioned criticisms and limitations, however, do not necessarily detract from the value and utility of the observational method. There are examples of technology supported observation systems for conducting short, frequent, formative classroom walkthroughs. Two common systems are the iObservation™ system (Marzano) and Power Walkthroughs (McREL). Both systems, which can be customized, provide forms with content-specific language and data reports that show trends and patterns.


DISTRICT WALKTHROUGH PROTOCOL (GENERIC)

A WALKTHROUGH IS: A WALKTHROUGH IS NOT: A process for giving and receiving non-threatening evidence-

based feedback to stimulate dialogue A strategy for promoting collegiality and breaking down isolation A snapshot of teaching and learning An integral part of the work with improvement plans and

accountability A powerful tool for continuous improvement A source of process data that answers the question “How are

we doing?”

A “gotcha” opportunity for supervisors or peers

A “dog and pony show” An evaluation of anyone, especially

individual teachers An isolated event An audit

DESIRED RESULTS OF WALKTHROUGHS

Frequent, formative progress High-quality feedback about teaching and learning Collegial dialog and collaboration Data-based decision-making Action planning with measurable results Focused professional learning On-going school improvement Enhanced ownership by all

WALKTHROUGH PROCESS – DLT RESPONSIBILITIES

BEFORE THE WALKTHROUGH IMPLEMENTING - DURING THE WALKTHROUGH

AFTER THE WALKTHROUGH

Introduce Walkthrough Protocol and Process to BLTs

Provide district-wide training/communication

Focus on identified curricular and instructional expectations as identified in the district plan

Identify configuration of walkthrough teams (e.g., DLT, other district staff, external partners, cross-building teachers and administrators)

Create a schedule: a minimum of one walkthrough per marking period

Walkthrough team meets with BLT to review the Walkthrough Protocol

BLT gives the walkthrough team an overview of current building conditions

Logistics are discussed, including observation form, classroom maps, scheduling and assignments

Conduct walkthroughs, e.g., first semester internal, second semester external, that are focused on the identified curricular and instructional expectations

Collect evidence using the focus questions/expectations on protocol

Walkthrough team analyzes observations with one another and generates building summary report before presenting to the BLT

Hold a meeting with BLT to share walkthrough summary report and plan next steps

BLT prepares a next steps report in response to the observation data

BLT shares walkthrough data and next steps with all building staff within two weeks

Option: DLT expands walkthroughs to include others, e.g., higher education, community, business partners, parents or a combination of participants

For follow-up support, DLT assigns partner schools to meet specific needs

Walkthrough Reminders for DLTs: * Be professional * Be courteous and respectful * Build trust and maintain confidentiality; don’t gossip *Be as unobtrusive as possible when visiting classrooms and asking questions *Enjoy the opportunity to grow professionally


LEARNING ROUNDS MODEL

Observation, which is often done by an individual, usually offers a snapshot of current practice and does not always lead to a better understanding of strategies which can be employed to effect large-scale practice improvement. Some districts and schools are now developing opportunities for peer observation, or observation where a colleague(s), regardless of role, conducts the observation. Learning Rounds borrows from the system of hospital ‘medical rounds and owes its inspiration to Richard Elmore of Harvard University. Using a ‘medical rounds’ model, Elmore brings groups of staff to investigate specific aspects of school improvement in order to generate strategies that can be used to improve practice. The major benefits include:

A focus on generating evidence-based solutions to problems/concerns faced by teachers, schools and districts around issues of system-wide improvement,

An opportunity for colleagues to get close to instructional practice as part of a supportive and joint learning inquiry,

A collaborative learning process to assist the group to explore practice and challenge assumptions, and

A unique and powerful form of personal PD for each colleague that will develop a self-reflective approach.

Learning Rounds is a process that is designed to deliver system-wide improvement across schools and authorities. It involves groups of staff observing and learning about and from school practice. The unique feature of this approach is that observation and inquiry are not carried out by a lone individual but by a team of colleagues who create a base of evidence around what they have seen. This evidence is descriptive only. It is not shaped by evaluative comments or value-laden points. While the collection of evidence can be done through talking to the teacher and pupils, looking at pupil work or doing a ‘walk-thru’, the most common form of inquiry is through lesson observation. It is important to note that everyone involved in this process, whether observing or being observed, are volunteers, and that the focus is on the learning of the observing group, not on the feedback given to an individual observed teacher.

Learning Rounds involves a team of staff visiting a school to explore an issue that has been identified as important, e.g., implementing research-based instructional practices. Groups of staff watch a sequence of class lessons from different teachers over part of a day. For instance, a group of three colleagues might spend the morning in a school observing segments of lessons, focusing on an agreed area of practice, such as the level of challenge through teacher questioning. If more people are involved they can be split into two groups and each group can visit a different class for the first half of the lesson and then swap round for the second half. This alternating can continue through the morning until a number of episodes of class teaching have been observed. By the end of the morning, both groups will have seen the same classes with the same teacher but at different times.

Each set of observations is followed by rich discussion in the afternoon that develops further learning. Here members of the group share their collective experiences. Because staff have observed the same lesson doesn’t mean that they have understood and interpreted the same things. It is this process of joint reflection that is so important to the learning of all involved and moves Learning Rounds away from the tradition of an observer giving an observed teacher advice and guidance, onto the equally important area of shared learning, and of next steps


which are identified from agreed evidence. It is advisable to have someone facilitate to give structure to the whole process. This person:

Mediates the conversation Guides the group to key questions Does not give solutions, and May introduce evidence that has been missed.

At this stage group members are not allowed to give opinions or suggestions as Elmore has found that this can lead to early disagreement and get in the way of next steps. Instead the process concentrates on getting consensus on the exact nature of the evidence in order to provide an agreed and descriptive platform to generate improvement. The visiting team then presents their evidence to the BLT. At this stage there is no evaluative comment, no suggestions as to next steps or what should be considered for change. The team’s evidence is all about what has been observed and only what has been observed.

In response to the evidence the BLT draws up a list of next steps which is presented on a return visit of the team. In this joint forum, DLT/BLT members discuss the next steps and offer evaluative opinions on the plans. This meeting is designed to be a significant driver for change in order to lay a platform for improvement and can lead to an amended plan which continues to have the engagement of the team and which can change in light of new evidence though further optional visits.

The emphasis throughout is on an evidence-based process, which leads to a continuous updating of practice. The key is that a group of colleagues are learning in a joint inquiry through mediated discussions and support. As a result of being involved in the process, colleagues, both those observed and those observing, develop and deepen their understanding of how to deliver effective system-wide change. This has major benefits to the school(s) visited as well as those doing the visiting. More importantly the process generates evidence that can be applied across a number of schools.


SUPERVISOR OBSERVATION AND TEACHER SELF-REPORT USING A COMMON RUBRIC BASED ON INSTRUCTIONAL DESIGN

QUESTIONSMarzano & Associates recommends a model for monitoring instruction in which every teacher is provided with feedback regarding the effectiveness of their instruction with the intent of capitalizing on strengths and improving on weaknesses. They emphasize that the criterion for effective teaching should be student learning as opposed to the rigid use of instructional strategies. The basis of this approach is that while individual teachers should be allowed flexibility in the instructional strategies they employ, all teachers should be expected to produce “learning” in their classrooms. To this end, it is recommended that data are systematically collected on students as well as teachers. For teachers, data would be collected through supervisor observation and teacher self-report on the extent to which the teacher uses elements of an instructional model. Marzano proposes that ten design questions be used with a rubric developed for each one. Following is a modified example from Marzano’s 2008 white paper of what a rubric for one of the ten design questions would look like and how it would be applied as a monitoring tool.

Teacher Rubric for Design Question 1—What will I do to establish and communicate learning goals, track student progress and celebrate success?

Score 4.0: In addition to score 3.0 behaviors, adaptations that enhance students’ learning.

Score 3.0: While engaged in classroom activities that involve establishing and communicating learning goals, tracking student progress and celebrating success, the teacher makes no major errors or omissions regarding the following behaviors:

o Presents students with a clearly defined scale or rubric for each learning goal,o Allows students to identify their own learning goals in addition to those

presented to them,o Designs and administers formative assessments for each learning goal,o Displays progress on learning goals for the whole class and facilitates

students tracking their own progress, ando Recognizes individual student status and progress as well as the whole class.

Score 2.0: No major errors or omissions regarding the following simpler behaviors:o Makes a distinction between learning goals and learning activities,o Presents learning goals, but does not design a scale for each,o Designs and administers assessments for each learning goal, but does not use a

formative system,o Tracks student progress, but does not facilitate students tracking their own progress

or does not display progress for the whole class, ando Recognizes and celebrates individual status and progress or group status and

progress, but not both.

Score 1.0: With help, partial success at some of the simpler behaviors (score 2.0 performance) and some of the more complex behaviors (score 3.0 performance).

Score 0.0: Even with help, no success with the score 2.0 or 3.0 behaviors.

Using rubrics such as that above, a profile of each teacher is compiled through teacher self-report and observations by supervisors. Regarding self-reports, teachers can rate themselves on a systematic basis and compile these ratings. Additionally, supervisors can make systematic observations of teachers. These two sources of data can be combined to construct a profile for each teacher regarding their use of the instructional model.


Teacher Code Design Question: What Will I Do to……… Pre/

PostEngage-ment

Student Learning

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10

Esta

blish

and

com

mun

icate

lear

ning

goals

, tra

ck st

uden

t pro

gres

s and

ce

lebra

te su

cces

s

Help

stude

nts i

nter

act e

ffecti

vely

with

ne

w kn

owled

ge

Help

stude

nts

prac

tice

and

deep

en

their

und

ersta

nding

of n

ew kn

owled

ge

Help

stude

nts

gene

rate

and

test

hypo

thes

es a

bout

new

know

ledge

Enga

ge st

uden

ts

Esta

blish

clas

sroo

m ru

les o

r pr

oced

ures

Reco

gnize

and

ack

nowl

edge

cla

ssro

om ru

les a

nd p

roce

dure

s

Esta

blish

and

main

tain

effe

ctive

re

lation

ships

with

stud

ents

Com

mun

icate

high

exp

ecta

tions

for a

ll stu

dent

s

Deve

lop e

ffecti

ve le

sson

s or

ganiz

ed

into

a co

hesiv

e un

it

Teacher ASupervisorTeacher BSupervisor

School Avg.District Avg.

A report similar to the one above would be generated for each school within a district. Each pair of rows in the chart represents the self-report and supervisor report data for a specific teacher regarding the design questions. The teacher provides self-report scores for each of the 10 design questions using rubrics. Supervisor ratings are right below teacher ratings. Supervisors would identify a few design questions based on what the district and/or building determined were priority based on the improvement plans. These may change as the plan is revised. For example, in Tactful Local, questioning was one of the instructional strategies identified in the district and school plan. Therefore, the principal in Tactful Local Elementary would select question 4 and possibly question 10 to observe. Note that teacher names are not used. This is primarily because the purpose is to aggregate school-wide and district-wide data, not to evaluate individual teachers. Also note that the last two rows report district and school averages. This allows comparison of individual teacher profile data with that for the entire school and the entire district. To complete the profile for each teacher, student data must be collected. It is recommended that for each type of data, all teachers use a common metric, such as a 0-4 scale. The following three types of student data are recommended:

1) Pre-test/post-test data from common formative assessments for a specific unit of instruction, 2) Student self-report engagement data, and 3) Student self-report on their learning.

The last two columns in the above chart records class averages for each teacher on the three types of student data. Comparison between individual teacher profiles and school or district averages should form the basis for discussion between teachers and supervisors. More specifically, individual teachers in consultation with their supervisors should identify specific goals for improvement. The focal point of such deliberation should always be student learning and engagement.

More information can be found at www.marzanoresearch.com


http://www.marzanoresearch.com/

OBSERVATION CHECKLIST PROTOCOL

Developing an observation checklist is not as easy as going to the Web and finding one that the DLT/BLT likes. The actual instrument used for the observation must be specific to the curriculum, instructional and assessment practices identified in the district and/or school plans. Steps to follow in creating an observation checklist are as follows:

1) Agree on the purpose of the observation, e.g.,

Contribute to the development and improvement of teaching and learning, To develop the teacher as a reflective practitioner, and Provide a set of data for the purpose of self-assessment which enable the

district/school to measure its overall performance against the district indicators.

2) Identify how observers will receive appropriate training/instruction to ensure validity, consistency and objectivity in the process.

3) Determine a schedule for observing all teachers.

4) Decide feedback mechanisms, individual and/or group, e.g., grade level. Feedback may include:

Key strengths and opportunities for improvement observed, Specific information about types of instruction, student engagement, bloom’s

taxonomy, content, etc., and Specific information relative to the “Look Fors.”

5) Create a standard observation form for recording data. Be sure data collected can be aggregated across classrooms.

a) Agree on the “Look Fors”.

b) Identify background information that should be collected, e.g., name of teacher, location of class, grade level, date, start/end time, number of students, gender or racial make-up of class, subject or course.

c) Identify contextual background or activities necessary, e.g., description of classroom lesson observed; classroom setting including seating arrangement, teacher location and space.

d) Identify any information, e.g., lesson plan, which needs to be collected prior to observation.

e) Determine coding or rating of “Look Fors”. Options in addition to those provided as examples below, are to mark evident/not evident or observed/not observed.


Example of Rating

Rating “Look Fors” by different categories, e.g., from 1 (not at all) to 4 (to a great extent). Consider adding a place to record “supporting evidence” for the rating. Since any one observation is not likely to provide evidence for every single “Look For”, use “DN” (don’t know) for when there is not enough evidence to make a judgment. Use “NA” (not applicable) when the indicator is inappropriate given the purpose and context of a lesson. For example,

Look Fors…. 1 2 3 4 DN NA

The design of the lesson encouraged a collaborative approach to learning among the students.

Adequate time and structure were provided for “sense-making.”

The teacher’s questioning strategies emphasized higher order questions.

The teacher appropriately used “wait time.”

The teacher displayed an understanding of mathematics/science concepts (e.g., in dialogue with students).

Intellectual rigor, constructive criticism and the challenging of ideas were evident.

Examples of Coding

Types of Instruction

L Lecture/presentation CL Cooperative learning (roles) T Transition time LC Learning center/station SP Student presentation (formal) TIS Teacher interacting w/ student D Discussion LWD Lecture with discussionCD Class discussion A Assessment (describe) WW Writing work (if in groups, add SGD) AD Administrative tasks RSW Reading seat work (if in groups, add

SGD) OOC Out-of-class experience

HOA Hands-on activity/materials I Interruption SGD Small group discussion (inc. pairs) UT Using technology and/or digital

education mediaBe sure to include an “other” with “please describe.”

Student Engagement LE low engagement, 50% or more of the students off-task HE High or mixed engagement,51% or more of the students engaged

Cognitive Activity (Bloom’s Taxonomy) K Knowledge (e.g., observation and recall of information, knowledge of dates, events and places) C Comprehension (e.g., understanding information, translating knowledge into new context) AP Application (e.g., use information, methods, concepts, and theories in new situations) AN Analysis (e.g., seeing patterns, organization of parts, identification of components) S Synthesis (e.g., generalize from given facts, predict, draw conclusions) E Evaluation (e.g., make choices based on reasoned argument, verify value of evidence)

For example:Look For….. Type of

InstructionStudent Engagement

Cognitive Activity

The design of the lesson encouraged a collaborative approach to learning among the students.

Adequate time and structure were provided for “sense-making.”


The teacher’s questioning strategies emphasized higher order questions.

The teacher appropriately used “wait time.”

The teacher displayed an understanding of mathematics/science concepts (e.g., in dialogue with students).

Students provided constructive criticism to each other.

Students challenged ideas.

f) Identify the timeframe for which observation data will be recorded, e.g., in five minute increments, after whole lessons, or at the beginning, middle and end.

6) Synthesize ratings for each “Look For” across observers (if more than one observer is assigned to the same classroom) and across classrooms. Consider including a section called “Supporting Evidence for Synthesis Ratings” that identifies factors most influential in determining a synthesis rating and to give specific examples and/or quotes to illustrate those factors.

7) Create a way to visually represent the data, such as on a graph or chart. Data needs to be collected in short intervals, e.g., once per week, at the building level in order to provide sufficient information for trend analysis within a month (based on the monitoring schedule the building has set) or across a quarter (based on the monitoring schedule the district has set). Monitoring procedures generate a large quantity of data that must be systematically recorded and analyzed. A visual representation of data such as a graph or chart provides a visual depiction of expected (as described on the graph by the indicator line) and actual student performance.

8) Define a process for mid-course correction prior to implementation of the actions/tasks Course correction should allow for the systematic interpretation of trends with regard to progress toward the identified strategy. Course correction should include a rule for raising the strategy indicator if teacher practice exceeds expectations. Conversely, there also should be a rule which prompts the alteration of the actions/tasks if insufficient progress is demonstrated.

If an action must be modified, only one instructional factor at a time should be altered. If more than one factor at a time is modified, and if teacher practice subsequently changes, the collaborative team will not be able to determine which factor was responsible for the change. Small changes or refinements in plans should usually be attempted before major alterations in the plan are made. At the same time, however, the modifications must be ones that are judged to be sufficiently substantial to result in improved performance.

Whenever a modification in the plan is made, this modification should be noted on the graph. A vertical line should be drawn on the graph at the point in time when the plan was changed. The collaborative team meeting notes should record the specific procedures and materials that are modified. This will assist in determining what action/tasks should be generalized across a building/grade level/district.


SELECTED DOCUMENT REVIEW

Sometimes referred to as a desk audit, a district can review documents using a structured review sheet to monitor building level implementation of its plan. Documents also can be reviewed as part of a targeted academic review. Documents typically reviewed are:

BLT Minutes/Notes Collaborative/Teacher-Based Team Minutes/Notes Formative Assessment Results Lesson Plans Building Level Observation Summaries PD Agendas, Materials and Results Other Documents Specific to Areas in a SIP, e.g., attendance records

When reviewing the above documents, the following provides guidance on what to look for:

BLT Minutes/NotesThe reviewer may ask to see meeting minutes/notes for a designated period of time, e.g., monthly, quarterly. The reviewer is looking for evidence of:

Regular meeting schedule; Full and active participation of members; A summary of the discussion, decisions and assignments after each meeting that

demonstrate progress and completion of assignments; Regular monitoring of adult implementation and student performance indicators; Use of monitoring data to make course corrections; and Reports on plan action implementation, including evidence to support level of

implementation.

Collaborative/Teacher-Based Team Minutes/NotesThe reviewer may ask to see meeting minutes/notes for a designated period of time, e.g., once per month for each teacher-based team, sequence of minutes/notes for a designated month early in the school year, mid school year and late in the school year. The reviewer is looking for evidence of:

A regular meeting schedule; A summary of the discussion, decisions and assignments after each meeting that also

demonstrate progress and completion of assignments; Full and active participation of team members; Analysis of student work and data from (common) formative assessments; Implementation of instructional strategies consistently across all classrooms; Use of protocols to guide meeting discussions; Use of monitoring techniques for determining implementation of the plan’s designated

instructional strategy(ies); and Discussion of levels of progress and activities to respond accordingly to the progress, or

lack thereof.


Formative Assessment ResultsThe reviewer may ask to see common formative assessment data by month/quarter, pre-interim-post, year to year, etc. The reviewer is looking for evidence of:

Charts of (common) formative assessment data (classroom, building) relative to priority standards/skills identified in the plan;

Comparison of data (student, classroom, building) across time; Trend data (classroom, building) across time; Analysis of data (classroom, building); and Interpretation of data (classroom, building).

Lesson PlansThe reviewer may ask to see lesson plans for a designated week(s) of the year for the different subject areas or grades. The reviewer is looking for evidence of:

Planning across the entire school and/or district; Use of the district/school lesson structure, e.g., review, anticipatory set/class opener, whole

class - direct instruction/modeling, guided practice, small group learning/independent work, closure/student reflection;

Implementation of state standards and district curriculum; Use of district and/or school plan designated instructional strategies; Implementation of district and/or school formative assessments at the appropriate time; Consistent adherence to the pacing and scheduling of academic program delivery, e.g.,

reading program, mathematics program; Collaboration between classes, across subject areas and/or interdisciplinary work between

subject areas; and A connection between what happens in a school from one grade to another and from school

to school at the same grade/subject level.

Building Level Observation SummariesThe reviewer may ask for monthly observation summaries from each building/classroom. The reviewer is looking for evidence of:

A consistent process for making observations, including who conducts the observations, when they are conducted and a common tool(s) for collecting data;

Use of district “Look Fors” as the basis for the observations; Analysis of observation results; Interpretation of observation results; and Use of observation results to make course corrections.

PD Agendas, Materials and ResultsThe reviewer may ask for all PD information or choose one specific area of focus. The reviewer is looking for evidence of:

A variety of opportunities for teachers/staff to learn, apply and receive feedback on the targeted area(s); and

Evaluation results that show that what is learned is being consistently used in all classrooms.

Other Documents Specific to Areas in a SIP, e.g., attendance records


FOCUSED ACADEMIC REVIEW

Targeted Academic Review, sometimes referred to as academic audit or quality review, information is offered as an option to provide insight into areas of demonstrated strength and weakness based on the district and school plans. Academic review is designed to be a formative or summative evaluation of a school’s progress in developing a culture of data-driven continuous improvement that focuses on increased student achievement. The process also provides:

A structured opportunity for constructive feedback, and A springboard for school improvement planning and continued self-evaluation.

The district will likely find as academic reviews are instituted over a period of time that conversations within schools about student performance and progress and differentiated student needs will become focused on every child. These conversations will lead also to developing strategies in collaborative teams and elsewhere to address the needs of all students. Academic review can take from one to three days, depending on the decisions that districts make during each part of the process. The Academic Audit contains three parts:

Part 1: Pre-Review WorkPart 2: On-Site VisitPart 3: The Academic Review Report

Part 1: Pre-Review Work

This part may include several components. The district will need to determine which components need to occur prior to the on-site visit and to what extent they need to occur. Components include:

A) School Self-Evaluation

The district may choose for the school to complete a self-evaluation prior to an on-site review. This form may be standardized for all schools or modified based on unique characteristics of the school, e.g., grade levels, status of improvement. Some standardized questions could be:

BLT Self-Evaluation Questions:

1) How do planned results compare to actual outcomes in student performance based on formative assessment data?

2) What is the impact of PD on teacher practice and student achievement?

3) To what degree has the school plan been implemented with integrity, on-time and within budget?

4) What lessons has the school learned as a result of implementing this plan (and how will they be applied to the next plan)?

5) What should the school do to institutionalize successes and eliminate unsuccessful practices?

The BLT should assume responsibility for submitting the responses to the designated district person, e.g., liaison assigned to the school. The BLT should reflect on each question posed by the district and provide evidence that supports the response. The BLT should draw on evidence


specifically identified in the task implementation details (Resource 29). The self-evaluation can be submitted in the form of a quarterly progress report (formative evaluation) or annual progress report (summative evaluation) to the district.

In addition to providing the standard questions all schools should use as the basis for its self-evaluation, the district also may want to provide guidelines so that the school responses are sufficiently detailed without becoming overwhelming for the school or district. These guidelines may include:

Use evaluative, rather than only descriptive, language. Focus the response on how these practices impact students and teachers.

Include references to where evidence of the self-evaluation can be found. Use bullet points to list multiple evaluative points. Limit the responses to no more than six pages. Draw on a wide evidence base, taking the views of all stakeholders into account. Be analytical, explaining the basis for actions and the resulting outcomes. Be evaluative, using selective examples to support the evaluation and link cause and

effect clearly. Explain succinctly how the school has tackled the areas for improvement.

B) On-site Visit Schedule

The DLT will need to establish a schedule for school reviews. The schedule can vary, e.g., all schools are visited within a year, staggered schedule based on specific criteria. When a school is scheduled for an on-site review, the DLT/school liaison should contact the principal approximately two weeks before the review date to indicate the review time. The BLT will develop a proposed schedule for the review. The district review team and principal will discuss this proposed schedule and make any necessary amendments during their pre-review communication. An on-site schedule that includes meetings, observations and other activities could look as follows:

Meetings B

LT T

eachers: group St

udents: group C

ollaborative team(s) P

arents/families

Observations Classroom visits (specific to content

and/or instructional focus areas) Collaborative team meeting(s) Building tour (optional) Before and after school activities

Other BLT debrief

(end of day 1) Feedback

presentation (end of the review with BLT and/or whole staff)

Additional evidence gathering

Collection of additional data

Academic Review- Two Day Sample ScheduleDay 1 Day 2

Time Activity Time Activity8:00 – 8:30 8:30 – 9:00 9:00 – 11:3011:30 –12.30 12:30 – 1:001:00 – 3:00 3:00 – 3:303:30 – 4:00

Meeting with BLTBuilding TourClassroom VisitsMeeting with Parents/FamiliesFlexible TimeClassroom VisitsStudent Group MeetingAfter School Activities

7:30 – 8:00 8:00 – 8:308:30 – 9:30 9:30 – 12:0012:00 – 12:30 12:30 – 1:301:30 – 3:00 3:00 – 3:30

Before School ActivitiesTeacher Group MeetingCollaborative Team MeetingsClassroom VisitsFlexible TimeCollaborative Team MeetingsClassroom VisitsDistrict Review Team Reflection


3:30 – 4:00 4:00 – 5:00

BLT DebriefDistrict Review Team Reflection

3:30 – 4:30 BLT and/or Staff Feedback Presentation

Considerations for developing the schedule include:

Starting and ending times of students and teachers; School and classroom schedules, e.g., reading block, math block; Percent of time spent in direct classroom observation versus other activities; Number of teachers, students and parents/families necessary for input; Benefits and limitations to group versus individual interviews; Use of common protocols or tools for consistent gathering of data and information

(observation and interview); Identification of an “on-site manager” of the process (possibly the district liaison); and Representative sampling of teachers, students, parents/families and collaborative teams.

C) Selection of District Review Team

Depending on the district size and capacity, it may need to draw reviewers beyond those available in the district, e.g., building level administrators and teachers, district partners, staff from institutions of higher education. It is important that each reviewer has a background that equips them for effective school review work. Some districts have reviewers sign and commit to a Code of Conduct that guides their work. The primary work of the review team is to:

Listen, observe and obtain an objective understanding of the school in relation to the district goals, strategies and indicators and the school actions, and

Provide the school with useful feedback for improvement.

In order to ensure consistency reviewers should receive training and on-going support in the appropriate use of district interview and observation processes and tools.

D) Orientation of District Review Team

The district should allow time to orient the district review team prior to the on-site review. It is not recommended that the review be conducted by an individual but a minimum of two individuals. Review team members may include staff from buildings other than the one being reviewed. The orientation should include a detailed discussion of the information provided in Part 1, a copy of the school plan, results of the building DF, and any recent progress reports submitted by the school. This also is the time when the on-site review schedule should be shared. This schedule should clearly show the classes identified for review, the room numbers and start and end times for the classes and subject areas. Prior to the school visit, a designated district person will need to develop a preliminary selection of classrooms that will be visited. Review team members may offer feedback to the review team leader about the selected classes, as well as reaffirm what the reviewer should expect to see in the classrooms, i.e., classroom “Look-Fors”.

Part 2: On-Site Visit

A) Classroom Visits

The focus of classroom visits is to look at student engagement, instructional practice, student work and student learning described in the district “Look-Fors”. Depending on how structured the district chooses to conduct the on-site review, each reviewer will either have a designated set of classrooms to visit or the reviewer will select which classrooms s/he will observe.


Generally, a reviewer will visit 10-15 classes during a typical two day visit and spend approximately 20 minutes in each class. The district may wish to establish some guidelines for the on-site visit. For example, should the reviewer ask the teacher for a lesson plan? When should the reviewer talk to teachers and students…openly, only if the opportunity presents itself or not at all? Should the reviewer give feedback to the teacher?

B) Document Review

The review team will need to determine which documents need to be reviewed during the on-site visit. This may include collaborative team meeting agendas/notes, curriculum plans, data and other school documentation. The types of data to review will be based on the district and school plan strategies and actions. Guidelines for conducting document reviews can be found elsewhere in Resource 36. Note that documents can be reviewed during Part 1 or Part 2 of the process.

C) Teacher Meetings

Teachers may meet as a group with one or more members of the review team. It is sometimes helpful to have at least two persons so that one individual can interview and the other one can serve as the recorder. The building administration should be reminded to provide coverage if a teacher doesn’t have a common preparation or planning time as the other teachers in the group. Generally, a group should be no more than 6 -8 teacher participants that represent the school’s range of teachers. The review team will want to consider whether the bargaining unit representative should be a member of a group, be interviewed independently or not at all. Interview questions should correspond to the district “Look-Fors” and plan actions.

D) Student Meetings

The reviewer team should select and meet with a group of approximately 8 – 10 students at a time, again focusing on the district “Look-Fors”. Another option is for the review team to select and have a short discussion with 2 – 4 students about their portfolios, notebooks or other student work as it relates to the specific curricular, instructional or assessment practices included in the focused plan. Interview questions should correspond to the district “look-fors” and implementation of instructional strategies; and student responsibility for/use of assessment results.

E) Collaborative Team Meetings

The review team may choose to interview collaborative teams and/or observe collaborative team meetings in action. The purpose of both is to understand how instructional decisions are made and how data is used to inform these decisions. The reviewer is looking to understand how the teamwork influences school-wide practices in the collection, analyzing and interpretation of data. The reviewer is looking for evidence that these meetings are aligned with goals, strategies and actions which the team addresses regularly.

F) Parent/Family Meetings

The BLT and/or principal will likely select this group for interview. A small number of parents, generally no more than 12, which represent the school’s diverse population and grade levels, will participate in a focused discussion with members of the review team. Consideration should be given to including a PTA/PTSA/PTO leader as a participant. Interview questions should focus on communication about the plan (e.g., development, implementation, monitoring) and the role


parents/families have in supporting the plan’s targeted curricular, instructional and assessment strategies and actions.

G) Review Feedback

Schools will be anxious to receive feedback; however, the review team may need time to assimilate all the information/data collected and/or obtain additional information or data before giving formal feedback. Providing provisional feedback should be given on the final day with a summation of the review protocol, provisional draft bullets on what the school does well, opportunities for improvement and any contingencies. Feedback may be given to the BLT and/or the entire staff. The review team may ask for comments. The review team should note that this provision report is subject to change before a report is returned to the school for an accuracy check. Timelines for completing the review should be given.

Part 3: The Academic Review Report

A) Report Format and Distribution

The Academic Review Report may take many forms. It can provide quantitative data, e.g., analysis of “Look-Fors” across classrooms, comparison of formative assessment results over a period of time compared to overall district performance or indicators. It can be qualitative, e.g., narrative summary or bullets of strengths and/or opportunities for improvement. Or, the report can be a combination of quantitative and qualitative information. The report should be detailed enough to provide the reader with background information about the school; a summary of the school’s overall current performance, including strengths and areas for improvement; and a specific evaluation of how well it is doing based on the district strategy indicators. The report may either be sent to the BLT with a timeline for accuracy check and response or the district may choose to deliver the report via a meeting with the BLT to not only explain the report findings and verify accuracy but also to discuss changes the school could make and what the district can do to support the school in its improvement efforts. It will be important to the district to have a consistent way of reporting results so the data from all schools can be aggregated to gauge where the district as a whole is in relation to the plan indicators. The district should be reminded that the final report is public information so every effort must be made to ensure an accurate, well-written, balanced report.

B) Frequency and Differentiation

The district may choose to differentiate the frequency and type of review for schools. Considerations for differentiation may include:

Status of schools in corrective action or school improvement, Schools with new principals/administration, Schools that have been re-configured, The type and number of strategy indicators that are not reaching the desired level of

progress, or Schools with significant staff turn-over.


Resource 28: Recording/Charting Monitoring DataThe following resource offers sample templates for recording monitoring data, both for student performance and adult implementation indicators. These forms can easily be adapted for use in the following ways:

a) Collaborative teams can complete the forms on a specific schedule, e.g., monthly, and submit them to the BLT or DLT, as appropriate. Charts/graphs can be created that show differences/similarities between classrooms/departments. The BLT can aggregate data to show total numbers/percents for the building.

b) BLTs can complete the forms on a quarterly basis and submit them to the DLT. The DLT can then create graphs that show differences/similarities between buildings, grade levels, etc. The DLT can aggregate data to show total numbers/percents for the entire district.

c) Teacher-based teams can use the form aligned to the 5-Step Process for reporting to the BLT at the end of the process and/or they can summarize the data on the form according to a schedule prescribed by the BLT.

A) BLT Summary Report BUILDING NAME BUILDING ADMINISTRATOR/PRINCIPAL DATE

BLT Membership Name(s) Representation

o Principal/building level administrator(s)

o Teachers

o Non-administrative certified staff

o Non-certified staff

o Parents

o Businesses and/or community organizations

o Teacher association/union representation

o Chair/co-chair

IMPACT OF PROCESS: List dates and the primary topic discussed at each BLT meeting. Include 2-3 samples of meeting agendas/notes.

IMPACT OF PLAN: ACTION PLAN IMPLEMENTATION DATA: Include a copy of the completed IMM and respond to the following questions: TO WHAT DEGREE ARE THE ACTIONS BEING IMPLEMENTED AS DESIGNED WITH ACCURACY AND CONSISTENCY (FIDELITY OF IMPLEMENTATION)?

DESCRIBE HOW THE ACTIONS ARE CHANGING ADULT PRACTICES AND STUDENT ACHIEVEMENT.

TO WHAT DEGREE IS THE ALLOCATION OF RESOURCES (TIME, PEOPLE, MONEY) SUFFICIENT?

Resource 28: Recording/Charting Monitoring Data September 2010 OIP Resources

STUDENT PERFORMANCE DATA Goal Focus: Strategy Focus: Student Performance Indicator:

Grade Level

# of Teachers

# of Students Who Took Assessment # of Students Below Standard # of Students Approaching

Standard# of Students Meeting

Standard# of Students Exceeding

Standard

Base-line Date Date Date Base-

line Date Date Date Base-line Date Date Date Base-

line Date Date Date Base-line Date Date Date

SUMMARY NARRATIVE OF FINDINGS: Bullet any explanation of irregularities or variation in the data. Include charts as appropriate.

ADULT IMPLEMENTATION DATAADULT IMPLEMENTATION INDICATOR:

DATA AND INFORMATION FOR INDICATOR: Describe monitoring processes and results. Results may be provided in table format (e.g., teacher or student “look fors” by grade level and the degree to which the “look fors” were observed.

SUMMARY NARRATIVE OF FINDINGS (BULLETS OF KEY FINDINGS)

PROGRESS SUMMARY: Respond to the following questions:

1. IDENTIFY OBSTACLES THAT WERE CONFRONTED IN THE IMPLEMENTATION OF THE ACTIONS. WHAT DID THE TEAM DO? DID THAT MAKE A DIFFERENCE?

2. BRIEFLY SUMMARIZE FROM THE MONITORING EVIDENCE IF THE CHANGES MADE AFTER PREVIOUS REVIEWS WERE IMPROVING STUDENT PERFORMANCE AND ADULT PRACTICES.

3. DESCRIBE IF THERE HAS BEEN A STRONG CAUSE AND EFFECT RELATIONSHIP BETWEEN ADULT IMPLEMENTATION AND STUDENT PERFORMANCE? IN WHAT WAYS? WHY OR WHY NOT?

4. DOES THE TEAM BELIEVE THAT EACH ACTION WHEN IMPLEMENTED WITH INTEGRITY WILL COLLECTIVELY BUILD THE STRENGTHS NECESSARY TO REACH THE INTENDED STRATEGY?

CORRECTIONS MADE BY BLT TO PLAN BASED ON ABOVE DATA: List in bullet format changes made to the SIP with explanation.


B) TBT Monitoring Form Aligned to Ohio 5-Step Process Building Name/Grade: Content Area: Strategy Focus:

Step 1: Collect and Chart Data (after common formative pre-assessment has been given)

Instructions: Each teacher comes to the TBT meeting with these sections completed for their class.

Learning Target (Standard/Indicator) = Note: Students with IEPs are underlined. Students with multiple-risk factors are in bold.

Teacher# Students who took test

# Students Proficient & Higher = (Blue & Green)

# Students NOT Proficient = (Yellow & Red)

# and names of students that need high support with

intensive intervention and extra time to become proficient with

this contentRed - Intensive

# and names of students that need additional support to become proficient with this

contentYellow - Targeted

# and names of students that are proficient with this content

and could use reinforcement to reach mastery

Green-Advanced

# and names of students that have mastered this content and

need enriched learning activities

Blue-Advanced

Step 2: Analyze Student Work Specific to the Data (after completing an item analysis and examining student work, identify strengths and obstacles)

IF THEN (What are the next learning targets?)Strengths (The 3 highest scoring questions in my class were..)

1.

2.

3. Obstacles (The 3 lowest scoring questions in my class were…)

1.

2.

3.

Prioritized Student Needs:1. Enrich2. Reinforce3. Target4. Intensive

Steps 3 & 4: Establish Shared Expectations for Adult Implementation Specific Changes in the Classroom and Across All Classrooms


Step 3: Intervention Strategies …. Step 4: Adults who will implement the strategies by name

1. EnrichDelivers Instruction: Administers Post-Assessment:

2. ReinforceDelivers Instruction:

Administers Post-Assessment:

3. TargetDelivers Instruction:


4. IntensiveDelivers Instruction:


Note: Learning from team members, colleagues, consultants, etc. how to implement the specific intervention strategies in the classroom through job embedded professional development may be needed. If so,

Step 3: “IF we learn how to use this strategy well, we will need:

What (content to learn)

How (method, e.g., modeling, coaching, workshop, demonstration, co-teaching)

When (timeframe for learning within school day)

Follow-Up (continued support to internalize learning)

Step 5: Collect, Chart and Analyze Post-Assessment Data (after common formative post-assessment has been given)Instructions: Each teacher comes to the TBT meeting with these sections completed for their class.

Learning Target (Standard/Indicator) =

Teacher

# Students who took test

# Students Proficient & Higher = (Blue & Green)

# Students NOT Proficient = (Yellow & Red)

# and names of students that need high support with

intensive intervention and extra time to become proficient with

this contentRed - Intensive

# and names of students that need additional support to become proficient with this

contentYellow - Targeted

# and names of students that are proficient with this content

and could use reinforcement to reach mastery

Green-Advanced

# and names of students that have mastered this content and need enriched learning

activitiesBlue-Advanced

Pre

Post

Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post

Expected Outcome

Actual Outcome

Level of Implementation

Reasons Expected Outcome Did or Did Not Occur (Successes and Obstacles) – What

Recommendations Reflections


We Learned

Fully

Parti

ally

Not A

t All

Cont

inue

Impl

emen

tation

&

Get A

dditio

nal P

D

Aban

don

Selec

t Alte

rnat

ive

Adap

t

1. Enrich

2. Reinforce

3. Target

4. Intensive


Resource 29: Decision-Making Parameters for Plan Course CorrectionPurpose

This document defines the roles, responsibilities and authorities for decision-making within the DLT/BLT as it applies to making course corrections to the district and building plans, i.e., decisions to keep, adjust, change, delete, or replace strategies, actions or tasks.

Parameters and Instructions for Use

In the complexity of educational systems, almost all decisions the DLT/BLT make have an impact upon other staff or teams. Effective decision-making takes this into consideration and avoids unexpected impacts. Decision-making parameters help to make sound decisions and may change as the DLT/BLT learns from experience. Decision-making processes and districts and buildings are not static; they constantly evolve in response to internal and external influences and organizational learning. Consequently, the decision-making parameters are not a set of explicit rules, policies or procedures. The following is a basic set of parameters for decision making and a checklist that can be used by the DLT/BLT in reviewing its plan against the parameters in order to make course corrections. Consider all evidence (data and information) in order to make a judgment about each strategy/action at the designated time (e.g., quarterly for the district, monthly for the building).

Note: Add or delete rows or columns as necessaryInstructions: Identify the Goal Focus Area GOAL 1 FOCUS: GOAL 2 FOCUS:

Instructions: Identify the Strategy Focus Area: Strategy A: Strategy B: Strategy A: Strategy B:

Instructions: Insert the Action Number:

Instructions: For each of the following decision-making parameters place a “+” if information and data show a positive impact, “-“ if negative, “o” if neutral or undetermined.

1. Effect on Student Learning

2. Effect on Teacher Practice

3. Effect on Leadership

4. Time Requirements

5. Staffing Implications, e.g.,(re)assignment, teams

6. Funding Allocation/Consequences

7. Effect on Teacher, Administrator or Other Staff Learning (Professional Development)

8. Effect on Public Relations and/or Communication

9. Legal Implications/Compliance

10. Plan, Policy or Procedure Alignment

Based on the above review of strategies, actions and tasks, the following changes to the plan are recommended:

Resource 29: Decision-Making Parameters for Plan Course Correction September 2010 OIP Resources