Continual Improvement Process

Oregon Department of EducationApril, 2012

AGENDA

GOAL: This informational session will review requirements for the evaluation of the 2011-12 improvement plans and some principles of Effective Planning for Continual District/School Program Improvement.

1. Review Rubric for evaluation of implementation of Title III Improvement Plans.

2. Lessons Learned3. Section A– Planning, 4. Section B Implementing, 5. Section C Monitoring.

2

3

SY 2007-08

Parent Notification SY 2008-09

• Parent Notification

• After Two years:

Improvement Plan; TA

SY 2009-10Parent NotificationAfter Two years: Improvement Plan; TA

SY 2010-11

•Parent Notification

•After four years: Modify Curriculum/ Instruction or

•Funding?

•Replacement of Personnel

4 yrs.2 yrs.

TIMELINE OF ACCOUNTABILITY FOR LEAS

3 yrs.

1 yr

4

Example of a timeline for Implementing Plans/Provisions

Implementation of 2 year plans / 4 year provisions(throughout the

year the subgrantee did not

meet AMAOs)

Development of 2 year

plans / 4 year provisions(mid fall)

AMAO letters sent to parents

(within 30 days)

AMAO notifications

sent to districts /

subgrantees(late summer /

early fall)

5

PROFESSIONAL DEVELOPMENT

GO

AL

S

INQ

UIR

Y

PLANNING

MASTER PLAN

FR

EQ

UE

NC

Y

MA

ST

ER

P

LA

N

Sustainable District/School Improvement

Cycle

EVALUATION

INSTRUTIONAL STRATEGIES

6

EVALUATION OF IP PLANS 2011-12

PROCESS:1. Follow Guidelines in Evaluation Rubric2. Evaluation will be the first item on the

submission of 2012-13 IP Plans3. Missing items from the evaluation will result in

returning the 2012-13 IP Plans for review.4. Why? The 2012-13 plans are a follow-up of the

2011-12 IPs5. All plans should be sent to Leslie Casebeer by

June 29, 2012.

EVALUATION RUBRIC – 3 GUIDELINES

1) Procedures (A series of actions or steps taken to achieve an end) are provided to monitor to assess the implementation for all activities set forth in the master plan;

2) The formative (short-term) evaluation procedures should seek to determine if the strategy had the expected effect on student achievement;

3) The summative (long-term) evaluation procedures should seek to determine if the goals and objectives (activities) have been attained.

Adapted from the Louisiana Department of Education7

EVALUATION RUBRIC

Guideline I. The evaluation procedures to monitor and assess the indicators of implementation for all activities should include  at least three of the four of the following criteria:1. What data instrument(s) was used to

collect information and what kind of feedback was obtained as a result of the data analysis (inquiry)?

2. What was measured or assessed, and how was this information used?

3. Who conducted the evaluation? Please include the position title, i.e. Curriculum Director, Principal, etc.

4. How often were the goals/activities monitored (frequency)?

8

EVALUATION RUBRICGuideline II. The formative (short-term) evaluation procedures should seek to determine if the strategy had the expected effect on student achievement.

• Did the evaluation procedures provide sufficient evidence to evaluate the short-term effects for each strategy on student achievement?

• Short-term effects may have included student portfolios, comparison of student’s work throughout the year, teacher-made tests, or other similar type evidence.

9

EVALUATION RUBRICGuideline III. The summative (long-term) evaluation procedures should seek to determine if the goals and objectives have been attained.

Did the summative evaluation adequately convey if the school/district is improving?

The summative evaluation should include the applicable testing instruments with descriptions of how they were used to determine if the goals and objectives were attained.

This evaluation should include a comparison and/or analysis of test data but may also include other types of assessment and/or qualitative data. 10

11

QUESTIONS

SCHOOL/DISTRICT IMPROVEMENT: THE LEADERSHIP AND LEARNING CENTER

Comprehensive Needs Assessment: Planning, Inquiry, SMART Goals

12

Monitoring: Master Plan, Frequency, Measuring Progress, Evaluation

Implementation: Research-based strategies, design of master plan, professional learning, parental engagement

COMPREHENSIVE NEEDS ASSESSMENT PLANNING

Assessment Results

Gather and reflect on school/district data: external and internal.

13

14

DATA PORTFOLIO

The following items could make up the Data Portfolio (to be kept on file at the school):

o ELPA data for the last five years.

o AYP report for ELs for the last three-five years.

o Summary of Findings of Survey Data and other source documents. (Teachers, Parents,

Students, Principal) If Parent sample size is inadequate, you might consider Parent Focus

Group(s).

o Summary of Findings of Interview Data and all source documents. (Principal, Counselor,

and Teachers)

o Summary of Findings of Focus Group Data and all source documents. (Teachers, Students,

and Parents)

o Copy of Data Triangulation

o Comprehensive Needs Assessment: Final Report

o Summary of internal assessment data

o Data Analysis (Trend Data history etc.)

o Cognitive Summary Data (PSAT, SAT etc.)

o Citation from monitoring of Federal Programs

Adapted from Improvement Planning Louisiana Department of Education,

15

DATA COMPREHENSIVE NEEDS ASSESSMENT: SUMMARY REPORT

: Rank-order the identified areas of strength (3-5) from the EL student

performance (cognitive data), data, and/or graduation index and indicate

the supporting data sources:

STRENGTHS DATA SOURCE/INSTRUMENT

1.

2.

3.

4.

5.

Adapted from Improvement Planning, Louisiana Department of Education

16

Part IIa. List the contributing factors from the cognitive, attitudinal/perceptual, data of the previously identified strengths:

CONTRIBUTING FACTORS TO THE STRENGTHS

DATA SOURCE

1. 2.

3.

4. 5.

17

DATA COMPREHENSIVE NEEDS ASSESSMENT: SUMMARY REPORT

Rank-order the identified areas of weakness (3-5) from the student

performance (cognitive data), dropout data, and/or graduation index and

indicate the supporting data sources:

CHALLENGES DATA SOURCE/INSTRUMENT

1.

2.

3.

4.

5.

18

DATA Part IIb. List the contributing factors from the cognitive,

attitudinal/perceptual, behavioral, and archival data of the previously

identified challenges:

CONTRIBUTING FACTORS TO THE

CHALLENGES

DATA SOURCE/INSTRUMENT

1.

2.

3.

4.

5.

The identified challenges will lead to the goals. The contributing factors of

the challenges will lead to the strategies.

19

Comprehensive Needs AssessmentPlanning

Teacher PracticesFocuses conversation on research-based best practices

Educators talk about their practices, share their knowledge and skills and support one another in the specific contexts in which they work.

Teaching Practices

Current Curricular Practices

________

(identify

practice)

________

(identify

practice)

________

(identify

practice)

Evidence of Practice (State in definitive/tangible terms)

Is the current practice research-based?

Is it a principle & practice of effective ELD programs?

Has the current practice been effective or ineffective? For how long?

What data source(s) do you have that support your answer? (identify all

applicable sources)

Evidence of effectiveness or ineffectiveness (State in terms of

quantifiable improvement)

Evidence of equitable school support for this practice. Do all teachers

have they need to faithfully implement the practice?

Next Step (changes or continuations)

Adapted from School and District Improvement Planning Template, Tennessee Department of Education

SCHOOL IMPROVEMENT PLANNING FORTHE SUCCESS OF ENGLISH LANGUAGE LEARNERS

Leadership Performance: Implementing Innovations:Develops, articulates, and communicates a shared vision of the intended change.

Investing in professional learning.Monitoring— Checking on progress.Providing continual assistance.Creating a context supportive of change.

21

22

STOP

23

QUESTIONS?

STRENGTHS: THE LEADERSHIP TEAMReflect and celebrate program strengths;Determine what is currently working well and why;

Assist in building program capacity in identifying strengths, areas which require attention to sustain strengths;

Foster introspection, reflection and analysis;

Lead to better planning with precision and intentionality;

Act as a catalyst for collaborative and collegial conversations about sustainability from within. 24

AREAS OF IMPROVEMENT: THE LEADERSHIP TEAM

Identify areas in need of improvement;Identify a “vital few” key areas upon which to focus improvement efforts ;

Acts as a catalyst for collaborative and collegial conversations about improvement from within;

Provides a forum for consensus building around program improvement;

Develops a deeper understanding of the unique improvement needs of schools and facilitates the communication of the improvement goals;

Identify possible cause and effect scenarios. 25

SMART GoalsSpecific and Strategic – Have you articulated precisely what you want to achieve and have priorities been strategically selected based on a comprehensive needs assessment?Measurable- Are you able to assess/ measure your progress?Achievable- Is the goal within your reach and within your control? Are targets ambitious yet attainable?Results-Based – Have established base-line data and targets of where you want to end up?Time-bound- What is the dead-line for completing your goal?

26

Specific and StrategicAsk questions such as: In what area are a significant number of

students experiencing difficulty?What specific aspect of this area of

concern would make the biggest impact for students if improvement occurred?

If students could change and do “X” in this area, would they benefit greatly – would it positively impact other areas of their learning?

If our goal was achieved and the specific area identified was significantly improved, what would students be doing? In other words, what would be the indicators?

27

Measurable

Ask questions such as.....

What tool(s) will best measure if targets have been achieved?

Can the tool(s) be used to establish a baseline?

What is the achievement target for your students?

28

Attainable

Ask questions such as..... Is what we are expecting reasonable?Do we have the capacity to make the

desired change?If “yes”, how do we most effectively use our capacity to make the changes?If “no”, how do we prepare ourselves so we have the capacity to make the changes?

29

Setting Attainable Targets

According to The Leadership and Learning Centre:If student performance is in the:

1st quartile the percentage increase should be at least 20%

2nd quartile at least 12%3rd quartile – around 7%Top quartile – around 4%

(Dr. Douglas Reeves, 2007)

30

Results -Oriented

Ask these questions...

Why is it important for staff to achieve this goal?

Have ambitious yet attainable targets been set?

31

Time Bound

Ask questions such as....

What is the timeframe for achieving this goal?

What strategies are in place to keep us on track (monitoring strategies and time)

32

Major Components of the Professional Development Plan:

SMART Goal:

Identify the focused professional development Planning - The Professional Development System must be based upon”:

Dates of delivery

Dates for systematic consultation with internal trainers, regional or state personnel designated to provide on-going technical assistance

Dates for monitoring of local professional development programs.

Analyses of student achievement data

Student needs Personnel needs (describe your processes for collecting data on personnel needs)

Adapted from North Carolina Improvement Planning

33

34

Major Components of the Professional Development Plan: SMART Goal: Identify the focused professional development

Delivery - “In-service activities for instructional personnel shall focus on:

analysis of student achievement data,

on-going formal and informal assessments of student achievement, identification and use of instructional strategies that emphasize rigor, relevance.

enhancement of subject content expertise (ELD and other core content areas),

integrated use of classroom technology that enhances teaching and learning.

35

PARENT PARTICIPATION

.

.

36

1. Convene an annual meeting at a time convenient for parents of Els:

All parents shall be invited and encouraged to attend

The school will provide information and explain the requirements of

Title III and the rights of parents

Activity/Task

(What will

be done )

Staff

Responsible

(Who will

be doing it)

Timeline

(When

it will be

done)

Steps

(How

it

will

be

done)

Accountability/Evaluations

(How success will be

measured)

Adapted from Effective Planning for Continuous Improvement, Ontario, Canada

.

.

37

3. Involve parents in an organized, ongoing, and timely manner, in the

planning, review, and improvement of the Title III program.

Activity/Task

(What will

be done )

Staff

Responsible

(Who will

be doing it)

Timeline

(When

it will be

done)

Steps

(How

it

will

be

done)

Accountability/Evaluations

(How success will be

measured)

38

4. Provide parents of participating children:

Timely information about the Title III programs

Description and explanation of the curriculum at the school, the forms of

academic assessment used to measure student progress, and the proficiency

levels students are expected to meet

Opportunities for regular meetings to formulate suggestions and to

participate, as appropriate, in decisions relating to the education of their

children

Activity/Task

(What will

be done )

Staff

Responsible

(Who will

be doing it)

Timeline

(When

it will be

done)

Steps

(How

it

will

be

done)

Accountability/Evaluations

(How success will be

measured)

39

Monitoring and Adjusting Evaluation

(Rubric Indicator Sec C)

Evidence of Monitoring Dates – Narrative response required

What are the calendar dates (Nov/Dec, May/June) when the

School/District Leadership Team will meet to sustain the

Improvement Planning Process? Identify the person(s)

responsible for monitoring and the role they will play in the

monitoring process.

Evidence of a Process for Monitoring Plan – Narrative response required What will be the process that the School Leadership Team

will use to review the analysis of the data from the

assessments and determine if adjustments need to be made in

our plan? Adapted from School Improvement Planning, Tennessee Department of Education

40

Monitoring and Adjusting EvaluationEvidence of a Process for Adjusting Plan – Narrative response required What will be the process that the School Leadership

Team will use for adjusting our plan (person(s)

responsible, timeline, actions steps, resources,

evaluation strategies) when needed? Evidence of a Plan for Communicating to All Stakeholders – Narrative response required How will the School Leadership Team communicate

success/adjustments of the plan to stakeholders and

solicit ongoing input from stakeholders?

41

QUESTIONS?

42

43

Title III Contacts

Kim A. Miller Education Specialist Education Improvement and Innovation kim.a.miller@state.or.us (503) 947-5712

Leslie CasebeerLeslie.Casebeer@state.or.us Phone: (503) 947-5648Fax: (503) 378-5156

44

Title III Contacts

Carmen West Program Specialist forTitle III, ESL, Bilingual Programs Education Improvement and Innovation carmen.west@state.or.us 503-947-5669

44