the impact of the mmp on student achievement

The Impact of the MMP on Student Achievement

Cindy M. Walker, PhD

Jacqueline Gosz, MS

University of Wisconsin - Milwaukee

Primary Goal

• Within a school it is ultimately the teacher that is responsible for teaching mathematics that has the greatest capacity to impact student achievement in mathematics.

• How is the MMP influencing what is going on in mathematics classrooms across the district?

Structure of MMP in MPSMath SpecialistsMath Teacher LeadersLearning TeamsLiteracy CoachesMath TeachersStudent Achievment

Concerns

• Much of the effect of the MMP is not under the direct control of the MMP

• The effect of the MMP on student achievement depends on– School Climate– Quality of Learning Team– Quality of MTL– Support of other Teachers

Solutions

• Capture school level variability in variables that are not under the direct control of the MMP

• Compare and contrast schools to determine if under the best conditions there is an increase in student achievement in mathematics

Methodology

• On-line survey made available to all MPS participants in the MMP that may have an effect on increasing student achievement in mathematics including– Learning Team Members– Literacy Coaches– Math Teacher Leaders– Math Teachers

Participation Rate

• A total of 1,107 MPS employees from 129 schools in the district responded to the survey

• 93% of the elementary schools, 90% of the middle schools, and 9% of the high schools were represented in varying degrees.

Participation Rate by Role

School N LC LT MTL MT %MT

Elementary 1,007 38 228 95 645 28.4%

Middle 87 9 19 18 41 22.2%

High 13 0 2 2 9

Instrumentation

• Six primary areas were evaluated1. Professional Development

2. School/District Functioning

3. Learning Team Functioning

4. Classroom Practice

5. Role of the MTL

6. Role of the LC

Professional Development

• How involved were respondents in organized professional development activities geared toward mathematics this year?

Role in the MMP N Mean SD

Literacy Coach 44 16.06 16.08

Learning Team Member 56 10.67 13.10

LT Member & Math Teacher 123 15.15 19.16

Math Teacher Leader 82 54.19 33.21

Math Teacher Only 513 12.36 15.07

Professional Development• How often did respondents participate

in professional learning experiences related to the teaching and learning of mathematics this year?

Role of Respondent N Mean SD


Learning Team Member 56 3.22 2.83

LT Member & Math Teacher 123 4.42 3.08




• How involved were respondents in informal professional development activities geared toward mathematics this year?



Learning Team member only 74 38.41 56.32

LT member & Math Teacher 162 85.09 101.58




• How useful were professional development activities, in terms of helping teachers improve their classroom teaching of mathematics?


Learning Team Member 162 3.08 .53

Math Teacher Leader 110 3.39 .46

Math Teacher Only 654 2.91 .52

School Functioning

• How strong is the focus on increasing student achievement in mathematics in MPS schools?

There is a Strong Focus on Increasing Student Achievement in Mathematics at My School

Role of Respondent SD D A SA

LC (n = 47) .0% 17.4% 32.6% 50.0%

LT (n = 83) 1.3% 6.3% 38.8% 53.8%

LT_math (n = 170) 4.2% 4.2% 34.3% 57.2%

MTL (n = 126) 3.2% 12.0% 40.0% 44.8%

MT (n = 692) 3.7% 8.2% 42.9% 45.2%

School Functioning• How cohesive are the mathematics

curriculum and pedagogical approaches used in MPS schools? SD D A SA

Teachers have different expectations about what students can learn

7.6% 42.0% 37.9% 12.5%

Teachers often individually choose the content they teach

12.6% 42.1% 36.0% 9.3%

Teachers are encouraged to use similar instructional practices

3.2% 13.8% 58.1% 24.9%

Within each grade, teachers are expected to follow the same curriculum

3.1% 8.4% 51.9% 36.6%

Across grades, teachers are expected to follow the same curriculum

3.9% 19.4% 48.9% 27.9%

School Functioning

• To what extent do respondents participate in school improvement efforts?


Literacy Coach 45 115.20 76

Learning Team Member 75 103.96 80

LT Member & Math Teacher 160 130.20 81

Math Teacher Leader 120 133.18 72

Math Teacher Only 659 81.61 70

School Functioning

• How aligned do respondents feel their mathematics program is with standards and assessments?

D A SA

My school’s math program is aligned with MPS learning targets

9.2% 57.5% 28.6%

My school’s math program is aligned with WI state standards

8.9% 58.2% 28.2%

My school’s math program is aligned with the Comprehensive Math Framework

10.5% 59.0% 23.5%

My school’s math program is aligned with state/district assessments (WKCE/Terra Nova)

14.0% 56.6% 22.4%

School Functioning• What information is used to plan

and evaluate school improvement activities?

Terra Nova/WKCE

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Not at all To a smallextent

To a moderateextent

To a greatextent

LC (N=47)

LT (N=83)

LT_ MT (N=170)

MTL (N=126)

MT (N=692)

School Functioning

Student Work Samples

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


To a moderateextent

To a greatextent

LC (N=47)

LT (N=83)

LT_ MT (N=170)

MTL (N=126)

MT (N=692)

School Functioning

Classroom Observations

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


To a moderateextent

To a greatextent

LC (N=47)

LT (N=83)

LT_ MT (N=170)

MTL (N=126)

MT (N=692)

School Functioning

Information Learned About Effective Instructional Practices

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


To a moderateextent

To a greatextent

LC (N=47)

LT (N=83)

LT_ MT (N=170)

MTL (N=126)

MT (N=692)

School Functioning

Information Learned About Student Learning Processes

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


To a moderateextent

To a greatextent

LC (N=47)

LT (N=83)

LT_ MT (N=170)

MTL (N=126)

MT (N=692)

District Functioning• How much support do the district’s

learning targets provide?

– Regardless of role in the MMP, the majority of respondents felt that the district’s math learning targets

1. Provide clear information about what students are expected to know and be able to do and

2. Drive their school’s math improvement agenda

District Functioning• How much support do the district’s

math teaching specialists provide? – The majority of LC, MTL, and LT

members that do not teach math felt that the district’s math teaching specialists provided ample support

– Nearly 40% of respondents who teach math teachers did not feel that the district’s math teaching specialists provided ample support. Most likely because they have the least amount of contact with them

Learning Team Functioning

• Who sets the agenda for LT meetings?– 76.5% of LT members indicated that

the principal sets the agenda – 61.6% indicated that the LC sets the

agenda– 37.4% reported that the MTL sets the

agenda– 28.9% reported that someone other

than the LC, principal or MTL set the agenda


• Who provides input into setting the LT agenda?– 86.3% of LT members respondents

reported that the principal provides input

– 77.3% of respondents reported that the LC provides input

– 69.7% of respondents reported that the MTL provides input

– 55.5% reported that others provide input


• How often is mathematics discussed at LT meetings? – 19.6% of LT members (n = 419)

reported that math was discussed all of the time

– 16.1% reported three-fourths of the time– 23.3% reported half of the time– 37.3% reported one-fourth of the time– 3.6% LT members reported never


• How often is mathematics a formal agenda item for LT meetings?

Role of Respondent

MTL(n = 123)

Other LT Member(n = 286)

Never 2.4% 2.1%

Some of the time 36.6% 22.7%

Most of the time 36.6% 32.5%

Always 24.4% 42.7%


• Does the LT regularly meet to plan and problem solve about issues surrounding mathematics?

Role of Respondent

MTL (n = 123)

Other LT Member (n = 292)

Strongly Disagree 6.5% 3.4%

Disagree 32.5% 18.8%

Agree 46.3% 47.3%

Strongly Agree 14.6% 30.5%


• What are the work priorities of LT across the district?– Much variability across the district– Supporting the implementation of

improvement efforts in math most often reported as high or top priority

– Monitoring the implementation of improvement efforts, developing and implementing staff development programs was least often reported as high or top priority

Learning Team Functioning• How often are specific math issues

discussed at LT meetings?Useful Discussions About How to Help Students Succeed

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Some of the time Most of the time Always

MTL

LT Member


Able to Accomplish Goals Related to Math

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


MTL

LT Member


MTL Shares Info Gained from District Meetings

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


MTL

LT Member


Collaborate to Construct and Implement Action Plan

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


MTL

LT Member

Learning Team Functioning• How much of an asset is the LC in

helping schools focus improvement in math?

LC Viewed as Asset for Increasing Achievment in Math

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Strongly Disagree Disagree Agree Strongly Agree

MTL

LC

LT

LT_MT

Learning Team Functioning• How much of an asset is the MTL in

helping schools focus improvement in math?

MTL Viewed as Asset for Increasing Achievment in Math

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


MTL

LC

LT

LT_MT


MTL and LC Collaborated

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


MTL

LC

LT

LT_MT

Learning Team Functioning• How collaborative is the LT?

– In general, the majority of all LT members reported that they usually felt comfortable expressing their opinions and the atmosphere of LT meetings was typically respectful, open, and friendly.

– To a lesser extent LT members reported a shared responsibility in decision making regarding the implementation of math programs and development in mathematics.

Classroom Practice

• What pedagogical approaches are being utilized in mathematics classrooms in MPS?– The majority of math teachers in the

district reported utilizing all of the pedagogical approaches described which were designed to on at least a weekly basis

– These approaches were designed to contrast didactic versus interactive approaches to mathematics instruction

Classroom Practice

• Compared to other math teachers, a significantly smaller percentage of MTLs reported regularly spending time – Having students solve problems using a

method taught by the teacher – Having students complete textbook

exercises for review

Classroom Practice

• A significantly greater percentage of MTLs reported regularly spending time having students– Analyze the similarities among

representations, solutions, or methods– Justify why a particular solution

strategy in mathematics worked– Work on math problems that had

multiple answers or solution methods

Classroom Practice• A significantly greater percentage of

MTLs and LT members that teach math reported regularly spending time having students– discuss mathematical ideas,

problems, or solutions in small groups– write explanations of math ideas,

solutions, or methods– practice computational skills

Classroom Practice• What content is being taught in

mathematics classrooms in MPS?

MTL(n = 113)

LT_MT(n = 170)

MT(n = 692)

Number concepts and computation

Less than 10 lessons 9.9% 7.2% 12.8%

More than 10 lessons 90.1% 92.8% 87.2%

Geometry Less than 10 lessons 27.5% 40.0% 47.7%


Measurement Less than 10 lessons 37.6% 55.1% 63.0%


Data analysis/statistics Less than 10 lessons 33.0% 41.4% 49.2%


Probability Less than 10 lessons 66.4% 72.6% 79.0%


Mathematical patterns/algebra

Less than 10 lessons 22.5% 36.5% 38.6%


MTL Functioning• How often do MTLs interact with

others to discuss teaching and learning in math?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

LT Other MathTeachers

LC Principal

None of the time One-fourth of the timeHalf of the time Three-fourths of the timeAll of the time

MTL Functioning• How supported do MTLs feel in

their role?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

LT Other Math Teachers LC Principal Math TeachingSpecialist

Strongly DisagreeDisagree Agree Strongly Agree

MTL Functioning• How closely are MTLs working with

Math Teaching Specialists?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Attends LT meetings Facilitates PD at School Dependable Resource

SD D A SA

LC Functioning• How involved is the LC in helping school

improve in mathematics?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Collaborated with MTLWorked with MathTeachers

Modeled MathTeaching

Modeled LiteracyStrategies During

Math

Covered MTL's Class

Never Once this year A few times during the yearOnce per month A few times per month 1-2 days per week or more

Discussion

• It is apparent that quite an impact has been made by the MMP in the majority of elementary and middle schools in MPS this year

• It is also apparent that there is quite a lot of school level variability in the level of impact that has been made in the district.

Discussion

• In some schools within the district, improving student achievement in mathematics appears to be a top priority

• In other schools in the district improving student achievement in mathematics does not appear to be a priority

Discussion

• This level of variability is to be expected with any educational improvement plan that is implemented on such a large scale

• Only by capturing such variability can the impact of the MMP on increasing student achievement in mathematics be evaluated

Discussion• Is there an increase in student

achievement in mathematics at schools in the district that are

1. Utilizing all the resources provided to them though the MMP?

2. Doing everything within their control to collaborate, plan and implement procedures designed to improve the teaching and learning of mathematics?

Discussion

• This is the direction in which the evaluation of the effect of the MMP on student achievement will proceed

• It is expected that the variability that was observed in school level implementation will help to predict increases in student achievement in mathematics

the impact of the mmp on student achievement

Documents