1

Developing an Evaluation Plan_____________________

The Mathematically-Connected Communities MSP

Developed for the February, 2008 - MSP ConferenceDr. Karin WiburgDr. Ted Stanford

MC2 – New Mexico

Evolution of a strong MSPIn the early days: Summer Academies – often disconnected from what teachers are expected to teach, only some schools and some teachersIsolated well-meaning efforts- diverse stakeholdersDecision to create a mathematically- connected community (8 research mathematicians, 8 math educators, a research & evaluation team)- 4th yearDecision to work with only those school districts who will be full partners in implementing math reform district-wide Decision to begin to systematically evaluate the effect of the MSP on teacher knowledge and student learning and achievement

MC22 (Mathematically-Connected Communities)

3

MC2 partners provide professional development and district leadership support throughout the state of New Mexico. You can learn more about MC2 and get many mathematics teaching resources, information on lesson study, and a copy of the latest evaluation plan by going to the MC2 web site

mc2.nmsu.edu

Need for Strong Research Models for Evaluation of Mathematics Reform Efforts

4

Evolution in New Mexico:1.Lots of descriptive research, who did what, when, numbers served, pd provided, some student scores. (systemic change initiatives) 2.Evolution of Teacher Action Research through Lesson Study – 3 years, Takahashi, Lewis 3.NSF required student outcomes studies on another project, we learned to connect PD to student achievement, evolution of mixed-effects model (Remmenga & Wiburg ) 4.An evolving focus on systems research and change of whole districts.

Developing our Evaluation Plan

The Overarching Question for MSP Professional Development Projects: What is the effect of our professional development on teacher learning, classroom teaching, and student learning? You need an evaluation team with qualitative and quantitative research abilities

Developing Research Questions

1. Does the MC2 PD partnership program positively affect student achievement (including all sub-groups) as measured on our state Standards-Based Assessment? 2. Does the MC2 PD partnership program result in growth in teacher pedagogical content knowledge?3. Does the level of implementation of the Professional Development in the classroom affect student achievement ?

Turn the Questions into Objectives and Design Measurements/ Analysis

1. Does the MC2 PD partnership program positively affect student achievement?Objective 1: Provide PD that supports teaching standards-based mathematicsMeasurements: demographic data, hours and types of PD, workshop evaluations, summer academy evaluations

Turn the Questions into Objectives and Design Measurements

1. Does the MC2 PD partnership program positively affect student achievement?Obj. 4: Students will show growth in achievement on the NM SBA. Measurements: Use SBA data by district, school, and teacher. Analyze relationships between amount and time of PD and achievement. Analyze relationship between growth in teacher knowledge and achievement

Turn the Questions into Objectives and Design Measurements

3. Does the level of implementation of the Professional Development in the classroom affect student achievement ?Obj. 3: Districts will integrate PD into the school culture (classroom level)Measurements: Classroom snapshots (engagement, type of learning & teaching), Levels of Use (based on CBAM and SEDL guidelines), Horizon Classroom Observation by trained observers .

Turn the Questions into Objectives and Design Measurements

2. Does the MC2 PD partnership program result in growth in teacher pedagogical content knowledge?Obj. 2: Teachers in partner districts will participate in PD to build pedagogical content knowledgeMeasurements: Survey for Teacher Knowledge for Teaching Mathematics (Ball et.al, University of Michigan). Mathematicians’ assessment of student growth in summer academy and classroom work.

A Mathematician talks about Pedagogical Content Knowledge

1. Multiple ways to present mathematical topics to meet diverse learning needs.

2. Sensitivity to common student errors and how to address them.

3. Recognizing valid student thinking even when it is not in the lesson plan.

A Mathematician talks about Pedagogical Content Knowledge

4. Choices about remediation: Re-teach, or review in the context of new material?5. Helping students move on with their learning, even when their background is lacking.6. Understanding vertical alignment – how does what I teach fit in to the big picture?

How we assess teacher knowledge for teaching mathematics

Mathematicians, writing and the one and two week academies

The Teacher Survey from Michigan

Time is Up, We will discuss

To contact us or the NM MSP write

Dr. Ted Stanford (stanford@nmsu.edu)

Or

Dr. Karin Wiburg

(wiburg@nmsu.edu)

We are happy to share instruments and evaluation plans and strategies.

Disclaimer

The instructional practices and assessments discussed or shown in this presentation are not intended as an endorsement by the U.S. Department of Education.