aft 7/12/04 marywood university using data for decision support and planning
TRANSCRIPT
AFT 7/12/04 Marywood University
Using Datafor
Decision Supportand
Planning
AFT 7/12/04 Marywood University
Presenters
Dr. Marie Loftus Dr. Barbara R. Sadowski
Marywood Universitywww.marywood.edu
AFT 7/12/04 Marywood University
Session Overview
Group activity followed by discussion
Starting the research process
Break
Group activity – questionnaire analysis and discussion
Research results
Question and answer
AFT 7/12/04 Marywood University
Group Task 1 Develop 2-3 questions of interest
For each question list:
– a) what data could be used – b) factors that might affect the process of
obtaining data
_ c) potential problems with existing data
– d) environmental factors that might bias the data or interpretation of the results
– e) relevant existing data sources
AFT 7/12/04 Marywood University
To determine if a teacher’s Thinking Math training has an effect on student achievement in Mathematics.
Research Purpose
AFT 7/12/04 Marywood University
Questions Considered
Does the mathematics training of teachers have an effect on the math achievement of students they teach?
Does professional development training of teachers translate into higher student achievement?
AFT 7/12/04 Marywood University
Practicality of Research Options
Question 1:
– Difficult to get data on teacher’s background in mathematics
– Measurement difficulty in defining college course work with in-service professional development
Question 2: - Teacher training data available- Non-transient teacher population- Well-documented training program- Student achievement data available
AFT 7/12/04 Marywood University
Other Variables Considered
What constituted teacher training? How many TM workshops?
Did TM trained teachers actually implement the principles? Which ones were implemented? How confident did the teachers feel about their ability to implement?
Was years of teaching experience a factor in how well TM was implemented?
AFT 7/12/04 Marywood University
Other Variables Considered
Was number of years teaching at grade level a factor?
Was the math background of the teacher a factor in how well TM was implemented?
How much of the time did the teacher actually implement TM when teaching math?
AFT 7/12/04 Marywood University
Other Variables Considered
What student variables might affect the score on the achievement test? Reading level? SES? Title 1 support? The number of TM trained teachers they had before reaching fifth grade?
Would school climate affect how the teachers felt about implementing TM math principles? Did the principal support their use of TM?
Ability grouping in the assignment of students
AFT 7/12/04 Marywood University
Research Question 1
Is the number of years students have a TM trained math teacher a predictor of how they will perform on a math achievement test?
AFT 7/12/04 Marywood University
Total enrollment 8,673 students
Economically disadvantaged 54%
Mobility 25.2%
Schools 18–13 elementary (Pre K-5)
Teachers 712–strong collective bargaining history with the Scranton Federation of Teachers
Scranton PA School District
AFT 7/12/04 Marywood University
Data Collection
Survey instrument developed collaboratively by Thinking Math trainers and researchers
Information session held for teachers and administrators
Surveys administered to TM teachers
Student data collected at school sites
AFT 7/12/04 Marywood University
Sample
The study sample was teachers and 5th grade students in:
4 of 13 elementary schools
–3 schools with TM trained teachers at the 5th grade level. –1 school with TM trained teachers, none at the 5th grade level
AFT 7/12/04 Marywood University
Sample Schools
All offer Title 1 reading, two also offer math
Fifth grade enrollments are 70, 55, 44, and 34
AFT 7/12/04 Marywood University
Sample Schools
Percentages of low income students: 28, 36, 46, and 77% (PA Profiles, 1999- 2000)
Class organization
– heterogeneous and homogeneous– departmentalized and self-contained
AFT 7/12/04 Marywood University
Group Task 2
Review/analyze teacher questionnaire results
Consider possible reasons for the results of the scale item means
Are the scales correlated? Which ones?
What other variables are related to the scales?
What changes would you make to improve the instrument?
AFT 7/12/04 Marywood University
Teacher Survey Results
AFT 7/12/04 Marywood University
Teachers Surveyed
33 of 37 are female
Pre-K - 5, Title 1, Learning Support
10% have < ten years teaching experience
70% have twenty or more years of experience
average of 1.5 ER&D courses beyond TM
AFT 7/12/04 Marywood UniversityNCTM 4/ 10/ 03 Scranton SD - Marywood University
Teacher Survey Scales
Confidence Scale ( 7 items)
Rate yourself on your confidence in... e.g., accepting multiple solutions.
1 = Not confident 10 = Very confident
Scale Mean = 8.57 s.d. = 0.80
AFT 7/12/04 Marywood University
Teacher Survey Scales
Implementation Scale (7 items)
To what extent do you... e.g., have your students explain and justify their solutions to problems?
1= Not at all 10 = To a great extent
Scale Mean = 8.1 s.d. = 0.89
AFT 7/12/04 Marywood University
Influence of TM Scale Items
1. How frequently do you incorporate the TM principles in math instruction?
2. To what degree has your delivery of math instruction changed as a result of TM training?
3. How has the depth of the math content you are teaching changed since your TM training?
4. How much has TM research added to your content knowledge of mathematics?
Scale Mean = 8.19 s.d. = 0.97
AFT 7/12/04 Marywood University
Correlations
Confidence Implementation
Influence of TM .395* .429**
Confidence .877**
* p<.05 **p < .01
n = 37, two-tailed
AFT 7/12/04 Marywood University
Regression
Implementation as the dependent variable
Influence of TM as independent variable
Confidence in TM Principles as independent variable
89% of the variance in implementation is accounted for by the two independent variables
AFT 7/12/04 Marywood University
Experience and Implementation
Teacher Experience N Mean s.d.
Less than 20 years 12 7.86 0.81
More than 20 years 25 8.33 0.95
AFT 7/12/04 Marywood University
Teacher Comments on TM
“I now change problems to reflect my class, linking school math to real world experiences.”
“I now use more manipulatives.”
“It has given me new insight for teaching multiple strategies and meeting individual needs.”
AFT 7/12/04 Marywood University
Student Achievement Results
AFT 7/12/04 Marywood University
SAT Math Problem Solving
PSSA Math
SAT Math Procedures
.798** .777**
SAT Math Problem Solving
.819**
**p < .01, (two-tailed); N = 167
Correlations: SAT – PSSA Math
AFT 7/12/04 Marywood University
SAT
Reading PSSA
Reading SAT Math Procedures .626** .620**
SAT Math Problem Solving .694** .648**
PSSA Mathematics .765** .693** **p < .01, (two-tailed); N = 167
Correlations: Reading - Math
AFT 7/12/04 Marywood University
Research Hypotheses
Students with 5th grade TM trained teachers will perform significantly better on PSSA 5th grade Math tests compared to the control group.
Students with 5th grade TM trained teachers will perform significantly better on PSSA 5th grade Math tests than the district and state.
AFT 7/12/04 Marywood University
Math Performance Levels
Level*** Thinking
Math District
Advanced 19.2% 14.8%
Proficient 26.3 25.9
Basic 36.5 31.2
Below Basic 18.0 28.0
***2 = 14.6; df =3, p < .005
AFT 7/12/04 Marywood University
Math Performance Levels
Level Thinking
Math State
Advanced 19.2% 22.5%
Proficient 26.3 30.5
Basic 36.5 24.6
Below Basic 18.0 22.4
Chi-square is not significant.
AFT 7/12/04 Marywood University
Mean Comparisons
Thinking
Math Control
PSSA Math** 1307 1216
SAT Math Procedures*
672 644
SAT Problem Solving***
665 631
*t = 2.40 ; df = 39, p < .05
**t = 2.74 ; df = 195, p < .01
***t = 4.0 ; df = 41, p < .000
AFT 7/12/04 Marywood University
Mean Comparisons
Thinking
Math District
PSSA Math** 1307 1270
SAT Math
Procedures*** 672 594
SAT Problem Solving***
665 609
**t = 2.91 ; df = 166, p < .005
***t > 20 ; df = 167, p < .000
AFT 7/12/04 Marywood University
Mean Comparisons
Thinking
Math State
PSSA Math 1307 1310
Mean difference is not significant.
AFT 7/12/04 Marywood University
Teacher-Student Results
AFT 7/12/04 Marywood University
Mean Comparisons > 2 TM teachers 2 or fewer
SAT Problem Solving** 665 655
SAT Math Procedures* 671 664
Reading Comprehension*** 671 660
*t = -1.76 ; df = 197, p = .081
**t = -0.91 ; df = 197 p = .081
***t = -2.04; df = 197, p < .05
AFT 7/12/04 Marywood University
Research Question
Can math achievement be predicted from teacher experience, professional development and implementation of the TM principles?
AFT 7/12/04 Marywood University
Analysis
A multiple regression was run with SAT math procedures as the criterion variable, and three independent variables: years of experience at grade level, professional development and scale score on implementation of TM principles.
A second multiple regression used SAT problem solving as the dependent variable.
AFT 7/12/04 Marywood University
Math Procedures Regression
Standardized Coefficients
t
Implementation of TM Principles .497 3.52***
Experience -.158 -1.85*
Professional Development -.191 N.S.
*p =.066
*** p < .001
AFT 7/12/04 Marywood University
Math Problem Solving Regression
Standardized Coefficients
t
Implementation of TM Principles
.451 3.18**
Experience -.130 N.S.
Professional Development
-.151 N.S.
** p < .05
AFT 7/12/04 Marywood University
Reading Comprehension Regression
Standardized Coefficients
t
Implementation of TM Principles .549 3.88***
Experience -.077 N.S.
Professional Development -.311 -2.30**
* * p = .023
*** p < .000
AFT 7/12/04 Marywood University
Conclusions
Teacher confidence in TM principles and perception of the importance of TM are predictors of implementation which is in turn a predictor of student mathematics achievement.
The number of years a student has a TM trained teacher is a predictor of student achievement in mathematics.
AFT 7/12/04 Marywood University
Conclusions
Students who have a 5th grade TM trained teacher performed significantly better on the PSSA 5Th grade Math Test than the control group and the district.
AFT 7/12/04 Marywood University
Research Purpose
The research results suggest that TM training does have a significant effect on student achievement.
AFT 7/12/04 Marywood University
Topics for Further Research
Does TM training affect student achievement in Reading Comprehension?
What is the relationship between TM training, student reading comprehension and student mathematics problem-solving achievement?
AFT 7/12/04 Marywood University
Questions?