Teaching and Learning Physical Science in Urban

Secondary Schools: Assessing the Assessments

Joan Whipp, Michael Politano, Michele Korb – Marquette UniversityMel Sabella, Chicago State UniversityDavid Hammer, University of MarylandKristi Gettelman, Wisconsin Conservatory of Lifelong LearningKelly Kushner, Shorewood Intermediate SchoolMegan Ferger, Roosevelt Middle School

Modeling in Physical Science for Urban Teachers Grades

6-10• 2004-2006• 18 middle and high school science

teachers from ten urban Milwaukee schools

• 2 summer institutes; 8 follow-up half-day modeling workshops; an online community; classroom mentoring

• Physical Science Modeling Workshop curriculum developed at Arizona State

• Focus on increasing teacher content knowledge and instructional strategies

Assessing Content Knowledge

Physical Science Concept Inventory (PSCI)

Multiple-choice diagnostic test based on:

– Lawson Test of Scientific Reasoning (Lawson),

– Chemistry Concept Inventory (Mulford and Robinson),

– Trends in International Mathematics and Science Study (TIMSS),

– Energy Concept Inventory (Swackhamer et al.)

served as an assessment of student and teacher content understanding of topics in Physical Science

Gains on PSCI

• gain calculated using the formula developed by Hake1.

Participants

• Pre-/post-test during 1st summer workshop

Students in subset of participants’ classes

• Baseline established in April 2004• Given as pre-/post-test in Fall 2004/Spring 2005 and

in Fall 2005/Spring 2006

1R. Hake, “Interactive engagement versus traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses,: Am. J. Phys., 65 (5) 418-428 (1997).

Student Performance on PSCI

baseline (N=213)

(% correct)

pre-test (F04)(N=610)

(% correct)

post-test (S05)(N=395)

(% correct)

pre-test (F05)

(N=298)(% correct)

post-test (S06)

(N=262)(% correct)

31% 30% 31% 30% 36%

•Student performance was essentially the same for the baseline, the pre-test, and the post-test during the 1st year of implementation.

•A mean post-test score of 36% was achieved at the end of the 2nd year of implementation. Although this is a small gain, it does indicate some improvement over what had been achieved in the past. The 2nd year data focused on the teachers who used the modeling curriculum at a significant level.

SummaryImplementation 1 (F04-S05) Looking at the entire set of PSCI data from year 1, there

is little evidence for improvement in performance on the diagnostic.

There does seem to be clear improvement in student responses to question 13, which deals with the concept of area.

Looking at the data for individual sections, one section does seem to show some significant gain – scores went from 37% to 47%.

Implementation 2 (F05-S06). Looking at the entire set of data from year 2, there is a greater degree of improvement, but the gain is still small. In one case (section 9), students went from a pre-test score of 33% to a post-test score of 60%, a 39% gain. There does seem to be clear improvement on a greater number of individual questions.

Assessing Attitudes about Science

Student Attitudes• Attitudes Survey

– consists of twenty-two statements, in which participants are asked whether they agree or disagree on a five point Lickert Scale.

– Given as a pre and a post test.

• Statements from the Attitudes survey were obtained from the: – Maryland Physics Expectations Survey

(MPEX, University of Maryland PERG), – the Epistemological Beliefs Assessment for

Physical Science (EBAPS, Elby) and – the Epistemological Questionnaire (M.

Schommer.)

Sample Statements from the Attitudes Survey

• I developed a better understanding of science by discussing my ideas with my classmates and my teachers.

• Learning science makes me change some of my ideas about how the physical world works.

Summary of ResultsThere was little change toward expert-like views after approximately one year of modeling instruction.

Researchers have found that attitudes toward science and science learning science are typically very difficult to change in the course of a semester or a year.E. F. Redish, J. M. Saul, and R. N. Steinberg, “Student expectations in introductory physics,” Am. J. Phys. 66 (3), 212-224 (1998).

Middle School Student Responses (revised version of

instrument)Results from six sections employing the Modeling Method.

– over 64% of the students stated that this course was different than other science courses they have taken (Q2). (Ranged between 64% to 81%.)

– Generally students seemed to feel that discussing ideas with their classmates and teachers helped them learn the material (Q11). (Ranged between 50% to 100%.)

– Students also seemed to feel that they were ready for their next science course (Q17) and many felt that they were capable of learning science (Q20). (In most cases more than half the class responded favorably to these statements.)

Teacher Self-Assessment

InterviewsSurveys

Interviews with focus group: Summary

• Teachers– who participated in the focus group were very

committed to the Modeling approach– were comfortable modifying the instructional

materials so that it better fit their students – felt that students did have a better understanding -

based on classroom observations and alternate forms of assessment.

• the modeling method created a classroom environment in which:– the classroom was more interactive, – their students began to realize that that they were

partly responsible for their learning – the teachers recognized that their students were

capable of constructing their own understanding – the teachers were able to better see where their

students were in their understanding.

Changes in TeachingDescribe how your teaching has changed as a result of the modeling curriculum.

The two statements that the teachers agreed with the most:– my knowledge of concepts in physical

science has improved.– I now engage my students in more

group-work in the classroom.

Changes in TeachingDescribe how your teaching has changed as a result of the modeling curriculum.

• Almost all teachers seem to have adopted the technique of using whiteboards in in their classes to stimulate group interaction.

• In addition, the use of a standard text book also decreased. 47% of the teachers stated that they used the textbook regularly or frequently before being engaged in modeling and 18% stated regular or frequent use after.

In general, these results, show a shift toward the approach emphasized in the modeling curriculum.

Using an Observation Tool to Assess Teaching Reform: The

RTOP

Purpose of using the RTOPs

• To maintain a standard for observation of modeling in the classrooms

• To provide a common language as reference when evaluating the effectiveness of and implementation of modeling curriculum.

• To provide a standard for evaluation of dialogue among teachers and mentors (in summer workshops and during online discussions)

Typical Evaluation Items on the RTOP and how they informed the evaluation:

Lesson Design and Implementation (1 & 2)Content (6 & 7)

RTOP example questionsProcedural Knowledge (11- 14)

Other typical RTOP evaluation pointsClassroom Culture

Communicative Interactions (16 & 17)Student/ Teacher Interactions (22-24)

Results from RTOP and RTOP discussion

• Teachers – improved most on understanding the modeling approach– struggled with initial implementation due to student

resistance

• Students– use of hands-on manipulatives and experimental design

improved– student ability to communicate results did not improve in

most cases.

• The RTOP was only a “snap shot” of teacher implementation of modeling, yet fostered discussion and understanding of this method of instruction.

Fascinating results• Classroom observations showed

that fifth graders’ response to Modeling were very mature and developed. This occurred in a public school setting.

• Highest gains in the PSCI were in a classroom where a teacher (according to RTOP data) scored the lowest on modeling skills and implementation of curriculum.

Using Action Research to Assess the Improvement of

Teaching and Learning Physical Science

Three Action Research Projects

Kushner, K.M. (2006). A comparison of constructivist and traditional methods on students’ scientific measurement skills. Unpublished master’s action research study, Cardinal Stritch University, Milwaukee, Wisconsin.

Gettelman, K. (2006). The use of a modeling method to teach physical science concepts at the elementary/intermediate level. Unpublished master ’s action research study, Marquette University, Milwaukee, Wisconsin.

Ferger, M.L. (2007). Traditional vs. modeling instruction of eighth grade chemistry students. Unpublished master’s action research study, Cardinal Stritch University, Milwaukee, Wisconsin.

Questions• How did student understanding of physical

science and related math concepts change?• How, if at all, did student attitudes toward

science change?• In what ways, if any, has student

engagement n the scientific process increased – i.e. making hypotheses, collecting and analyzing data, and forming conclusions to understand and explain physical science concepts?

• What is the impact of a modeling curriculum on student achievement, in comparison to a traditional science curriculum?

Action Research Designs

Researcher Kelly Kushner

Kristi Gettelman

Megan Ferger

Subjects &SamplePopulation

45 – 8th Grade Students

30- 5th Grade Students

100 – Eighth Grade Students in 4 Science classes

School at which study took place

midwestern suburban public intermediate school

Midwestern urban public elementary school

midwestern urban public middle school

Researcher Kelly Kushner

Kristi Gettelman

Megan Ferger

Content Taught

Measurement

Physical Science Chemistry

Teaching Techniques Used

Modeling vs. Traditional

Modeling Modeling vs. Traditional

Assessment instruments implemented

Measurement Pre and Post Test

Student Science Attitudes Pre/Post Survey – all participantsProperties of Matter Pre/Post-Assessment – all participantsClassroom Observations and ReflectionsStudent Reflections in Science Log – all participantsStudent Interviews – 6 participants

A combined fact and critical thinking pre and post-test

Findings•

Students showed greatest gains when they were taught with a blended approach of traditional and modeling methods

• Students showed greatest gains in their understanding of principles of matter and the least in their understanding and ability to apply advanced measurement principles

• Student resistance to modeling was a barrier that needed to be addressed initially; students were too used to being given right answers rather than thinking for themselves.

• Student attitudes toward science became more positive.

• Student engagement in scientific processes has increased.

What We Have Learned through Action Research

• Better understanding of the physical science concepts we are teaching

• Constructivist principles which underlie the modeling approach to teaching science

Understanding Our Learners• Common student

misconceptions about physical science and how to address those misconceptions in teaching

• How changing our practice can impact student learning;

• Middle school students vary greatly in ability to reason abstractly

Understanding Our Instruction• A blended model of traditional and

constructivist methods is most effective• Became more conscious of our own

instructional design and planning and how to use student work samples and classroom discussion to guide and inform planning

• Changed our focus from teacher to student-driven instruction and assessment.

• Applied scientific principles to instruction, measured benefits to student learning

Where do we go from here?

What will we, as science educators, take forward from

this experience?

Ongoing Action ResearchWe will continue to use action

research methodology to monitor student learning and inform our teaching

•Pre/post testing•Multiple data sources•Comprehensive data analysis

Continuing the Modeling Method

• Using modeling methods to teach variety of science content, rather than just physical science

• Using whiteboard presentations in other areas of curriculum to promote student-to-student questioning and understanding

Extending the Modeling Method

• Sharing findings with colleagues, in order to improve science instruction in other areas of school/district

• Connecting physical science and math instruction, particularly measurement and geometry strands, to promote deeper student understanding