Computation-based versus concept-based test Computation-based versus concept-based test questions: High school teachers’ perceptionsquestions: High school teachers’ perceptions

Luanna Gomez and Daniel MacIsaacPhysics

Joseph ZawickiEarth Sciences & Science Education

SUNY College at BuffaloBuffalo, NY 14222

<gomezls@buffalostate.edu>

1

AbstractAbstract

The New York State Regents examination in physics is a standardized

assessment of high school students' competencies after completing a year of

introductory physics. The analysis of select items taken from 1500 students

will be provided. We have compared traditional computational problems to

less traditional conceptual problems to examine the extent to which the

response pattern provides insight to the difficulty of the two types of questions.

This discussion will form a context in which teachers' perceptions of the nature

of numeric and concept questions, of their relation to physics understanding,

and of their implications to physics instruction.

2

False dichotomiesFalse dichotomies

We examined select items from recent New York State

Regents examinations, including response analysis from about 4500

high school student papers. Student response data revealed the

conceptual items were amongst the most difficult, which was surprising

to several vociferous high school physics teachers interviewed.

3

Student population and contextStudent population and context

Urban, suburban, and rural school districts

NYS Regents exam in physics

The secondary physics exam is administered in June to Grade 11 & 12 (N = 4500+ students)

4

Regents’ examination backgroundRegents’ examination background The exam‡:

• is aligned with New York State core curriculum¶;• has been administered since 2007; and• contains 3 sections (~ 60 items):

• multiple choice,• constructed response (e.g. requires a short answer or

calculation), and• extended constructed (e.g. requires a written passage

or multi-step calculation)

‡See, for example, http://www.nysedregents.org/testing/scires/regentsphys.html

¶The NYS core curriculum may be viewed at http:///www.nysed.gov

5

June 2008 secondary-level student data June 2008 secondary-level student data (Example item 1)(Example item 1)

Item 1Major learning goal: To be able to discern that measured quantities can be classified as either a vector or a scalar.

Discrimination index: D = 0.80

Concept multiple-choice questionThe speedometer in a car does not measure the car’s velocity because velocity is a

(1) vector quantity and has magnitude and direction (correct).

(2) vector quantity and does not have a direction associated with it.

(3) scalar quantity and has a direction associated with it.

(4) scalar quantity and does not have and does not have a direction associated with it.

6

June 2008 secondary-level student data June 2008 secondary-level student data (Example item 2)(Example item 2)

Item 2Major learning goals: To recognize that an object or system has a kinetic energy associated with its velocity, and that its gravitational potential energy depends solely on the relative positions of the objects in that system.

Discrimination index: D = 0.48

Concept multiple-choice questionA car travels from point A to point B at constant speed up a hill. As the car travels its gravitation potential energy

(1) increases and its kinetic energy decreases.

(2) increases and its kinetic energy remains the same (correct).

(3) remains the same and its kinetic energy decreases.

(4) remains the same and its kinetic energy remains the same.

7

June 2008 secondary-level student data June 2008 secondary-level student data (Example item 3)(Example item 3)

Item 3Major learning goals: To recognize that an object or system has a kinetic energy associated with its velocity, and that its gravitational potential energy depends solely on the relative positions of the objects in that system.

Discrimination index: D = 0.75

Short calculation itemA 65kg pole vaulter wishes to vault at a height of 5.5m. Calculate the minimum amount of kinetic energy the vaulter needs to reach the height if air friction is neglected and all the vaulting energy is derived from kinetic energy. [Show all work, including the equation and substitution with units.]Correct answer: KE = 3500 J (rounded to 2 significant figures)

8

June 2008 secondary-level student data June 2008 secondary-level student data (Example items 4-6)(Example items 4-6)

Major learning goal: Interpretation of graphs

Item 4 Discrimination index: D = 0.97

Item 5 Discrimination index: D = 0.93

Item 6 Discrimination index: D = 0.45

Multi-step calculation itemItem 4: Plot the data points for the dart’s maximum vertical displacement versus spring compression. [Use the information in the data table (not shown).]Item 5: Draw the line or curve of best fit.Item 6: Using information from your graph, calculate the energy provided by the compressed spring that causes the dart to achieve a maximum vertical displacement of 3.50m. [Show all your work, including equation and substitution with units.]

(Solution not shown)

9

Teacher data collection (preliminary)Teacher data collection (preliminary)

Individual teacher interviews (N ~ 3)Teachers were asked whether they believed

computation-based (i.e., formula-driven) problems were more or less challenging for students than concept-based (i.e., qualitative) problems on the Regents’ exam in physics/physical science—they believed that the calculation problems were more

difficult than conceptual ones!

10

Tentative conclusionTentative conclusion

Conceptual items may be more difficult than most calculation-driven ones

Teachers may not appreciate the value of concept questions and dismiss the results because they are perceived as “trick” questions.

On the basis of these results, teachers may be spending more time preparing students for the Regents exam in physics with equations rather than concepts.

11

ImplicationsImplications

Some skill sets, such as inscription, remain more difficult for students.

Mundane skill sets, such as plotting points and solving for commonly rehearsed variables, appear to be readily achieved

12

AcknowledgementsAcknowledgements

SUNY Buffalo State CollegeKathleen Falconer, Elementary Education and Reading

Western New York Regional Information Center

Timothy Johnson, Erie 1 BOCES

13