Copyright © 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail permissions@acm.org. ETRA 2010, Austin, TX, March 22 – 24, 2010. © 2010 ACM 978-1-60558-994-7/10/0003 $10.00

Gaze Scribing in Physics Problem Solving

David Rosengrant Kennesaw State University

1000 Chastain Road, Kennesaw GA, 30144 drosengr@kennesaw.edu

Abstract

Eye-tracking has been widely used for research purposes in fields such as linguistics and marketing. However, there are many possibilities of how eye-trackers could be used in other disciplines like physics. A part of physics education research deals with the differences between novices and experts, specifi-cally how each group solves problems. Though there has been a great deal of research about these differences there has been no research that focuses on noticing exactly where experts and no-vices look while solving the problems. Thus, to complement the past research, I have created a new technique called gaze scrib-ing. Subjects wear a head mounted eye-tracker while solving electrical circuit problems on a graphics monitor. I monitor both scan patterns of the subjects and combine that with videotapes of their work while solving the problems. This new technique has yielded new information and elaborated on previous studies.

CR Categories: J.2 [Computer Applications]: Physical Science and Engineering - Physics

Keywords: gaze scribing, education research, physics problem solving

1 Introduction

One of the goals of many education researchers is to narrow the gap between experts and novices [Chi et al 1981; Feltovich and Glaser 1981; Kindfield 1994; Kozma and Russell 1997; Styliano and Silver 2004]. In Physics, the gaps include but are not li-mited to how both groups understand and comprehend the con-tent material, how they learn new material and how they solve problems. Problem solving is not always finding a numerical answer; it can include qualitative solutions such as using simula-tions to explain how microwave ovens work. For this study, I focus specifically on the problem solving gap..

First, we must understand how experts and novices learn and solve problems in order to help students (novices) become ex-perts. Traditionally, problem solving studies have involved case studies with one-on-one interviews with students. The inter-viewer would give the subject a task and use a think aloud pro-tocol [Ericsson and Simon 1998] to determine the subjects thought processes while solving the problems.

Combining this methodology with eye-trackers creates a new research method I call gaze scribing. Subjects wear a head mounted eye-tracker while they write out solutions on a graphic monitor. Videotaping and audiotaping the student’s work, sav-ing their written data, listening to their discussion and analysis of their scan paths [Duchowksi 2007] are all necessary compo-nents for this gaze scribing.

This initial study focuses specifically on electrical circuits. Problem solving in physics varies depending on the sub-disciplines (mechanics, dynamics, electricity, et al). I have fo-cused on electrical circuits for this study because there is a unique combination of qualitative understanding of representa-tions and quantitative reasoning abilities. Furthermore, there has been no work to date utilizing eye-trackers and expert-novice differences with electrical circuits.

2 Previous Work

Novices and experts approach problem solving differently. One such difference is the search techniques they use to solve a prob-lem [Larkin et al 1980] or the strategy they use to solve a prob-lem [Chi et al 1981]. Novices typically write down the known and unknown variables. Next, they use a backward inference technique - a search for equations involving variables they think they can use. This is commonly called plug and chug. Experts use a forward inference technique. They first determine the con-cept and the key features associated with the problem to deter-mine how they want to solve the problem. Furthermore, novices categorize problems by the surface features of the problem as opposed to experts who again focus on underlying concepts [Kozma and Russell 1997].

What is also important is what happens during the problem solv-ing process. For example, if an expert or novice gets stuck dur-ing a problem, how they get unstuck is vastly different. Experts typically can perform a qualitative analysis of their work or they can use some other method to aid them in the problem solving process. If novices get stuck, they typically only manipulate equations or seek outside help to get them unstuck [Gerace 2001]. Another key difference is that experts are able to check their solutions, possibly via another method such as multiple representations or alternate mathematical equations. Novices however are only able to find a solution via one method.

These differences are common to all areas of physics, but since this study focuses on electrical circuits there are key differences between experts and novices that are specific to this sub-discipline. In direct current (DC) circuits, current is a major conceptual challenge to novices. They tend to believe that cur-rent gets “consumed” when moving through a circuit and that parallel branches split current equally throughout the branches regardless of the arrangement of the resistors [Duit and Rhonek 1997]. Novices also believe that the battery is a constant source

45

of current [Engelhardt and Beichner 2004]. Their difficulties with current are compounded by the fact that they interchange the ideas of “current” and “voltage” [Metioui 1996]. They be-lieve that circuits are a system of pipes that allow a fluid called electricity to flow through them [Johsua 1984]. These difficul-ties become even more noticeable when one incorporates series and parallel sections of circuits. Even identifying which compo-nents of a circuit are in parallel or in series are a challenge for many students.

3 Sample and Setting

This study was conducted at a suburban university of about 21,000 students in the second semester of a two-semester alge-bra based physics course. The students were typically Biology or Health and Exercise Science majors. The students in this course had already been taught about electrical circuits.

Eleven subjects participated in the case study. Nine of the sub-jects were considered novices. They were students in the above mentioned course. Two others were considered experts who were physics faculty at the university.

4 Methodology

Each subject received a series of questions based on the circuits found in Figure 1. Each subject received each of the four cir-cuits one at a time. The circuits were in a Microsoft paint doc-ument. This allowed the subjects to write their work next to the circuit since the monitor was a graphics display monitor. Sub-jects also received a calculator on the computer screen to assist

them with any quantitative analysis if needed. Subjects received simulations of the last circuit to assist with qualitative questions. Figure 2 is an example of one subject’s work and a visual of what the subject’s workspace looked like.

Some of the questions we asked the subjects only required an auditory response. For example, how does the current compare going through specific resistors. However, others required a numerical response. In every case, the subjects needed to find the net resistance of the circuit. All of the subjects’ work was conducted on the graphic tablet monitor in the paint file.

Each subject wore a head mounted eye-tracker while they ans-wered the questions. The eye tracker was an Applied Science Laboratories Model 6000 Mobile Control Unit that included an Applied Science Laboratories head-mounted optics unit with scene camera. The subjects sat at arm’s length from the screen so that they could write the answers on the monitor. The setup is show in Figure 3. A head mounted unit assured that the arm would not block any cameras while writing answers.

The graphics monitor was part of a 2 monitor setup. The inter-viewer sat at the second display monitor. This allowed the in-terviewer to easily change the circuits the subjects worked on as well as what tools the subjects could use.

A video camera (in addition to the camera from the eye-tracker) recorded the entire interview. Each interview was also audio-taped. The use of the multiple recording devices allowed me to analyze the data several ways. First, I was able to listen to what the students verbalized as they solved the problem. I was also able to tell what the students were calculating / drawing on the monitor while simultaneously knowing where subjects looked while they solved the problem.

4

5

4

5

3

9 9

2 2

1

4

55

0.5

0.5

1

1

0.5A B

Figure 1: 4 Circuits given to the students

Figure 2: Screenshot of student’s work.

Figure 3: Screenshot of setup.

46

5 Findings

The information uncovered about expert-novice differences on electrical circuit problems both reinforced previous studies and uncovered new data. However, the focus of the findings here is on how the gaze scribing and eye-tracking supplemented the previous work of other physics education researchers. More information about the other differences can be found in Rosen-grant et al [Press].

One of the first questions I asked the subjects was to calculate the net resistance of the circuit. This involved a quantitative answer with a qualitative understanding of how circuits worked. One of the traits I investigated was the idea that experts evaluate their work while solving problems while novices do not. The gaze scribing reinforced this finding. Both the experts and the novices used the provided extra space to write out their calcula-tions and eventually a solution. Both the experts and the novices would look back at the circuit to double check the value of the resistors in their mathematical formulas. However, from that point on the novices would only focus on their mathematical work until they arrived at a solution. They would look back and forth in their work, but not back to the circuit. The experts on the other hand would gaze back and forth between the given circuit, their work and circuits they may have redrawn to help them solve the problem. This difference is not something that could be noticed during a normal problem solving session.

There were other more specific patterns that emerged during this evaluation. For example, in a parallel circuit containing two or more resistors, the total resistance of that portion of the circuit must be less than any of the resistors making up the parallel portion of the circuit. After experts calculated the net resistance for the parallel portion of the circuit they would look back at the circuit. Specifically they would look back and forth at the val-ues of the resistors in parallel and their answer. Novices did not evaluate their work in this respect; they simply focused on what they were writing and then continued on to the next step in sim-plifying the circuit. They would look back and forth among their calculations and formulas, but not back to the original cir-cuit.

When the experts found the resistance for the total circuit, they then looked back at all of their work, back at any circuits they constructed and then to the original circuit while also looking back at their calculated answer before they exhibited signs of being finished with the problem. These signs include but are not limited to leaning back in their seat, looking away from the computer screen or saying that was their final answer or that

they were done. Novices, in some cases only looked at the last bit of mathematical work but generally did not show any evi-dence of comparing their answer with their work or the given circuit.

Another difference between the experts and the novices was how they initially looked at the circuit. Most of the subjects looked at circuit 2 in a fashion similar to what is shown in Figure 4. When the subjects analyzed the circuit, they would simply go from one resistor to the next following the shortest path between the resistors. This was common among both the experts and the novices. However, one of the experts sometimes exhibited a different behavior. This expert was also a stronger expert in this field because he taught the electronics course at the university. The expert followed a path similar to what is shown in Figure 5. In this circuit, it appears that the expert followed the path of the current throughout the circuit.

This is important because none of the novices exhibited this behavior. This scan-path ties in with previous research that states that one of the issues novices have with circuits is simply understanding how current flows through a circuit.

Figure 5: Expert partial gaze path of circuit 2

A final piece of interesting information that figure 4 also shows us is that both the experts and the novices would group resistors together when they first analyzed the circuit. The resistors would be grouped according to how they were arranged, either in series (such as the 8 and 16 ohm resistors shown in figure 4) or in parallel (the 3 and the 5 ohm resistors are connected in parallel to the 4 ohm resistor). As the groups were writing out their mathematical equations on how to add the resistors in pa-rallel or in series both groups would look back to the relevant parts of the circuit.

Figure 4: Sample gaze path of circuit 2

6 Discussion

The gaze scribing provides a unique opportunity to analyze problem solving behaviors. Instead of relying on verbal res-ponses and written work from experts and novices, now we can also monitor their scan paths while they are solving problems. The differences in the scan paths while subjects are solving physics problems can lead to new interpretations and under-standings of the differences between the two groups.

This preliminary study is only focused on one type of problem solving scenario. Solving problems with electrical circuits al-lows subjects to find a quantitative solution but it also involves

47

the construction of multiple representations (other circuits). Representations also have many differences between how ex-perts and novices use and construct them [Rosengrant et al 2006].

There are many types of problems students solve in physics. Gaze scribing is a research technique that can be implemented for these other problem types such as mechanics problems, ray tracing with thin lens, online simulations and how students use them, etc. These are all areas of future research that combine eye-tracking with physics education research. This type of me-thodology can also extend beyond just physics. Gaze scribing could be used in any discipline where subjects need to write out solutions.

References

CHI, M. FELTOVICH, P., AND GLASER R. 1981. Categoriza-tion and Representation of Physics Problems by Experts and Novices. Cognitive Science 5, 121-152.

DUCHOWSKI, A. 2007. Eye Tracking Methodology: Theory and Practice, 2nd Ed. Springer-Verlag.

DUIT, R., AND RHONEK, C. 1997 Learning and Understand-ing Key Concepts in Electricity. In Connecting Research in Physics Education with Teacher Education, A. Tiberghien, E.L.Jossem, and J. Barojas, Eds., The International Commission on Physics Education, 1-6

ENGELHARDT, P., AND BEICHNER, R 2004. Students’ Understanding of Direct Current Resistive Electrical Circuits. American Journal of Physics, 72, 98-115.

ERICSSON, K., AND SIMON, H. 1984. Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press

GERACE, W. 2001. Problem Solving and Conceptual Under-standing. In Proceedings of the 2001 Physics Education Re-search Conference, S. Franklin, J. Marx, K. Cummings, Eds., Physics Education Research Conference, Annual Conference, 33-36.

JOHSUA, S. 1984. Students’ Interpretation of Simple Electrical Circuit Diagrams, International Journal of Science Education, 6, 12. 271-275.

KINDFIELD, A. 1994. Understanding a Basic Biological Process: Expert and Novice Models of Meiosis, Science Educa-tion, 78, 255-283.

KOZMA, R. AND RUSSELL, J. 1997. Multimedia and Under-standing: Expert and Novice Responses to Different Chemical Representations of Chemical Phenomena. Journal of Research in Science Teaching, 34, 9, 949-968.

LARKIN, J., MCDERMOTT, J., SIMON, D. AND SIMON, H. 1980. Expert and Novice Performance in Solving Physics Prob-lems. Science, 208, 1335-1342.

METIOUI, A., BRASSARD, C., LEVASSEUR, J. AND LA-VOICE, M. 1996. The Persistence of Students’ Unfounded Beliefs About Electrical Circuits: The Case of Ohm’s Law. International Journal of Science Education, 18, 2, 193-212.

ROSENGRANT, D., ETKINA, E., AND VAN HEUVELEN, A. 2007. An Overview of Recent Research on Multiple Represen-tations. In Proceedings of the 2006 Physics Education Research Conference, L. McCullough, P. Heron, and L.Hsu, Eds., Physics Education Research Conference, Annual Conference, 149-152.

ROSENGRANT, D., THOMSON, C., AND MZOUGHI, T., (PRESS). Comparing Experts and Novices in Solving Electrical Circuit Problems With the Help of Eye-Tracking. Submitted to the Proceedings of the 2009 Physics Education Research Confe-rence.

STYLIANO, D. AND SILVER, E. 2004. The Role of Visual Representations in Advanced Mathematical Problem Solving: An Examination of Expert-Novice Similarities and Differences. Mathematical Thinking and Learning, 6, 4, 353-387.

48