the effects of non-cas graphing calculators on student...

16

The Effects of Non-CAS Graphing Calculators on StudentAchievement and Attitude Levels in Mathematics:

A Meta-Analysis

Aimee J. EttingtonVirginia Commonwealth University

Eorty-two studies comparing students with access to graphing calculators during instruction tostudents who did not have access to graphing calculators during instruction are the subject of thismeta-analysis. The results on the achievement and attitude levels of students are presented. Thestudies evaluated cover middle and high school mathematics courses, as well as college coursesthrough first semester calculus. When calculators were part of instruction but not testing, students'benefited from using calculators while developing the skills necessary to understand mathematicsconcepts. When calculators were included in testing and instruction, the procedural, conceptual,and overall achievement skills of students improved.

The evolution oftechnology over the last 15 yearshas resulted in graphing calculators becoming everydaytools in mathematics classrooms. They have becomestandard instruments at every level ofthe educationalsystem and in almost every subject across the math-ematics spectrum.

In a survey of 600 colleges and universities, 85% ofthe mathematics departments reported that facultymembers utilized graphing calculators in the teaching ofmathematics in at least one course (Laughbaum, 1999).The most featured type of calculator at the institutionswas the graphing calculator without computer algebrasystem capabilities. A regional survey of 146 teachersinmiddleandhighschoolsfoundthat 74% ofthe schoolsallowed the use of graphing calculators (Milou, 1999).The inclusion of graphing calculators in mathematicsinstruction has been encouraged by various mathemati-cal organizations.

The National Council ofTeachers of Mathematics(NCTM, 1989) noted the significance ofthe role ofthegraphing calculator in "the emergence of a newclassroom dynamic in which teachers and studentsbecome natural partners in developing mathematicalideas and solving mathematical problems" (p. 128). TheAmerican Mathematical Association of Two-YearColleges (AMATYC, 1995) encouraged extensive useof graphing calculators in all lower level mathematicscourses and emphasized that the inclusion oftechnologyallows students to be part of an exploratory leamingenvironment. With the support for graphing calculators

continuing to grow, the question is, are graphingcalculators helping students develop the skillsnecessaryto do well in their mathematics courses? This questionis addressed and some answers are provided by a meta-analysis conducted on the use of graphing calculators inmathematics courses and described in this paper.

Results of Previous Reviews and Meta-Analyses

Educators have been researching the role of calcu-lators in the leaming of mathematics for the last 30years. Although most summaries of research havebeen narrative, meta-analysis was used in three publi-cations (Ellington, 2003; Hembree &. Dessart, 1986,1992) to synthesize and evaluate research on basic andscientific calculators. Hembree and Dessart (1986,1992) conducted a statistical evaluation ofthe effects ofcalculators on students' achievement and attitude inprecollege classrooms with 88 research reports span-ning the years 1969 through 1992. Since some studiesallowed students to have access to calculators duringtesting while other studies did not, they separated thestudies by mode of testing and provided results for eachgroup. Regardless ofthe mode of testing, they foundthat calculators had no meaningful effect on students'understanding of mathematical concepts. When calcu-lators were part of testing and instruction, students oflow or average ability improved in their paper-and-pencil skills and problem solving skills. The paper-and-

School Science and Mathematics

Graphing Calculator Meta-Analysis 17

pencil skillsofhigh ability students were neither helpednor hindered by calculator use, but these students didbenefit in the development of problem solving skills.

When students did not have access to calculatorsduring testing, Hembree and Dessart (1986) found thatstudents of average ability, in all grades except fourthgrade, improved in both paper-and-pencil and problemsolving skills. For average fourth graders, calculatorshad a negative impact on paper-and-pencil skills. Basedon this negative result, Hembree and Dessart (1992)concluded that calculators may not be appropriate forall educational situations or all mathematical subjects.Last, Hembree and Dessart (1986,1992) reported thatthe inclusion of calculators in the study of mathematicshad a positive influence on students' attitudes towardmathematics. It should be noted that most ofthe studiesanalyzed featured functional use of calculators (i.e.,drill and practice, checking work), and in only a fewstudies were calculators an integral part ofthe instruc-tion process.

A second calculator meta-analysis (Ellington, 2003)supported many of the Hembree and Dessart resultsand also provided limited details on the graphingcalculator's effect on achievement in precollege math-ematics classrooms. In the same manner as Hembreeand Dessart, the 54 studies conducted between 1983and 2002 were separated according to mode of testing.When calculators were part of instruction but not usedin testing, the skills needed to solve problems onmathematics achievement tests improved. On the otherhand, paper-and-pencil skills and the skills necessary tounderstand mathematical concepts were maintainedbut did not get better as a result of using calculators.When calculators were included in both testing andinstruction, students experienced improvement in over-all mathematics achievement. The meta-analysis re-vealed particularly meaningful results for students'problem solving skills. The benefit to achievement inmathematics was greatest from long-term calculatoruse (i.e., 9 weeks or more). Based on a small numberof studies, the improvement to problem solving skillswas most significant when (a) special curriculummaterials were designed for use with the calculator and(b) the type of calculator used was the graphingcalculator. This meta-analysis also showed that stu-dents' attitudes toward mathematics improved as aresult of working with calculators.

Summary techniques were used in two other re-views (Burrill et al., 2002; Penglase & Amold, 1996) tooutline the impact of graphing calculators in mathemat-ics. Penglase and Amold (1996) used research studiesconducted between 1990 and 1995 to determine the

impact of the graphing calculator in high school andcollege mathematics classrooms. They attempted todetermine how the graphing calculator benefited stu-dent achievement in mathematics and what kind ofleaming environment allowed maximum benefits to beattained. The researchers found the available researchto be "elusive and conflicting" (p. 59). Because tradi-tional skill-based testing procedures were used toevaluate student achievement, many studies reportedinconclusive findings. A slight majority ofthe studiesrevealed that graphing calculator use is beneficial tostudents in precalculus and calculus courses. How-ever, in most studies, the differences between calcula-tor and noncalculator groups were not significant. Instudies featuring students' understanding ofthe con-cept of function, the results were more favorable forthe calculator but still mixed.

Students' understanding of graphical concepts andtheir capabilities with spatial visualization skills weretwo areas that provided conclusive results in favor ofgraphing technology. The positive correlation betweenthe development of spatial visualization skills and math-ematical achievement was especially significant forfemale students who had notable difficulties in thisarea. The mixed results in their review led Penglaseand Amold (1996) to conclude that many pedagogicalissues need to be resolved before students will achievemaximum benefits from calculator use in the study ofmathematics.

In a review commissioned by Texas Instruments(Burrill et al., 2002), a group of researchers synthesizedthe results of 43 quantitative and qualitative researchstudies on the use of handheld graphing technology inhigh school mathematics. The reports were chosenaccording to specific criteria, including that the reportwas published in a peer-reviewed joumal, the reportedresearch was conducted under rigorous conditions, andthe results were based on scientific evidence. Thereview provides research findings, weaknesses in studydesign, areas ofthe relationship between mathematicsand handheld technology in which rigorous researchhas not been conducted, and implications for classroompractice. The findings for mathematics achievementinclude the following:

1. When compared with students without access tographing calculators, students using calculators had abetter understanding of mathematical concepts includ-ing functions, variables, applications of algebra, and theinterpretation of graphs.

2. The improvements in achievement were morepronounced for low ability students when compared toaverage or high ability students.

Volume 106(1), January 2006

18 Graphing Calculator Meta-Analysis

3. Students with graphing technology spent moretime in mathematical investigations and problem solv-ing activities than did their noncalculator counterparts.

4. Students were likely to use a graphing calculatorwhen they believed that a graph would help the problemsolving process, but when they felt the situation did notrequire looking at a graph they were less likely toincorporate other features ofthe graphing calculator inthe working ofthe problem.

Purpose

The meta-analysis outlined in this report was con-ducted to determine the effect of graphing calculatorsin helping students develop the skills necessary to dowell in mathematics courses. Specifically, the meta-analysis provides information on the effect of graphingcalculators on student development of procedural skills,conceptual skills, and overall mathematics achieve-ment. Since prior reviews (Burrill et al., 2002; Penglase& Amold, 1996) didnot use statistical methods, a meta-analysis of the available research in this area waswarranted. One focus of this meta-analysis was use ofgraphing calculators in the application of mathematicalformulas and procedures; therefore, only studies thatused graphing calculators without computer algebrasystem (CAS) capabilities were included (see theappendix for a complete bibliography of studies in-cluded in this meta-analysis). Under this condition, allofthe studies available for analysis were conducted inmiddle and high school mathematics courses and lowerlevel college mathematics classes through first semes-ter calculus. The majority ofthe studies featured theTexas Instruments TI-82, TI-83, TI-83 Plus, or theCasio fx7400, fx7500, or fx7700.

The classroom standards currently are the TI-83Plus and the recently released TI-84 Plus. The majordifference between these new graphing calculatorsand the earlier versions is memory capacity and theincluded or downloadable applications.

The capabilities of the TI-82 and TI-83 are themost frequently used graphing calculator functions inlower level mathematics classes like developmentalmathematics, college algebra, and precalculus, wherestudents do not need computer algebra system capabili-ties. The TI-82 and TI-83 (and the equivalent calcula-tors produced by other companies engaged in thedevelopment of educational technology) allow studentsto graph and manipulate functions, as well as paramet-ric curves, polar equations, and sequences. They havezoom and trace features and provide students with atable of function values. They include applications that

allow for easy access to a function's roots and maxi-mum and minimum values. Among other features, theyalso allow for the collection of data with a calculator-based laboratory (CBL). Therefore, this meta-analysisprovides results on mathematics achievement follow-ing student access to the basic graphing calculatorfeatures that are applicable to the study of lower levelmathematics courses. These features are found in theTI-82, TI-83, and TI-84 calculator models.

Research Method

Study Gathering and Data OrganizationThe search for studies began with a thorough

perusal of education-related databases, including Edu-cation Resources Information Center (ERIC),PsycINFO, Social Sciences Citation Index, Disserta-tion Abstracts Intemational, and the British EducationIndex with the keywords graphing calculator, graphicscalculator, programmable calculator, mathematicsachievement, and mathematics attitude. As a researchreport was read forpossible inclusion, its bibliographywas scanned for other appropriate studies.

A study was included if it (a) consisted of atreatment group with access to graphing calculatorsduring instruction and a control group studying the samesubj ect without graphing calculators, (b) provided meansand standard deviations or other data from which aneffect size could be calculated, and (c) was based onresearch conducted in a mainstream precollege orcollege classroom. These criteria yielded 42 studies, ofwhich four studies did not state whether a calculator(scientific or graphing) was allowed duringtesting. Theauthors of these studies were contacted for the neces-sary information before data analysis began.

Once the studies were collected, data was gleanedfrom each study for the calculation of effect sizes. Thestandardized mean difference - the difference in treat-ment and control group means divided by a pooledstandard deviation (Lipsey & Wilson, 2001) - was theeffect size measure used. All but one of the studiesprovided data necessary for calculation of an effectsize. The remaining study contained correlation coeffi-cients that were converted to effect sizes by a formulaoutlined in Lipsey and Wilson (2001). Hedges andOlkin (1985) proved that raw effect sizes havedistribution bias. Therefore, the effect sizes werecorrected for this difficulty, and the resulting valueswere used in the analysis.

Effect sizes were organized into categories basedon the achievement and attitude constmcts covered bythe studies gathered for evaluation. Twenty-eight studies



provided data for analysis in more than one category.With one exception, all studies provided only one effectsize for each achievement or attitude category. Shore(1999) studied the procedural and conceptualachievement of students in two different developmentalmathematics classes - Elementary Algebra andIntermediate Algebra. Since achievement tests for twodifferent classes provided data for two pairs of treatmentand control groups, each pair was considered a separateprimary study for analysis purposes.

With respect to the achievement category, databased on student development of procedural skills wereanalyzed separately from the data gathered on studentdevelopment of conceptual skills. Procedural skillswere those featuring the application of an algorithm,rule, or procedure to complete the problem at hand.Conceptual skills were skills used to understand math-ematics concepts and to apply those concepts to avariety of problem solving situations.

Data were included in these categories only if theauthor (a) provided data from an achievement test thatcontained only questions pertaining speci fically to stu-dents' procedural skills or conceptual skills or (b)separated the scores from the procedural and concep-tual questions of an overall achievement test andprovided data from which two separate effect sizescould be calculated. Data from an assessment of acombination of procedural and conceptual skills wereanalyzed in a separate category. The studies thatprovided data for this category were those that con-ducted an evaluation of overall mathematics achieve-ment and did not state that the skills being assessedwere solely procedural or conceptual.

Another significant factor in the organization ofdata was the role of the calculator in the assessmentprocess. The researchers of 26 studies allowed stu-dents in the treatment group to use graphing calculatorsduring assessment, but the remaining 16 studies did notallow the use of calculators during testing. Therefore,two separate analyses were conducted in each achieve-ment category (procedural, conceptual, and combined).One analysis, using the studies in which the calculatorwas part ofthe assessment process, was to determinewhether the calculator helped extend student achieve-ment beyond the actual treatment. The other analysis,based on the studies that did not include calculators intesting, determined whether students were able to main-tain the skills they leamed during the treatment phase.

Attitude data was gathered for the six factorsoutlined by the Mathematics Attitude Inventory (Sand-man, 1980). The factors are (a) attitude toward math-ematics (b) anxiety toward mathematics (c) self-concept

in mathematics (d) motivation to increase mathemati-cal knowledge (e) perception of mathematics teachersand (f) value of mathematics in society. Based on itsgeneral nature, most ofthe attitude data gathered wereevaluated in the first category and were used todetermine if the students who used graphing calcula-tors had better attitudes toward mathematics than didtheirnoncalculator counterparts.

All ofthe studies providing data for the other fivecategories either used the Mathematics Attitude In-ventory to assess a particular factor or used otheravailable measures like Fennema and Sherman's (1976)Mathematics Attitude Scales. Four studies provideddata on student attitudes toward the use of calculatorsin mathematics classrooms. These data were analyzedin a seventh category to determine whether studentsenjoyed having access to graphing calculators duringmathematics instruction.

Once the data were organized into the achievementand attitude categories, a statistical analysis was con-ducted with the data in each category. The proceduresoutlined in Practical Meta-Analysis by Lipsey andWilson (2001) were followed.

Statistical AnalysisHomogeneous data best represent the population

from which they were gathered, and each value in thesample is an estimate ofthe population's effect size(Hedges & Olkin, 1985). Hedges' Q statistic was usedto determine whether the data gathered for a constmctwere homogeneous. This statistic has a chi-squaredistribution with n - 1 degrees of freedom, where n isthe number of effect sizes. A statistically significantHedge's Q statistic (i.e.,/j < .01) refiects significantheterogeneity in the data and, therefore, the effect sizevalues are not estimates ofthe population mean. If thiswas the case in the evaluation of data for an achieve-ment or attitude category, then outliers were removedone at a time until the remaining effect sizes werehomogeneous.

Huffcutt and Arthur' s (1995) sample-adj usted meta-analytic deviancy statistic (SAMD) was used to findoutliers. This process takes both effect size magnitudeand sample size into account when determining whetheror not a value is an outlier. SAMD is the differencebetween a given effect size and the weighted meaneffect size (each value is weighted by its variance) ofthe other values divided by standard errors of thedifference for all ofthe studies. A graph of all SAMDvalues revealed possible outliers.

Once a homogeneous data set was ready forevaluation the meta-analytic process continued.



Sampling error is the primary reason for effect sizevariance, but since there was the possibility that othersources of variability were present, a random effectsmodel (Lipsey & Wilson, 2001) was used. Next, theweighted mean effect size and corresponding confidenceinterval for a category of effect sizes was calculated.The weighted mean effect size is the best representativeofthe population average effect size for a homogeneousset. Cohen (1988) has provided some general size limitsfor evaluating effect sizes, with the small, medium, andlarge categories being effect size values near 0.2,0.5,and 0.8, respectively.

The statistical significance of the weighted meaneffect size was determined by considering the corre-sponding confidence interval. If the confidence intervaldid not contain 0, then the difference between the meanassessment score of the treatment and control groupwas significant in favor of one of the groups. If thestatistically significant weighted mean effect size waspositive, then the studies for that category showed thatthe treatment group performed better than the controlgroup on assessment instmments. A negative weightedmean effect size and confidence interval revealed thatthe control groups outperformed the treatment groupson assessment measures in that particular category.Lastly, Cohen's (1988) U^ statistic was calculated foreach statistically significantweighted mean effect size.To give a clear interpretation of an effect size value, theU3 statistic converts the value to the percentage of areafalling below it on the standard normal curve (Cohen,1988). The median score lying at the exact middle ofthestandard normal curve is the value at which 50% ofstudents in the control group scored higher. Ujis thepercentage of students in the treatment group whoscored higher than the median score of the controlgroup.

Results

Overview ofthe StudiesForty-two studies satisfied the criteria for meta-

analysis and provided 97 effect sizes for evaluation.The achievement constructs yielded 74 effect sizes;the remaining 23 were generated for the attitudeconstructs.

Fifty-two percent of the researchers randomlyassigned classes to participate in the treatment. Sinceall of these studies involved intact classes, they wereconducted with quasi-experimental research designs,and none were conducted under tme experimentalconditions. The remaining 20 studies used intact classes,but they did not use random techniques to assign

classes to treatments. Eight ofthe studies appeared asjoumal articles, while the rest were doctoral disserta-tions or appeared in some other nonpublished form.

Over 90% of the researchers used teacher orresearcher designed tools for assessment of at leastone construct. Seven studies tested achievement orattitudes with a standardized assessment instrument.Three studies used a teacher-designed instrument toassess one construct and a standardized instmment foranother construct.

Half of the studies were conducted in colleges (2-and 4-year) or universities. High school students werethe participants in 40% of the research studies. Theremaining four studies were conducted with students ingrades 6-8. Forty-eight percent ofthe studies lasted 7weeks or less. Forty percent ofthe studies were 8 to 15weeks in duration, and six researchers conductedstudies that lasted more than half of the school year.

With respect to ability level, all but four of thestudies were conducted in mixed ability classrooms.One researcher characterized the students who par-ticipated as students of low ability, while three reportsstated that high ability students were the focus of thosestudies. The treatment groups in 17 studies used specialcurriculum materials designed for the graphing calcu-lator. The special materials took one of two forms: (a)supplemental materials designed by the researcher or(b) a textbook with a graphing calculator focus. For theremaining 63% ofthe studies, the treatment and controlgroups used the same, more traditional, curriculummaterials, including the same textbook.

The subjects covered by these studies varied fromalgebra I to calculus. All four middle school studies and12 ofthe high school studies tookplace in algebra I andn classrooms, while six high school studies wereconducted in precalculus classrooms. The subjectsfeatured in the community college classes were devel-opmental mathematics and college algebra. Collegealgebra, precalculus, and calculus were the focus ofthe4-year college or university studies.

In all studies, the treatment groups used graphingcalculators that did not have a keyboard or computeralgebra system capabilities. The students in the controlgroups were not allowed to use graphing calculators inclass or on homework assignments, but the teachers in18 studies required them to use scientific calculators.Two researchers stated that the control groups did nothave access to any type of calculator.

Results of Statistical AnalysisFor each constmct, the statistical analysis was

conducted twice, once with all ofthe studies and once



with outliers removed resulting in a homogeneous dataset. The weighted mean effect size, corresponding 95%confidence interval, Cohen'sUj statistic, and the homo-geneity statistic are presented for each analysis.

Achievement ResultsThe top of Table 1 contains a summary of the

statistical analysis results for studies in which graphingcalculators were part of instmction but not allowed intesting. The six studies assessing procedural skills werehomogeneous and yielded a weighted mean effect size(g = -.21) which according to Cohen's (1988) criteriawas small. The 95% confidence interval contained 0, sothe effect size value was not statistically significant infavor of either the treatment or control group.

Withrespect to conceptual skills. Hedges' Q statis-tic revealed that the 10 studies were heterogeneous.Two studies (Tolias, 1993;Upshaw, 1994) were deemedoutliers. After their removal, the remaining 8 studieswere homogeneous with a weighted mean effect size(g = .29) that represents a statistically significantdifference in favor of students who had access tographing calculators during instmction. This value isrepresented by a U3 statistic of 61. Sixty-one percent ofstudents using graphing calculators during instmction

scored higher than the median score of their non-graphing calculator counterparts on tests of conceptualskills. When separating the studies according to re-search design (random versus nonrandom assignmentto treatment), the weighted mean effect size for thestudies usingrandomassignment was slightly smaller (g= .24) than the weighted mean effect size for the studiesusing nonrandom assignment (g = .31).

The value for the random assignment studies wasnot statistically significant in favor ofthe treatment orcontrol groups, but the value for the nonrandom assign-ment studies was statistically significant in favor ofthetreatment group. Since the ideal research design in-cludes random assignment of subjects to treatments,the results for the randomly assigned studies are mostapplicable to the overall question of whether graphingcalculators help improve students' conceptual under-standing in mathematics.

Twelve studies assessed students' overall math-ematical achievement after having access to graphingcalculators during instmction. The weighted mean ef-fect size for this homogeneous data set (g=. 19) was notstatistically significant in favor of the treatment orcontrol groups. An analysis of random assignmentstudies compared to nonrandom assignment studies

Table 1Analysis of Mathematical

Construct

Testing Without CalcutatorsProcedural Skills

All studiesConceptual skills

All studiesOutliers removed

Combined SkillsAll studies

Testing With CalcutatorsProcedural skills

All studiesOutliers removed

Conceptual skillsAll studiesOutliers removed

Combined SkillsAll studiesOutliers removed

Skills

k

6

108

12

107

1411

1916

8

-.21

.3829

.19

.52

.32

.72

.42

.3529

a

(-.54,. 13)

(-.01,.78)(.01,.58)

(-.02, .39)

(.11,.92)(.04, .59)

(.37,1.06)(.18, .67)

(.09, ,61)(.14, .45)

61

7063

7666

(A61

Q

15.7

63.6*20.3

22.2

60.8*13.2

79.9*16.1

151.8*30.9

244

396314

470

404245

498280

1,115805

Nc

233

366284

400

336253

442282

1,236769

Note, k = number of studies; g = weighted mean effect size; CI = 95% confidence interval; U3 = percentage of area belowgon the standard normal curve; Q = homogeneity statistic; Ng= combined experimental group sample size; N ,̂ = combinedcontrol group sample size.*p<.Q\



yielded similar results to those presented previously forthe conceptual skills. The weighted mean effect size forthe studies using random assignment (g = .02) was notstatistically significant in favor of the treatment orcontrol groups, while the weighted mean effect size forthe studies using nonrandom assignment (g = .46) wasstatistically significant in favor ofthe treatment group.Based on the fact that random assignment is theresearch standard, the results for the randomly as-signed studies are most applicable to the question ofwhether the use of graphing calculators during instruc-tion helps students on assessments requiring a combina-tion of procedural and conceptual skills and not allowingcalculators.

The results for studies in which calculators wereused in testing and instruction are outlined at the bottomof Table 1. With respect to procedural skills, the 10effect sizes were heterogeneous, and three studies(Ruthven, 1990; Shore, 1999; Wilkins, 1995) weredeemed outliers. The weighted mean effect size for theremaining seven studies was reduced slightly in size (g= .32), but the value was statistically significant in favorof students who used graphing calculators. The Ujstatistic for this category revealed that 63% of studentsusing graphing calculators during testing and instructionscored higher than the median score of the controlgroup on examinations requiring the application ofmathematical formulas and procedures. Separatingthestudies according to research design (random versusnonrandom assignment to treatment) did not yielddifferent results.

The results for conceptual skills were similar to theresults forprocedural skills. Three studies were outliers(Cassity, 1997; Rodgers 1995; Wilkins, 1995), but theremaining studies revealed that the students who hadaccess to graphing calculators during testing and in-struction performed better than did their noncalculatorcounterparts on assessments of conceptual skills. Thecombined skills category contained the most effectsizes of any category evaluated in this meta-analysis.The 19 effect sizes were heterogeneous, but afterremoving three outliers (Harskamp, Suhre, & VanStreun, 1998;O'Neill 1994; Quesada & Maxwell 1994)the test for homogeneity revealed that the remaining 16studies were representative of the population fromwhich they were gathered. The statistically significantweighted mean effect size (g = .29) revealed that thestudents who had access to graphing calculators duringtesting and instruction outperformed the students with-out access to graphing calculators on assessments ofoverall mathematics achievement. An analysis ofthestudies separated according to whether or not the

researchers randomly assigned classes to treatmentsdid not yield different results.

Attitude ResultsTable 2 contains a summary ofthe results for the

attitude constructs. Three constructs (motivation toincrease mathematical knowledge; perception of math-ematics teachers; value of mathematics in society) didnot provide enough data for analysis. Ofthe constructsthat were analyzed, the attitude toward mathematicsconstruct was most featured in the research studies.The 13 studies in this category were heterogeneousaccording to Hedges' Q statistic. After the removal ofone outlier (Giamati, 1991), the weighted mean effectsize was slightly smaller (g = .21) but was statisticallysignificant in favor of the treatment group. The stu-dents who used graphing calculators in these studieswhile leaming mathematics had more positive attitudestoward mathematics at the conclusion of instructionwhen compared to their noncalculator counterparts.

Only two studies provided information to be ana-lyzed for the anxiety toward mathematics construct.The weighted mean effect size of 0.05 for this homo-geneous data was not statistically significant. At theend of graphing calculator treatment in these studies,the anxiety level of students in the treatment groupswas no different than the anxiety level of students in thecontrol groups. The data from four studies generated aweighted mean effect size of 0.45 for the self-conceptin mathematics construct. This medium sized valuewas based on heterogeneous data, and after the re-moval of an outlier (Scott, 1994) the weighted meaneffect size was smaller at 0.06 but not statisticallysignificant. This value reflects that there was no differ-ence in the mathematical self-concept of students whoused graphing calculators while leaming mathematics,compared to the mathematical self-conceptof studentswho did not have access to graphing calculators whileleaming mathematics.

With respect to student attitudes toward the use ofcalculators in mathematics, four studies generated aweighted mean effect size of 0.48. Hedges' Q statisticrevealed that this value was generated from homoge-neous data. It was also statistically significant in favorofthe treatment group, reflecting that the students whohad access to graphing calculators while leamingmathematics liked using them.

Discussion

The 42 studies gathered for this meta-analysisrepresent all ofthe research reports the located through



Table 2Analysis of Attitude Constructs

Construct

Attitude toward mathematicsAll studiesOutliers removed

Anxiety toward mathematicsAll Studies

Self-concept in mathematicsAll studiesOutlier removed

k

1312

2

43

Attitude toward use of calculatorsin mathematics

All studies 4

g

2921

.05

.45

.06

.48

a

(.02,.56)(.01,.40)

(-.37.47)

(-.32,1.22)(-.18,.3O)

(.27, .70)

6158

68

Q

51.1*22.7

1.3

43.4*0.6

2.7

611560

69

228142

274

Nc

504444

58

215131

141

Note, k=number of studies; g = weighted mean effect size; CI = 95% confidence interval; U3 = percentage of area belowg onthe standard normal curve; Q = homogeneity statistic; Ng= combined experimental group sample size; N ,̂ = combined controlgroup sample size.*p<.Ol

a thorough search of computer databases, inquiries ofmathematics educators, and scans of research report s.The generalizations outlined in this section are basedon these 42 studies. Since 91% of the researchanalyzed was conducted with high school or collegestudents, the results are most applicable to studentsbeyond the eighth year of education. All but threestudies were conducted in mixed ability classrooms.Therefore, these results are best applied to similarclasses, not classes containing solely low ability or highability students. Ninety-three percent ofthe studieswere conducted in algebra (high school and college)and precalculus classes, so the results are mostapplicable to students studying these subjects. Themost prominent graphing calculator versions in thesestudies were the TI-82, TI-83, TI-83 Plus, Casiofx7500, or Casio fx7700. Therefore, the results arebased on student access to the capabilities found inthese calculators.

When graphing calculators are included inmathematics instruction but not in the testing process,the calculator does not help students develop the skillsnecessary to apply mathematical formulas andprocedures. There is also no data to support that the useof calculators hurts student development of these typesof skills. The graphing calculator is an aid to studentleaming with respect to the skills necessary to understandmathematical concepts, even if the calculator is notpart of assessment. Under these same testing and

instruction conditions, the calculator is neither a helpnor a hindrance to students' overall mathematicsachievement, based on their use of a combination ofprocedural and conceptual skills.

When graphing calculators are included in bothtesting and instruction, students benefit from the use ofcalculators in development of mathematical skills in allthree categories that were analyzed (procedural, con-ceptual, and overall mathematics achievement). Basedon the U3 statistics for each category, students usinggraphing calculators in testing and instruction per-formed better on achievement tests than did at least60% ofthe students taught without access to graphingcalculators. Therefore, in all aspects of achievementand all types of skills, students benefit from usinggraphing calculators in mathematics instruction andhaving access to them while being tested.

This meta-analysis also reveals that the graphingcalculator has a positive effect on students' attitudestoward mathematics. Students who had access to agraphing calculator during instruction reported an atti-tude toward mathematics that was better than 58% ofthe students who did not have access to calculatorsduring instruction. Based on data from a small numberof studies, the graphing calculator does not have aneffect on students' anxiety toward mathematics andself-concept in doing mathematics. The research alsosupports that students who have access to graphingcalculators like using them while leamingmathematics.



Conclusion

Based on the results of this meta-analysis, studentsbenefit from using graphing calculators in the study ofmathematics. The details outlined in this report providesupport for the position taken by many mathematicalorganizations, including the NCTM, that graphing cal-culators should be an integral part of the study ofmathematics. Regardless ofthe mode of testing, graph-ing calculators help students with understanding math-ematical concepts. Withrespect to overall achievement,the results are most prevalent when calculators areallowed during assessment. This detail is particularlyrelevant to teachers and curriculum developers and isapparent when comparing the results of studies thatallowed calculators in testing and instruction with theresults from studies in which assessments were con-ducted under different circumstances than those underwhich leaming took place.

There were no circumstances under which thestudents taught without calculators performed betterthan the students with access to calculators. However,students receive the most benefit from graphing calcu-lators when they have access to them during bothaspects ofthe leamingprocess(instructionand testing).Based on the studies evaluated, teachers should decidewhat the role of the calculator will be in assessmentwhen deciding when and how to use them in instruction.

The search for studies revealed several areas inwhich quantitative research is lacking. In particular,more research is needed on retention of mathematicalskills after graphing calculator use. Further research isneeded on the general use of graphing calculators at themiddleschooUevel.

References

American Mathematical Association of Two-Year Colleges.(1995). Crossroads in mathematics: Standards for introductorycollege mathematics below calculus. Memphis, TN: Author.

Burrill, G., Allison, J., Breaux, G., Kastberg, S., Leatham, K.,& Sanchez, W. (2002). Handheld technology in secondarymathematics: Research findings and implications for classroompractice. Dallas, TX: Texas Instruments.

Cohen, J. (1988). Statistical power analysisforthebehavioralsciences. Hillsdale, NJ: Eribaum.

Ellington, A. (2003). Ameta-analysis of theeffects of calculatorson students in precollege mathematics classes. Journal for Researchin Mathematics Education, 34,433-463.

Fennema, E., & Sherman, J. (1976). Mathematics attitudescales: Instruments designed to measure attitudes toward theleaming of mathematics by males and females. Journal for Researchin Mathematics Education, 7,324-326,

Giamati, C. (1991). The effect of graphing calculator use onstudents' understanding of variations on a family of equations and

the transformations of their graphs. Dissertation AbstractsInternational, 52, 0103A. (UMI No, AAC91-16100)

Harskamp, E,, Suhre, C , & Van Streun, A, (1998). The graphicscalculator in mathematics education: An experiment in the Netherlands,Hiroshima Journal of Mathematics Education, 6,13-31.

Hedges, L, & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.

Hembree, R., & Dessart, D. (1986), Effects of hand-heldcalculators in precollege mathematics education: A meta-analysis.Journal for Research in Mathematics Education, 17, 83-99.

Hembree, R,, & Dessart, D, (1992), Research on calculators inmathematics education. In J, Fey(Ed,), Calcutators in mathematicseducation (pp, 23-32), Reston, VA: National Council ofTeachersof Mathematics.

Huffcutt, A,, & Arthur, W, (1995), Development of a newoutlier statistic formeta-analyticdata,youA-na/o/>lpp//erf/'ij'c/io/ogy,80,327-334,

Laughbaum, E, (1999), Hand-held technology in mathematicseducation at the college level [online]. Available: http://www,math,ohio-state.edu/~elaughba/chapters/98survey.pdf

Lipsey, M,, & Wilson, D, (2001), Practical meta-analysis.Thousand Oaks, CA: Sage,

Milou, E. (1999), The graphing calculator: A surveyof classroomusage. School Science and Mathematics, 99,133-137.

National Council of Teachers of Mathematics. (1989).Curriculum and evaluation standards for school mathematics.Reston, VA: Author,

O'Neill, C, (1994), Incorporating the graphing calculator intocollege algebra (Doctoral dissertation, Kansas State University,1994), Disseration A bstracts International, 56,3491.

Penglase, M,, & Amold, S, (1996), The graphics calculator inmathematics education: A critical review of recent research.Mathematics Education Research Journal, 8,58-90.

Quesada, A,, & Maxwell, M, (1994). The effects of usinggraphing calculators to enhance college students' performance inprecalculus. Educational Studies in Mathematics, 27,205-215,

Ruthven, K, (1990), The influence of graphic calculator use ontranslation from graphic to symbolic forms. Educational Studies inMathematics, 21,43\ -450,

Sandman, R. (1980), The mathematics attitude inventory:Instrumenxanduser'smanual. JournalforResearchinMathematicsEducation, I l,l4S-\49.

Scott, B. (1994), The effect of graphing calculators in algebraII classrooms: A study comparing achievement, attitude, andconfidence (Doctoral dissertation. University of North Texas,1994), Dissertation Abstracts International, 55,2755,

Shore, M, (1999), The effect of graphing calculators on collegestudents' ability to solve procedural and conceptual problems indevelopmental algebra (Doctoral dissertation. West VirginiaUniversity, 1999), Dissertation Abstracts International, 61,3932,

Tolias, G, (1993), The effects of using graphing technology incollege precalculus (Doctoral dissertation. University of Texas atAustin, 1993). Dissertation Abstracts International, 54, MIA.

Upshaw, J, (1994), The effect ofthe calculator-based, graph-exploration method of instruction on advanced placement calculusachievement (Doctoral dissertation. University of South Carolina,1993), Dissertation Abstracts International, 54,4023 A,

Wilkins, C, (1995). The effect ofthe graphing calculator onstudent achievement in factoring quadratic equations (Doctoraldissertation, Mississippi State University, 1995). DissertationAbstracts International, 56,2159A,



AppendixBibliography of Studies Included in the Meta-Analysis

Abuloum, K. (1996). Graphing calculators: Teachersperceptions, training, and attitude (Doctoral dissertation. UniversityofNebraska, \996). Dissertation Abstracts International, 57,1063 A.

Adams, T. (1996). Addressing students' difficulties with theconcept of function: Applying graphing calculators and a model ofconceptual change. Focus on Learning Problems in Mathematics,79,43-57.

Austin, J. (1996). Effect of graphing calculator use on studentachievement in college algebra: Gender and age-related differences(Doctoral dissertation. University ofMissouri,l 996). Diwertfl/ionAbstracts International, 57,3433.

Autin, N. (2001). The effects of graphing calculators onsecondary students' understanding of the inverse trigonometricfunctions. Dissertation Abstracts International, 62,0890A. (UMlNo. AAT30-09261)

Bucher, S. (2002). Graphingcalculators andquadratic functions(Masters thesis, California State University, 2002). MastersAbstracts International, 40,1110.

Caldwell, F. (1994). Effect of graphics calculators on collegestudents' learning of mathematical functions and graphs. Paperpresented at the Annual Conference of the American MathematicalAssociation of Two-year Colleges, Tulsa, Oklahoma, USA, ERICDocument Reproduction Service No. ED 393 669.

Carter, H. (1995). A visual approach to understanding thefunction concept using graphing calculators (Doctoral dissertation,Georgia State University, 1995). Dissertation AbstractsInternational, 56,3869.

Cassity, C. (1997). The relation of gender, spatial visualization,mathematical confidence, and classroom graphing calculatorutilization to conceptual mathematical performance: Learning withtechnology (Doctoral dissertation. University of Wyoming, 1997).Dissertation Abstracts International, 58,3095.

Chandler, P. (1992). The effect of the graphing calculator onhigh school students' mathematical achievement (Doctoraldissertation. University of Houston, 1992). Dissertation AbstractsInternational, 53,3832A.

Devantier, A. (1992). The impact of graphing calculators onthe understanding of functions and their graphs (Master's thesis.Central Michigan University, 1992). Masters AbstractsInternational, 31,57)5 K.

Drottar, J. (1999). An analysis of the effect of the graphingcalculator on student performance in algebra 11 (Doctoral dissertation,Boston College, 1999). Dissertation Abstracts International, 60,56A.

Fox, L. (1998) The effect of a graphing calculator used in anactive learning environment on intermediate algebra students'achievement and attitude (Doctoral dissertation. University ofSouth Florida, 1998). Dissertation Abstracts International, 59,0761.

Giamati, C. (1991). The effect of graphing calculator use onstudents' understanding of variations on a family of equations andthe transformations of their graphs. Dissertation AbstractsInternational, 52, 0103A. (UMI No. AAC91-16100)

Graham, A., & Thomas, M. (2000). Building a versatileunderstanding of algebraic variables with a graphic calculator.Educational Studies in Mathematics, 4I,165-1%1.

Graham, A., & Thomas, M. (2000). A graphic calculatorapproach to understanding algebraic variables. Paper presented atthe TIME 2000 International Conference, Auckland, New Zealand.

Harskamp, E., Suhre, C , c& Van Streun, A. (1998). The graphicscalculator in mathematics education: An experiment in the Netherlands.Hiroshima Journal of Mathematics Education, 6,13-31.

Hollar, J., & Norwood, K. (1999). The effects of a graphing-approach intermediate algebracurriculum on students' understandingof function. Journal for Research in Mathematics Education, 30,220-226.

Hylton-Lindsay, A. (1997). The effect of the graphing calculatoron metacogniti ve aspects of student performance in precalculus forbusiness (Doctoral dissertation, Columbia University TeachersCollege, 1997). Dissertation Abstracts International, 58,3449.

Lesmeister, L. (1997). The effect of graphing calculators onsecondary mathematics achievement (Master's thesis. Universityof Houston Clear Lake, 1996). Masters Abstracts International, 35,0039A.

Merckling, W. (1999). Relationship(s) between the perceptualpreferences of secondary school students and their achievement infunctions using a graphing calculator (Doctoral dissertation, St.Johns University, 1999). Dissertation Abstracts International, 60,0371.

Mustafa, A. (1997). An investigation of the understanding ofthe numerical experience associated with the global behavior ofpolynomial functions for students in graphing and non-graphingcalculator college algebra courses (Doctoral dissertation. Universityof New Orleans, 1997). Dissertation Abstracts International, 58,1629.

Nimmons, L. (1997). Spatial ability and dispositions towardmathematics in college algebra: Gender-related differences (Doctoraldissertation, Georgia State University, 1997). Dissertation AbstractsInternational, 58,3054.

O'Neill, C. (1994). Incorporating the graphing calculator intocollege algebra (Doctoral dissertation, Kansas State University,1994). Disseration Abstracts International, 56,3491.

Oster, J. (1994). Graphics technology and calculus readiness:The effects of instruction using a programmable scientific graphingcalculator on conceptual and procedural understanding in precalculus(Doctoral dissertation, Florida Institute of Technology, 1994).Dissertation Abstracts International, 55,3466.

Ottinger, T. (1994). Conceptual and procedural learning infirst-year algebra using graphing calculators and computers (Doctoraldissertation, Georgia State University, 1993). D/Merta/(ony4i5<rac/5International, 54,2934A.

Pankow, C. (1994). The effects of college student use ofgraphics calculators on the learning of algebraic concepts (Doctoraldissertation. University of North Dakota, 1994). DissertationAbstracts International, 55,7)11 A.

Quesada, A., & Maxwell, M. (1994). The effects of usinggraphing calculators to enhance college students' performance inprecalculus. Educational Studies in Mathematics, 27,205-215.

Rich, S. (1990). The effect of the use of graphing calculators onthe learning of function concepts in precalculus mathematics(Doctoraldissertation. The University of Iowa, 1990). Dissertation AbstractsInternational, 52,0835.

Ritz, C. (1999). The effect of using graphing calculators onstudent achievement in introductory algebra and geometry (Mastersthesis, California State University, 1999). Masters AbstractsInternational, 38,525.

Rodgers, K. (1995). The effects on achievement, retention ofmathematical knowledge, and attitudes toward mathematics as aresult of supplementing the traditional algebra II curriculum withgraphing calculator activities(Doctoraldissertation, Southern IllinoisUniversity, 1995). Dissertation Abstracts International, 57,0091.

Ruthven, K. (1990). The influence of graphic calculator use on



translation from graphic to symbolic forms. Educational Studies inMathematics, 2I,43\ -450.

Scott, B. (1994). The effect of graphing calculators in algebraII classrooms: A study comparing achievement, attitude, andconfidence (Doctoral dissertation. University of North Texas,1994). Dissertation Abstracts International, 55,2755.

Seavertson, P. (1995). Comparing the use of the graphicscalculator and scientific calculator in college algebra (Doctoraldissertation. University of Kansas, 1995). Dissertation AbstractsInternational, 56,4309.

Shoaf-Grubbs, M. (1994). The effect of the graphing calculatoron female students' spatial visualization skills and level-of-understanding in elementary graphing and algebra concepts. In E.Dubinsky, Schoenfeld, A., & Kaput, J. (Eds.), CBMS issues inmathematics education (Vol. 4; pp. 169-194). Providence, RhodeIsland: American Mathematical Society.

Shore, M. (1999). The effect of graphing calculators on collegestudents' ability to solve procedural and conceptual problems indevelopmental algebra (Doctoral dissertation. West VirginiaUniversity, \999). Dissertation Abstracts International, 61,3932.

Thomasson, S. (1992). The effects of the graphing calculatoron the achievement and attitude of college students enrolled inelementary algebra (Doctoral dissertation. University ofTennessee,\992). Dissertation Abstracts Internationa.\, 53,3S35.

Tolias, G. (1993). The effects ofusing graphing technology incollege precalculus (Doctoral dissertation. University of Texas atAustin, \993). Dissertation Abstracts International, 54, MIA.

Upshaw, J. (1994). The effect of the calculator-based, graph-exploration method of instruction on advanced placement calculus

achievement (Doctoral dissertation. University of South Carolina,1993). Dissertation Abstracts International, 54,4023A.

Vasquez, S., & McCabe, T. (2002). The effect of calculatorusage in the learning of basic skills. Research and Teaching inDevelopmental Education, 19,33-40.

Vazquez, J. (1991). The effect of the calculator on studentachievement in graphing linear functions (Doctoral dissertation.University of Florida, 1991). Dissertation Abstracts International,J/,3660.

Weber, T. (1998). Graphing technology and its effect onsolving inequalities (Doctoral dissertation, Wayne State University,1998). Dissertation Abstracts International, 60,0088.

Wilkins, C. (1995). The effect of the graphing calculator onstudent achievement in factoring quadratic equations (Doctoraldissertation, Mississippi State University, 1995). DissertationAbstracts International, 56,2159A.

; Correspondence concerning this article shouldbe addressed to Aimee J. Ellington, Virginia CommonwealthUniversity, P.O. Box 842014,1001 W. Main Street, Richmond, VA23284-2014. Electronic mail may be sent via Internet [email protected]


the effects of non-cas graphing calculators on student...

Documents