pursuing excellence: comparisons of international eighth ...nces.ed.gov/pubs2001/2001028a.pdf ·...

INITIAL FINDINGS FROM THE

THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY – REPEAT

OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT

U.S. DEPARTMENT OF EDUCATION

NCES 2001–028

Pursuing Excellence:

Comparisons of International Eighth-Grade Mathematics and

Science Achievement from a U.S.Perspective, 1995 and 1999

NATIONAL CENTER FOR EDUCATION STATISTICSStatistical Analysis Report December 2000

Pursuing Excellence: Comparisons of International Eighth-Grade Mathematics and

Science Achievement from a U.S. Perspective,1995 and 1999

Initial Findings from the Third International Mathematics and Science Study–Repeat

Patrick GonzalesChristopher Calsyn

Leslie JocelynKitty Mak

David KastbergSousan Arafeh

Trevor WilliamsWinnie Tsen

United States TIMSS–R Technical Review Panel

Margaret Cozzens, co-chair Susan Fuhrman, co-chair

Gordon Ambach Jerry PineRuben Carriedo Andrew PorterColette Chabbott Francisco RamirezDenis P. Doyle Linda RosenRamesh Gangolli William SchmidtGerry House* James W. StiglerJeremy Kilpatrick Lisa Towne*Paul Kimmelman Susan TraimanShirley Malcom

*through June 2000

U.S. Department of Education

Richard W. RileySecretary

Office of Educational Research and Improvement

C. Kent McGuireAssistant Secretary

National Center for Education Statistics

Gary W. PhillipsActing Commissioner

The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting datarelated to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, andreport full and complete statistics on the condition of education in the United States; conduct and publish reports and specializedanalyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statisticalsystems; and review and report on education activities in foreign nations.

NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurateindicators of education status and trends; and report timely, useful, and high-quality data to the U.S. Department of Education,the Congress, the states, other education policymakers, practitioners, data users, and the general public.

We strive to make our products available in a variety of formats and in language that is appropriate to a variety of audiences. You,as our customer, are the best judge of our success in communicating information effectively. If you have any comments or sugges-tions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to

National Center for Education StatisticsOffice of Educational Research and ImprovementU.S. Department of Education1990 K Street NWWashington, DC 20006

December 2000

The NCES World Wide Web Home Page is http://nces.ed.gov

The NCES World Wide Web Electronic Catalog is http://nces.ed.gov/pubsearch

Suggested Citation

U.S. Department of Education. National Center for Education Statistics. Pursuing Excellence: Comparisons of International Eighth-Grade Mathematics and Science Achievement from a U.S. Perspective, 1995 and 1999, NCES 2001–028, by Patrick Gonzales,Christopher Calsyn, Leslie Jocelyn, Kitty Mak, David Kastberg, Sousan Arafeh, Trevor Williams, and Winnie Tsen. Washington,DC: U.S. Government Printing Office, 2000.

For ordering information on this report, write:

U.S. Department of EducationED PubsP.O. Box 1398Jessup, MD 20794-1398

Or call toll free 1–877–4ED–PUBS or go to the Internet: http://www.ed.gov/pubs/edpubs.html

Contact: Patrick Gonzales (202) 502–7346

ii

This report, Pursuing Excellence: Comparisons of International Eighth-Grade Mathematics andScience Achievement from a U.S. Perspective, 1995 and 1999, is a collaborative effort by theNational Center for Education Statistics (NCES), the National Science Foundation (NSF), andthe Office of Educational Research and Improvement (OERI). NCES is responsible for theanalyses presented in this report. Funding for the U.S. portion of the Third InternationalMathematics and Science Study–Repeat (TIMSS–R) was provided by NCES and NSF, with addi-tional funding from OERI.

iii

ACKNOWLEDGMENTSA report of this nature and scope is never only the work of its authors. There are teams of peoplewho, behind the scenes, make substantial contributions of content and process that move areport like this one toward completion. The authors wish to thank all those who contributed tothe design, writing, and production of this report for their thoughtful critique, insightful sugges-tions, and creativity.

Members of the TIMSS–R Interagency Management Team provided excellent ideas and direc-tion from the start. Members include Janice Earle, Larry Suter, and Elizabeth VanderPutten, ofthe National Science Foundation (NSF); Carol Fromboluti, Jill Edwards Staton, and PatriciaRoss of OERI; Laura Lippman, Eugene Owen, and Val Plisko of NCES; and Maggie McNeely,formerly of OERI and now with the Office of Elementary and Secondary Education. The Teamreceived valuable support from Naoko Kataoka, Jay Moskowitz, Yasmin Shaffi, and MariaStephens of the American Institutes for Research (AIR).

Ellen Bradburn of NCES, Laura Salganik of the Education Statistics Services Institute (ESSI),and Sally Dillow of AIR provided excellent technical and editorial advice. Invited reviewers whogave of their time and expertise include the members of the TIMSS–R Technical Review Panel,Senta Raizen of the National Center for the Improvement of Science Education, John Dossey ofIllinois State University, Mary Lindquist of Columbus State University, and Arnold Goldstein,Patrick Rooney, Jeffrey Owings, Laura Lippman, and Marilyn McMillen, all of NCES.

Finally, the graphics and layout of the report would not have been possible without the creativ-ity of Brian Henigin and Karen Moyes of Westat.

ON THE COVER: World 2000 mural, Copyright 2000, International Child Art Foundation. Kindpermission to reproduce the artwork was granted by the International Child Art Foundation(ICAF), a Washington, DC-based nonprofit organization that nurtures, promotes, and cele-brates children’s art and creativity locally, nationally, and internationally. The mural was createdon the National Mall in Washington, DC, on June 30, 1999, by child artists from 50 nationsaround the world. The original mural is 16 feet by 24 feet and was created as part of the ChildArt2000 Festival. For details, visit www.icaf.org.

iv

COMMISSIONER’S STATEMENTThe Third International Mathematics and Science Study–Repeat (TIMSS–R) is the latestchapter in one of the most comprehensive and rigorous international studies of school-ing and student achievement ever conducted. TIMSS–R, conducted in 1999, comes fouryears after TIMSS, and was designed to focus on the mathematics and science achieve-ment of eighth-grade students. NCES and the National Science Foundation (NSF)supported the United States’ participation in TIMSS–R to provide an update on themathematics and science performance of U.S. eighth-grade students originally detailedin the 1995 TIMSS study. This report, Pursuing Excellence: Comparisons of InternationalMathematics and Science Achievement from a U.S. Perspective, 1995 and 1999, presentsinitial findings on how our eighth-grade students fared on TIMSS–R and whether therehave been significant changes in achievement in the four years since TIMSS.

TIMSS–R addresses the mission of NCES to gather and publish information on thestatus and progress of education in the United States and other nations, and continuesthe tradition of U.S. participation in international comparative studies of mathematicsand science education since the 1960s. TIMSS–R represents an advancement in tradi-tional studies because it is the first international study specifically designed to trackchanges in achievement. The data on mathematics and science achievement collected inTIMSS–R can be compared to the 1995 TIMSS data to identify changes between theeighth-grade students of yesterday and today, and relative changes between fourth-gradestudents 4 years earlier and their classmates 4 years later. While the same students didnot participate in both studies, a scientific sampling of the two groups of studentsprovides the most accurate picture available of their mathematics and science perform-ance from an international comparative perspective. Information from TIMSS–R, incombination with what we have learned from the National Assessment of EducationalProgress (NAEP), provides an opportunity to take stock of mathematics and scienceperformance of our students.

One of the most important steps in making good decisions is to have good data.TIMSS–R fills that need and is one of the many surveys and assessments conducted byNCES that can be used by U.S. educators, parents, policymakers, and business leaders tomake important decisions that will improve student learning. In addition to data onstudent performance, TIMSS–R includes a wealth of information on the context withinwhich student learning takes place, such as teaching practices, students’ study habits,teacher training and professional development, and school policies. Taken into consid-eration with other knowledge about the education systems of participating nations,TIMSS and TIMSS–R provide a thoughtful and in-depth look into what our eighth-grade mathematics and science teachers teach and what our eighth-grade students learnin comparison to their counterparts in other nations of the world.

In conclusion, TIMSS–R is a learning experience. The information presented in thisreport is presented in a straightforward way, and is not intended to determine whetherU.S. performance is good or bad. Rather, it is intended to provide you, the reader, with

v

the most accurate and up-to-date information available. The importance of this infor-mation, and its impact on American education, will depend on how it is used to improveour mathematics and science education. My colleagues and I invite everyone dedicatedto enhancing the quality of our nation’s mathematics and science education to make thefullest possible use of this rich resource.

Gary W. Phillips December 2000Acting Commissioner of Education Statistics

vi

NSF DIRECTOR’S STATEMENTIt is critical that students in the United States achieve at high levels in mathematics andscience. The position of the U.S. in the world economy, the continuing demand for well-trained mathematicians and scientists, and the need for an informed citizenry able tomake intelligent public-policy decisions about important economic, medical, and envi-ronmental issues all depend upon it.

Studies such as TIMSS–R help us place the achievement of U.S. students into an inter-national context and thus provide important additional sources of information forevaluation of student abilities. The National Science Foundation (NSF) has co-fundedthe TIMSS–R study and has actively participated in its management for this reason.

The careful design of the TIMSS–R study provides an opportunity to analyze trends inthe achievement of eighth-grade students in the 23 countries that participated in both1995 and 1999. The results show that U.S. eighth-grade students continue to perform atthe international average in science and just below the international average in mathe-matics, with no statistically significant changes in their level of achievement from 1995to 1999. Indeed, this is true for most of the countries participating in both years,although some countries (e.g., Canada) did make significant gains. A thorough analysisof the reasons for these exceptional gains may provide insight into possible strategies forimproving education in the United States.

The timing of TIMSS–R allows us to compare results across grades in the 17 nations thatparticipated in both the fourth-grade TIMSS in 1995 and the eighth-grade TIMSS–R in1999. It is disturbing that the international ranking across these 17 nations of the U.S.eighth-grade students is relatively poor in both mathematics and science whencompared with that of U.S. fourth-graders in 1995. This confirms the disappointingshowing of our eighth-grade students in international comparisons, and demonstratesthat the decline in relative performance during the middle school years is a continuingand serious problem.

The initial TIMSS study indicated that student achievement is the result of multiplefactors. In schools, curriculum, teacher qualifications, and high expectations for allstudents are critical. Other factors, such as the educational resources available to thefamily, also may be key to student success. For example, achievement differences foundbetween student groups or by type of school may be narrowed or eliminated whenparent education and home resources are used in the analyses.

This first TIMSS–R report does not analyze the relationships between contextual vari-ables and student achievement. However, it contains a preliminary comparison of theU.S. with other nations on a number of factors. For example, U.S. eighth-grade teachersare less likely to have majors and minors in mathematics and science than their coun-terparts in most other countries. This finding is consistent with other reports such asBefore It’s Too Late: A Report to the Nation from the National Commission of Mathematicsand Science Teaching for the 21st Century.

vii

We look forward to further analysis of the data in this report, the release of data from 27U.S. benchmarking jurisdictions that engaged in TIMSS–R as if they were separatenations, and the companion classroom video studies. These will enrich our under-standing of the factors that contribute to the disappointing achievement levels of U.S.eighth-grade students. Similar detail from the 1995 TIMSS study revealed the impor-tance of rigorous mathematics and science curricula and alerted researchers to the needfor teachers to have deep content knowledge in order to use those curricula successfullyand achieve high standards for all students.

NSF is pleased to have supported this important study and report. The data containedwithin the TIMSS–R study will be used for years to understand issues and trends in theteaching of mathematics and science. Simply said, it is an invaluable resource.

Rita R. Colwell December 2000DirectorNational Science Foundation

viii

TABLE OF CONTENTS

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iv

Commissioner’s Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v

NSF Director’s Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vii

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xii

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiv

Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Why are international comparisons of education important? . . . . . . . . . . . . . . .2

Why a repeat of TIMSS? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2

What questions does this report address? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

What issues does this report not address? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

What is TIMSS–R? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Which nations participated in TIMSS–R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

How was TIMSS–R conducted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

Are the results from TIMSS and TIMSS–R comparable? . . . . . . . . . . . . . . . . . . .7

How can we be sure the data are comparable across nations? . . . . . . . . . . . . . . .7

How does TIMSS–R relate to other large-scale studies of mathematics and science achievement? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8

How is the rest of the report organized? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Chapter 2: Achievement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Key Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

What do the test scores mean? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

The Mathematics and Science Achievement of Eighth-Graders in 1999 . . . . . . . . .12

How well did U.S. eighth-graders perform in 1999? . . . . . . . . . . . . . . . . . . . . .14

What percentage of our students scored at or above the international top 10 percent benchmark in 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

What percentage of our students scored at or above the international top 25 percent benchmark in 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

How well did U.S. eighth-graders perform in the different content areas in 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

What were students asked to do on the TIMSS–R assessment? . . . . . . . . . . . . .20

ix

How did different groups of students within the United States perform? . . . . .29

Was there a difference in the mathematics and science achievement ofU.S. eighth-grade boys and girls? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

Did the achievement of U.S. students differ by race and ethnicity? . . . . . . . . . .31

Did the achievement of students in U.S. public and nonpublic schoolsdiffer? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

Did the achievement of U.S. students of different national origins differ? . . . .31

Did the achievement of U.S. students differ by the level of their parents’education? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

The Mathematics and Science Achievement of Eighth-Graders Between1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Did the performance of U.S. eighth-graders change between 1995 and 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Did the percentage of U.S. students at or above the international top 10 percent benchmark change over the 4 years? . . . . . . . . . . . . . . . . . .35

Did the percentage of U.S. students at or above the international top 25 percent benchmark change over the 4 years? . . . . . . . . . . . . . . . . . .38

Did the performance of U.S. eighth-graders in the content areas change between 1995 and 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

Did the performance of U.S. population groups change between 1995 and 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

The Mathematics and Science Achievement of the 1995 Fourth-Grade Cohort in 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

Has the relative performance of the United States changed between fourth and eighth grade over the 4 years? . . . . . . . . . . . . . . . . . . . . . . . . . .40

Chapter 3: Teaching and Curriculum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Key Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Teacher Preparation, Qualifications, and Professional Development . . . . . . . . . . . .44

What educational backgrounds did our mathematics teachers havein 1999? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

What educational backgrounds did our science teachers have in 1999? . . . . . .46

How confident were mathematics teachers in their preparation to teach mathematics subjects? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47

How confident were science teachers in their preparation to teachscience subjects? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

In what types of professional development activities did our mathematics teachers participate? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

In what types of professional development activities did our scienceteachers participate? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

Did our mathematics teachers observe one another teaching? . . . . . . . . . . . . .50

Did our science teachers observe one another teaching? . . . . . . . . . . . . . . . . . .50

x

What topics were emphasized in professional development activities for U.S. mathematics teachers? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50

What topics were emphasized in professional development activities for U.S. science teachers? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51

Curriculum, Content Coverage, and Emphases . . . . . . . . . . . . . . . . . . . . . . . . . . . .51

Who sets the curriculum in TIMSS–R nations? . . . . . . . . . . . . . . . . . . . . . . . . .52

How much of each TIMSS–R content area did the intended U.S. curriculum Cover? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52

How much of the mathematics curriculum was taught? . . . . . . . . . . . . . . . . . .53

How much of the science curriculum was taught? . . . . . . . . . . . . . . . . . . . . . . .54

Which topics were emphasized most in U.S. eighth-grade curricula? . . . . . . . .54

Did the TIMSS–R nations’ curricula accommodate students with varying degrees of interests and abilities? . . . . . . . . . . . . . . . . . . . . . .55

Classroom Practices and Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55

What kinds of skills did U.S. mathematics and science teachers report asking their students to use during lessons? . . . . . . . . . . . . . . . . . . .55

What activities did U.S. students report occurring in their mathematics and science classes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56

How often did U.S. students use calculators in their mathematics lessons? . . .58

Did students have access to computers and the Internet, and how did schools, teachers, and students report using these tools? . . . . . . . . . . .58

How often did U.S. students discuss completed homework or begin homework in their mathematics and science classes? . . . . . . . . . . . . . . . . .60

How much time did U.S. students spend studying mathematics or doing mathematics homework outside of school? . . . . . . . . . . . . . . . . . . .61

Chapter 4: Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

Works Cited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67

Appendix 1: TIMSS Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71

Appendix 2: Technical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Appendix 3: Supporting Data for Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85

Appendix 4: Supporting Data for Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107

Appendix 5: Comparisons of All TIMSS and TIMSS–R Nations . . . . . . . . . . . . . . . . . . . . .115

xi

LIST OF TABLESA2.1: Coverage of target population, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

A2.2: Student and school samples and participation rates, by nation: 1999 . . . . . . . . . . .78

A2.3: Number of items by item format in main survey . . . . . . . . . . . . . . . . . . . . . . . . . . .80

A2.4: Number of mathematics items by content area in main survey . . . . . . . . . . . . . . . .81

A2.5: Number of science items by content area in main survey . . . . . . . . . . . . . . . . . . . .81

A2.6: Fourth- and eighth-grade nations in TIMSS: 1995 . . . . . . . . . . . . . . . . . . . . . . . . . .83

A3.1: Average mathematics and science achievement of eighth-grade students with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . .86

A3.2: Percentiles of achievement in eighth-grade mathematics with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87

A3.3: Percentiles of achievement in eighth-grade science with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88

A3.4: Average eighth-grade achievement in mathematics content areas with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89

A3.5: Average eighth-grade achievement in science content areas with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90

A3.6: Percent correct on mathematics assessment item examples with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91

A3.7: Percent correct on science assessment item examples with standard errors, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92

A3.8: U.S. eighth-grade mathematics and science achievement with standard errors, by selected characteristics: 1999 . . . . . . . . . . . . . . . . . . . . . . . .93

A3.9: Average mathematics and science achievement of eighth-grade students with standard errors, by sex, by nation: 1999 . . . . . . . . . . . . . . . . . . . .94

A3.10: Comparisons of eighth-grade mathematics achievement with standard errors, by nation: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

A3.11: Comparisons of eighth-grade science achievement with standard errors,by nation: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .96

A3.12: Comparisons of percentages of eighth-grade students reaching the TIMSS–R1999 top 10 percent international benchmark of mathematics achievement with standard errors: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . .97

A3.13: Comparisons of percentages of eighth-grade students reaching the TIMSS–R 1999 top 10 percent international benchmark of science achievement with standard errors: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . .98

A3.14: Comparisons of percent correct in mathematics content areas withstandard errors: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

xii

A3.15: Comparisons of percent correct in science content areas with standard errors:1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101

A3.16: U.S. mathematics and science achievement with standard errors,by selected characteristics: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . .103

A3.17: Mathematics achievement of TIMSS–R 1999 nations that participated in 1995 at both the fourth and eighth grades relative to the averageacross these nations with standard errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104

A3.18: Science achievement of TIMSS–R 1999 nations that participated in 1995 at both the fourth and eighth grades relative to the average across these nations with standard errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105

A4.1: Organization of science instruction at grade 8, by nation: 1999 . . . . . . . . . . . . . .108

A4.2: Eighth-grade mathematics teachers’ reports of their main areaof study with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109

A4.3: Eighth-grade science teachers’ reports of their main area of studywith standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109

A4.4 Teachers’ beliefs about their preparation to teach mathematics and science with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110

A4.5: Percentage of U.S. eighth-grade students taught by teachers that participated in professional development activities that emphasizeddifferent topics with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . .111

A4.6: Percentages of eighth-grade students “taught” mathematics content areas with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111

A4.7: Percentages of eighth-grade students “taught” science content areaswith standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111

A4.8: Eighth-grade students’ reports of the occurrence of selected activitiesin their mathematics class “almost always” or “pretty often” with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112

A4.9: Eighth-grade students’ reports of the occurrence of selected activitiesin their science class “almost always” or “pretty often” with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112

A4.10: Eighth-grade students’ reports of access to computers and the Internet with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112

A4.11: Eighth-grade students’ reports of using computers in mathematics and science classes “almost always” or “pretty often” with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113

A4.12 Eighth-grade students’ reports of discussing or beginning homework in mathematics and science classes “almost always” or “pretty often”with standard errors: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113

A5.1 Mathematics and science achievement of TIMSS–R and TIMSSnations with standard errors: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . .117

xiii

LIST OF FIGURESFigure 1: Participation in TIMSS and TIMSS–R: 1995 and 1999 . . . . . . . . . . . . . . . . . . . .6

Figure 2: Average mathematics and science achievement of eighth-grade students, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

Figure 3: Percentages of eighth-grade students reaching the TIMSS–R 1999 top 10 percent in mathematics and science achievement,by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

Figure 4: Average eighth-grade achievement in mathematics contentareas, by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Figure 5: Average eighth-grade achievement in science content areas,by nation: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Figure 6: Example mathematics item 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20





Figure 11: Example science item 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24



Figure 14 Example science item 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27



Figure 17: U.S. eighth-grade mathematics and science achievement,by selected characteristics: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Figure 18: Comparisons of eighth-grade mathematics achievement, by nation:1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Figure 19: Comparisons of eighth-grade science achievement, by nation: 1995and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Figure 20: Comparisons of percentages of eighth-grade mathematics students reaching the TIMSS–R 1999 top 10 percent in mathematics achievement, by nation: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . .36

Figure 21: Comparisons of percentages of eighth-grade science students reaching the TIMSS–R 1999 top 10 percent in science achievement,by nation: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

Figure 22: Changes in U.S. eighth-grade mathematics and science achievement,by U.S. selected characteristics: 1995 and 1999 . . . . . . . . . . . . . . . . . . . . . .39

xiv

Figure 23: Mathematics achievement for TIMSS–R 1999 nations that participated in 1995 at both the fourth and eighth gradesrelative to the average across these nations . . . . . . . . . . . . . . . . . . . . . . . . .41

Figure 24: Science achievement for TIMSS–R 1999 nations that participatedin 1995 at both the fourth and eighth grades relative to the average across these nations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

Figure 25: Eighth-grade mathematics teachers’ reports on their main area of study: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45

Figure 26: Eighth-grade science teachers’ reports on their main area ofstudy: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46

Figure 27: Teachers’ beliefs about their preparation to teach mathematics and science: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48

Figure 28: Percentages of U.S. eighth-grade students taught by teachers that participated in professional development activities that emphasized different topics: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51

Figure 29: Percentage of U.S. eighth-grade students “taught” mathematics content areas: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53

Figure 30: Percentage of U.S. eighth-grade students “taught” science content areas: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54

Figure 31: Eighth-grade students’ reports of the occurrence of selected activities in their mathematics class “almost always” or “pretty often”: 1999 . . . . . .56

Figure 32: Eighth-grade students’ reports of the occurrence of selected activities in their science class “almost always” or “pretty often”: 1999 . . . . . . . . . . .57

Figure 33: Eighth-grade students’ reports of access to computer and the Internet: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58

Figure 34: Eighth-grade students’ reports on using computer in mathematics and science classes “almost always” or “pretty often”: 1999 . . . . . . . . . . . .59

Figure 35: Eighth-grade students’ reports of discussing or beginning homeworkin mathematics and science classes “almost always” or “pretty often”: 1999 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61

xv

CHAPTER 1INTRODUCTION

The National Science Foundation (NSF), the U.S.Department of Education’s Office of EducationalResearch and Improvement (OERI), and theNational Center for Education Statistics (NCES)joined together to support the participation of theUnited States in the Third InternationalMathematics and Science Study–Repeat(TIMSS–R), a successor to the 1995 ThirdInternational Mathematics and Science Study(TIMSS).1 The joint research effort has producedrich information on the mathematics and scienceperformance of U.S. eighth-grade students. Thisreport, Pursuing Excellence: Comparisons ofInternational Eighth-Grade Mathematics andScience Achievement from a U.S. Perspective, 1995and 1999 presents initial findings from theTIMSS–R study.

Why are international comparisons of educationimportant?International comparisons of student achieve-ment and various background factors related toteaching and learning have been conducted forover 30 years. Many observers believe that suchcomparisons can help policymakers, researchers,teachers, and parents understand what othernations do to further the educational achievementof their populations. Some also believe that if theUnited States wants to remain internationallycompetitive, we need to better understand howour students perform in critical areas such asmathematics and science. Moreover, some are ofthe opinion that international assessments are oneway of seeing what our national, state, and localstandards mean in a world context. In short, inter-national assessments can expand comparisons ofeducational achievement to other systems outsidethe United States; aid in our understanding of thepossible reasons for observed differences inachievement; document the many varied educa-tion and learning practices around the world; get asense of resources available to students in differentnations; and improve the study of education itself(Board on International Comparative Studies inEducation, 1990; Medrich and Griffith 1992).

Why a repeat of TIMSS?The series of NCES reports on the 1995 TIMSSstudy described the mathematics and scienceperformance of U.S. students in comparison totheir peers at three different grade levels (NCES1996, 1997c, 1998, 2000a).2 The 1995 TIMSSassessments revealed that U.S. fourth-gradersperformed well in both mathematics and sciencein comparison to students in other nations, U.S.eighth-grade students performed near the inter-national average in both mathematics and science,and U.S. twelfth-graders scored below the interna-tional average and among the lowest of the TIMSSnations in mathematics and science generalknowledge, as well as in physics and advancedmathematics.

The participation of the United States in TIMSSheightened the nation’s interest in improvingmathematics and science education. Althoughwork on improving mathematics and scienceeducation began years before TIMSS, results fromTIMSS have had an impact on the way the UnitedStates thinks about mathematics and scienceeducation (Welch 2000).

TIMSS–R continues the tradition of internationalcomparative study of mathematics and scienceeducation begun in the 1960s. The contribution ofTIMSS–R is unique, however, because its designmakes it possible to track changes in achievementand certain background factors from the earlierTIMSS study—a first for any international study.Moreover, TIMSS–R is the first internationalassessment that provides some indication of thepace of educational change across nations,informing expectations as to what can beachieved. TIMSS–R provides valuable informationon the state of education in the United States andother nations in 1999.

Thirty-eight nations chose to compare the mathe-matics and science performance of their studentsin 1999. However, unlike TIMSS, the 1999TIMSS–R study focused on eighth-grade studentsonly. TIMSS–R allows the United States tocompare the achievement of its eighth-graders in

2

CHAPTER 1—INTRODUCTION

1TIMSS collected data during the 1994–95 school year. TIMSS–R collected data during the 1998–99 school year. For convenience,reference will be made to 1995 and 1999, respectively, throughout this report.

2See appendix 1 for a brief list of TIMSS-related publications.

the original TIMSS to the scores of its eighth-graders four years later in TIMSS–R. It alsoprovides an opportunity for the United States tocompare the relative performance of a cohort offourth-graders in 1995 to the relative performanceof a cohort of eighth-graders 4 years later in 1999.3

In short, TIMSS–R should help us understand theoverall progress that our schools, teachers, andstudents are making toward achieving excellencein mathematics and science.

What questions does thisreport address?This report highlights initial findings on theperformance of U.S. eighth-grade students relativeto students in other nations on the TIMSS–Rassessment. This report also describes the mathe-matics and science performance of students inparticipating nations at two points in time: 1995and 1999.

In general, this report addresses the followingquestions:

� How does the mathematics and scienceknowledge of U.S. eighth-grade studentscompare to that of students in other nations?

� Has the level of mathematics and scienceknowledge of eighth-grade students changedsince 1995, and has the relative internationalstanding of U.S. eighth-grade studentschanged in the 4 years since the originalTIMSS?

� How does the relative performance of U.S.eighth-grade students in 1999 compare to therelative performance of U.S. fourth-gradestudents 4 years earlier, in 1995?

� How do nations compare on education-related background factors studied inTIMSS–R?

Performance in the United States is presentedrelative to that of other nations that participatedin each assessment.4 Comparisons in this reportare made among the 38 nations that participatedin TIMSS–R in 1999; among 23 nations thatparticipated in both TIMSS and TIMSS–R at theeighth-grade level; and among the 17 nations thatparticipated at the fourth-grade level in TIMSSand at the eighth-grade level in TIMSS–R.5 Thisreport is based on the comparative data publishedin the reports TIMSS 1999 InternationalMathematics Report: Findings from IEA’s Repeat ofthe Third International Mathematics and ScienceStudy at the Eighth Grade (Mullis et al. 2000) andTIMSS 1999 International Science Report: Findingsfrom IEA’s Repeat of the Third InternationalMathematics and Science Study at the Eighth Grade(Martin et al. 2000).

What issues does this reportnot address?Findings from comparisons between the results ofTIMSS and TIMSS–R cannot be interpreted toindicate the success or failure of mathematics andscience reform efforts in the United States.TIMSS–R was designed to specifications detailedin the TIMSS curriculum frameworks (Robitailleet al. 1993). International experts developed theTIMSS curriculum frameworks to portray thestructure of the intended school mathematics andscience curricula from many different nations, notspecifically the United States. Thus, when inter-preting the findings, it is important to take intoaccount the mathematics and science curriculalikely encountered by U.S. students in school.TIMSS and TIMSS–R results are most useful whenthey are considered in light of other knowledgeabout education systems, including not onlycurricula, but also factors such as trends in educa-tion reform, changes in the school-agepopulations, and societal demands and expecta-tions.

3


3Comparisons of fourth- and eighth-graders between TIMSS and TIMSS–R are made on the basis of two sets of cross-sectional,nationally representative samples.

4Participants in TIMSS and TIMSS–R are referred to as nations throughout the text. However, several of the participants are notindependent jurisdictions, as is the case for Hong Kong, Special Administrative Region (SAR), Belgium-Flemish, and ChineseTaipei.

5Throughout the text “grade 8” refers to the middle-school grade sampled for TIMSS-R as well as the higher of the two middle-school grades sampled for TIMSS; “grade 4” refers to the higher of the two elementary school grades sampled for TIMSS. This isan accurate characterization of the samples for the United States and many of the other nations. Detailed information on the gradessampled can be found in appendix 2 of this report for TIMSS–R and in Beaton et al. (1996a and 1996b) for TIMSS.

Change efforts in the United States began yearsbefore TIMSS and TIMSS–R. These efforts tocreate change in U.S. schools have been under-taken at the state and local levels, making itdifficult to determine by solely examiningnational-level statistics the extent to which theseefforts have been implemented and the degree anddepth of the changes made. The 4 years betweenTIMSS and TIMSS–R is a relatively short amountof time to expect to see significant change. Finally,this report focuses on variability in achievementamong nations. It is important to keep in mindthat the range of achievement observed amongnations could also be expected to be observedwithin the United States (NCES 1997a and 1997b;Johnson and Siegendorf 1998). Thus, as will beshown later in the report, there are U.S. eighth-grade students who perform among thetop-performing students in the world, and thereare U.S. eighth-grade students who performamong the lowest performing students in theworld.

This report should also not be construed tosuggest that specific school policies, professionaldevelopment techniques, instructional practices,curricula or change strategies, or combinations ofthese will lead to higher levels of achievement. Thefactors that may contribute to high achievementcan vary from nation to nation. Nonetheless,TIMSS–R provides valuable information that canhelp the United States reflect on its own perform-ance relative to other nations as we strive toimprove educational opportunities for allstudents.

What is TIMSS–R?TIMSS–R is the fourth comparison of mathemat-ics and science achievement carried out by theInternational Association for the Evaluation ofEducational Achievement (IEA). IEA conductedstudies of mathematics and science as separatesubjects at various times during the 1960s, 1970s,and 1980s. The United States participated in eachof these studies. The Third InternationalMathematics and Science Study (TIMSS) collecteddata during the 1994–95 school year. TIMSSprovided an update on the performance of U.S.students in mathematics and science during themid-1990s and a starting point for a regular cycle

of international assessments in mathematics andscience. Funded by the U.S. Department ofEducation, NSF, the Government of Canada, theWorld Bank, and participating nations, TIMSSwas the first IEA study to combine both mathe-matics and science in the same assessment. TIMSSwas also the largest and most comprehensiveinternational study of educational achievementever undertaken.

TIMSS–R follows the earlier TIMSS study by 4years and focused on the mathematics and scienceachievement of eighth-grade students. Mostimportantly perhaps, TIMSS–R provides a seconddata point in a regular cycle of internationalassessments of mathematics and science that areplanned to chart trends in achievement over time,much like the regular cycle of national assessmentsin this nation, such as the National Assessment ofEducational Progress (NAEP), or longitudinalstudies such as the National EducationalLongitudinal Study (NELS:88).

The United States sponsored three additionalcomponents of TIMSS–R that will enrich ourknowledge of education in an internationalcontext:

� TIMSS–R Benchmarking Project—Twenty-seven states, districts, and consortia ofdistricts throughout the United States partici-pated as their own “nations” in this project,following the same guidelines as the partici-pating nations. When the findings from theBenchmarking Project are released in April2001, these 27 participating jurisdictions willbe able to assess their comparative interna-tional standing and judge their mathematicsand science programs in an internationalcontext.

� Videotape Classroom Study—the first TIMSSVideotape Classroom Study examined eighth-grade mathematics teaching in three nations.Building on the work of the first TIMSS video-tape study (Stigler et al. 1999), the TIMSS–RVideotape Classroom Study has beenexpanded in scope to examine nationalsamples of eighth-grade mathematics andscience instructional practices in sevennations. The study is designed to presentnational-level portraits of mathematics andscience teaching practices that can provide amore detailed context for understanding

4


mathematics and science teaching andlearning in the classroom. The first set ofresults from the Videotape Classroom Study isanticipated in late 2001.

� NAEP/TIMSS–R Linking Study—A subsam-ple of students taking the 2000 state NAEPmathematics and science assessment also tookthe TIMSS–R assessment. This provides anopportunity to compare students’ perform-ance on NAEP to their performance onTIMSS–R, and allows for estimates of howstates participating in NAEP 2000 would haveperformed had they participated in TIMSS–R.Results from the TIMSS–R BenchmarkingProject will be used to check the results of thislinking study. Results will be released in late2001.

With many states and districts creating contentand performance standards targeted at boostingstudent achievement to “world class” levels inmathematics and science, the BenchmarkingProject can provide reliable data on how state anddistrict students compare internationally in theseareas. Results from the TIMSS–R VideotapeClassroom Study should also add to our under-standing of mathematics and science instructionalpractices in nations with high student achieve-ment levels on assessments such as TIMSS.Findings from the NAEP/TIMSS–R Linking Studywill provide states the opportunity to comparetheir students to their peers in other nations.

Which nations participated inTIMSS–R?The IEA invited all nations that participated in the1995 TIMSS as well as other nations to participatein the 1999 TIMSS–R. Interested nations met atinternational meetings where study plans andguidelines were discussed. Thirty-eight nationscollected data for TIMSS–R, including 26 that hadparticipated in TIMSS and 12 that were participat-ing for the first time. Therefore, depending on theanalysis, the number of nations being comparedbetween TIMSS and TIMSS–R will change. The 38nations that participated in TIMSS–R are shownin figure 1. In addition, figure 1 lists the nationsthat participated in both TIMSS and TIMSS–R.

How was TIMSS–R conducted?The IEA, a Netherlands-based organization ofeducation and research institutions from itsmember nations, conducted TIMSS–R. The IEAdelegated responsibility for the overall coordina-tion and management of the project to theInternational Study Center at Boston College. TheUnited States, the World Bank, and participatingnations paid for and carried out data collectionaccording to international guidelines.

NCES and NSF funded the collection of data inthe United States and also contributed towardsupport of the international project. OERI hascontributed additional funding towards the U.S.portion of the study. Westat, Inc., a privateresearch firm, handled the data collection in theUnited States under contract to the Department ofEducation. To help guide the study, NCES andNSF established a TIMSS–R Technical ReviewPanel (TRP). The members of the TRP are expertsin mathematics and science education, assess-ment, and international comparative studies.

TIMSS–R included two types of data collectioninstruments: mathematics and science assessmentitems in multiple-choice (77 percent) and free-response (23 percent) formats; and school,teacher, and student questionnaires that requestedinformation to help provide a context for theperformance scores. An international panel ofassessment and content experts, following thesame assessment framework established forTIMSS, developed the mathematics and scienceitems in TIMSS–R. Like the TIMSS assessmentitems, the TIMSS–R items represent a range ofmathematics and science topics that are includedin the curricula of many different nations and,thus, not aligned to any particular curriculum. Seeappendix 2 for more details on the composition ofthe TIMSS and TIMSS–R assessments and howthe achievement scores were derived.

5


6


Figure 1.—Participation in TIMSS and TIMSS–R: 1995 and 1999TIMSS-R nations (1999)

8th grade

TIMSS-R nations thatparticipated at 8th grade in

TIMSS (1995)

TIMSS-R nations thatparticipated at 4th grade in

TIMSS (1995)Australia Australia AustraliaBelgium-Flemish1 Belgium-Flemish1

Bulgaria BulgariaCanada Canada CanadaChileChinese TaipeiCyprus Cyprus CyprusCzech Republic Czech Republic Czech RepublicEngland England EnglandFinlandHong Kong SAR Hong Kong SAR Hong Kong SARHungary Hungary HungaryIndonesiaIran, Islamic Republic of Iran, Islamic Republic of Iran, Islamic Republic ofIsrael IsraelItaly Italy2 Italy2

Japan Japan JapanJordanKorea, Republic of Korea, Republic of Korea, Republic ofLatvia-LSS3 Latvia-LSS3 Latvia-LSS3

Lithuania LithuaniaMacedonia, Republic ofMalaysiaMoldovaMoroccoNetherlands Netherlands NetherlandsNew Zealand New Zealand New ZealandPhilippinesRomania RomaniaRussian Federation Russian FederationSingapore Singapore SingaporeSlovak Republic Slovak RepublicSlovenia Slovenia SloveniaSouth Africa South AfricaThailand ThailandTunisiaTurkeyUnited States United States United States

Total Nations 38 26 17

1The Flemish and French educational systems in Belgium participated separately in TIMSS 1995. The Flemish educational system in Belgium participated in TIMSS-R 1999.2Italy was unable to provide the International Study Center at Boston College with its data in time for these data to be included in the international reports for both the fourth and eighth grade in TIMSS 1995. However, its data for TIMSS 1995 are included in this report.3Designated LSS because only Latvian-speaking schools were tested.

NOTE: Only nations that completed the necessary steps for their data to appear in the reports from the InternationalStudy Center at Boston College are listed.SOURCE: Mullis et al. (2000). TIMSS 1999 International Mathematics Report: Findings from IEA’s Repeat of the ThirdInternational Mathematics and Science Study at the Eighth Grade. Exhibit A.1. Chestnut Hill, MA: Boston College.

The questionnaires asked for information ontopics such as students’ attitudes and beliefs aboutlearning, study habits and homework, and theirlives both in and outside of school; teachers’ atti-tudes and beliefs about teaching and learning,teaching assignments, class size and organization,instructional practices, and participation inprofessional development activities; and princi-pals’ viewpoints on policy and budgetresponsibilities, curriculum and instructionissues, student behavior problems, as well asdescriptions of the organization of schools andcourses.

Both public and nonpublic school students in allparticipating nations received the TIMSS–Rassessments and questionnaires. Most nations,including the United States, conducted the assess-ment 2 to 3 months before the end of the 1998–99school year. Students with special needs anddisabilities that would make it very difficult forthem to take the test were excused from the assess-ment as accommodations were not offered inTIMSS–R in the United States. Each participatingnation documented such exclusions, including theUnited States. Each nation translated the assess-ments and questionnaires into the primarylanguage or languages of instruction. In theUnited States, all materials were in English. Thestudent assessment portion required approxi-mately one and a half hours to complete.

All participating nations drew nationally represen-tative samples of students. In the United States, thesample consisted of 221 schools and 9,072 eighth-grade students, which ensured a representativesample of eighth-grade students in the UnitedStates as a whole. Detailed information onsampling is provided in appendix 2.

Are the results from TIMSSand TIMSS–R comparable?The data collected for TIMSS in 1995 and the datacollected for TIMSS–R in 1999 are comparablebecause comparability was built into the designand implementation. Through a careful process ofreview, analysis, and refinement, the assessmentand questionnaire items were purposefully devel-oped and field tested for similarity and for reliablecomparisons between TIMSS and TIMSS–R. After

careful review of all available data, including a testfor item reliability between old and new items, theTIMSS and TIMSS–R assessments were found tobe very similar in format, content, and difficultylevel. Moreover, TIMSS and TIMSS–R data are onthe same eighth-grade scale to allow for reliablecomparisons between the two eighth-gradecohorts over time. Procedures for conducting theassessments were the same. Appendix 2 containsmore detailed information on these and othertechnical aspects of TIMSS–R.

How can we be sure the dataare comparable acrossnations?TIMSS–R continues the tradition of fair andaccurate international comparisons of studentachievement and other educational factors. It isnot a comparison of other nations’ best studentsto our nation’s average students. Moreover,through the refinement of the scaling process thatallows comparisons within and across nations, theTIMSS and TIMSS–R achievement scores can bereliably compared. To ensure the comparability ofdata across nations, the International StudyCenter at Boston College instituted a series ofstrict quality-control procedures. National schooland student samples were rigorously reviewed forbias and international comparability by theTIMSS–R Sampling Referee. A professional trans-lation agency verified the accuracy of translatedmaterials. Project coordinators in each nationreceived thorough training in data collection andscoring procedures and their work was monitoredfor scoring reliability. Quality control staffconducted site visits in each participating nationduring the testing period to further ensure thatinternational data collection procedures werefollowed. Data from each nation were extensivelyreviewed for internal and cross-country consis-tency.

Nations collected data from a representativenational sample of students, but were permitted tosupplement their student samples to allow for theanalysis of data by variables of national interest. Toobtain reliable comparisons among nations, thedata were appropriately weighted to account forsampling designs. Sampling and participation rate

7


irregularities arose in some nations. These irregu-larities are clearly noted in this and otherTIMSS–R reports. The United States met all inter-national sampling and participation guidelines.More detailed information on quality control canbe found in appendix 2 and the TIMSS 1999Technical Report from Boston College (Martin andGregory 2000).

Finally, it should be noted that in addition to the38 nations that participated in TIMSS–R in 1999,this report separately discusses the 23 that partici-pated in TIMSS at the eighth-grade level,6 and the17 TIMSS-R nations that participated in TIMSS atthe fourth-grade level (see figure 1).7 In order tomake a fair comparison of how U.S. eighth-gradestudents in 1999 compared to the eighth-gradersof 1995 or the fourth-graders of 1995, analyseswere conducted only among those nations thatparticipated in both TIMSS and TIMSS–R.

How does TIMSS–R relate toother large-scale studies ofmathematics and scienceachievement?TIMSS–R is one of several large-scale studiesdesigned to examine the mathematics and scienceperformance of students. Two other large-scalestudies of mathematics and science achievementare the National Assessment of EducationalProgress (NAEP) and the Program forInternational Student Assessment (PISA). NAEP isan ongoing program that has reported on themathematics and science achievement of U.S.students for some 30 years. PISA is a relatively newinternational project and will report results for thefirst time in late 2001. These three assessmentswere designed with different purposes in mind,

and this is evident in the types of assessment itemsas well as the content areas and topics covered ineach assessment.

TIMSS–R and NAEP assess students at the eighthgrade. TIMSS–R is based on the curricula thatstudents in participating nations are likely to haveencountered by grade 8, while NAEP is based onan expert consensus of what students in theUnited States should know and be able to do invarious academic subjects at that grade. PISA, onthe other hand, focuses on 15-year-old students(most often tenth-graders in the United States)and is designed to measure students’ mathematicsand science literacy—that is, students’ ability torespond to “real life” situations both in andoutside of school. In contrast, TIMSS–R andNAEP tend to focus on mathematics and scienceas it is generally presented in classrooms and text-books.

All three assessments cover a range of mathemat-ics and science content areas and topics, but todifferent degrees. In mathematics, for example,TIMSS–R appears to place more emphasis onnumber sense, properties and operations than theother two studies; PISA tends to emphasize dataanalysis more than the other two studies; andNAEP appears to distribute its focus across thecontent areas included in its assessment frame-work more than the other two studies. In science,TIMSS–R appears to emphasize physical sciencesmore than the other two assessments; PISA seemsto have a stronger emphasis on earth science thanTIMSS–R and NAEP; and NAEP appears todistribute most science items among three contentareas: physical science, earth science, and lifescience. As findings from these studies arereleased, it is important to understand the differ-ences and similarities among them to be able tomake sense of the findings in relation to eachother.

8


6Twenty-six nations participated in the eighth-grade level in TIMSS 1995 and TIMSS–R 1999. Three of the 26 nations—Israel, SouthAfrica, and Thailand—experienced significant difficulties with meeting international sampling or participation guidelines in 1995.Therefore, these 3 nations are not included in analyses comparing achievement at the eighth-grade level between 1995 and 1999,nor are they included in the international averages associated with these comparative analyses.

7Of the 42 nations that participated in TIMSS 1995 at the eighth-grade level, 26 also participated in TIMSS–R. Of the 26 nationsthat participated in TIMSS 1995 at the fourth-grade level, 17 also participated in TIMSS–R. See table A2.6 in appendix 2 for a com-plete list of nations.

How is the rest of the reportorganized?The remainder of the report includes three addi-tional chapters and several appendices:

Chapter 2 describes the relative performance ofU.S. eighth-grade students in mathematics andscience in comparison to their peers in participat-ing nations. The chapter is divided into threesections. First, achievement results for TIMSS–Rare described for the United States and the other37 participating nations, including overall mathe-matics and science achievement, achievement infive mathematics content areas and six sciencecontent areas, and proportions of students in thetop 10 percent and top 25 percent of all students.Sample mathematics and science items areincluded to acquaint the reader with the TIMSS–Rassessment. The second section focuses on the 23nations that participated in TIMSS and TIMSS–Rat the eighth-grade level, describing changes inmathematics and science achievement over the 4intervening years. The third section compares the17 nations that participated in fourth-gradeTIMSS and eighth-grade TIMSS–R, examiningchanges in the relative standing of the U.S. 1995fourth-graders and 1999 eighth-graders.

Chapter 3 focuses on the education-relatedcontextual factors related to teaching and curricu-lum that were examined in TIMSS–R. Thechapter is divided into four sections. The first

section describes mathematics and science teacherpreparation, qualifications, and ongoing profes-sional development activities. The next sectionexamines the curriculum in the participatingnations, including the topics covered and empha-sized in mathematics and science lessons. Thethird section provides information on classroompractices as reported by teachers and students.The chapter ends with a brief discussion of howmuch time eighth-grade students spend studyingmathematics and science outside of school.

Chapter 4 discusses future directions that theanalyses of TIMSS and TIMSS–R data could take.Several appendices are included in this report toprovide additional information on the technicalaspects of the study as well as more detailed infor-mation on the analyses presented in the mainchapters of the report.

In addition to the text of this report, supplementalinformation is provided in the five appendices.Appendix 1 contains a selection of publicationsthat have been produced in relation to TIMSS1995. Appendix 2 discusses several technicalaspects of the TIMSS and TIMSS–R studies. Thetables in Appendices 3 and 4 provide additionalinformation on the figures in Chapters 2 and 3,respectively. Lastly, Appendix 5 provides a supple-mental table containing comparisons ofmathematics and science achievement of the 54nations that participated at the eighth-grade levelin either TIMSS, TIMSS–R, or both studies.

9


CHAPTER 2MATHEMATICS AND SCIENCE ACHIEVEMENT

KEY POINTS

In 1999, U.S. eighth-graders exceeded the international average in mathematics and science amongthe 38 participating nations.

Between 1995 and 1999, there was no change in eighth-grade mathematics or science achievementin the United States. Among the 22 other nations, there was no change in mathematics achievementfor 18 nations, and no change in science achievement for 17 nations.

There was an increase in mathematics achievement among U.S. eighth-grade black students between1995 and 1999. There was no change in science achievement for this group of students over thesame period. U.S. eighth-grade white and Hispanic students showed no change in their mathematicsor science achievement over the 4 years.

No differences in performance were found between U.S. eighth-grade girls and boys in mathematicsin 1999, but boys outperformed girls in science.

The relative performance of the United States in mathematics and science was lower for eighth-graders in 1999 than it was for the cohort of fourth-graders 4 years earlier in 1995.

As indicated in the previous chapter, the primaryintent of conducting TIMSS in 1995 andTIMSS–R in 1999 was to take the first step inmeasuring change in both achievement andeducational context at the international level. Thischapter describes the mathematics and scienceachievement of students in the participatingnations. It is divided into three main sections, inthe following order:

� findings for the 38 nations that participated inTIMSS–R;

� findings for the 23 nations that participated atthe eighth grade in both TIMSS andTIMSS–R1; and

� findings for the 17 nations that participated atthe fourth grade in TIMSS and eighth grade inTIMSS–R.

To assist the reader, the number of nations beingcompared in each analysis will be made explicit.This is important, as the number of nationsincluded in the international average can varydepending on the frame of reference in theanalysis.

What do the test scores mean? TIMSS–R test scores are on a scale of 1 to 1,000,with a standard deviation of 100.2 TIMSS–R testscores indicate where on the scale a group ofstudents would fall. In general, the higher thescore on TIMSS or TIMSS–R, the more itemscorrectly answered by a larger percentage of anation’s students. The lower the score on TIMSSor TIMSS–R, the fewer items correctly answeredby a larger percentage of a nation’s students.TIMSS and TIMSS–R used item response theoryto create the scale scores. The scales used in TIMSSand TIMSS–R account for differences in the diffi-culty of items and allow students’ performance tobe summarized on a common metric. The scalesare thus a simplified method for making compar-isons between nations. The scales measureachievement on mathematics and science items

judged by international experts to be appropriatefor eighth-grade students in the participatingnations. Thus, higher performance indicates thatstudents are more proficient at middle-schoolmathematics or science.

For all analyses presented in this report, differ-ences between averages or percentages that arestatistically significant are discussed usingcomparative terms such as “higher” and “lower.”Differences that are not statistically significantlyare discussed as “similar to” or “not differentfrom” each other. To determine whether differ-ences reported are statistically significant,two-tailed t-tests, at the .05 level, were used.Bonferroni adjustments are made when more thantwo groups are compared simultaneously (e.g.,black, white, and Hispanic students).

THE MATHEMATICS ANDSCIENCE ACHIEVEMENT OFEIGHTH-GRADERS IN 1999This section presents results for the 38 nations thatparticipated in TIMSS–R in 1999.

National averages for mathematics and sciencefrom the 1999 TIMSS–R assessment are presented,beginning with figure 2. Though tempting, it isnot correct to report U.S. scores by rank. This isbecause the process of estimating each nation’sscore from the sample of students who took thetest produces only an estimate of the range withinwhich the nation’s real score lies. To conduct a faircomparison of the United States to other nations,nations are grouped according to whether theirperformance is higher than, not different from, orlower than the United States, given the margin oferror for the survey. Nations with a nationalaverage higher than the U.S. average are indicatedin the uppermost band of shading. Nations with anational average lower than the U.S. average are

12

CHAPTER 2—ACHIEVEMENT

1Twenty-six nations participated in TIMSS and TIMSS–R at the eighth grade. Of the 26 nations, 3 nations experienced significantirregularities in their participation in 1995: Israel, South Africa, and Thailand. Findings for the other 23 nations are reported here.

2Because the standard deviation is 100, raw differences between scores can be translated into effect sizes by dividing the raw differ-ence by the standard deviation. For example, if the raw difference between the scores of two nations is 75, this translates to an effectsize of 0.75 in TIMSS–R. The TIMSS–R scale was developed once a majority of nations had submitted data. At that time, the meanwas set to 500, with a standard deviation of 100. Once the remaining data was submitted by nations, it was fitted to the developedscale, resulting in an actual mean slightly different than 500.

indicated in the lowermost band of shading.Nations with a national average not different fromthe U.S. average are shown unshaded and, for themost part, lie between these shaded areas. Notethat the international average—the average of the

national average scores for all nations combined—can be compared to the U.S. average in the sameway as a national average and is shaded to indicatethe significance of the difference.

13


Figure 2.—Average mathematics and science achievement of eighth-grade students, by nation: 1999

MATHEMATICS SCIENCENation Average Nation AverageSingapore 604 Chinese Taipei 569Korea, Republic of 587 Singapore 568Chinese Taipei 585 Hungary 552Hong Kong SAR 582 Japan 550Japan 579 Korea, Republic of 549Belgium-Flemish 558 Netherlands 545Netherlands 540 Australia 540Slovak Republic 534 Czech Republic 539Hungary 532 England 538Canada 531 Finland 535Slovenia 530 Slovak Republic 535Russian Federation 526 Belgium-Flemish 535Australia 525 Slovenia 533Finland1 520 Canada 533Czech Republic 520 Hong Kong SAR 530Malaysia 519 Russian Federation 529Bulgaria 511 Bulgaria 518Latvia-LSS2 505 United States 515United States 502 New Zealand 510England 496 Latvia-LSS2 503New Zealand 491 Italy 493Lithuania3 482 Malaysia 492

Italy 479 Lithuania3 488Cyprus 476 Thailand 482Romania 472 Romania 472Moldova 469 (Israel) 468Thailand 467 Cyprus 460(Israel) 466 Moldova 459Tunisia 448 Macedonia, Republic of 458Macedonia, Republic of 447 Jordan 450Turkey 429 Iran, Islamic Republic of 448Jordan 428 Indonesia 435Iran, Islamic Republic of 422 Turkey 433Indonesia 403 Tunisia 430Chile 392 Chile 420Philippines 345 Philippines 345Morocco 337 Morocco 323South Africa 275 South Africa 243

International average of 38 nations

487International average of 38 nations

488

Average is significantly higher than the U.S. averageAverage does not differ significantly from the U.S. averageAverage is significantly lower than the U.S. average

1The shading of Finland may appear incorrect; however, statistically, its placement is correct.2Designated LSS because only Latvian-speaking schools were tested which represents 61 percent of the population.3Lithuania tested the same cohort of students as other nations, but later in 1999, at the beginning of the next school year.

NOTE: Eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines. See appendix 2 for details.The international average is the average of the national averages of the 38 nations.

SOURCE: Martin et al. (2000). TIMSS 1999 International Science Report: Findings from IEA’s Repeat of the Third International Mathematicsand Science Study at the Eighth Grade. Exhibit 1.1. Chestnut Hill, MA: Boston College; Mullis et al. (2000). TIMSS 1999 InternationalMathematics Report: Findings from IEA’s Repeat of the Third International Mathematics and Science Study at the Eighth Grade. Exhibit 1.1.Chestnut Hill, MA: Boston College.

How well did U.S. eighth-graders perform in 1999?In mathematics, U.S. eighth-graders exceeded theinternational average, outperforming their peersin 17 of the 37 other TIMSS–R nations, perform-ing similarly to students in 6 nations, andperforming lower than their peers in 14 nations.In 1999, the U.S. average score was 502, with othernations’ average mathematics scores ranging from604 for Singapore to 275 for South Africa. Amongthe top performing nations in 1999 were five Asianindustrialized nations—Singapore, Korea,Chinese Taipei, Hong Kong SAR, and Japan.Comparisons with five of the Group of Eight (G8)nations are possible as well: in 1999, the UnitedStates performed significantly better in mathe-matics than Italy, performed similarly to England,but was outperformed by Japan, Canada, and theRussian Federation.3

In science, U.S. eighth-graders exceeded the inter-national average, outperforming their peers in 18of the 37 other nations, performing similarly tostudents in 5 nations, and performing lower thantheir peers in 14 nations. In 1999, the U.S. averagescore was 515, with other nations’ average sciencescores ranging from 569 for Chinese Taipei to 243for South Africa. Among the top performingnations in science were four Asian industrializednations—Chinese Taipei, Singapore, Korea, andJapan—and Hungary. Comparisons with otherparticipating G8 nations show that the UnitedStates performed significantly better than Italy,performed on par with the Russian Federation,but performed lower than Japan, England andCanada.

When looking across mathematics and scienceachievement in 1999, 12 nations outperformed theUnited States in both subjects: Australia, Belgium-Flemish, Canada, Chinese Taipei, Finland,Hungary, Japan, Korea, the Netherlands,Singapore, the Slovak Republic, and Slovenia.

Likewise, three nations performed similarly to theUnited States in both subjects: Bulgaria, Latvia-LSS, and New Zealand. Finally, U.S. eighth-gradersoutperformed their peers in 17 nations acrossboth mathematics and science in 1999.4

What percentage of ourstudents scored at or abovethe international top 10percent benchmark in 1999?Average achievement scores indicate how theaverage student performs, but say little about theperformance of the nation’s students at differentlevels. International benchmarks were devised toprovide a view of what proportion of a nation’sstudents scored at or near various levels ofachievement. These international benchmarksgive a general indication of the relative distribu-tion of scores within and across nations. Forexample, if a nation has a high average score and alarge percentage of its students at or above theupper international benchmarks, this indicatesthat the nation’s students are concentrated amongthe highest achieving students internationally.

TIMSS–R uses four benchmarks: the top 10percent, the top 25 percent, the upper 50 percent,and the upper 75 percent. Each benchmark isbased on all eighth-graders from all 38 nations in1999. This report discusses two benchmarks indetail: the top 10 percent benchmark, which refersto the cutoff score that separates the top 10percent of all students in 1999, and the similar top25 percent benchmark. In 1999, the top 10 percentof all students scored 616 or higher in mathemat-ics and 616 or higher in science (data not shown).The top 25 percent of all students scored 555 orhigher in mathematics and 558 or higher inscience (data not shown). Detailed information onthese two benchmarks, as well as the upper 50 andupper 75 percent benchmarks, is found in tablesA3.2 (mathematics) and A3.3 (science) inappendix 3.

14


3The United Kingdom, a member of the G8, is represented here by the score for England. France and Germany, the other two mem-bers of the G8, did not participate in TIMSS–R.

4An analysis of the overall mathematics and science achievement of the 54 nations that participated in TIMSS or TIMSS–R is pro-vided in appendix 5.

15


In mathematics, 9 percent of U.S. eighth-gradersscored 616 or higher, placing them among the top10 percent of all eighth-graders in the 38 nationsin 1999. This is a lower percentage of studentsthan in 8 nations, a similar percentage as in 13nations, and a higher percentage than in 16nations (figure 3). In contrast, 46 percent of

Singapore’s eighth-grade students scored 616 orhigher in mathematics in 1999. Among the fiveparticipating G8 nations, only Japan had a signifi-cantly higher percentage of students who scored ator above the international top 10 percent bench-mark (33 percent) than the United States inmathematics.

Figure 3.—Percentages of eighth-grade students reaching the TIMSS-R 1999 top 10 percent in mathematics and scienceachievement, by nation: 1999

MATHEMATICS SCIENCENation Percent Nation PercentSingapore 46 Singapore 32Chinese Taipei 41 Chinese Taipei 31Korea, Republic of 37 Hungary 22Hong Kong SAR 33 Korea, Republic of 22Japan 33 England 19Belgium-Flemish 23 Australia 19Hungary 16 Japan 19Slovenia1 15 Russian Federation 17Russian Federation 15 Czech Republic 17Netherlands 14 Netherlands 16Slovak Republic 14 Slovenia 16Australia 12 United States 15Malaysia 12 Finland 14Canada 12 Slovak Republic 14Czech Republic 11 Bulgaria 14Bulgaria 11 Canada 14United States 9 New Zealand 12New Zealand 8 Belgium-Flemish 11Latvia-LSS2 7 Hong Kong SAR 10England 7 Italy 7Finland 6 Latvia-LSS2 7Romania 5 (Israel) 7Italy1 5 Malaysia 6

(Israel)1 5 Romania 6

Thailand 4 Lithuania3 6

Lithuania3 4 Jordan 4Moldova 4 Moldova 4Cyprus 3 Macedonia, Republic of 4Jordan 3 Thailand 3Macedonia, Republic of 3 Cyprus 2Indonesia 2 Iran, Islamic Republic of 2Turkey 1 Indonesia 1Iran, Islamic Republic of 1 Chile 1Chile 1 Turkey 1Tunisia 0 Philippines 1South Africa 0 South Africa 0Philippines 0 Tunisia 0Morocco 0 Morocco 0

Average is significantly higher than the U.S. averageAverage does not differ significantly from the U.S. averageAverage is significantly lower than the U.S. average

1The shading of Italy, Israel, and Slovenia in mathematics may appear incorrect; however, statistically, their placement is correct.2Designated LSS because only Latvian-speaking schools were tested which represents 61 percent of the population.3Lithuania tested the same cohort of students as other nations, but later in 1999, at the beginning of the next school year.

NOTE: Eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines. See appendix 2 for details.

SOURCE: Martin et al. (2000). TIMSS 1999 International Science Report: Findings from IEA’s Repeat of the Third International Mathematicsand Science Study at the Eighth Grade. Exhibit 1.6. Chestnut Hill, MA: Boston College; Mullis et al. (2000). TIMSS 1999 InternationalMathematics Report: Findings from IEA’s Repeat of the Third International Mathematics and Science Study at the Eighth Grade. Exhibit 1.6.Chestnut Hill, MA: Boston College.

In science, 15 percent of U.S. eighth-gradersscored 616 or higher, placing them among the top10 percent of all students internationally in 1999.This was a lower percentage of students than in 4nations, a similar percentage as in 13 nations, anda higher percentage than in 20 nations (figure 3).In contrast, 32 percent of Singapore’s eighth-gradestudents scored 616 or higher in science in 1999.Among the five participating G8 nations, nonehad a significantly higher percentage of studentswho scored at or above the international top 10percent benchmark than the United States inscience.

What percentage of ourstudents scored at or abovethe international top 25percent benchmark in 1999?An examination of the top 25 percent interna-tional benchmark offers yet another opportunityto understand the performance of our eighth-grade students in mathematics and science in1999. In mathematics, 28 percent of U.S. eighth-grade students scored 555 or higher, placing themamong the top 25 percent of all students interna-tionally in 1999. This was a lower percentage thanin 11 nations, a similar percentage as in 9 nations,and a higher percentage than in 17 nations. Incontrast, 75 percent of eighth-grade students inSingapore scored 555 or higher in mathematics in1999.

In science, 34 percent of U.S. eighth-gradersscored 558 or higher, placing them among the top25 percent of all students internationally in 1999.This was a lower percentage than in 5 nations, asimilar percentage as in 13 nations, and a higherpercentage than in 19 nations. In contrast, 58percent of eighth-grade students in Chinese Taipeiscored 558 or higher in science in 1999.

How well did U.S. eighth-graders perform in thedifferent content areas in1999?An overall score is a useful summary of generalmathematics and science performance. However,mathematics and science comprise a range ofcontent areas that can be conceptually distinct,differ in levels of complexity, enter the curriculumat different times, and be taught by differentteachers in separate courses. TIMSS–R assessedfive mathematics and six science content areas:

Mathematics5

� Fractions and number sense

� Measurement

� Data representation, analysis, and probability

� Geometry

� Algebra

Science

� Earth science

� Life science

� Physics

� Chemistry

� Environmental and resource issues

� Scientific inquiry and the nature of science

U.S. eighth-graders’ average score was higher thanthe international average in three of the five math-ematics content areas assessed in 1999: fractionsand number sense; data representation, analysis,and probability; and algebra. They performed atthe international average in measurement andgeometry.

Figure 4 displays mathematics content area scoresfor all 38 nations based on the TIMSS–R assess-ment. Six nations outperformed the United Statesacross all five mathematics content areas in 1999:Belgium-Flemish, Chinese Taipei, Hong KongSAR, Japan, Korea, and Singapore. New Zealand isthe only nation in TIMSS–R that performedsimiarly to the United States in all five contentareas. Seven nations performed below the UnitedStates in all five mathematics content areas: Chile,

16


5TIMSS 1995 included proportionality among the mathematics content areas. After careful consideration, the proportionality itemswere redistributed among several of the other mathematics content areas for the TIMSS and TIMSS–R data.

17

CHAPTER 2—ACHIEVEMENTFi

gure

4.—

Aver

age

eigh

th-g

rade

ach

ieve

men

t in

mat

hem

atic

s co

nten

t are

as, b

y na

tion:

199

9Fr

acti

ons

and

nu

mbe

r se

nse

Mea

sure

men

tD

ata

repr

esen

tati

on,

anal

ysis

, an

d pr

obab

ility

Geo

met

ryA

lgeb

ra

Nat

ion

Ave

rage

Nat

ion

Ave

rage

Nat

ion

Ave

rage

Nat

ion

Ave

rage

Nat

ion

Ave

rage

Sin

gapo

re60

8Si

nga

pore

599

Kor

ea, R

epu

blic

of

576

Japa

n57

5C

hin

ese

Taip

ei58

6H

ong

Kon

g SA

R57

9K

orea

, Rep

ubl

ic o

f57

1Si

nga

pore

562

Kor

ea, R

epu

blic

of

573

Kor

ea, R

epu

blic

of

585

Ch

ines

e Ta

ipei

576

Hon

g K

ong

SAR

567

Ch

ines

e Ta

ipei

559

Sin

gapo

re56

0Si

nga

pore

576

Kor

ea, R

epu

blic

of

570

Ch

ines

e Ta

ipei

566

Japa

n55

5C

hin

ese

Taip

ei55

7Ja

pan

569

Japa

n57

0Ja

pan

558

Hon

g K

ong

SAR

547

Hon

g K

ong

SAR

556

Hon

g K

ong

SAR

569

Bel

giu

m-F

lem

ish

557

Bel

giu

m-F

lem

ish

549

Bel

giu

m-F

lem

ish

544

Bel

giu

m-F

lem

ish

535

Bel

giu

m-F

lem

ish

540

Net

her

lan

ds54

5H

un

gary

538

Net

her

lan

ds53

8Sl

ovak

Rep

ubl

ic52

7H

un

gary

536

Can

ada

533

Net

her

lan

ds53

8Sl

oven

ia53

0B

ulg

aria

524

Ru

ssia

n F

eder

atio

n52

9M

alay

sia

532

Slov

ak R

epu

blic

537

Fin

lan

d52

5La

tvia

-LSS

152

2Sl

ovak

Rep

ubl

ic52

5Fi

nla

nd

531

Cze

ch R

epu

blic

535

Au

stra

lia52

2R

uss

ian

Fed

erat

ion

522

Slov

enia

352

5Sl

oven

ia52

7A

ust

ralia

529

Slov

ak R

epu

blic

521

Net

her

lan

ds51

5C

anad

a352

5H

un

gary

526

Ru

ssia

n F

eder

atio

n52

7C

anad

a52

1C

zech

Rep

ubl

ic51

3N

eth

erla

nds

522

Slov

ak R

epu

blic

525

Slov

enia

523

Hu

nga

ry52

0C

anad

a50

7A

ust

ralia

520

Au

stra

lia51

9C

anad

a52

1C

zech

Rep

ubl

ic51

3Sl

oven

ia50

6C

zech

Rep

ubl

ic51

4R

uss

ian

Fed

erat

ion

513

Fin

lan

d52

1E

ngl

and

506

Au

stra

lia49

7B

ulg

aria

512

Un

ited

Sta

tes

509

Mal

aysi

a51

4U

nit

ed S

tate

s50

649

7U

nit

ed S

tate

s50

6C

zech

Rep

ubl

ic50

7E

ngl

and

507

Ru

ssia

n F

eder

atio

n50

1Li

thu

ania

249

6M

alay

sia

505

Bu

lgar

ia50

3L

atvi

a-LS

S150

5N

ew Z

eala

nd

497

Fin

lan

d49

4La

tvia

-LSS

149

9E

ngl

and

497

Ital

y50

1La

tvia

-LSS

149

5H

un

gary

489

En

glan

d49

8La

tvia

-LSS

149

6B

ulg

aria

497

Lith

uan

ia2

493

Rom

ania

487

Fin

lan

d49

8N

ew Z

eala

nd

493

New

Zea

lan

d49

6B

ulg

aria

493

Th

aila

nd

484

New

Zea

lan

d49

7C

ypru

s48

1R

oman

ia49

1M

alay

sia

491

Cyp

rus

484

Lith

uan

ia2

487

Lith

uan

ia2

479

Un

ited

Sta

tes

482

Ital

y48

4Tu

nis

ia48

4It

aly

481

(Isr

ael)

472

Mol

dova

479

Th

aila

nd

476

Ital

y48

2R

oman

ia48

1T

hai

lan

d47

1C

ypru

s47

1C

ypru

s47

2M

oldo

va48

1(I

srae

l)47

9It

aly

471

Lit

huan

ia2

467

(Isr

ael)

468

New

Zea

lan

d47

8C

ypru

s47

9M

oldo

va46

5T

hai

lan

d46

3R

oman

ia45

3U

nit

ed S

tate

s47

3M

oldo

va47

7R

oman

ia45

8(I

srae

l)45

7M

oldo

va45

0E

ngl

and

471

Mac

edon

ia, R

epu

blic

of

465

Tun

isia

443

Mac

edon

ia, R

epu

blic

of

451

Tun

isia

446

(Isr

ael)

462

Th

aila

nd

456

Iran

, Isl

amic

Rep

ubl

ic o

f43

7Tu

nis

ia44

2Tu

rkey

446

Mac

edon

ia, R

epu

blic

of

460

Tun

isia

455

Mac

edon

ia, R

epu

blic

of

437

Jord

an43

8M

aced

onia

, Rep

ubl

ic o

f44

2Jo

rdan

449

Jord

an43

9Jo

rdan

432

Turk

ey43

6Jo

rdan

436

Iran

, Isl

amic

Rep

ubl

ic o

f44

7Ir

an, I

slam

ic R

epu

blic

of

434

Turk

ey43

0C

hile

412

Iran

, Isl

amic

Rep

ubl

ic o

f43

0In

don

esia

441

Turk

ey43

2In

don

esia

406

Iran

, Isl

amic

Rep

ubl

ic o

f40

1C

hile

429

Turk

ey42

8In

don

esia

424

Ch

ile40

3In

don

esia

395

Indo

nes

ia42

3C

hile

412

Ch

ile39

9P

hili

ppin

es37

8P

hili

ppin

es35

5P

hili

ppin

es40

6M

oroc

co40

7M

oroc

co35

3M

oroc

co33

5M

oroc

co34

8M

oroc

co38

3P

hili

ppin

es38

3P

hili

ppin

es34

5So

uth

Afr

ica

300

Sou

th A

fric

a32

9So

uth

Afr

ica

356

Sou

th A

fric

a33

5So

uth

Afr

ica

293

Inte

rnat

ion

al a

vera

ge

of

38 n

atio

ns

487

Inte

rnat

ion

al a

vera

ge

of

38 n

atio

ns

487

Inte

rnat

ion

al a

vera

ge

of

38 n

atio

ns

487

Inte

rnat

ion

al a

vera

ge

of

38 n

atio

ns

487

Inte

rnat

ion

al a

vera

ge o

f38

nat

ion

s48

7

Ave

rage

is s

ign

ific

antl

y h

igh

er t

han

th

e U

.S. a

vera

geA

vera

ge d

oes

not

dif

fer

sign

ific

antl

y fr

om t

he

U.S

. ave

rage

Ave

rage

is s

ign

ific

antl

y lo

wer

th

an t

he

U.S

. ave

rage

1 Des

ign

ated

LSS

bec

ause

on

ly L

atvi

an-s

peak

ing

sch

ools

wer

e te

sted

wh

ich

rep

rese

nts

61

perc

ent

of t

he

popu

lati

on.

2 Lit

huan

ia te

sted

th

e sa

me

coh

ort

of s

tude

nts

as

oth

er n

atio

ns,

bu

t la

ter

in 1

999,

at

the

begi

nn

ing

of t

he

nex

t sc

hoo

l yea

r.3 T

he

shad

ing

of S

love

nia

an

d C

anad

a m

ay a

ppea

r in

corr

ect;

how

ever

, sta

tist

ical

ly, t

hei

r pl

acem

ent

is c

orre

ct.

NO

TE

: E

igh

th g

rade

in m

ost

nat

ion

s. S

ee a

ppen

dix

2 fo

r de

tails

.Pa

ren

thes

es in

dica

te n

atio

ns

not

mee

tin

g in

tern

atio

nal

sam

plin

g an

d/or

oth

er g

uid

elin

es.

See

appe

ndi

x 2

for

deta

ils.

Th

e in

tern

atio

nal

ave

rage

is t

he

aver

age

of t

he

nat

ion

al a

vera

ges

of t

he

38 n

atio

ns.

SOU

RC

E:

Mu

llis

et a

l. (

2000

).

TIM

SS 1

999

Inte

rnat

iona

l Mat

hem

atic

s R

epor

t: F

indi

ngs

from

IE

A’s

Rep

eat o

f the

Thi

rd I

nter

nati

onal

Mat

hem

atic

s an

d Sc

ienc

e St

udy

at th

e E

ight

h G

rade

. E

xhib

it3.

1. C

hes

tnu

t H

ill, M

A: B

osto

n C

olle

ge.

Mal

aysi

a

Indonesia, Iran, Morocco, the Philippines, SouthAfrica, and Turkey. Geometry and measurement arethe content areas in which the United Statesperformed lowest in terms of the number ofnations that outperformed the United States, butthe U.S. average was similar to the internationalaverage in both content areas.

In interpreting these results, it is important toconsider the mathematics content areas and topicsthat students have likely encountered in the yearsleading up to and including eighth grade. Forexample, if students in the United States were notprovided the opportunity to learn a particularmathematics topic or content area by the time ofthe assessment, it would be less likely that thestudents would perform well in comparison totheir international peers in that area. Informationon the coverage of mathematics content areas, aswell as many other aspects of eighth-grade mathe-matics teaching and learning, is discussed in thenext chapter.

U.S. eighth-graders’ average score was higher thanthe international average in five of the six sciencecontent areas assessed in 1999: earth science; lifescience; chemistry; environmental and resourceissues; and scientific inquiry and the nature ofscience. They performed at the internationalaverage of the 38 nations in physics.

Figure 5 displays science content area scores forthe 38 TIMSS–R nations in 1999. As with mathe-matics, the international performance of nationsdiffers when examining science by the six sciencecontent areas. The international performance ofthe United States is highest for life science; environ-mental and resource issues; and scientific inquiryand the nature of science. Only two nations scoredhigher than the United States in each of thesethree content areas. Chinese Taipei outperformedthe United States in five of the six content areas,however. As in mathematics, New Zealand is theonly nation that performed similarly to the UnitedStates across all six content areas in science.Finally, 12 nations performed below the UnitedStates in all six science content areas: Chile,Cyprus, Iran, Jordan, Macedonia, Moldova,Morocco, the Philippines, Romania, South Africa,Tunisia, and Turkey. Physics was the sciencecontent area that the United States performedlowest in terms of the number of nations thatoutperformed the United States, but the U.S.average was similar to the international average.

As with mathematics, it is important to under-stand the context within which science learningoccurs when interpreting these results. Thisincludes the science content areas and topics thatstudents have likely encountered in their sciencelessons. Information on the coverage of the sixscience content areas, as well as many otheraspects of eighth-grade science teaching andlearning, is covered in the following chapter.

18


19

CHAPTER 2—ACHIEVEMENTFi

gure

5.—

Aver

age

eigh

th-g

rade

ach

ieve

men

t in

scie

nce

cont

ent a

reas

, by

natio

n: 1

999

Ear

th s

cien

ceLi

fe s

cien

ceP

hysi

csC

hem

istr

yE

nvir

onm

enta

l an

d re

sou

rce

issu

esSc

ien

tifi

c in

quir

y an

d th

e n

atu

re o

f sc

ien

ceN

atio

nA

vera

geN

atio

nA

vera

geN

atio

nA

vera

geN

atio

nA

vera

geN

atio

nA

vera

geN

atio

nA

vera

geH

un

gary

560

Ch

ines

e Ta

ipei

550

Sin

gapo

re57

0C

hin

ese

Taip

ei56

3Si

nga

pore

577

550

Slov

enia

541

544

Ch

ines

e Ta

ipei

552

Hu

nga

ry54

856

7K

orea

, Rep

ubl

ic o

f54

5C

hin

ese

Taip

ei53

8Si

nga

pore

541

Japa

n54

4Si

nga

pore

545

Au

stra

lia53

0Ja

pan

354

3Sl

ovak

Rep

ubl

ic53

7N

eth

erla

nds

536

Kor

ea, R

epu

blic

of

544

Fin

lan

d53

5N

eth

erla

nds

526

Ch

ines

e Ta

ipei

540

Net

her

lan

ds53

4Sl

ovak

Rep

ubl

ic53

5H

un

gary

543

530

Kor

ea, R

epu

blic

of

523

En

glan

d53

8Ja

pan

533

Hu

nga

ry53

5N

eth

erla

nds

537

Bu

lgar

ia52

7C

anad

a52

1A

ust

ralia

535

Bel

giu

m-F

lem

ish

533

Bel

giu

m-F

lem

ish

535

Au

stra

lia53

1Sl

ovak

Rep

ubl

ic52

5Sl

oven

ia51

9N

eth

erla

nds

534

Cze

ch R

epu

blic

533

Japa

n53

4B

elgi

um

-Fle

mis

h53

0E

ngl

and

524

Hon

g K

ong

SAR

518

Can

ada

532

Kor

ea, R

epu

blic

of

532

En

glan

d53

3R

uss

ian

Fed

erat

ion

529

Kor

ea, R

epu

blic

of

523

En

glan

d51

8H

ong

Kon

g SA

R53

1R

uss

ian

Fed

erat

ion

529

Au

stra

lia53

0E

ngl

and

528

Ru

ssia

n F

eder

atio

n52

3C

zech

Rep

ubl

ic51

6Fi

nla

nd

528

525

Kor

ea, R

epu

blic

of

528

Cze

ch R

epu

blic

526

Can

ada

521

Fin

lan

d51

4B

elgi

um

-Fle

mis

h52

6Si

nga

pore

521

Can

ada

523

Slov

enia

525

Au

stra

lia52

0B

elgi

um

-Fle

mis

h51

3H

un

gary

526

Fin

lan

d52

0Sl

oven

ia52

1H

ong

Kon

g SA

R52

3H

ong

Kon

g SA

R51

5Sl

ovak

Rep

ubl

ic51

2U

nit

ed S

tate

s52

2B

ulg

aria

520

Fin

lan

d52

0C

anad

a52

1N

eth

erla

nds

515

Un

ited

Sta

tes

509

Cze

ch R

epu

blic

522

Au

stra

lia51

9U

nit

ed S

tate

s52

0Fi

nla

nd

520

Cze

ch R

epu

blic

512

Th

aila

nd

507

New

Zea

lan

d52

1C

anad

a51

9R

uss

ian

Fed

erat

ion

517

Slov

ak R

epu

blic

518

Slov

enia

509

Japa

n50

6Sl

oven

ia51

3

Hon

g K

ong

SAR

506

Hon

g K

ong

SAR

516

Lith

uan

ia2

510

Un

ited

Sta

tes

508

New

Zea

lan

d50

3Sl

ovak

Rep

ubl

ic50

7

New

Zea

lan

d50

4B

ulg

aria

514

Bu

lgar

ia50

5B

elgi

um

-Fle

mis

h50

8M

alay

sia

502

Latv

ia-L

SS1

495

Un

ited

Sta

tes

504

Latv

ia-L

SS1

509

New

Zea

lan

d49

9N

ew Z

eala

nd

503

Hu

nga

ry50

1R

uss

ian

Fed

erat

ion

491

Ital

y50

2T

hai

lan

d50

8U

nit

ed S

tate

s49

8It

aly

493

Ru

ssia

n F

eder

atio

n49

5It

aly

489

Latv

ia-L

SS1

495

New

Zea

lan

d50

1La

tvia

-LSS

149

5La

tvia

-LSS

149

0La

tvia

-LSS

149

3M

alay

sia

488

Mal

aysi

a49

1Li

thu

ania

249

4M

alay

sia

494

Iran

, Isl

amic

Rep

ubl

ic o

f48

7It

aly

491

Lith

uan

ia2

483

Lith

uan

ia2

476

Ital

y48

8(I

srae

l)48

4Li

thu

ania

248

5In

don

esia

489

Bu

lgar

ia47

9

Rom

ania

475

Mal

aysi

a47

9It

aly

480

Mal

aysi

a48

548

3(I

srae

l)47

6(I

srae

l)47

2M

oldo

va47

7T

hai

lan

d47

5Jo

rdan

483

Jord

an47

6M

oldo

va47

1T

hai

lan

d47

0R

oman

ia47

5R

oman

ia46

5R

oman

ia48

1C

ypru

s47

5C

ypru

s46

7M

oldo

va46

6C

ypru

s46

8M

aced

onia

, Rep

ubl

ic o

f46

3M

aced

onia

, Rep

ubl

ic o

f48

1R

oman

ia47

3M

aced

onia

, Rep

ubl

ic o

f46

4M

aced

onia

, Rep

ubl

ic o

f46

4M

aced

onia

, Rep

ubl

ic o

f46

8C

ypru

s45

9(I

srae

l)47

9Ir

an, I

slam

ic R

epu

blic

of

470

Th

aila

nd

462

Cyp

rus

459

(Isr

ael)

463

Jord

an45

9C

ypru

s47

0Tu

nis

ia46

2R

oman

ia45

6Ir

an, I

slam

ic R

epu

blic

of

459

Jord

an44

8M

oldo

va45

7M

oldo

va45

1Tu

rkey

461

Tun

isia

451

Jord

an44

6In

don

esia

448

Indo

nes

ia45

2Tu

nis

ia43

9Li

thu

ania

245

8Ir

an, I

slam

ic R

epu

blic

of

446

Tun

isia

442

Turk

ey44

4Ir

an, I

slam

ic R

epu

blic

of

445

Th

aila

nd

439

(Isr

ael)

458

Indo

nes

ia44

6C

hile

435

Tun

isia

441

Turk

ey44

1Tu

rkey

437

Ch

ile44

9Tu

rkey

445

Turk

ey43

5Ir

an, I

slam

ic R

epu

blic

of

437

Ch

ile42

8C

hile

435

Mol

dova

444

Ch

ile44

1In

don

esia

431

Ch

ile43

1Tu

nis

ia42

5In

don

esia

425

Mac

edon

ia, R

epu

blic

of

432

Jord

an44

0P

hili

ppin

es39

0P

hili

ppin

es37

8P

hili

ppin

es39

3P

hili

ppin

es39

4M

oroc

co39

6P

hili

ppin

es40

3M

oroc

co36

3M

oroc

co34

7M

oroc

co35

2M

oroc

co37

2P

hili

ppin

es39

1M

oroc

co39

1So

uth

Afr

ica

348

Sou

th A

fric

a28

9So

uth

Afr

ica

308

Sou

th A

fric

a35

0So

uth

Afr

ica

350

Sou

th A

fric

a32

9

Inte

rnat

ion

al a

vera

ge o

f38

nat

ion

s48

8In

tern

atio

nal

ave

rage

of

38 n

atio

ns

488

Inte

rnat

ion

al a

vera

ge o

f38

nat

ion

s48

8In

tern

atio

nal

ave

rage

of

38 n

atio

ns

488

Inte

rnat

ion

al a

vera

ge o

f38

nat

ion

s48

8In

tern

atio

nal

ave

rage

of

38 n

atio

ns

488

Ave

rage

is s

ign

ific

antl

y h

igh

er t

han

th

e U

.S. a

vera

geA

vera

ge d

oes

not

dif

fer

sign

ific

antl

y fr

om t

he

U.S

. ave

rage

Ave

rage

is s

ign

ific

antl

y lo

wer

th

an t

he

U.S

. ave

rage

1 Des

ign

ated

LSS

bec

ause

on

ly L

atvi

an-s

peak

ing

sch

ools

wer

e te

sted

wh

ich

rep

rese

nts

61

perc

ent

of t

he

popu

lati

on.

2 Lit

huan

ia te

sted

th

e sa

me

coh

ort

of s

tude

nts

as

oth

er n

atio

ns,

bu

t la

ter

in 1

999,

at

the

begi

nn

ing

of t

he

nex

t sc

hoo

l yea

r.3 T

he

shad

ing

of J

apan

may

app

ear

inco

rrec

t; h

owev

er, s

tati

stic

ally

, its

pla

cem

ent

is c

orre

ct.

NO

TE

: E

igh

th g

rade

in m

ost

nat

ion

s. S

ee a

ppen

dix

2 fo

r de

tails

.Pa

ren

thes

es in

dica

te n

atio

ns

not

mee

tin

g in

tern

atio

nal

sam

plin

g an

d/or

oth

er g

uid

elin

es.

See

appe

ndi

x 2

for

deta

ils.

Th

e in

tern

atio

nal

ave

rage

is t

he

aver

age

of t

he

nat

ion

al a

vera

ges

of t

he

38 n

atio

ns.

SOU

RC

E:

Mar

tin

et

al.

(200

0).

TIM

SS 1

999

Inte

rnat

iona

l Sci

ence

Rep

ort:

Fin

ding

s fr

om I

EA

’s R

epea

t of t

he T

hird

Int

erna

tion

al M

athe

mat

ics

and

Scie

nce

Stud

y at

the

Eig

hth

Gra

de.

Exh

ibit

3.1

. C

hes

tnu

t H

ill, M

A: B

osto

nC

olle

ge.

En

glan

d

Cze

ch R

epu

blic

Bu

lgar

ia

Japa

n

Ch

ines

e Ta

ipei

Sin

gapo

re

What were students asked to do on the TIMSS–R assessment?This section contains an example test item fromeach of the five mathematics and six sciencecontent areas assessed in 1999. Included are bothmultiple-choice and free-response item formats.Each example item is introduced with a briefdescription, the content area it represents, thecorrect answer or an example of a writtenresponse that was marked as correct, the U.S.percent correct, and the international averagepercent correct.

Information on the percent correct for each of the38 TIMSS–R nations is provided in tables A3.6(mathematics example items) and A3.7 (scienceexample items) in appendix 3.

Figure 6 shows an example of a mathematics itemthat relates to fractions and number sense. Thisitem asked students to choose the expression thatbest estimated the sum of two three-digitnumbers using rounding. Ninety-three percent ofU.S. students correctly chose B as the answer. Theinternational average was 80 percent.

20


Figure 6.—Example mathematics item 1

The sum 691 + 208 is closest to the sum

A. 600 + 200

B. 700 + 200

C. 700 + 300

D. 900 + 200

Correct answer: B U.S. percent correct: 93 International average: 80

SOURCE: Mullis et al. (2000). TIMSS 1999 International Mathematics Report: Findings from IEA’s Repeat of the Third InternationalMathematics and Science Study at the Eighth Grade. Exhibit 2.18. Chestnut Hill, MA: Boston College.

o

In this example of a measurement item (figure 7),students were asked to find the area of a rectanglecontained in a given parallelogram. Thirty-four

percent of U.S. students correctly answered thisitem, while the international average was 43percent.

21




The figure shows a shaded rectangle inside a parallelogram.

What is the area of the shaded rectangle in square centimeters?

8-3 = 520 cmAnswer: _______________________

4 4 .5= 205

Correct answer: 20 cm2 U.S. percent correct: 34 International average: 43

3 cm

8 cm 5

4 cm

Figure 8 is an example of an item from the datarepresentation, analysis, and probability contentarea. In this item, students were asked to deter-mine which of the two magazines was lessexpensive, given the number of issues and cost ofeach issue. In order to receive full credit for this

item, students had to calculate the cost of 24 issuesfor each magazine and arrive at the answer of TeenLife being 3 ceds less expensive than Teen News. Inthe United States, 26 percent of students receivedfull credit for this item; the international averagewas 24 percent.

Chris plans to order 24 issues of a magazine. He reads the following advertisementsfor two magazines. Ceds are the units of currency in Chris’ country.

Which magazine is the least expensive for 24 issues? How much less expensive?Show your work.

20 18x 3 x 3.5- -$60 .90

+540-$63.0

3 ceds cheaper

Correct answer: Teen Life, 3 ceds cheaper U.S. percent correct: 26 International average: 24

22




Teen LifeMagazine

24 issuesFirst four issues FREEThe remaining issues

3 ceds each.

Teen NewsMagazine

24 issuesFirst six issues FREEThe remaining issues

3.5 ceds each.

24

Figure 9 is an example of an item from thegeometry content area. In this item, students wereasked to determine the measure of the fourthangle of a quadrilateral, given the measurementsof the other three (figure 9). In order to correctlyanswer this item, students needed the knowledgethat the sum of the four angles of a quadrilateralalways equals 360 degrees. Twenty percent of U.S.students answered this item correctly. The interna-tional average was 40 percent.

Figure 10, an algebra item, asked students to deter-mine the number of girls and the number of boysin the fictitious club, given the total number ofmembers and the information that there were 14more girls than boys. Full credit was given ifstudents gave the correct response of 36 boys and50 girls and showed their work. Numerical, alge-braic, and “guess and check” methods were allaccepted for full credit. Twenty-nine percent ofU.S. students received full credit on this item. Theinternational average was 33 percent.

A club has 86 members, and there are 14 more girls than boys. How many boys and how many girls are members of the club?

Show your work.

x+(14+x) = 86 86-36=50

2x+14 = 86

2x+14- 14=86- 14

2x = 72- -2 2x = 36

There are 36 boys and 50 girls .Correct answer: 36 boys and 50 girls U.S. percent correct: 29 International average: 33

23


SOURCE: Boston College, International Study Center, Third International Mathematics and Science Study–Repeat (TIMSS–R),unpublished tabulations, 1999.


In a quadrilateral, each of two angles has a measure of 115º. If the measure of a third angle is 70º, what is the measure of the remaining angle?

A. 60º

B. 70º

C. 130º

D. 140º

E. None of the above

Correct answer: A U.S. percent correct: 19 International average: 40

o



On the diagram, hills and valleys are shown by means of contour lines. Eachcontour line indicates that all points on the line have the same elevation abovesea level.

In which direction does the river flow?

A. Northeast

B. Southeast

C. Northwest

D. Southwest

E. It is not possible to tell from the map.


Figure 11 is an example of an earth science item.This item asked students to read a contour mapand determine which direction a river is flowing.

In the U.S., 48 percent of students answered thisitem correctly; the international average was 37percent.

24



Figure 11.—Example science item 1

N

River

150 m

125 m

100 m

75 m

50 m

25m

100 m75 m

50 m

25m LAKE

o

In this life science item, students were given a foodweb and asked to explain the effects of one part ofthe web on another part (figure 12). Specifically,they were asked to describe the consequences ofcrop failure on the population of robins. Severaltypes of responses were given full credit. Forexample, students could have answered that therobin population would decrease due to predators

eating more robins if mice die. They could havealso answered that the robin population wouldincrease based on predators dying due to lack offood (mice). Other feasible explanations, such asthe robin population being unaffected becausemice would find other sources of grain, were alsogiven full credit.

25




Look at the food web above. If the corn crop failed one year what would mostlikely happen to the robin population? Explain your answer.

The population would go down because, the miceeat corn and if there is no corn for the miceto eat they will die and if the mice die thesnake will have to eat more robins to stayalive .

U.S. percent correct: 35 International average: 26

Sunlight

Corn Mouse

Oak tree Caterpillar

Snake

Robin

Hawk

Figure 13 shows an example of a science item thatrelates to physics. Given data on fuel consumptionand work accomplished, students were asked todetermine and explain which of two machines wasmore efficient by converting the information into

common units or measures that could then becompared. Thirty percent of U.S. eighth-gradestudents answered both parts of this itemcorrectly. The international average was 31percent.

26


SOURCE: Martin et al. (2000). TIMSS 1999 International Science Report: Findings from IEA’s Repeat of the Third InternationalMathematics and Science Study at the Eighth Grade. Exhibit 2.3. Chestnut Hill, MA: Boston College.


Volume of Water Gasoline UsedRemoved in 1 Hour in 1 Hour

(liters) (liters)

Machine A 1000 1.25

Machine B 500 0.5

B

Machine A and Machine B are each used to pump water from a river. The tableshows what volume of water each machine removed in one hour and how muchgasoline each of them used.

a) Which machine is more efficient in converting the energy in gasoline to work?

Answer:__________________________

b) Explain your answer:

l000 -: l.25 = 800500 -: .5 = l000

Machine B is more efficientbecause for every liter of gasolineused it removed l000L of water.With lL of gasoline Machine Aonly removes 800L of water.


Figure 14 shows an example of a science item thatrelates to chemistry. This item asked students torecall that when exposed to moisture and oxygen,iron rusts, and that painting the iron couldprevent this reaction from happening. Sixty-sixpercent of U.S. eighth-grade students correctlyanswered this item. The international average was67 percent.

In figure 15, an environmental and resource issuesitem, students were asked to choose the best expla-nation for why insecticides become ineffectiveover time. Sixty-two percent of U.S. studentsanswered this item correctly; the internationalaverage was 48 percent.

27




Paint applied to an iron surface prevents the iron from rusting. Which ONE ofthe following provides the best reason?

A. It prevents nitrogen from coming in contact with the iron.

B. It reacts chemically with the iron.

C. It prevents carbon dioxide from coming in contact with the iron.

D. It makes the surface of the iron smoother.

E. It prevents oxygen and moisture from coming in contact with the iron.

Correct answer: E U.S. percent correct: 66 International average: 67

o



Insecticides are used to control insect populations so that they do not destroy the crops. Over time, some insecticides become less effective at killing insects, and newinsecticidesmust be developed. What is the most likely reason insecticides become

less effective over time?

A. Surviving insects have learned to include insecticides as a food source.

B. Surviving insects pass their resistance to insecticides to their offspring.

C. Insecticides build up in the soil.

D. Insecticides are concentrated at the bottom of the food chain.


o

Figure 16 is an example of an item that relates toscientific inquiry and the nature of science. In thisitem, students were asked to describe a procedurethat could be used to determine the time it takesfor a person’s heart rate to return to normal afterexercising. They were also asked to list the materi-als needed for their procedure. In order to receivefull credit, students needed to include all of the

following: somebody (or self) measuring“normal” pulse rate with a timer or watch; havingthe subject exercise; and measuring the timeinterval between the completion of exercise andthe pulse rate returning to “normal.” Twenty-onepercent of U.S. students answered this itemcorrectly. The international average was 12percent.

28




Suppose you want to investigate how long it takes for the heart rate to return tonormal after exercising. What materials would you use and what procedureswould you follow?

Materials-Stop watch

procedure-l. check heart rate2. exercise3. stop exercising, begin timing4. check heart rate. when heart rate returns to original rate, stop timing.

U.S. percent correct: 21 International average: 12

How did different groups ofstudents within the UnitedStates perform?Comparisons of U.S. population group perform-ance are common in the literature on studentachievement, especially comparisons by sex andrace/ethnicity. The Condition of Education (NCES2000b), the Digest of Education Statistics (NCES1999), Science and Engineering Indicators–2000(National Science Board 2000), and the variousreports associated with each NAEP assessment(e.g., NCES 1997a, 1997b, and 2000c) routinelyprovide comparisons of the achievement ofselected population groups.

Population groups tend to be defined by demo-graphic attributes such as sex, race/ethnicity,language, and the like. Interest in the comparativeperformance of population groups reflects aconcern that all students—regardless of race,ethnicity, sex, or family background, among otherthings—receive equitable educational opportuni-ties. A national average score cannot describe therange of achievement within a nation and whetherpatterns of performance are associated with differ-ent subgroups.

The analyses that follow focus on five categories ofpopulation groups in the United States; thesegroups are defined by: sex, race/ethnicity, nationalorigin of parents, level of parental education, andtype of school attended.6 These analyses examinethe relationship between specific group character-istics and achievement. These are preliminaryanalyses of the data from TIMSS–R. Futureanalyses will examine the same relationships whileaccounting for other factors.

Figure 17 shows the average mathematics andscience performance for the population groupsnoted above. The results of testing the statisticalsignificance of the difference between groupaverages are described to the right of the groupaverages.7

Was there a difference in themathematics and scienceachievement of U.S. eighth-grade boys and girls?In mathematics, there was no evidence of a differ-ence in achievement between U.S. eighth-gradeboys and girls in 1999. The average score for girlswas similar to the average score for boys. Of theother nations in 1999, only four—the CzechRepublic, Iran, Israel, and Tunisia—showed differ-ences in the achievement of boys and girls inmathematics, all in favor of boys (see table A3.9,appendix 3 for details).

In science, U.S. eighth-grade boys outperformedeighth-grade girls in 1999. In all, the United Statesand 15 other nations showed differences betweenthe average achievement of boys and girls, and alldifferences favored boys.8 Twenty-two nationsshowed no differences between boys and girls inscience. In addition to the United States, Canada,Chile, Chinese Taipei, the Czech Republic,England, Hungary, Iran, Korea, Latvia-LSS,Lithuania, the Netherlands, Russian Federation,Slovak Republic, Slovenia, and Tunisia showeddifferences in science achievement between boysand girls (see table A3.9, appendix 3).

The TIMSS–R findings in mathematics are consis-tent with other studies conducted at this gradelevel, such as NAEP (NCES, 1997a, 2000c). TheTIMSS–R findings for the United States in sciencediffer from the most recent results for NAEP andlong term trend NAEP (NCES, 1997b, 2000c)where no difference in science achievement wasfound between eighth-grade boys and girls.Reasons for the different results in TIMSS–R andNAEP may relate to the differences in the sciencetopics and content areas emphasized in the twoassessment frameworks and the relationship of theframeworks to U.S. science curricula through theeighth grade. Differences and similarities between

29


6Data are analyzed based on students’ reports of sex, race/ethnicity, national origin of parents, and level of parental education. Dataon type of school attended based on school sample.

7Other factors are not controlled for in these analyses.8Readers may recall that there was no difference found in TIMSS 1995 between the science performance of U.S. eighth-grade boysand girls (NCES 1996). As a result of rescaling the TIMSS data, the data show that U.S. eighth-grade boys outperformed girls inscience in 1995.

30


Figure 17.—U.S. eighth-grade mathematics and science achievement,by selected characteristics: 1999Characteristics

Mathematicsaverage

Scienceaverage

Significance

Sex

Boys 505 524 Boys and girls performed similarly in mathematics. Boysoutperformed girls in science.Girls 498 505

Race/ethnicity

White students 525 547 White students outperformed black and Hispanic studentsin mathematics and science. Black and Hispanic students performed similarly to each other in mathematics. Hispanic students outperformed black students in science.

Black students 444 438

Hispanic students 457 462

Public/nonpublic school

Public school students

498 510Nonpublic school students outperformed public schoolstudents in mathematics and science.Nonpublic

school students526 548

National origin of parents

Both U.S. born 510 527In mathematics and science, students whose parents wereboth U.S. born outperformed students whose parents wereboth foreign born. In mathematics and science, studentswhose parents were both U.S. born and students with oneU.S. born parent and one foreign born parent performedsimilarly. In science, students with one U.S. born parentand one foreign born parent outperformed students whoseparents were both foreign born.

Both foreign born 477 472

1 U.S. born, 1 foreign born

496 509

Mother’s education

High school or less 484 499In mathematics and science, students whose motherscompleted college outperformed students whose motherscompleted high school or less. In mathematics and science,students whose mothers completed college outperformedstudents whose mothers attended some college. Inmathematics and science, students whose mothersattended some college outperformed students whosemothers completed high school or less.

Some college 511 525

Completed college 539 554

Father’s education

High school or less 482 495In mathematics and science, students whose fatherscompleted college outperformed students whose fatherscompleted high school or less. In mathematics and science,students whose fathers completed college outperformedstudents whose fathers attended some college. Inmathematics and science, students whose fathers attendedsome college outperformed students whose fatherscompleted high school or less.

Some college 512 529

Completed college 543 560

NOTE: Other factors are not controlled for in these analyses.

SOURCE: U.S. Department of Education, National Center for Education Statistics, Third International Mathematics andScience Study–Repeat (TIMSS–R), unpublished tabluations, 1999.

TIMSS–R and NAEP, as well as PISA, are discussedin chapter 1. A more thorough analysis ofTIMSS–R science data for U.S. boys and girls mayreveal important insights into the differencesnoted here.

Did the achievement of U.S.students differ by race andethnicity?Studies have regularly shown that white studentsoutperform the two largest minority groups in theUnited States—namely, black students andHispanic students—in mathematics and science.TIMSS-R results and other large-scale studies,such as NAEP (NCES 1997a, 1997b, and 2000c),present a similar picture of the achievement ofeighth-grade white students, black students, andHispanic students in the United States. In 1999,the average score for white students was higherthan for either black students or Hispanic studentsin mathematics. Black students and Hispanicstudents performed similarly (see figure 17).

In science, the average 1999 score for U.S. eighth-grade white students was higher than for eitherblack students or Hispanic students, and Hispanicstudents outperformed black students (see figure17). The research literature offers several explana-tions for differences in the performance ofparticular populations, generally suggesting thatvarious forms of inequality of opportunity resultin differences in achievement (Wilson 1987 and1996; Jencks and Phillips 1998). These possibleexplanations are not explored in the analysespresented here.

Did the achievement ofstudents in U.S. public andnonpublic schools differ? In both mathematics and science in 1999, theaverage achievement score of U.S. eighth-gradenonpublic school students was higher than theaverage of their peers in U.S. public schools (figure17).9 Competing explanations for differences inthe achievement of public and nonpublic students

in the United States are found in the research liter-ature. One possible explanation is that the twotypes of schools differ in the quality of the educa-tion offered to students (Coleman, Hoffer, andKilgore 1981, 1982). The rationale here is thathigher quality offerings lead to higher achieve-ment. Another possible explanation offered in theliterature is that differences in achievementbetween public and nonpublic school students arethe result of differences in the socioeconomicstatus of the students recruited into each type ofschool (Jimenez and Lockheed 1991). The ration-ale behind this argument is that differentopportunities for learning are created or nurturedamong students from different socioeconomicbackgrounds. The findings for public andnonpublic students from TIMSS–R are consistentwith findings from NAEP (NCES 1997a, 1997b,and 2000c). Indeed, in nations with sizablenumbers of nonpublic schools (e.g., Australia, theUnited Kingdom, and the United States), onaverage, students who attended nonpublic schoolsdid better than those who attended public schools(Coleman, Hoffer, and Kilgore 1982; Williams andCarpenter 1990; Halsey, Heath, and Ridge 1984).The analyses presented here do not offer anypossible explanation for the observed differences;rather, the analyses simply document achievementdifferences between eighth-grade students in thesetwo types of schools. More thorough analysis ofthe data, taking into account such factors asrace/ethnicity or socioeconomic status, may revealimportant insights into possible reasons for theobserved differences.

Did the achievement of U.S.students of different nationalorigins differ? TIMSS–R asked students to indicate whether theirparents were U.S. or foreign-born. There is aninterest in the birthplace of students’ parentsbecause a sizeable proportion of students withparents born outside the United States may notspeak English as their first language or may notspeak English at home with great frequency, if atall. Since English is generally the language of

31


9Forty-four of the 221 schools sampled in the United States were nonpublic schools. Among these 44 nonpublic schools, 26 wereCatholic, 13 were Protestant/other religious, 4 were non-religious independent schools, and 1 was unspecified.

instruction in U.S. classrooms, students’ facilitywith language may play a role in their ability toadequately understand school subjects. Moreover,immigrant status is often associated with lowersocioeconomic status and more limited educa-tional opportunities. The average 1999mathematics score of eighth-grade students whoseparents were both foreign-born was lower than thescore of students whose parents were both U.S.born (figure 17). In science in 1999, the averagescore of eighth-graders whose parents were bothforeign-born was lower than the score of studentswith at least one parent born in the United States.

Did the achievement of U.S.students differ by the level oftheir parents’ education? The average mathematics performance of eighth-grade students in 1999 differed by their parents’level of education. Students who reported thattheir parents had completed college had a higheraverage score in mathematics than students whoreported that their parents completed somecollege and, in turn, these students had a higherscore than students whose parents had no morethan a high school education (figure 17).

The pattern in science is similar to mathematics in1999. As the level of parental education rises, so dothe test scores of students. On average, in science,eighth-grade students whose parents hadcompleted college outperformed students whoseparents had attended some college and thesestudents, in turn, outperformed students whoseparents had no more than a high school education(figure 17).

The TIMSS–R results indicate that as parentaleducation levels increased so did the mathematicsand science performance of U.S. eighth-gradestudents. The relationship between level ofparental education and the educational achieve-ment of children is well-documented (Sewell,Hauser, and Wolf 1976; Featherman 1981; Riordan1997; NCES 1997a and 1997b).

THE MATHEMATICS ANDSCIENCE ACHIEVEMENT OFEIGHTH-GRADERS BETWEEN1995 AND 1999This section presents results for the 23 nationswith comparable data that participated at theeighth grade in both TIMSS and TIMSS–R.10 Tocompare the performance of eighth-gradestudents on TIMSS and TIMSS–R, both eighth-grade assessments used the same scale.11

Did the performance of U.S.eighth-graders changebetween 1995 and 1999?For the 23 nations that participated in bothTIMSS and TIMSS–R, there was little change inmathematics average scores over the 4-yearperiod. There was no change in eighth-grademathematics achievement between 1995 and 1999in the United States as well as 18 other nations(figure 18).12 Three nations—Canada, Cyprus,and Latvia-LSS—showed an increase in overallmathematics achievement between 1995 and1999. One nation, the Czech Republic, experi-enced a decrease in overall achievement over the

32


10Twenty-six nations participated in TIMSS and TIMSS–R at the eighth grade. Of the 26 nations, 3 nations experienced significantirregularities in their participation in 1995: Israel, South Africa, and Thailand. Findings for the other 23 nations are reported here.Results for the 3 nations that experienced irregularities are provided in appendix 3, tables A3.10 and A3.11.

11The national averages presented here for the TIMSS grade 8 assessment differ a little from the averages appearing in previousTIMSS reports published over the past several years. This is a result of rescaling the TIMSS 1995 grade 8 data to allow for reliablecomparisons to the TIMSS–R 1999 grade 8 data.

12The finding that there has been no change in the overall mathematics score from 1995, when the United States performed at theinternational average, to 1999, when the United States performed above the international average, may appear to be inconsistent.However, readers are cautioned from drawing conclusions based on the relative position of the United States in comparison to theinternational average for all 42 nations in 1995 and all 38 nations in 1999. A more accurate analysis of change in achievement overthe 4 years is the one presented above: a comparison between only the 23 nations that participated in both 1995 and 1999, and theinternational average of scores for these nations.

same period.13 The reader is cautioned againstcomparing the relative change in one nation to therelative change in another nation.

In the United States and 17 other nations, therewas no change in the science achievement score ofeighth-graders between 1995 and 1999. Fournations documented an increase in science

33


Figure 18.—Comparisons of eighth-grade mathematics achievement, by nation: 1995 and 1999

Nation 1995 average 1999 average1995–1999difference3

(Latvia-LSS)1 488 505 �

Canada 521 531 �

Cyprus 468 476 �

Hong Kong SAR 569 582 �

(Netherlands) 529 540 �

(Lithuania)2 472 482 �

United States 492 502 �

Belgium-Flemish 550 558 �

Korea, Republic of 581 587 �

(Australia) 519 525 �

Hungary 527 532 �

Iran, Islamic Republic of 418 422 �

Russian Federation 524 526 �

Slovak Republic 534 534 �

(Slovenia) 531 530 �

(Romania) 474 472 �

(England) 498 496 �

Japan 581 579 �

Singapore 609 604 �

Italy 491 485 �

New Zealand 501 491 �

(Bulgaria) 527 511 �

Czech Republic 546 520 �

International average of 23 nations 519 521 �

The 1999 average is significantly higher than the 1995 average The 1999 average does not differ significantly from the 1995 average The 1999 average is significantly lower than the 1995 average

1Designated LSS because only Latvian-speaking schools were tested.2Lithuania tested the same cohort of students as other nations, but later in 1999, at the beginning of the next school year.3Difference is calculated by subtracting the 1995 score from the 1999 score. Detail may not sum to totals due to rounding.

NOTE: Eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines in 1995, 1999, or bothyears. See appendix 2 for details regarding 1999 data. See NCES (1996) for details for 1995 data.The international average is the average of the national averages of the 23 nations with approved samplingprocedures.The tests for significance take into account the standard error for the reported differences. Thus, a small differencebetween the 1995 and 1999 averages for one nation may be significant while a large difference for another nationmay not be significant.The 1995 scores are based on re-scaled data.

SOURCE: Mullis et al. (2000). TIMSS 1999 International Mathematics Report: Findings from IEA’s Repeat of theThird International Mathematics and Science Study at the Eighth Grade. Exhibit 1.3. Chestnut Hill, MA: BostonCollege.

�

�

�

1710

9131110

98665420

-1-1-1-2-4-6

-10-16-26

2

13In a separate analysis of just those 48 mathematics items (out of 155) in common between TIMSS and TIMSS–R, the same pic-ture of overall eighth-grade mathematics achievement emerges. Results of this separate analysis revealed that 3 nations—Canada,Cyprus, and Latvia-LSS—experienced increases in their mathematics performance over the 4 years on the in-common items. Onenation, the Czech Republic, experienced a decrease in its mathematics performance over the same period of time. The remaining19 nations, including the United States, experienced no change in overall mathematics achievement on the set of 48 in-commonitems between TIMSS and TIMSS–R.

achievement between 1995 and 1999: Canada,Hungary, Latvia-LSS, and Lithuania (figure 19).One nation, Bulgaria, showed a decline in science

over the 4 years.14 Again, the reader is cautionedagainst comparing the relative change in onenation to the relative change in another nation.

34


Figure 19.—Comparisons of eighth-grade science achievement, bynation: 1995 and 1999

Nation 1995 average 1999 average1995–1999difference3

(Latvia-LSS)1 476 503(Lithuania)2 464 488Canada 514 533Hungary 537 552Hong Kong SAR 510 530(Australia) 527 540Cyprus 452 460Russian Federation 523 529(England) 533 538(Netherlands) 541 545Slovak Republic 532 535Korea, Republic of 546 549United States 513 515Belgium-Flemish 533 535(Romania) 471 472 Italy 497 498New Zealand 511 510Japan 554 550(Slovenia) 541 533Singapore 580 568Iran, Islamic Republic of 463 448Czech Republic 555 539(Bulgaria) 545 518

International average of 23 nations 518 521

The 1999 average is significantly higher than the 1995 average The 1999 average does not differ significantly from the 1995 average The 1999 average is significantly lower than the 1995 average


NOTE: Eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines in 1995, 1999, or bothyears. See appendix 2 for details regarding 1999 data. See NCES (1996) for details for 1995 data.The international average is the average of the national averages of the 23 nations with approved samplingprocedures.

The tests for significance take into account the standard error for the reported differences. Thus, a small differencebetween the 1995 and 1999 averages for one nation may be significant while a large difference for another nationmay not be significant.

The 1995 scores are based on re-scaled data.

SOURCE: Martin et al. (2000). TIMSS 1999 International Science Report: Findings from IEA’s Repeat of the ThirdInternational Mathematics and Science Study at the Eighth Grade. Exhibit 1.3. Chestnut Hill, MA: Boston College.

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

272519162014

8753332211

-1-5-8

-12-15-16-27

3

14In a separate analysis of just those 48 science items (out of 143) in common between TIMSS and TIMSS–R, a similar picture ofoverall eighth-grade science achievement emerges. Results of this separate analysis revealed that 3 nations—Canada, Hungary, andLatvia-LSS—experienced increases in science performance over the 4 years on the in-common items. The remaining 20 nations—including the United States—experienced no change in overall mathematics achievement on the set of 48 in-common itemsbetween TIMSS and TIMSS–R.

In sum, eighth-grade mathematics and sciencescores in the United States showed no changesbetween 1995 and 1999. The lack of change innational averages over a relatively short period of4 years may indicate that longer periods of moni-toring achievement may be necessary to detectchange. It may also indicate that change effortsimplemented at the local level may not yet beimpacting achievement measured at the nationallevel. Of course, careful consideration of TIMSSand TIMSS–R data as well as other data on theteaching and learning of mathematics and sciencein middle school is needed to better address thepossible reasons why change was not evident overthe 4 years.

Did the percentage of U.S.students at or above the international top 10 percentbenchmark change over the 4 years?As was discussed earlier in this chapter, averageachievement scores indicate how the averagestudent performs, but reveal little about theperformance of a nation’s top students. Thefollowing analyses document changes in thepercentages of students who scored at or above theinternational top 10 percent and top 25 percentbenchmarks. Detailed information on changes inthese two international benchmarks is provided intables A3.12 (mathematics) and A3.13 (science) inappendix 3.

The percentage of U.S. eighth-graders who scoredat or above the international top 10 percentbenchmark of students in mathematics showed nochange between 1995 and 1999. None of the other22 nations documented a change either. The 1999top 10 percent cut-off score was 616 in mathemat-ics. Applied to the 1995 TIMSS data, 6 percent ofU.S. eighth-graders scored 616 or higher in math-ematics in 1995, placing them among the top 10percent of all students internationally.15 In 1999,this percentage was 9 percent (figure 20).

35


15Readers may note that previous reports on TIMSS indicated that 5 percent of U.S. eighth-grade students were included among allstudents internationally who scored at or above the international top 10 percent benchmark in mathematics, whereas the per-centage reported here is 6 percent. This difference is due to the way that the percentage of students in mathematics in 1995 is cal-culated for comparative purposes. To compare the percentage of students who scored at or above the international top 10 percentbenchmark in mathematics in 1995 to those in 1999, the score point used to determine the top 10 percent in 1999 was also appliedto the 1995 data. This, of course, was not the case when the data was initially reported for TIMSS. This procedure was applied tothe science data as well.

36


Figure 20.—Comparisons of percentages of eighth-grade mathematics students reaching the TIMSS-R 1999 top 10 percent in mathematics achievement, by nation: 1995 and 1999

Nation1995 percentage of

students1999 percentage of

students1995–1999difference3

Hong Kong SAR �

Belgium-Flemish �

Canada �

United States �

Hungary �

(Latvia-LSS)1 �

(Netherlands) �

(Slovenia) �

Russian Federation �

Korea, Republic of �

(Australia) �

(Lithuania)2 �

Iran, Islamic Republic of �

(Romania) �

Singapore �

(England) �

New Zealand �

Japan �

Cyprus �

Slovak Republic �

Italy �

Czech Republic �

(Bulgaria) �

International average of 23 nations �

The 1999 average is significantly higher than the 1995 average The 1999 average does not differ significantly from the 1995 average

The 1999 average is significantly lower than the 1995 average


NOTE: Eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines in 1995, 1999, or bothyears. See appendix 2 for details regarding 1999 data. See NCES (1996) for details for 1995 data.The international average is the average of the national averages of the 23 nations with approved samplingprocedures.1995 scores are based on re-scaled data.1995 percentage of students reaching the top 10 percent is based on 1999 top 10 percent calculations.

SOURCE: Mullis et al. (2000). TIMSS 1999 International Mathematics Report: Findings from IEA’s Repeat of theThird International Mathematics and Science Study at the Eighth Grade. Exhibit 1.7. Chestnut Hill, MA: BostonCollege.

�

�

�

2819

96

135

1213123611

305

4688

344

147

1919

14

543333322211000000

-1-1-1-8-8

1

332312

916

71415153712

415

4678

333

146

1111

15

As in mathematics, the percentage of U.S. eighth-graders who scored at or above the internationaltop 10 percent benchmark of students in scienceshowed no change between 1995 and 1999. The1999 top 10 percent cut-off score was 616 inscience. Applied to the 1995 TIMSS data, 13percent of U.S. eighth-graders scored 616 orhigher in science in 1995, placing them among thetop 10 percent of all students internationally. In

1999, this percentage was 15 percent (figure 21).Among the 22 other nations that participated inTIMSS and TIMSS-R at the eighth-grade level,only 2 nations showed a change in the proportionof students scoring at or above the internationaltop 10 percent benchmark over the same four-yearperiod: Hungary documented an increase whileBulgaria documented a decrease.

37


Figure 21.—Comparisons of percentages of eighth-grade sciencestudents reaching the TIMSS-R 1999 top 10 percent in scienceachievement, by nation: 1995 and 1999

Nation1995 percentage of

students1999 percentage of

students1995–1999difference3

HungaryRussian FederationCanada(Latvia-LSS)1

(Lithuania)2

(Australia)(England)United StatesKorea, Republic of(Netherlands)ItalyHong Kong SARIran, Islamic Republic ofNew Zealand(Romania)(Slovenia)CyprusSlovak RepublicBelgium-FlemishSingaporeJapanCzech Republic(Bulgaria)


The 1999 average is significantly higher than the 1995 average

The 1999 average does not differ significantly from the 1995 average

The 1999 average is significantly lower than the 1995 average1Designated LSS because only Latvian-speaking schools were tested.2Lithuania tested the same cohort of students as other nations, but later in 1999, at the beginning of the next school year.3Difference is calculated by subtracting the 1995 score from the 1999 score. Detail may not sum to totals due to rounding.

NOTE: Eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines in 1995, 1999, or bothyears. See appendix 2 for details regarding 1999 data. See NCES (1996) for details for 1995 data.The international average is the average of the national averages of the 23 nations with approved samplingprocedures.1995 scores are based on re-scaled data.1995 percentage of students reaching the top 10 percent is based on 1999 top 10 percent calculations.


�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

8433

33222111000000

-1-1-2-4

-10

1

221714

7

61919152216

810

212

616

2141132191714

14

141311

4

31717132015

792

116

163

151233212124

13

Did the percentage of U.S.students at or above the international top 25 percentbenchmark change over the 4years?The percentage of U.S. eighth-graders who scoredat or above the international top 25 percentbenchmark of students in mathematics showed nochange between 1995 and 1999. The 1999 interna-tional top 25 percent cut-off score was 555 inmathematics. Applied to the 1995 TIMSS data, 24percent of U.S. eighth-graders scored 555 orhigher in mathematics in 1995, placing themamong the top 25 percent of all students interna-tionally.16 In 1999, this percentage was 28percent. Only one nation showed a change in thepercentage of its students who scored at or abovethe international top 25 benchmark over this sameperiod of time—the Czech Republic documenteda decrease.

The percentage of U.S. eighth-graders who scoredat or above the international top 25 percentbenchmark of students in science showed nochange between 1995 and 1999. The 1999 interna-tional top 25 percent cut-off score was 558 inscience. Applied to the 1995 TIMSS data, 34percent of U.S. eighth-graders scored 558 orhigher in science in 1995, placing them among thetop 25 percent of all students internationally. In1999, this percentage was also 34 percent. Fournations—Canada, Hungary, Latvia-LSS, andLithuania—showed an increase in the percentageof students who scored at or above the interna-tional top 25 benchmark over this same period oftime.

Did the performance of U.S.eighth-graders in the contentareas change between 1995and 1999?Comparisons of performance on the mathematicsand science content areas can be made among the23 nations that participated in TIMSS andTIMSS–R at the eighth-grade level. Detailed infor-mation on changes in performance in themathematics and science content areas between1995 and 1999 is provided in tables A3.14 andA3.15 in appendix 3.

In the five mathematics content areas in commonbetween TIMSS and TIMSS–R, there was nochange in the performance of U.S. eighth-gradersnor of their peers in most of the other 22 nations.However, Canada and Latvia-LSS documentedincreases in performance in four of the five math-ematics content areas over the 4-year period:fractions and number sense; data representation,analysis and probability; geometry; and algebra. Nonation showed a change in the performance of itsstudents in measurement. On the other hand, theCzech Republic showed a decrease in threecontent areas: fractions and number sense;geometry; and algebra. The only other nation toshow a decrease over the four years was Bulgaria inthe area of data representation, analysis, and prob-ability.

In the four science content areas in commonbetween TIMSS and TIMSS–R,17 there was nochange in the performance of U.S. eighth-gradersnor of their peers in most of the other 22 nations.Only one nation, Canada, recorded an increase inthe performance of its eighth-graders in all fourscience content areas over the 4 years. Hungaryand Latvia-LSS showed increases in the perform-ance of their students in two of the four sciencecontent areas. The Czech Republic and SlovakRepublic experienced decreases in physics over thesame four years, and Slovenia documented adecrease in earth science.

38


16To compare the percentage of students who scored at or above the international top 25 percent benchmarks in mathematics andscience in 1995 to those in 1999, the score point used to determine the top 25 percent in 1999 was also applied to the 1995 data.

17The TIMSS–R science assessment reflects the inclusion of 10 new items in the areas of environmental and resource issues, and sci-entific inquiry and the nature of science. In TIMSS, these areas were reported as a single content area. Therefore, there are four sci-ence content areas in common between the two studies that can be reported.

Did the performance of U.S.population groups changebetween 1995 and 1999?TIMSS and TIMSS–R data for several populationgroups showed an increase in performancebetween 1995 and 1999 in mathematics andscience.18 U.S. eighth-grade black studentsshowed an increase in their mathematics achieve-ment over the 4 years. Students whose parents

were both U.S. born also showed an increase inmathematics achievement between 1995 and1999. Students whose mothers or fathers attendedsome college or completed college also showed anincrease in their mathematics performance overthe 4 years. Finally, U.S. eighth-grade studentswhose mothers or fathers completed collegeshowed an increase in science achievement overthe 4 years (figure 22). There was no change foundfor the other groups of students shown in figure22 over the 4 years in mathematics or science.

39


18The U.S. sample for TIMSS did not include sufficient numbers of nonpublic school students to reliably calculate achievementscores for this group.

Figure 22.—Changes in U.S. eighth-grade mathematics and scienceachievement, by U.S. selected characteristics: 1995 and 1999

MATHEMATICS SCIENCE

1995average

1999average

1995–1999difference*

1995average

1999average

1995–1999difference*

Sex SexBoys 495 505 10 Boys 520 524 5Girls 490 498 8 Girls 505 505 0

Race/ethnicity Race/ethnicityWhite students 516 525 9 White students 544 547 3Black students 419 444 25 Black students 422 438 16Hispanic students 443 457 14 Hispanic students 446 462 16



Both U.S. born 496 510 13 Both U.S. born 521 527 6Both foreign born 474 477 2 Both foreign born 465 472 61 U.S. born, 1 foreign born 482 496 13

1 U.S. born, 1 foreign born 498 509 11

Mother’s education Mother’s educationHigh school or less 479 484 6

High school or less 497 499 2

Some college 498 511 13 Some college 522 525 3Completed college 511 539 27 Completed college 531 554 23

Father’s education Father’s educationHigh school or less 474 482 8

High school or less 494 495 1

Some college 498 512 14 Some college 521 529 8Completed college 515 543 28 Completed college 534 560 25

The 1999 average is significantly higher than the 1995 average The 1999 average is not significantly different from the 1995 average The 1999 average is significantly below the 1995 average

*Difference is calculated by subtracting the 1995 average from the 1999 average. Detail may not sum to totals due to rounding.

NOTE: Other factors are not controlled for in these analyses.

SOURCE: U.S. Department of Education, National Center for Education Statistics, Third International Mathematics andScience Study–Repeat (TIMSS–R), unpublished tabulations, 1999.

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

THE MATHEMATICS ANDSCIENCE ACHIEVEMENT OFTHE 1995 FOURTH-GRADECOHORT IN 1999TIMSS and other studies before it have suggestedthat the international performance of the UnitedStates relative to other nations appears lower atgrade 8 in both mathematics and science than atgrade 4. TIMSS–R provides data about the cohortof fourth-grade students in 1995 in comparison tothe cohort of eighth-grade students four yearslater in 1999. However, direct comparisonsbetween the 1995 fourth-grade assessment and the1999 eighth-grade assessment are complicated byseveral factors: First, the fourth-grade and eighth-grade assessments include different test questions.By necessity, the kind of mathematics and scienceitems that can be asked of an eighth-grader may beinappropriate for a fourth-grader. Second, becausemathematics and science differ between the twogrades, the content areas assessed also differ. Thatis, geometry and physics at grade 4 are differentfrom geometry and physics at grade 8, forexample. Without a sufficient set of in-commontest items between the grade 4 and grade 8 assess-ments, it can be difficult to construct a reliable andmeaningful scale on which to compare the 1995fourth-graders to 1999 eighth-graders. Thus, forpurposes of this report, comparisons betweenfourth and eighth grade are based on the perform-ance relative to the international average of the 17nations that participated in fourth-grade TIMSSand eighth-grade TIMSS–R.

Has the relative performanceof the United States changedbetween fourth and eighthgrade over the 4 years?Figures 23 and 24 display a comparison of theaverage scores of the 17 nations between fourth-grade TIMSS and eighth-grade TIMSS–R to theinternational averages at both grades for eachsubject. The numbers shown in the figures aredifferences from the international average for the17 nations. Nations are sorted into three groups:above the international average; similar to theinternational average; and below the internationalaverage.

In mathematics, the U.S. fourth-grade score in1995 was similar to the international average ofthe 17 nations in common between fourth-gradeTIMSS and eighth-grade TIMSS–R. At the eighthgrade in 1999, the U.S. average in mathematics wasbelow the international average of the 17 nations.Thus, U.S. fourth-graders performed at the inter-national average in 1995 and U.S. eighth-gradersperformed below the international average in 1999in mathematics, suggesting that the relativeperformance of the cohort of 1995 U.S. fourth-graders in mathematics was lower relative to thisgroup of nations 4 years later. The data alsosuggest that, in mathematics, the relative perform-ance of the cohort of 1995 fourth-graders inCanada was higher relative to this group ofnations in 1999; the relative performance of thecohort of 1995 fourth-graders in the CzechRepublic, Italy, and the Netherlands was lowerrelative to this group of nations 4 years later; andthe relative performance of the cohort of 1995fourth-graders in the 12 other nations wasunchanged relative to this group of nations 4 yearslater.

40


41


Figure 23.—Mathematics achievement for TIMSS-R 1999 nations thatparticipated in 1995 at both the fourth and eighth grades relative tothe average across these nations

1995 1999Fourth grade Eighth grade

Difference from average across 17 nations1 Difference from average across 17 nations1

Singapore SingaporeKorea, Republic of Korea, Republic ofJapan Hong Kong SARHong Kong SAR Japan(Netherlands) NetherlandsCzech Republic Hungary(Slovenia) Canada(Hungary) SloveniaUnited States Australia(Australia) Czech Republic(Italy) Latvia-LSS2

Canada United States(Latvia-LSS)2 England(England) New ZealandCyprus ItalyNew Zealand CyprusIran, Islamic Republic of Iran, Islamic Republic of



Average is significantly higher than the international averageAverage does not differ significantly from the international averageAverage is significantly lower than the international average

1Difference is calculated by subtracting the international average of the 17 nations from the national average of each nation.2Designated LSS because only Latvian-speaking schools were tested.

NOTE: Fourth and eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines at fourth grade in 1995. SeeNCES (1997c) for details.The international average is the average of the national averages of the 17 nations.

SOURCE: Mullis et al. (2000). TIMSS 1999 International Mathematics Report: Findings from IEA’s Repeat of the ThirdInternational Mathematics and Science Study at the Eighth Grade. Exhibit 1.4. Chestnut Hill, MA: Boston College.

736350403223

8400

-7-12-18-33-42-48

-130

517

8063585516

8761

-4-19-22-28-33-39-48

-102

524

In science, the U.S. fourth-grade score in 1995 wasabove the international average of the 17 nationsin common between fourth-grade TIMSS andeighth-grade TIMSS–R. At the eighth grade in1999, the U.S. average in science was similar to theinternational average of the 17 nations. Thus, U.S.fourth-graders performed above the internationalaverage in 1995 and U.S. eighth-gradersperformed similar to the international average in1999 in science. As in mathematics, this suggeststhat the relative performance of the cohort of U.S.fourth-graders in science was lower relative to thisgroup of nations 4 years later. The data alsosuggest that, in science, the relative performance ofthe cohort of 1995 fourth-graders in Singapore

and Hungary was higher relative to this group ofnations in 1999; the relative performance of thecohort of 1995 fourth-graders in Italy and theNew Zealand was lower relative to this group ofnations 4 years later; and the relative performanceof the cohort of 1995 fourth-graders in the 12other nations was unchanged relative to this groupof nations 4 years later.

The available evidence appears to confirm whathad been suggested 4 years ago: that the relativeperformance of U.S. students in mathematics andscience is lower at the eighth grade than at thefourth grade among this group of nations.

42


Figure 24.—Science achievement for TIMSS-R 1999 nations thatparticipated in 1995 at both the fourth and eighth grades relative tothe average across these nations

1995 1999Fourth grade Eighth grade

Difference from average across 17 nations1 Difference from average across 17 nations1

Korea, Republic of SingaporeJapan HungaryUnited States Japan(Australia) Korea, Republic ofCzech Republic Netherlands(Netherlands) Australia(England) Czech RepublicCanada England(Italy) SloveniaSingapore Canada3

(Slovenia) Hong Kong SARHong Kong SAR United States(Hungary) New ZealandNew Zealand Latvia-LSS2

(Latvia-LSS)2 ItalyCyprus CyprusIran, Islamic Republic of Iran, Islamic Republic of



Average is significantly higher than the international averageAverage does not differ significantly from the international averageAverage is significantly lower than the international average

1Difference is calculated by subtracting the international average of the 17 nations from the national average of each nation.2Designated LSS because only Latvian-speaking schools were tested.3The shading of Canada in eighth grade may appear incorrect; however, statistically, its placement in correct.

NOTE: Fourth and eighth grade in most nations. See appendix 2 for details.Parentheses indicate nations not meeting international sampling and/or other guidelines at fourth grade in 1995. SeeNCES (1997c) for details.The international average is the average of the national averages of the 17 nations.


62392828181714121010

8-6-6-9

-27-64

-134

514

4428252421161514

995

-9-15-21

-26-64-76

524

pursuing excellence: comparisons of international eighth ...nces.ed.gov/pubs2001/2001028a.pdf ·...

Documents