96p.mcconnell, eugene h. owen of nces, kitty mak of essi, patrick gonzales of essi, and john luczak...

DOCUMENT RESUME

ED 433 243 SE 062 784

AUTHOR Kimmelman, Paul; Kroeze, David; Schmidt, William; van derPloeg, Arie; McNeely, Maggie; Tan, Alexandra

TITLE A First Look at What We Can Learn from High PerformingSchool Districts: An Analysis of TIMSS Data from the Firstin the World Consortium.

INSTITUTION National Inst. on Student Achievement, Curriculum, andAssessment (ED/OERI), Washington, DC.

REPORT NO SAI-1999-3011PUB DATE 1999-08-00NOTE 96p.

AVAILABLE FROM ED Pubs, P.O. Box 1398, Jessup, MD 20794-1398; Tel:877-433-7827 (Toll Free); TTY/TTD: 877-57677734; Fax:301-470-1244; Web site: http://www.ed.gov/pubs/edpubs.html

PUB TYPE Reports - Research (143)EDRS PRICE MF01/PC04 Plus Postage.DESCRIPTORS *Academic Achievement; *Cross Cultural Studies; Elementary

Secondary Education; Foreign Countries; *MathematicsAchievement; Mathematics Education; *School Districts;Science Education; *Socioeconomic Influences; TeachingMethods

IDENTIFIERS *Third International Mathematics and Science Study

ABSTRACTInterest in the First in the World (FiW) Consortium's

activities has been intense for a number of reasons. Initial results showthat the FiW Consortium is well on its way to meeting its goal, with the FiWperforming at or near the top of the world on the international benchmarkchosen by the Consortium--the Third International Math and Science Study(TIMSS). A status report is provided on the FiW's initial activities. Thefirst section describes the districts that make up the FiW Consortium andoutlines the Consortium's history, purpose, goals, and plan of action tobecome the first in the world in math and science. The second sectiondocuments how the students from the FiW Consortium performed when benchmarkedagainst an international comparison, the TIMSS assessment. The next sectiondiscusses the impact that socio-economic variables could have on theperformance of FiW students. The fourth section presents data on the contextsfor teaching and learning mathematics in the FiW, exploring differencesbetween the FiW and the U.S. and, where data are readily available,differences between the contexts for teaching in the FiW and countries withhigh mathematics achievement. The fifth section describes some of theactivities being undertaken by the FiW Consortium to improve math and scienceeducation, highlighting a recent project that is using data from TIMSS andother sources to improve science instruction. The final section summarizesthe whole report and offers some possible questions for future research. Twoappendices provide: (1) contact information and characteristics of FiWConsortium Districts; and (2) 12 tables presenting FiW TIMSS AchievementResults by country. (Contains 19 references and 32 endnotes.) (ASK)

E

cnC1

res

4.1

4

a

add

D aQ'CiC>

f?,-C-Cji2E1 cz.i>Qpow7. 0-0-6:71

D rOAV ED

hac3 171-- 3

c'0 DB' \o

J1

is----( 1 rGL (\-1j

IQ

UM rartrilrgi0 ee

0 LSVlS u 0

U S DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

UCATIONAL RESOURCES INFORMATION

CENTER (ERIC)Th s document has been reproduced as

eived from the person or organizationoriginating itMinor changes have been made to

improve reproduction quality

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy

L

COPY AVAILABLE

- e- I

e-s. Si

To help us improve future editions of this publication and giveyou better customer service, we would appreciate yourcomments on this survey form. Please check the appropriateboxes below for each question. Responses will be keptcompletely confidential. You may return the survey by mail orFAX. It can be folded and taped closed to allow mailing to theaddress listed on the reverse side of this form, or it can bereturned by FAX to 202-219-1321. Many thanks for yourcustomer feedbackft is very Important to us!

1. Name of publication

2. How did you receive a copy of this publication?

Bought it

Borrowed it

Mailing list membership

Telephone request

Internet request

Other (please describe)

3. Was this publication easy to get?

Very Somewhat Not at all

4. How did you find out about this and other OERIpublications? (Check all that apply.)

Conferences

Journal articles

Teacher/educator

Professional associations

0 Internet (WWW)

Publication announcement

Received in mail

OERI staff contact

5. For what purposes did you use this OERI publication?(Check all that apply.)

Planning

Policy or legislation

Administrative decisions

Teaching, class material

Research/analysis

General information

Writing news articles, TV or radio material

Marketing, sales, or promotion


6. Did the publication help you accomplish whatever youneeded ft for?

Yes No Partially

7. What is your occupation?

Parent Teacher Administrator

Librarian Researcher Statistician

Journalist/writer Policy Analyst Student

Program Planner

Other (please specify)

8. How could this OERI publication (or other OERIpublications) better meet your needs?(Check all that apply).

0 More important topics in education

More timely release of data

More text introductions to each section

More research statistics

Shorter reports (less than 10 pages)


9. Overall, how satisfied are you with the following aspects of this publication?

Very Satisfied

a. Comprehensiveness of information

b. Clarity of writing (readability, interpretability)

c. Clarity of presentation (e.g., tables, charts)

d. Timeliness of information

e. Accuracy of information

f. Clarity of technical notes

g. Usefulness of resources and bibliography

h. Organization

i. Length

j. Format

Satisfied Dissatisfied

O

0

PAPERWORK BURDEN STATEMENTOffice of Educational Research and Improvement (OERI)

Publication Customer Survey

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of Information unless it displays a valid OMBcontrol number. The validOMB control number for this Information collection Is 1880-0529. The time required to complete this information collection is estimated to average 10 minutes per response,Including the time to review Instructions. search existing data resources, gather the data needed, and complete and review the Information collection. If you have any comments

concerning the accuracy of the time estimate(s), suggestions for improving this form, or concerns regarding the status of your Individual submission of this form, writedirectly to:

P. Quinn, Room 204, Media and Information Services, OERI, U.S. Department of Education, 555 New Jersey Avenue NW, Washington, DC 20208-5570.

3 BEST COPY AVAICABLE

OERI Publication Customer SurveyMedia and Information Services

555 New Jersey Avenue NWRm. 202Washington DC 20208-5570

OFFICIAL BUSINESSPENALTY FOR PRIVATE USE, $300

0 I N

BUSINESS REPLY MAILFIRST-CLASS MAIL PERMIT NO. 012935 WASHINGTON DC

POSTAGE WILL BE PAID BY US DEPARTMENT OF EDUCATION

A. ReedRoom 202Media and Information ServicesOffice of Educational Research and ImprovementUS Department of Education555 New Jersey Avenue NWWashington DC 20208-5570

NO POSTAGENECESSARY

IF MAILEDIN THE

UNITED STATES

Fold on line TAPE CLOSEDDO NOT STAPLE

10. Do you have any suggestions regarding the content or format of future editions of this publication or other comments?

4

MIS 1999-6532

A FIRST LOOK AT WHAT WE CAN LEARN FROMHIGH PERFORMING SCHOOL DISTRICTS:

An Analysis of TIMSS Data From the First in theWorld Consortium

Paul Kimmelman, President, First in the World ConsortiumDavid Kroeze, Superintendent, Northbrook School District 27

William Schmidt, Michigan State UniversityArie van der Ploeg, North Central Regional Educational Laboratory

Maggie McNeely, U.S. Department of EducationAlexandra Tan, Education Statistics Services Institute

U.S. Department of EducationRichard W. RileySecretary

Office of Educational Research and ImprovementC. Kent McGuireAssistant Secretary

National Institute on Student Achievement, Curriculum, and AssessmentIvor A. PritchardActing Director

Media and Information ServicesCynthia Hearn DorfmanDirector

August 1999

This study was funded by the Office of Educational Research and Improvement, U.S.content does not necessarily reflect the views of the Department or any other agency

To obtain copies of this report (while supplies last) or ordering information on other U.products, call toll free 1-877-4ED-Pubs (877-433-7827), or write:

U.S. Department of EducationED PubsP.O. Box 1398Jessup, MD 20794-1398

Copies are available in alternate formats upon request.

TTY/TTD: 1-877-576-7734FAX: 301-470-1244

Online ordering via the Internet: http://www.edgov/pubeedpubs.html

Content Contact:Maggie McNeely202-219-1568

Department of Education. Theof the U.S. Government.

S. Department of Education

Acknowledgments

The authors of this report represent a unique collaboration among local school districts,the U.S. Department of Education (ED), the Education Statistics Services Institute(ES SI), and the North Central Regional Educational Laboratory. We wish toacknowledge all those who contributed to this report, especially the teachers, students,administrators, and parents of the First in the World Consortium. Without theirdedication and commitment to improving math and science achievement, this reportwould not exist.

We would also like to thank Erin C. Massie, Joanna Wertheimer, and Mark White ofESSI and Blase Masini of NCREL who provided valuable research assistance. J.W.McConnell, Eugene H. Owen of NCES, Kitty Mak of ESSI, Patrick Gonzales of ESSI,and John Luczak of ED all gave their time and expertise by reviewing draft reports andproviding comments and suggestions. We are particularly grateful to Ron Varnum, fromColgate University, who did the final formatting and quality control work on the report.Christopher Sheedy, of ED, provided editorial services.

7

Contents

Introduction 1

What is the FiW Consortium? 3

History and Description 3

Goals and Objectives 4

Plan of Action 5

How Well Did the FiW Consortium Perform When Benchmarked Againstan International Measure of Math and Science Achievement? 7

Fourth-Grade Results 7

Eighth-Grade Results 8

Twelfth-Grade Results 8

General Knowledge 9

Advanced Math and Physics 10

AP Calculus and AP Physics Results 10

Summary 11

Do Home Factors Explain the High Achievement of the FiW Consortium? 13

What is the Context for Teaching and Learning Math in the FiW Consortium? 14

Curriculum 15

Instructional Topic Coverage 15

Textbook Use and Topic Coverage 20

Fourth Grade 21

Eighth Grade 21

Summary 22

Classroom Instructional Practices 22

Teachers' Reports on Class Activities 22

Teachers' Reports on Classroom Organization 26

Students' Reports on Classroom Organization and Class Activities 28

Summary 31

Teacher Engagement 32

Teacher Involvement in School-related Activities Outside the School Day 32

Frequency of Teacher Meetings 35

Teacher Influence Over Key School Decisions 36

Teacher Familiarity with Key Curriculum and Assessment Documents 39

Summary 40

Teaching Environment 41

Summary 43

What is the FiW Consortium Doing to Improve Math and Science? 45

Teacher Learning Networks: Collaborative Learning Communities 45

Teacher Network Grants 48

vii

Example of a FiW Teacher Learning Network: Analysis of FiW Physics Achievement 49Project Activities 50Initial Results 50Topics for Further Research Identified by the Physics Teachers' Learning Network 51Summary 52

Summary and Conclusion 53

Bibliography 57

Endnotes 59

Appendix A: Contact Information and Characteristics of First in the WorldConsortium Districts A-1

Appendix B: FiW TIMSS Achievement Results B-1

viii

List of Exhibits

Exhibit 1: Fourth-Grade Math Teachers' Reports on Instructional Topic Coverage 18

Exhibit 2: Eighth-Grade Math Teachers' Reports on Instructional Topic Coverage 19

Exhibit 3: Teachers' Reports on Use of Textbooks and Other Teaching Materials and Percent

of Mathematics Teaching Time based on Textbook 20

Exhibit 4: Teachers' Reports on How Frequently Students Are Asked to Complete Specific

Tasks 25

Exhibit 5: Teachers' Reports on Classroom Organization During Mathematics Lessons 27

Exhibit 6: Students' Reports on the Frequency of Math Class Activities 30

Exhibit 7: Teachers' Reports on Hours Spent Per Week on Activities Outside the Formal

School Day 34

Exhibit 8: Teachers' Reports on the Frequency of Meetings with Other Teachers in Their

Subject Area to Discuss and Plan Curriculum or Teaching Approaches 36

Exhibit 9: Teachers' Reports on Their Influence over School Decisions 37

Exhibit 10: Teachers' Reports on Their Familiarity with Key Curriculum Documents 38

Exhibit 11: Teachers' Reports on the Factors that Limit How They Teach Mathematics Class 42

Exhibit 12: First in the World Teacher Learning Networks 47

Appendix A: Contact Information and Characteristics of First in the World Consortium Districts

Exhibit A-1: List of First in the World Districts A-2

Exhibit A-2: First in the World District Characteristics A-4

Appendix B: FiW TIMSS Achievement Results

Exhibit B-1: Distributions of Mathematics Achievement in the Fourth Grade B-2

Exhibit B-2: Distributions of Science Achievement in the Fourth Grade B-3

Exhibit B-3: Fourth-Grade Mathematics Performance, by Subtopic B-4

Exhibit B-4: Distributions of Mathematics Achievement in the Eighth Grade B-5

Exhibit B-5: Distributions of Science Achievement in the Eighth Grade B-6

ix

10

Exhibit B-6: Eighth-Grade Mathematics Performance, by Subtopic B-7

Exhibit B-7: Distributions of General Mathematics Achievement in the Twelfth Grade B-8

Exhibit B-8: Distributions of General Science Achievement in the Twelfth Grade B-9

Exhibit B-9: Distributions of Advanced Mathematics Achievement in the Twelfth Grade B-10

Exhibit B-10: Distributions of Advanced Science (Physics) Achievement in the Twelfth Grade B-11

Exhibit B-11: Distributions of Advanced Mathematics Achievement for Twelfth-Grade FiW

AP Students B-12

Exhibit B-12: Distributions of Advanced Science (Physics) Achievement for Twelfth-Grade FiW

AP Students B-13

11x

Introduction

The efforts of a group of small schooldistricts north of Chicago to become first inthe world in math and science achievementhave recently captured the interest ofeducators, researchers, and policymakersacross the country.

Working collaboratively, rather thancompetitively, this unique consortium ofdistricts has begun to benchmark its studentsagainst an international standard ofachievement, identify and implement bestpractices for improving math and scienceachievement, and establish learningnetworks among the educators within itsdistricts.

Interest in the First in the World (FiW)Consortium's activities has been particularlyintense for a number of reasons. First, initialresults show that the FiW Consortium iswell on its way to meeting its goal, with theFiW performing at or near the top of theworld on the international benchmarkchosen by the Consortiumthe ThirdInternational Math and Science Study(TIMSS).'

Given the disappointing performance of U.S.students on the same international math andscience assessments, many believe that theapproaches taken by the FiW towards mathand science may offer some importantlessons for other schools and districts in theU.S.

Second, the results obtained by the FiWConsortium on this international benchmarkalso provide the first U.S. multi-district leveldata available from TIMSS. As such, theymay provide valuable insights into thecontexts for learning math and science inhigh-performing districts, and how they

might relate to world class standards, notonly in achievement but also ininstructional, curricular, and assessmentstandards. Finally, the Consortium'sactivities offer a rare opportunity to learnfrom this unique cross-district, multi-partnercollaborative effort to obtain world-classstandards.

This report provides a status report on theFiW's initial activities. It examines five keyquestions:

What is the FiW Consortium?

How well did the FiW Consortiumperform when benchmarked against an ,

international measure of math andscience achievement?

Do home factors explain the highachievement of the FiW Consortium?

What is the context for teaching andlearning in the FiW Consortium?

What is the FiW Consortium doing toimprove math and science?

It should be noted that this report constitutesone of the first comprehensive examinationsof how FiW students performed on TIMSSand an initial exploration of some of thereasons for their performance.

This paper is not based on an exhaustivereview of the possible reasons for the FiWperformance, but rather examines possiblefactors that might have had an importantrole. In addition, the paper does notinvestigate causal links between FiWachievement and the different topicsdiscussed later.

The remainder of the report is organized intomajor sections, which roughly correspond tothe key questions listed above.

The first section describes the districts thatmake up the FiW Consortium and outlinesthe Consortium's history, purpose, goals,and plan of action to become the first in theworld in math and science.

The second section documents how thestudents from the FiW Consortiumperformed when benchmarked against aninternational comparison, the TIMSSassessment.

The next section discusses the impact thatsocio-economic variables could have on theperformance of FiW students.

The fourth section presents data on thecontexts for teaching and learningmathematics in the FiW, exploringdifferences between the FiW and the U.S.and, where data are readily available,differences between the contexts forteaching in the FiW and countries with highmath achievement.

The fifth section describes some of theactivities being undertaken by the FiWConsortium to improve math and scienceeducation, highlighting a recent project thatis using data from TIMSS and other sourcesto improve science instruction.

The final section summarizes the report andoffers some possible questions for futureresearch.

13

What is the FiW Consortium?

This section describes the First in the WorldConsortium. It answers questions such as:

How and why was the Consortiumlaunched?

How many students are in theConsortium?

What are its goals?

How does it hope to achieve these goals?

It begins with a description of theConsortium's history and origins, and thendescribes its goal and objectives, as well asthe Consortium's plan of action forachieving these objectives.

History and Description

The FiW Consortium is currently made upof 17 school districts located in the northsuburbs of Chicago and the Illinois Mathand Science Academy. Together, they arepursuing a common goal of becoming firstin the world in math and scienceachievement.

The Consortium grew out of a study groupof superintendents which was formed tofulfill an administrative re-certificationrequirement. Members of this study groupmet regularly over the course of severalmonths. Discussions at these meetingscentered on contemporary education reformissues facing the administrators in theirvarious districts.

The FiW Consortium was launched at thefinal meeting of the superintendents' group,

3 14

during which the National Education Goalswere discussed.

Determined to take the national goalsseriously, the superintendents decided toform a consortium of districts committed toproviding a world class education for theirstudents. The group agreed to first focuscollectively on obtaining Goal 5, to be firstin the world in math and science by the year2000.

Reflecting their goal, the group calledthemselves the First in the WorldConsortium, a title with which they feltuncomfortable, but which accuratelycaptured their goals and aspirations. InMarch 1995, the Consortium entered intopartnership with the U.S. Department ofEducation and the North Central RegionalEducation Laboratory in its efforts to obtainthis goal.

As of winter 1999, the Consortium included13 elementary districts (grades K-8), 3 highschool districts (grades 9-12), the NorthSuburban Special Education District (whichserves most of the districts' specialeducation students), and the IllinoisMathematics and Science Academy(IMSA).2 IMSA is a publicly fundededucational laboratory and three-yearresidential secondary program for Illinoisstudents gifted in math and science. WhileIMSA is a state agency, the term "districts"is used throughout the report as aconvenience to the reader.

Together, these districts contain 49elementary or middle schools, 6 highschools, and one special education school.During the 1996-97 school year, totalstudent enrollment in these schools was

approximately 36,000 students. Nearly fourout of five students (78 percent) within theConsortium were white, non-Hispanic.Fourteen percent of the students wereAsian/Pacific Islanders, while seven percentwere Hispanic and two percent were black,non-Hispanic.

The Consortium contains relatively highwealth districts, and thus may have differentcharacteristics from many individual U.S.districts and the U.S. as a whole. In the1995-96 school year, average per-pupilexpenditures across the Consortium wasapproximately $8,9583 compared to $5,774in the U.S .4

Yet, not all of the Consortium's studentscome from high-income families. In 1996-97, seven percent of the Consortium'sstudents were classified as coming fromlow-income families.' Six percent ofConsortium students had limited-Englishproficiency (LEP).

Approximately 2,600 classroom and specialeducation teachers teach within theConsortium.6 In general, FiW teachers tendto have higher education levels than theirU.S. counterparts: Sixty-three percent ofFiW teachers have earned at least a master'sdegree versus 56 percent of U.S. teachers.'The average number of years of teachingexperience for FiW teachers is 14 years.'

Perhaps reflecting the education andexperience levels of FiW teachers, averageteacher salaries in Consortium districts arerelatively high. In 1995-96, the averagesalary for FiW elementary school teacherswas $47,339. The average salary for FiWhigh school teachers was $65,263.9

U.S. teacher salaries for the same timeperiod are lower. The average salary in the

U.S. during the 1995-96 school year was$39,976 for elementary school teachers and$38,423 for secondary school teachers."'See appendix A for a current list of FiWdistricts and more information on FiWdistrict characteristics.

Goals and Objectives

As noted earlier, the FiW Consortiumdecided to focus first on obtaining Goal 5 ofthe National Education Goals: to becomefirst in the world in math and science. TheConsortium's leadership set three objectivesto help them obtain this distinction. Theywere:

Benchmark Consortium schools'performance against an internationalmeasure of student achievement;

Create a forum to clarify world-classstandards for business leaders,policymakers, educators, and communitymembers; and

Establish networks of learningcommunities that actively involveeducators, parents, and communityleaders.

These objectives were chosen to provideConsortium leaders and educators with abaseline against which to measure theirprogress, as well as a better understandingand knowledge of the instructional,curricular, and assessment practices neededto obtain world-class achievement.

Recognizing that obtaining this ambitiousgoal would involve input, advice, andsupport from all members of theConsortium's community, they also soughtto actively involve educators, parents, andcommunity leaders.

15

Plan of Action

To obtain the consortium's goal andobjectives, the FiW leadership developedand embarked upon a three-step plan ofaction. These steps were:

(1) developing partnerships at the national,regional, and local levels;

(2) identifying and defining world classstandards in math and science; and

(3) working with the Consortium's partnersto implement exemplary math andscience programs.

Each of these steps is described brieflybelow.

First, the FiW Consortium sought to formpartnerships with key organizations in theeducation and business communities toobtain technical, administrative, andresearch support in achieving its goals.Accordingly, the FiW has establishedworking partnerships with numerousorganizations, including the U.S.Department of Education, the North CentralRegional Education Lab (NCREL), andpolicymakers at the national level, includingmembers of Congress.

Under its agreement with the U.S.Department of Education, the Consortiumcommitted to work with the Department toexplore general outcome and specific mathand science competencies in a study ofglobal competition. It also promised todevelop world class standards and alignthese standards with their local curricularand instructional programs and to supportthe acceleration of technologyimplementation as it pertains to math andscience achievement.

5.1 6

In addition, the Consortium agreed todevelop a math and science resource centeron the World Wide Web, as well ascollaborate with the Department todisseminate the Consortium's findings. Italso pledged to include all students in itsefforts, including students with disabilities.

The Consortium resolved to develop andimplement assessment instruments todetermine student achievement, and toimplement staff development training toassist teachers in mastering new content andinstructional strategies.

Further, the Consortium committed toentering into school and businesspartnerships to foster the identification ofthe needed skills and knowledge to achieveworld class standards.

Finally, the Consortium agreed to committhe needed resources to ensure that the jointeffort did not fail due to a lack of resources.Taken together, these commitments form aunique partnership between a group of localschool districts and the Department ofEducation.

Second, the FiVleadership is working withits partners to identify and define world-class standards in the areas of math andscience. As part of this effort, FiW studentsparticipated in TIMSS, the most ambitious,comprehensive, and rigorous internationalassessment of math and science everundertaken. With financial support from theindividual boards of education that make upthe Consortium, FiW fourth, eighth, andtwelfth grade students, their teachers, andtheir schools participated in the TIMSSstudy.

Although an invitation to participate inTIMSS was extended to all school districts

in the nation, the FiW Consortium was theonly group that took advantage of thisopportunity.

Third, FiW districts are working together todesign and implement exemplary programsin mathematics and science. To achieve thisgoal, FiW leaders have established teacherlearning networks which bring together stafffrom across the Consortium to identify,develop, and enact model programs in mathand science instruction.

Teams are self-identified and includeteachers, principals, superintendents, andother educational staff. Relying on datafrom TIMSS, as well as research on bestinstructional and curricular practices, theConsortium hopes to improve achievementby strengthening instruction, using moreeffective assessment tools, learning aboutnew curricula materials and techniques, andidentifying and addressing topics or areas

6

where their students demonstrateweaknesses.

In sum, the districts that make up the FiWConsortium have sparked the attention of theeducation community by agreeing to worktogether to become first in the world in mathand science. The following sectionsdescribe the efforts to enact their plan, aswell as some of the their preliminary results.

In particular, the next section describes theresults of the Consortium's effort to measureits performance against an internationalbenchmark. The following two sectionslook at what might account for the highachievement levels obtained, exploring firstthe effect of socio-economic factors, andthen the Consortium-wide context forteaching and learning. The subsequentsection describes the Consortium's efforts todevelop learning communities of educators,policymakers, and community leaders.

.17

How Well Did the FiW Consortium Perform When Benchmarked Against anInternational Measure of Math and Science Achievement?

As mentioned earlier, one of the FiWConsortium's three primary goals was tobenchmark its achievement against aninternational measure of studentachievement. The FiW chose TIMSS as itsmeasure. FiW student assessments wereadministered during 1996, and preliminaryresults were made available in January 1997.

This section discusses the results of theseassessments.

In general, FiW students did exceedinglywell on TIMSS, particularly in the fourthand eighth grades. In the twelfthgradeadvanced math and physics tests, the FiWAdvanced Placement (AP) students alsoscored among the top performing countries.FiW students performed among the highestperforming nations on the twelfth gradegeneral knowledge achievement tests andnear the international average on theadvanced math and physics tests.

Fourth, eighth, and twelfthgrade resultsare discussed below and reported in moredetail in appendix B.

FoUrthGrade Results

Only students in Singapore had scoressignificantly above those of FiW students onthe fourth grade math assessment. The FiWConsortium had average scores that were notsignificantly different from four othercountries (Korea, Japan, Hong Kong, andNetherlands). FiW students outperformedtheir counterparts in 21 of 26 countries.

In science, no nations outperformed fourthgrade students in the FiW. The FiW fourthgrade science score was not significantlydifferent from that of one other country

(Korea). FiW students outperformed theircounterparts in 25 of 26 countries. Seeexhibits B-1 and B-2 for more detail onscores and distributions.

Although FiW students did well on thefourth grade math test, they had strongerperformance in some topic areas than inothers. FiW fo. urthgrade math studentsperformed among the best in the world in 8of 14 content areas." They were:

(1) integers and whole number operations;(2) common fractions;(3) rounding and estimating computations;(4) geometry: position and shapes;(5) symmetry, congruence and similarity;(6) proportionality;(7) patterns, relations, and functions; and(8) data and statistics.

Topics where students from other nationsscored higher were:

(1) meaning of whole numbers;(2) decimal fractions;(3) estimating quantity and size;(4) measurement units;(5) perimeter, area, and volume; and(6) equations and formulas.

For more detail on how the FiW performedrelative to other TIMSS countries, seeexhibit B-3.

EighthGrade Results

Eighthgrade FiW students also performedvery well on the TIMSS assessmentcompared to students from the 41 countriesthat participated in this part of the study. Asin the fourth grade, only students inSingapore outperformed eighthgrade FiWstudents in math and no nationsoutperformed FiW students in science.

In math, only students in Singapore scoredsignificantly above FiW students on theeighthgrade assessment. FiW had anaverage score that was not significantlydifferent from six other countries (Korea,Japan, Hong Kong, Belgium-Flemish, CzechRepublic, and Slovak Republic). FiWeighthgrade math students outperformedtheir counterparts in the remaining 34 of 41countries.

In science, no nation outperformed FiW onthe eighthgrade assessment. The FiWscore was not significantly different fromeight other high-performing countries(Singapore, Czech Republic, Japan, Korea,Bulgaria, Netherlands, Slovenia, andAustria). See exhibits B-4 and B-5 for moredetail.

As in the fourth grade, eighthgrade mathachievement varied by topic. FiW eighthgrade math students performed among thebest in the world in 9 of 20 content areas.'2They were:

(1) decimal fractions and percentages;(2) relationships of fractions;(3) estimations of quantity and size;(4) rounding;(5) estimating computations;(6) three-dimensional geometry and

transformations;(7) patterns, relations, and functions;

8

(8) data representation and analysis; and(9) statistics and probability.

Students from other nations scored higher onthe following topics:

(1) whole numbers;(2) common fractions;(3) measurement units;(4) perimeter, area, and volume;(5) measurement estimations and errors;(6) two-dimensional geometry basics;(7) polygons and circles;(8) congruence and similarity;(9) proportionality concepts;(10) proportionality problems; and(11) equations and formulas.

See exhibit B-6 for more detail.

TwelfthGrade Results

In the twelfth grade, two sets of assessmentswere administered to the countriesparticipating in TIMSS. The first setmeasured student achievement in generalmath and science knowledge. The secondwas designed to measure achievement of themost advanced students in their final year ofsecondary school. Accordingly, theadvanced math exam covered advancedmath topics, including geometry, numbersand equations, and calculus. The advancedexam in science focused on physics.

In the United States, the advanced mathassessment was given to students who hadtaken, or were taking, a full year of a highschool course that included calculus in thetitle, including calculus, pre-calculus, APCalculus, and calculus and analyticgeometry.

Using the U.S. definition for advanced math,approximately 14 percent of the U.S. school-

19

leaving age cohort was covered by theTIMSS sample of advanced math students.Internationally, 19 percent of the school-leaving age cohort was covered, under thevarious definitions used by differentcountries to identify their most advancedstudents.

By comparison, the FiW advanced mathsample covered approximately 65 percent ofthe FiW school-leaving age cohort under thesame definition used by the U.S., clearly amuch larger percentage of students than theU.S. or its international peers."

To take the physics assessment, U.S.students had to be enrolled in, or have takenat least one year-long class of physics (thisincludes physics and AP Physics). Underthis definition, approximately 15 percent ofthe U.S. school-leaving age cohort wascovered by the TIMSS sample of physicsstudents. Internationally, approximately 13percent of the school-leaving age cohortwere covered by this assessment using thedifferent definitions of eligibility developedacross countries.

As with mathematics, a much largerpercentage of FiW twelfthgrade studentswere exposed to physics than U.S. orinternational students. In the FiW,approximately 67 percent of the school-leaving age cohort were covered by thephysics sample."

The large difference between the percent ofthe school-leaving age cohort covered in theU.S. and FiW samples is explained primarilyby differences in course taking patterns,rather than differences in the number ofstudents in this age cohort who are still inschool or differences in which componentsof the system they may have excluded fromtheir sample.

9

Eighty-three percent of FiW students takemathematics and 74 percent take science intheir last year of schooling. In the U.S.,however, less than two-thirds of all seniorsenroll in a math class and less than one-halfof U.S. seniors take a science class.

Furthermore, nearly all FiW students take atleast one of the following classes: pre-calculus, calculus, AP Calculus, physics, orAP Physics:5 Results for each set of examsare discussed below.

General Knowledge

In general mathematics knowledge, FiWtwelfth grade students' performance was not'significantly different from students in theseven highest performing countries(Netherlands, Sweden, Denmark,Switzerland, Iceland, Norway, andAustralia).

FiW students outperformed theircounterparts in the remaining 14 countries.It should be noted that no Asian countriesparticipated in the end of secondary schoolassessments. See exhibit B-7 for more detailon the countries participating in thisassessment and the distribution of theirscores.

In general science knowledge, theachievement of FiW twelfthgraders wassimilar to their achievement in mathematics.FiW twelfthgrade students' performanceon general science knowledge was notsignificantly different from students in theseven highest performing countries(Sweden, Netherlands, Iceland, Norway,Canada, New Zealand, and Australia). FiWstudents outperformed their counterparts inthe remaining 14 countries. See exhibit B-8for more detail.

Advanced Math and Physics

The TIMSS advanced math and physicsexams were designed to be administered tothe highest performing students in the worldin math and science as they are about toleave secondary school. Each countrydefined the groups of students that theythought most appropriate to be included inthese assessments, based on the generalcontent of the tests as well as practicalconsiderations.

In order to have large enough samples ofstudents taking the advanced math andscience exams, the U.S. included pre-calculus and general physics students in theadvanced groups. Other countries limitedtheir testing to calculus and advancedphysics students.

The Consortium followed the U.S. samplingparameters, even though it had a largeproportion of students who would qualifyunder the more restrictive internationalcriteria, as discussed below. We have,therefore, presented two different views ofthe advanced math and physics test data.

In advanced mathematics, for example, wepresent the FiW Advanced PlacementCalculus student score on the advancedexamination to provide comparison withother countries. We also provide the scorefor all students who took the advanced testfor comparison to the U.S. national score.

As discussed in the rest of the section, theConsortium's AP Calculus and AP Physicsstudents were first in the world. However,when the pre-calculus and general physicsstudents are incorporated into the FiWscores, the FiW averages drop below theinternational average. The section considersthe general results first, then provides the

10

results for the AP Calculus and AP Physicsstudents.

On the advanced math assessment, FiWstudents scored near the internationalaverage. Advanced FiW twelfthgrade mathstudents were outperformed by students inseven countries (France, Russian Federation,Switzerland, Denmark, Cyprus, Lithuania,and Sweden). FiW scores were notsignificantly different from those of sixcountries. FiW placed significantly abovethree countries. See exhibit B-9 for moredetail.

FiW physics students were significantlybelow students in twelve nations (Norway,Sweden, Russian Federation, Denmark,Slovenia, Germany, Australia, Cyprus,Switzerland, Greece, Canada, and France).FiW performance did not differ significantlyfrom three nations, outperforming only theUnited States in the TIMSS physicsassessment. See exhibit B-10 for moredetail.

AP Calculus and AP Physics Results

Many have wondered whether it wasappropriate to be comparing two-thirds ofthe students in FiW against one-fifth orfewer students in the U.S. or internationallybecause such large percentages of FiWtwelfth grade students were included in theadvanced math and science samples. It hasbeen suggested that AP Calculus or APPhysics students might make a bettercomparison group for the advanced mathand science assessments because thesestudents are enrolled in the most advancedcourses and similar percentages of FiW andinternational students would be coveredunder such a comparison.

21

AP courses are offered in all FiW highschools and are generally considered to bethe most advanced classes offered inmathematics or physics, since studentsparticipate in national AP exams and mayreceive college credit if they score well onthese exams. FiW AP Calculus studentsrepresent 28 percent of the FiW school-leaving population. Under this comparison,then, 28 percent of the FiW students arebeing compared against 19 percent ofinternational students and 14 percent of theirU.S. counterparts.

In AP Physics, 7 percent of the FiW school-leaving cohort is compared against 13percent internationally and 15 percent ofU.S. students.I6

FiW twelfthgrade AP Calculus studentswere first in the world, as no nationsoutperformed FiW AP Calculus students onthe advanced math assessment. The FiWscore was significantly above that of sixteennations. See exhibit B-11 for more detail.

Performance of AP Physics students in FiWConsortium was also first in the world. Nonations scored significantly above FiW APPhysics students. Five nations (Norway,Sweden, Russian Federation, Slovenia, andGermany) had scores that were notsignificantly different from FiW AP Physicsstudents. FiW AP Physics studentsoutperformed their counterparts in 11countries. See exhibit B-12 for more detail.

Summary

When benchmarked against an internationalmeasure of math and science achievement,FiW students performed exceptionally wellin all grades tested.

FiW students excelled on the fourth, eighth,and twelfth grade general knowledge tests,and scored among, or just below, the highestperforming countries worldwide. Althoughall Consortium students tested on theadvanced math and physics tests did notperform as well as expected (they scoredaround the international average), theperformance of AP students was

11

exceedingly high, with their scores placingthem in first place internationally.

The outstanding performance of the FiWstudents, particularly given thedisappointing results of their U.S.counterparts, has generated a lot of interestin examining what factors might havecontributed to this world-class performance.

The next two sections look at the influencethat home factors and differences in thecontexts for teaching and learning mighthave on these gaps.

22

Do Home Factors Explain the High Achievement of the FiW Consortium?

As mentioned earlier, the districts that makeup the FiW Consortium are high wealthdistricts. The question that natuarlly arisesis: Could student and family backgroundcharacteristics explain the differences inachievement between the FiW Consortiumand the U.S.?

To examine this question, an exploratoryanalysis was conducted using TIMSSachievement and student and familycharacteristics data." The analysisidentified a set of student and familyvariables included in the TIMSS'questionnaires that were found to be highlycorrelated with student math achievement.These variables included parents' educationlevel, whether the student's parents wereborn in the U.S., language spoken at home,and number of books in the home.

A set of regression analyses were run toestimate the difference between the FiW andU.S. scores not attributable to home andfamily characteristics (in scale points) forthe fourth and eighth grades in math andscience, and the twelfth grade for generalknowledge of math and science.

These exploratory analyses showed that thepoint differences between the FiW and U.S.students' scores could not be fully attributedto students' home and family characteristics.

In fourthgrade math and eighthgrade mathand science, socio-economic factorsexplained approximately 20 to 25 percent ofthe difference in scale scores, but left 75 to80 percent unexplained.

13 23

Family and parental characteristics couldaccount for slightly more of the difference ineighthgrade science, and twelfthgrademath and science. In these cases, half of thedifference is attributable to family and homefactors.

There are, however, some immeasurableeffects of resource rich districts that may notbe well measured by this analysisa morestable teaching force, high levels ofinvolvement from parents, and highexpectations for students.

These factors may play as important a roleas that of high financial resource levels inpromoting high achievement. In fact, somebelieve that one of the explanations for thehigh achievement levels in FiW districts ishow they use their wealth to supportteaching and learning, not the wealth itself

While further analysis may give moreprecise estimates of the relationship betweenachievement and socio-economic status inthe FiW Consortium, this analysis andresults from other preliminary analyses'indicate that other factors, such ascurriculum, classroom instructionalpractices, and teacher engagement, play animportant role in their high achievementlevels.

The next section explores some of the otherfactors that may have contributed to theConsortium's success.

What is the Context for Teaching and Learning Math in the FiWConsortium?

This section offers some possibleexplanations that might account for thedifferences in math achievement betweenthe FiW and the U.S. that cannot beattributed to home factors.

Drawing on data from the TIMSS teachersurveys, an analysis of textbooks done byresearchers at Michigan State University,and anecdotal information based on visits toFiW schools, this section discussesdifferences in the contexts for teaching andlearning between the FiW Consortium andthe U.S.

Where international data are readilyavailable, compariions are also madebetween the contexts for teaching andlearning in the FiW and those in countrieswith high math achievement to point outsimilarities or differences where they exist.

In order to simplify the analysis, the sectionconcentrates only on the differences in thecontexts for learning math. An examinationof differences in the contexts for learningscience may show different results.

In addition, the section only looks at thecontexts for learning math in the fourth andeighth grades. These grades were selectedbecause of the rich data available fromTIMSS for analyzing math instruction atthese levels:9

The findings presented in this section shouldbe viewed as an exploration of somepossible reasons for the high achievement inthe FiW Consortium. This report is a firstattempt to identify factors that may play animportant role.

14

Further analysis may reveal other factorsthat might explain the differences inachievement between the FiW and the U.S.In addition, the paper does not investigatepossible causal links between the factorsdiscussed and the achievement gaps.

Finally, because of data limitations, thispaper points only to interesting differencesinpatterns between the contexts for learningin the FiW, U.S. and, when data areavailable, other high performing countries.20

Drawing on data from TIMSS, this sectionlooks at differences in four key areas that arecommonly thought to have a large impact onachievement levels:

Curriculum. In particular, differences ininstructional topic coverage and thecoverage of different topics by textbooksare explored.

Classroom instructional practices. Thereport looks at both student and teacherreports on the most frequently used classactivities and classroom organizationalmethods.

Teacher engagement. Four factors areexamined: the time spent by teachers onschool-related activities during personaltime, frequency of teacher meetings,teacher influence over key schooldecisions, and teacher familiarity withkey curricular and assessmentdocuments.

Teaching environment. Finally, teacherreports on the factors that limit theirteaching ability are compared.

24

Each of these areas is discussed in moredetail in the next section.

Curriculum

It is a common sense conclusion that what istaught in classrooms around the world hasan impact on what is learned. Accordingly,differences in the math curriculum coveredin the FiW and the U.S. may account forsome of the variations in achievementbetween the FiW and U.S. students.

In particular, differences in the organizationand sequence of topics, the level and depthof coverage, and the timing of topicintroduction may reflect importantdifferences in the opportunities for FiW andU.S. students to master math and sciencematerial.

The textbooks and other curricular materialsused may also play a potentially importantrole, if they are found to be more closelyaligned with the curriculum of highachievement nations. This section presentsdata on teacher reports on instructional topiccoverage and textbook content and use.

Instructional Topic Coverage

One of the first insights from the TIMSSdata has been the lack of rigor in the U.S.math curriculum. Many educationalresearchers and policymakers haveexpressed concern that U.S. students are notbeing taught the concepts and skills neededto achieve world class standards.

Accordingly, one possible explanation fordifferences in math achievement betweenthe FiW and the U.S. might be variations ininstructional topic coverage, i.e., whatcontent teachers cover in class.

15

Differences in both the number and type ofmath topics covered throughout the schoolyear may indicate that fourthand eighthgrade FiW students are exposed to moreadvanced material than their U.S.counterparts.

According to TIMSS data, FiW students aremore likely than their U.S. peers to beintroduced to relatively more advanced mathtopics in earlier grades. This pattern isreflected in data reported by math teachersin the fourth and eighth grade.

Exhibit.1 presents detailed data oninstructional topic coverage for 21 mathtopics in fourth-grade math.21 In a numberof key subject areas, all or nearly all FiWand U.S. students are introduced to thesesubjects by the end of the fourth grade,according to their math teachers.

Some of these students cover these topics intheir entirety in earlier grades (indicated onthe exhibit as "learned earlier"), others begintopics in earlier grades and receivereinforcement on the material during fourthgrade (indicated as "reinforcedinformation"), while still others are firstintroduced to the material in fourth grade(indicated as "new information").

In topic areas where these three categoriessum to 100, teachers report that all fourthgrade math students will have beenintroduced to these subjects prior to, orduring, the current year.

All, or nearly all, FiW and U.S. fourth grademath students are introduced to thefollowing five basic concepts prior to, orduring, fourth grade, according to teacherreports (that is, the percentage of students in"learned earlier", "reinforced information"

and "new information" sum to 100 or near100):

(1) whole numbers;(2) common and decimal fractions;(3) estimation and number sense;(4) measurement units and processes; and(5) data representation and statistics.

In all of these subjects, FiW students weremore likely to have been exposed to thetopic earlier than their U.S. counterparts.For example, 75 percent of FiW studentshad either learned whole numbers prior tofourth grade (e.g., covered it in grade threeor earlier and were no longer spending timeon this concept) or were receivingreinforcement.

In contrast, less than half of U.S. students(43 percent) had been exposed to thismaterial earlier, according to their teachers.

FiW students are also more likely than theirU.S. fourthgrade counterparts to beintroduced to two more difficult mathconcepts during or prior to fourth grade:

(1) percentages, and(2) number sets and concepts.

Seventy-seven percent of FiW students havebeen introduced to percentages prior to orduring fourth grade (calculated by addingthe three categories indicating when topicsare introduced: e.g. "learned earlier" +"reinforced information" + "newinformation").

This compares to only 47 percent of theirU.S. peers. Sixty-three percent of FiWstudents have been introduced to numbersets and concepts prior to or during fourthgrade, compared to only 32 percent of theirU.S. peers.

16

As illustrated in exhibit 2, according toteacher reports, all or nearly all eighth gradeFiW and U.S. students had been introducedto the following eleven math topics:22

(1)(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

whole numbers;common and decimal fractions;percentages;number sets and concepts;number theory;estimation and number sense;measurement units and processes;perimeter, area, and volume;basics of one- and two- dimensionalgeometry;

(10) ratio and proportion; and(11) equations, inequalities and formulas.

However, as in the fourth grade, FiWstudents are more likely than their U.S.counterparts to have been exposed to mostof these basic math concepts in precedinggrades. As a result, FiW students are morelikely to either not cover these topics againin the eighth grade or spend time reinforcingthese concepts.

For example, according to their teachers,eighthgrade FiW math students were muchmore likely than their U.S. counterparts tohave covered whole number operations andmeaning in earlier grades and moved beyondthis material. Forty percent of FiW studentscovered this material in earlier grades; whileonly 16 percent of U.S. students covered thismaterial earlier and were not covering it ineighth grade according to their teachers.

For the other ten subjects, FiW studentswere much more likely than U.S. students tohave been exposed to the material in priorgrades and to spend time in their eighthgrade math classes reinforcing this material.

26

Eighthgrade students in the FiW were alsomore likely than their U.S. counterparts tohave been introduced to new, moreadvanced material during the eighth grade.The more advanced material includes:

(1) geometric congruence and similarity;(2) geometric transformations and

symmetry;(3) constructions and three-dimensional

geometry;(4) proportionality: slope, trigonometry, and

interpolation;(5) functions, relations and patterns; and(6) sets and logic.

17

While the data on instructional coveragegive us some clues as to when FiW and U.S.fourthand eighthgraders are introduced todifferent topics, the data do not providemuch insight into the level or depth ofcoverage. Further research in this area couldpotentially give a fuller understanding of thedifferences in implemented curriculumbetween the FiW and the U.S.

27

00

Exh

ibit

1: F

ourt

h-G

rade

Mat

h T

each

ers'

Rep

orts

on

Inst

ruct

iona

l Top

ic C

over

age

FiW

Who

le N

umbe

rsu.

s.

Com

mon

& D

ecim

alFi

W

Frac

tions

u.s.

FiW

Perc

enta

ges

U.S

.

Num

ber

Sets

& F

iWC

once

pts

u.s.

FiW

Num

ber

The

ory

u.s.

Est

imat

ion

& F

iWN

umbe

r Se

nse

u.s.

Mea

sure

men

t Uni

ts &

FiW

Proc

esse

su.

s.

Est

imat

ion

& E

rror

of

FiW

Mea

sure

men

tu.

s.

Peri

met

er, A

rea,

& F

iWV

olum

e u.

s.

Bas

ics

of O

ne &

Tw

o Fi

WD

imen

sion

al G

eom

etry

u.s.

28C

ongr

uenc

e &

FiW

Sim

ilari

tyu.

s.

0

Perc

ent o

f Fo

urth

-Gra

de S

tude

nts

2040

6080

100

45,7

1111

-,%

;;;

Tra

nsfo

rmat

ions

& F

iWSy

mm

etry

u.s

.

0

Perc

ent o

f Fo

urth

-Gra

de S

tude

nts

2040

6080

Thr

ee D

imen

sion

al F

igur

esFi

Vi

& C

onst

ruct

ions

u.s.

g

FiW

Rat

io &

Pro

port

ion

us

Func

tions

, Rel

atio

ns, &

FaV

Patte

rns

u.s.

FiW

Equ

atio

ns &

for

mul

asU

.S.

Dat

a R

epre

sent

atio

n &

Fw

In

Stat

istic

su.

s

FiW

Prob

abili

tyu.

s.

Sets

& L

ogic

FiW

1

u.s.

b

FiW

Prob

lem

Sol

ving

Str

ateg

ies

FiW

Oth

er M

athe

mat

ics

Con

tent

u.s.

Lea

rned

Ear

lier

El R

einf

orce

d In

form

atio

nN

ew I

nfor

mat

ion

SOU

RC

E: N

CR

EL

Ana

lysi

s of

Tea

cher

Que

stio

nnai

re D

ata.

29

Exh

ibit

2: E

ight

h-G

rade

Mat

h T

each

ers'

Rep

orts

on

Inst

ruct

iona

l Top

ic C

over

age

Who

le N

umbe

rsF

iW

U.S

.

Com

mon

& D

ecim

alF

iW

Frac

tions

u.s.

Perc

enta

ges

rox

U.S

.

Num

ber

Sets

& C

once

pts

riw

U.S

.

Num

ber

The

ory

FiW

U.S

.

Est

imat

ion

& N

umbe

rF

iW

Sens

eu.

s.

Mea

sure

men

t Uni

ts &

FN

Proc

esse

su.

s.

Est

imat

ion

& E

rror

of

EN

Mea

sure

men

tsu.

s.

Peri

met

er, A

rea,

& F

wV

olum

eu.

s.

Bas

ics

of O

ne &

Tw

oF

N

Dim

ensi

onal

Geo

met

ryu.

s.

Geo

met

ric

Con

grue

nce

&F

iW

Sim

ilari

tyu.

s.

30

0

Perc

ent o

f E

ight

h-G

rade

Stu

dent

s

2040

6080

100

EM

IL

j

1

1

1

1

Geo

met

ric

Tra

nsfo

rmat

ions

FiW

0

Perc

ent o

f E

ight

h-G

rade

Stu

dent

s

2040

6080

& S

ymm

etry

U.S

. Mr7

1

Con

stru

ctio

ns &

Thr

eeD

imen

sion

al G

eom

etry

Rat

io &

Pro

port

ion

FiW

U.S

.W

.;

FiW

U.S

.

Prop

ortio

nalit

y: S

lope

,F

iW

Tri

gono

met

ry &

Int

erpo

latio

nu.

s.

Func

tions

, Rel

atio

ns,

EN

& P

atte

rns

U.S

.

Equ

atio

ns, I

nequ

aliti

es, &

FiW

Form

ulas

u.s.

Stat

istic

s &

Dat

aF

iW

Prob

abili

ty &

Unc

erta

inty

U.S

.

FiW

U.S

.

Sets

& L

ogic

FiW

U.S

.

Prob

lem

Sol

ving

Str

ateg

ies

FiW

U.S

.

Oth

er M

athe

mat

ics

Con

tent

FiW us

.

I

IN L

earn

ed E

arlie

rE

a R

einf

orce

d In

form

atio

nN

ew I

nfor

mat

ion

SOU

RC

E: N

CR

EL

Ana

lysi

s of

Tea

cher

Que

stio

nnai

re D

ata.

3-0

Textbook Use and Topic Coverage

Could the difference in achievement beinfluenced by variations in the waytextbooks are used in the classroom or thetextbooks chosen? This section looks atTIMSS questionnaire data and a specialstudy commissioned by FiW to examine thedegree to which teachers rely on theirtextbooks to structure their teaching time, aswell as the types of topics covered in thesetextbooks and the relative emphasis given toeach topic.

TIMSS teacher questionnaires indicate thatnearly all students use a textbook in FiWand U.S. eighthgrade math classes (100percent of FiW students and 97 percent ofU.S. students). The degree to which eighthgrade math teachers use these books tostructure their teaching time varies

considerably, however, according to teacherreports. FiW students are more likely thanU.S. students to be in classes where a largepercentage of the teaching time is based onmaterial in the text. As seen in exhibit 3, 55percent of eighthgrade students are inclasses where more than three-fourths of theteaching time is based on material in thetextbook. In contrast, only 36 percent ofU.S. students are in classes that rely thisheavily on material in the textbook.

FiW teachers also report using aconsiderable amount of supplementarymaterial in addition to their main textbook.As shown in exhibit 3, 91 percent of FiWand all U.S. students have teachers who useother materials in the place of or in additionto their main textbook. In both cases,textbooks appear to be used as one resourceout of many rather than the sole resource.

Exhibit 3: Teachers' Reports on Use of Textbooks and Other TeachingMaterials and Percent of Mathematics Teaching Time Based on Textbook

Textbook UsePercent of Eighth

Grade StudentsFiW U.S.

Use a textbook at all 100 97Use supplementary materials 91 100

Percent of teaching time based on textbook1-25 percent 6 16

26-50 percent 4 17

51-75 percent 35 31

76-100 percent 55 36SOURCE: NCREL analysis of TIMSS Data; FiW Teacher Questionnaire results.

3220

Data from the TIMSS curriculum study doneby researchers at Michigan State Universitycatalogued the topics covered by differenttextbooks and the relative emphasis given toeach topic. Their analysis of math textbooksused by the FiW, U.S. and TIMSS23countries report the following conclusions:

"The number of topics in Consortiumtextbooks is similar to that in textbooksof the US composite at all populations.[grade levels]. "24

Fourth Grade

At the fourthgrade level, "theconsortium's textbooks are essentiallyno different in their content profiles fromthe US as a whole and [they also have] agreat deal of overlap in content withJapan."

The major exception is "the prominentpresence of decimals in the Japanesetextbooks, which is not the case in U.S.or FiW textbooks."25

Eighth Grade

"At the eighthgrade level, there arenotable differences in the topicsemphasized in the Consortium...with thetopic of 'Equations and Formulas' forexample, more emphasized in theConsortium than is the case for the mostcommonly used textbooks in the

"The emphasis of Consortium textbookson 'Equations and Formulas' in Grade 8is similar to that of TIMSS countries inwhich mean student achievement wassignificantly higher than mean studentachievement in the U.S."'

21 33

"This greater focus and emphasis onalgebra is further reflected by the factthat the Consortium's books for the non-algebra tracks do not have any of thestandard arithmetic topics among the top5 contained in the book. This is inmarked contrast to U.S. non-algebrabooks."

"The Consortium teachers clearlyemphasize algebra (linear equations) andgeometry (2D geometry basics) morethan is common for their U.S. peers, anemphasis that appears to be aided by thefact that the textbooks themselvesprovide more material in these areas."28

In sum, some differences in curriculum maycontribute to the differences in achievementbetween the FiW and the U.S. At both gradelevels, these differences do not seem to bedriven by differences in the number of topicsfound in textbooks used by FiW and U.S.students, as the number of topics covered byFiW math textbooks is very similar to thenumber covered by other U.S. mathtextbooks.

However, FiW eighthgrade textbooks tendto emphasize algebra and geometry moreheavily, while U.S. textbooks emphasizearithmetic. This is true even for the studentsin the non-algebra tracks. In addition, at theeighthgrade level, a higher percentage ofFiW students than U.S. students are likely tobe in classrooms where most of the teachingtime is centered on material in the book.

However, there is some anecdotalinformation that indicates FiW teachers aremore apt to customize the textbooks used,e.g. carefully selecting chapters andexercises to be completed. This might be incontrast to the typical use of textbooks and

could be an important component inunderstanding how FiW teachers can usetextbooks with a large number of topicseffectively.

Summary

Differences in achievement do not appear tobe driven by the number of topics covered inthe classroom or in the textbooks.According to teacher reports, FiW studentsseem to cover just as many topics as theirU.S. peers. Likewise, FiW textbooks coverthe same number of topics as U.S.textbooks.

However, FiW eighth graders are morelikely than their U.S. peers to use textbooksthat emphasize algebra and geometry. Inaddition, FiW students may be introduced tomore advanced topics earlier than their U.S.peers.

Additional analysis is necessary todetermine whether more advanced conceptsare actually covered or just presented in asimplistic way.

Nevertheless, according to teacher reports,FiW students receive more reinforcement oftopics introduced in earlier grades and covermore new material.

This pattern is found in both the fourthandeighthgrades, and as might be expected, ismore pronounced in the eighth grade.

The next section explores whether there arenotable differences between the instructionalstrategies used by FiW and U.S. teachersthat may also contribute to the differences inachievement.

22

Classroom Instructional Practices

There is a rich research base regarding whathappens in most classrooms in the U.S.While much of the literature documentsestablished patterns of teacher-studentinteractions, some studies have examinedthe relationship between classroominstructional practices and studentachievement. The indicators selected foranalysis in this section are based on thatliterature and some of the unique practices inFiW districts.

As discussed in the following three sections,data from the TIMSS teacher and studentsurveys suggest that some of the explanationfor the differences in achievement may bedue to differences between the methods thatFiW and U.S. teachers use to presentmaterial to their students.

In particular, data on teachers' reports onclass activities and classroom organization,as well as students' reports on the same,indicate that notable differences may existbetween the FiW and the U.S. in the contextfor learning in student classrooms.

Teachers' Reports on Class Activities

Differences in classroom activities, that is,the actual tasks that teachers require theirstudents to do in class, may account forsome of the variations in achievement.

For example, some teachers may ask theirstudents to tackle more challenging tasks orcomplex math problems that requirestudents to apply their skills to differentproblems. Other teachers may challengestudents to explain the reasoning behind newconcepts more frequently.

34

Some students may spend more time usingcomputers to apply new math concepts tosolve exercises or problems. Other studentsmay spend their time completing lesschallenging tasks, like completing drills orpracticing basic computational skills.

This section explores whether differencesexist between the types of activities usedfrequently in the FiW and U.S. math classes.

Data on the types of activities that teachersreport they ask their students to do in classindicate that important differences may existbetween the classroom activities in FiW andU.S. classes. However, the data also showmany similarities. Both the differences andsimilarities are discussed below.

According to teachers, FiW math students inboth the fourth and eighth grades are askedto perform reasoning tasks more frequentlyand complete drills less frequently than U.S.students.

As shown in exhibit 4, FiW students aremore likely than U.S. students to have mathteachers who ask them to explain thereasoning behind an idea during "everylesson." Thirty percent of FiW fourthgrade students have math teachers who askthem to explain their reasoning during"every lesson," 21 percent of U.S. fourthgraders fall into this category.

At the eighthgrade level, the difference ismore pronounced: 48 percent of FiWstudents have teachers who ask theirstudents to explain the reasoning behind anidea during "every lesson," while roughlyhalf as many (23 percent) of U.S. studentshave teachers that do.

FiW students are also more likely than U.S.students to be asked by their teachers to

write equations during "most lessons" or"every lesson." In eighth grade, threefourths of FiW students have math teacherswho ask their students to write equationsduring "most lessons" or "every lesson."

The percentage of U.S. students being askedto express relationships in equations in"most lessons" or "every class" is farloweronly 38 percent of eighthgraders.

While the difference between FiW and U.S.eighthgrade students may be a function ofthe higher percent of FiW students takingalgebra classes, the difference is notable.

In the fourth grade, FiW and U.S. studentsare asked to use equations to expressrelationships less frequently, however FiWstudents are more likely than their U.S.counterparts to be asked to write equations(94 percent of FiW students, 83 percent ofU.S. students).

Most fourthand eighthgrade students, inboth the FiW and the U.S., are only asked toundertake more complex or challengingactivitiessuch as representing andanalyzing relationships using tables, charts,or graphs or working on problems for whichthere is no immediate solutionduring"some lessons."

Roughly 80 percent of FiW and U.S. fourthgrade math students and 70 percent of FiWand U.S. eighth grade students spend timeduring "some lessons" representing andanalyzing relationships using tables, charts,or graphs.

In both the fourth and eighth grades,teachers reported roughly 65 percent of FiWand U.S. students are asked by their teachersto work on problems for which there is noimmediate solution during "some lessons."

23 35

According to teacher reports, FiW studentsalso practice computational skills in classless frequently than their U.S. counterparts.Teachers report that 58 percent of FiWfourth graders practice their computationskills during "most lessons" or "everylesson" compared to 70 percent of their U.S.counterparts.

In the eighth grade, students in both FiWschools and schools across the U.S. are lesslikely to practice computational skillsfrequently. However, the differencebetween the FiW and the U.S. eighth gradersis even more striking: 25 percent of FiWmath students practice computational skillsduring "most lessons" or "every lesson",compared to 59 percent of U.S. students.

The differences between how often FiW andU.S. students are asked to practicecomputational skills could be a function ofhigher expectations of mastery in earliergrades without the need for teaching and re-teaching the same topics year after year.

Classroom computer usage is another areawhere notable differences exist betweenFiW and the U.S. Neither FiW nor U.S.students use computers routinely to solveexercises or problems. However, in both thefourthand eighthgrades, FiW teachersreport that over half of the students usecomputers during "some lessons," whileonly a little over one-third of U.S. fourthgraders, and less than a quarter (21 percent)of U.S. eighthgraders, use computers.

While these data point out interestingcontrasts, the differences in computer usemay be a function of the availability oftechnology. A further look needs to betaken at how computers are actually used inFiW classrooms, as compared to the U.S.

24

Finally, one additional key point that can bemade after reviewing exhibit 4 is the pattern,or lack of pattern, that emerges in FiW andU.S. classrooms.

It is interesting to note that the mostpredominant activities in FiW fourth gradeclassroomsexplaining the reasoningbehind an idea, practicing computationalskills, and writing equationsare consistentwith the U.S. patterns.

At the eighthgrade level, however, there isa striking difference between FiW and U.S.reports in the areas of writing equations andpracticing computational skills.

Taken together, these data suggest that FiWand U.S. teachers ask their students to dosimilar things in their math classes.However, there are a number of noteworthydifferences.

In general, FiW students are challenged toperform reasoning tasks more frequently andcomplete drills less often than their U.S.peers. As noted above, the differences areparticularly evident at the eighthgradelevel, where FiW students are morefrequently required to write equations, ratherthan practice their computational skills.

One could reasonably conclude that studentsin FiW eighthgrade classrooms areexperiencing very different content andinstruction from students in U.S. classrooms.

36

Exhibit 4: Teachers' Reports on How Frequently StudentsAre Asked to Complete Specific Tasks

Response

Percent ofFourth

_Grade

Students

Percent ofEighthGrade

StudentsFiW U.S. FiW U.S.

Explain the reasoning behind an idea

Every lesson 30 21 48 23

Most lessons 56 50 28 44

Some lessons 14 28 24 32

Never or almost never 0 1 0 1

Represent and analyze relationships usingtables, charts, or graphs





Work on problems for which there is noimmediate...solution





Use computers to solve exercises orproblems





Write equations to represent relationships





Practice computational skills





SOURCE: NCREL analysis of lEA's Third International Mathematics and Science Study (TIMSS) data 1994-95;FiW Teacher Questionnaire results, NCREL; tables 5.10 and 5.17 in Mullis, I.V.S., et al. (1997). MathematicsAchievement in the Primary School. Years: lEA's Third International Mathematics and Science Study. Chestnut Hill,MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston College; tables 5.10 and 5.17 inBeaton, A.E., et al. (1996). Mathematics Achievement in the Middle School Years: lEA's Third InternationalMathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and EducationalPolicy, Boston College.

NOTE: Totals may not add to 100 due to rounding.

25 Q I

Teachers' Reports on ClassroomOrganization

The different methods that math teachersemploy to organize their classrooms and thedifferent pedagogical approaches they usemay have an impact on the achievement oftheir students.

In particular, the amount of time teachersspend standing up in front of their classintroducing new material, explaining newconcepts, and answering student questions,as opposed to having students work throughexercises on their own, may play animportant role in explaining the differencesin achievement between FiW and U.S.students.

Similarly, whether students work together asa class or break off into small groups mayaffect student achievement. Furthermore,the choice of organizational or pedagogicalapproach that may have the strongest impacton the achievement of younger students maybe different than the most effective choicefor older students.

Accordingly, this section examines TIMSSteacher data on classroom organization forany differences in patterns between the FiWand the U.S. for both fourthand eighthgraders.

Teachers' reports on classroom organizationshow some differences between the FiW andthe U.S., and some similarities between FiWand countries with high math achievement.

As shown in exhibit 5, FiW fourth grademath teachers reported that FiW students areless likely than their counterparts in both theU.S. and high performing countries to betaught by teachers that rely either heavily or

26

predominantly on any one classroomorganizational approach.

In fact, the method that is the most heavilyrelied upon for teaching FiW fourthgrademath studentsstudents working together asa class with their math teacher leading thewhole classis used during "most or everylesson" for less than half of fourthgradeFiW students.

FiW fourth graders are considerably lesslikely than both U.S. students and theircounterparts in Japan, Korea, and Singaporeto spend "most or every lesson" workingindividually with assistance from their mathteacher, according to teacher reports.

This is the one area where there are notabledifferences between FiW and the U.S.:twentyfour percent of FiW students areasked to take this approach, compared to 55percent of U.S. students, according to theirteachers.

In the eighth grade, classroomorganizational patterns are different fromthose in the fourth grade. FiW teachersreported that three quarters of their studentswork together as a class with the mathteacher teaching the whole class during"most or every lesson."

The strong emphasis on whole classinstruction was also reported in othercountries with high achievement. Accordingto teacher reports, over 60 percent of eighthgrade math students in Singapore, Korea,and Japan spend "most or every lesson"working together with the math teacherteaching the whole class.

By contrast, less than half of eighthgrademath students in the U.S. spend "most orevery lesson" receiving instruction in largegroups. U.S. teachers also report that half of

38

eighthgraders spend "most or every lesson"working individually with assistance fromteachers.

This is a much higher percentage than in theFiW Consortium, where only 35 percent arein classes where this technique is used asfrequently.

To summarize, TIMSS teacher data suggestthe existence of important differencesbetween FiW and U.S. and similaritiesbetween the FiW and high mathachievement countries in the context forteaching math.

According to these data, fourthgrade FiWstudents are more likely than both their U.S.

and international peers to be taught bymultiple methods of instruction.

FiW eighthgraders, however, spend moretime receiving whole class instructioninthis case, the pattern is similar to theirinternational peers but differs considerablyfrom that found in the U.S.

Again, one might conclude that the FiWstudents are being exposed to more newcontent in the eighth grade than U.S.students and that higher expectations areimposed on them to learn more complexmathematics.

Exhibit 5: Teachers' Reports on Classroom Organization During MathematicsLessons

Country

Percent of Students Whose Teachers Report Using Each OrganizationalApproach "Most or Every Lesson"

WorkTogether as a

Class withStudents

Respondingto One

Another

Workas

Class withTeacher

Teaching theWhole Class

WorkIndividually

withAssistance

from Teacher

WorkIndividually

withoutAssistance

from Teacher

Work inPairs orSmall

Groupswith

Assistancefrom Teacher

Work inPairs orSmall

Groupswithout

Assistancefrom Teacher

Fourth Grade *FiW 37 48 24 26 36 23

United States 32 54 55 15 20 11

Korea 50 77 57 37 30 20

Japan 50 78 34 25 7 2

Singapore 23 68 37 41 25 10

Eighth Grade *FiW 42 75 35 22 20 16

United States 22 49 50 19 26 12

Korea 39 89 41 30 12 11

Japan 22 78 27 15 7 1

Singapore 15 61 48 27 20 6

SOURCE: FiW Teacher Questionnaire results, NCREL; figure 5.5 in Mullis, I.V. ., et al. (1997). MathematicsAchievement in the Primary School Years: lEA's Third International Mathematics and Science Study. Chestnut Hill,MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston College; figure 5.3 in Beaton, A.E.,et al. (1996). Mathematics Achievement in the Middle School Years: lEA's Third International Mathematics andScience Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and Educational Policy, BostonCollege.

* Fourth/eighth grade in most countries. 27

39

Students' Reports on ClassroomOrganization and Class Activities

TIMSS also collected data from students onwhat happens in the classroom. As in theteacher questionnaires, the studentquestionnaires asked students to report onhow frequently different classroomorganization methods were used and howoften varying class activities wereundertaken

This section looks at FiW and U.S. studentreports on what happens in their mathclassrooms in order to get a fullerunderstanding of the varying classroomorganizational and activity patternscommonly used, as well as some insight intothe differences between teachers' andstudents' reports.

FiW and U.S. students' reports on classroomorganization and class activities provide aninteresting picture of the similaritiesbetween what goes on in FiW and U.S. mathclasses. Exhibit 6 presents data on students'reports on the frequency of math classactivities. As shown in the exhibit, fourthand eighth grade students in FiW and U.S.schools report similar patterns for the fourmost frequently emphasized activities intheir math classes.

These activities are:

(1) teacher demonstrations of how to domath problems;

(2) distribution of homework;(3) teacher checking of homework (fourth

grade)/class discussions of completedhomework (eighth grade); and

(4) students working from worksheets ortextbooks on their own.

28

The majority of students in both the FiWand the U.S. report that they perform theseactivities during "most lessons."

Although important similarities emerge,students also reported differences betweenthe FiW and U.S., and between the fourthand eighth grades.

The relative importance placed on twocommon activities, copying notes from theboard and taking quizzes or tests, illustratessome of these differences. In the fourthgrade, 32 percent of U.S. students reportedthat they copy notes from the board during"most lessons" compared to 22 percent ofFiW students.

In the eighth grade, this technique is reliedupon more frequently than in fourthgradeclassrooms, in both FiW and U.S. mathclasses. Interestingly, unlike in the fourthgrade, eighthgrade FiW math students aremore likely than their U.S. counterparts toreport that they copy notes from the boardduring "most lessons" (47 percent of FiWstudents versus 42 percent of U.S. students).

As for taking quizzes or tests, differencesalso exist between the FiW and the U.S. inboth the fourth and eighth grades. In thefourth grade, U.S. students are more likelythan FiW students to report that they aretested during "most lessons" (48 percent ofU.S. students compared to 32 percent ofFiW students).

In the eighth grade, however, FiW studentsreported that they are more likely to betested during "most lessons" than their U.S.counterparts (46 percent of FiW studentscompared to 39 percent of U.S. students).

FiW fourthgrade math students reportedthat they used calculators more frequentlythan their U.S. counterparts, however, most

40

FiW students (75 percent) reported thatcalculators were only used during "somelessons." For the U.S., 46 percent ofstudents reported that calculators are usedduring "some lessons," while almost fourout of ten U.S. students (39 percent)reported that they are never used.

Calculator usage was reported morefrequently in the eighth grade than in thefourth grade by both FiW and U.S. students,although the gap between FiW and the U.S.remains. Sixty-nine percent of eighthgradeFiW students reported that calculators wereused during "most lessons," while only 38percent of U.S. students reported thatcalculators were used "most lessons." Thiscould either be a resource issue, (i.e.availability of calculators), or a function ofthe content and topic coverage in mathclassrooms.

Other differences between the FiW and theU.S. exist in how homework is assigned andused. In both the fourth and eighth grades,FiW students are more likely than U.S.students to report that they have homeworkassigned regularly.

In the fourth grade, approximately three-fourths of FiW students, versus two-thirds ofU.S. students, report that their math teachergives them homework during "mostlessons." They are also more likely thantheir U.S. counterparts to be able to starttheir homework in class during "mostlessons" or "some lessons" (83 percent ofFiW fourthgraders versus 69 percent ofU.S. fourthgraders) and to discuss theircompleted homework in class during "mostlessons" or "some lessons" (87 percent ofFiW fourthgraders versus 72 percent ofU.S. fourthgraders).

In the eighth grade, differences persist inhow homework is handled in class. Exhibit6 also shows a higher percentage of FiWeighthgrade math students reported thathomework is assigned during "most lessons"(86 percent of FiW eighthgrade studentsversus 72 percent of U.S. eighth graders).

Classroom time spent on homework-relatedactivities follows a slightly different patternin eighth grade than in fourth grade. In theeighth grade, FiW students are much morelikely than U.S. students to report that theydiscuss their completed homework in classduring "most lessons" (72 percent of FiWeighthgraders versus 54 percent of U.S.eighthgraders).

In both FiW and U.S. math classes, eighthgrade math students reported that they areless likely to spend time in class working onhomework than in the fourth grade. But,unlike in the fourth grade, FiW eighthgradestudents are less likely than their U.S.counterparts to report that they can start theirhomework in class during "most lessons"(32 percent of FiW eighthgraders versus 50percent of U.S. eighthgraders).

As discussed earlier in this section, itappears as if most FiW fourthand eighthgrade students have the same types ofexperiences in their math classes as theirfellow U.S. students.

According to student accounts, the mostfrequently emphasized activities in the FiWand the U.S., in both fourth and eighthgrades, are:

(1) teacher demonstrations;(2) homework distribution, checking or

discussion of homework; and(3) independent work on worksheets or on

material in the textbook.

294 1

The three areas in eighth grade wherestudents report differences between the U.S.and the FiW are:

(1) calculator use;(2) beginning hoMework in class; and(3) discussing completed homework.

These reports support the findings reportedin the two preceding sections, and suggestthat important differences between whatFiW and U.S. students do in class may exist.

Exhibit 6: Students' Reports on the Frequency of Math Class Activities

Activity Frequency

Percent ofFourth-Grade ,

Students

Percent ofEighth-Grade


The teacher shows us how to do math problemsMost Lessons 72 73 83 78

Some Lessons 27 25 16 21

Never 1 2 2 1

We copy notes from the boardMost Lessons 22 32 47 42Some Lessons 61 48 43 49Never 17 20 10 .9

We have a quiz or testMost Lessons - 32 48 46 39Some Lessons 65 47 53 61

Never 2 5 1 1

We work from worksheets or textbooks on our ownMost Lessons 56 55 69 59Some Lessons 40 35 29 38Never 4 10. 2 3

We use calculatorsMost Lessons 16 15 69 38Some Lessons 75 46 28 50Never . 9 39 2 11

We use computersMost Lessons 6 16 3 4

Some Lessons 35 29 34 28Never 58 55 63 67

The teacher gives us homeworkMost Lessons 74 66 86 72

Some Lessons 26 30 14 27Never 1 4 1 2

We can begin our homework in classMost Lessons 40 36 32 50Some Lessons 43 33 64 44Never 16 31 4 7

The teacher checks our homeworkMost Lessons 64 67 58 56


Never 6 8 7 7

We discuss our completed homeworkMost Lessons 40 35 72 54


Never 13 28 4 9

SOURCE: NCREL analysis of TIMSS data; FiW Student Questionnaire results.NOTE: Totals may not add to 100 due to rounding.

*The fourth grade and eighth grade student questionnaires contained a set of slightly different possible responses tothis question. To make comparisons across grades, some categories from the grade 8 survey were combined. Thesenew categories, along with the remaining responses were then matched to similar items on the grade 4 survey.Accordingly, "Some Lessons" is the sum of responses to "Pretty Often" and "Once in a While" on the grade 8 survey.Also, "Most Lessons" is "Almost Always" on the grade 8 survey.

30 42

Summary

In summary, data on instructional practicesindicate that there are differences betweenFiW and U.S. fourth and eighthgrademath classes. According to students, themost common activities in both the FiW andthe U.S. show similarities, with studentsreporting that the four most frequently usedactivities were:

(1) teacher demonstrations of how to domath problems;

(2) teacher assigning of homework;(3) teacher checking of homework (grade

4)/class discussions of homework (grade8); and

(4) students working from worksheets ortextbooks on their own.

Teacher and student reports indicate thatFiW and U.S. math teachers rely on differentmethods when demonstrating how to domath problems.

In the fourth grade, FiW teachers rely on avariety of approaches for teaching, so thatno one method dominates. Perhapsreflecting the different way in whichyounger students learn and process material,some instructional time is spent in largegroups, some in small groups, and someindividually.

In the eighth grade, however, FiW teachersreport that the most frequently used methodinvolves the math teacher instructing thewhole class. This approach is used far morefrequently than in U.S.eighth grade mathclasses.

In addition, in the FiW, both fourthandeighthgrade students are more likely thanU.S. students to be asked to explain the

reasoning behind an idea, or write anequation to represent a relationship.

According to teacher reports, FiW studentsreceive instruction in large and small groupsmore frequently than their U.S. counterparts,with the form of instruction varyingaccording to the grade level. In all cases,FiW math teachers are more likely than U.S.math teachers to challenge their students todemonstrate their mastery of more difficultideas or concepts.

The data also indicate that differences existin how homework is assigned and used.FiW students are more likely than U.S.students to have homework assigned everyday and to discuss their completedhomework in class.

This pattern of homework assignment anduse may mean that FiW students spend moretime outside of class reinforcing newconcepts. Classroom discussions ofhomework may clarify common difficultiesand serve to solve outstanding problems.

In the fourthgrade, FiW math students arealso more likely to spend class time startingtheir homework in class. Until the contentof the homework is analyzed, however, it isdifficult to determine whether this timeenhances instruction by allowing teachers towork more directly with their students on aregular basis.

31 43

Teacher Engagement

Teacher engagement and involvement in theinstructional process as well as in generalschool activities may also have an impact onstudent achievement. Although it can bedifficult to measure teacher engagementdirectly, a number of factors can give usinsight into the overall level of engagementand commitment.

For example, the amount of time and effortthat math teachers put into preparing andplanning for their classes, both during theregular school day and outside of regularclassroom hours, may provide a goodindication of teacher engagement.

Similarly, the amount of influence thatteachers have over basic school budgetallocation and curriculum decisions provideanother useful measure.

Teacher familiarity with key curriculum andassessment documents may offer anindication of the level of knowledge ofsignificant reform efforts.

Data from the TIMSS teacher surveyssuggest that teachers in the FiW may bemore engaged in school activities than U.S.teachers. Accordingly, this section presentsdata on four measures of teacherengagement:

teacher involvement in school-relatedactivities outside the school day;

frequency of teacher meetings;

teacher influence over key schooldecisions; and

teacher familiarity with key curriculumand assessment documents.

32

Together, thee measures provide someinsight into the relative levels of teacherengagement in the FiW and the U.S.

Teacher Involvement in School-relatedActivities Outside the School Day

Many teachers spend time outside the schoolday involved in school-related activities.The types of activities that they undertakeare varied and range from preparing for classactivities (e.g., planning lessons and gradinghomework) to helping, teaching, or workingwith individual students (e.g., tutoringstudents, meeting with parents, or consultingwith other educational personnel on theprogress of a particular student).

Teachers may also spend time outside theschool day attending to administrative orother record-keeping tasks (e.g., attendingstaff meetings, updating class grade books).Teacher involvement in any of theseactivities are used as a proxy for the level ofteacher engagement.

FiW and U.S. teachers spend similaramounts of their own time outside theformal school day working on nearly alltypes of school-related activities, with FiWfourth and eighthgrade math teachersmore likely than U.S. teachers to spend theirown time on a few key activities.

Exhibit 7 contains data on teachers' reportsof hours spent per week on activities outsidethe formal school day. Larger percentagesof FiW students than U.S. studentsat boththe fourthand eighthgrade levelshaveteachers who report spending more thanthree hours per week planning lessons bythemselves.

In the fourth grade, 67 percent of FiWstudents have teachers who spend more than

44

three hours a week preparing lessons outsidethe formal school day, compared to 46percent in the U.S., according to teacher'sreports.

In the eighth grade, 48 percent of FiWstudents have teachers who spend over threehours per week preparing for classes outsideof the formal school day, as compared to 34percent of U.S. students.

In the eighth grade, larger percentages ofFiW students than U.S. students haveteachers who devote more than three hoursper week to preparing or grading studenttests or exams outside the classroom.

Seventy-two percent of FiW students haveteachers who spend over three hours a weekof their own time preparing or gradingstudent tests or exams, compared to 47percent of U.S. students. This difference isnot found in the fourth grade.

Differences also exist between FiW and U.S.eighthgrade students with regard to howmuch time their teachers spend meeting withstudents outside the classroom. On average,eighthgrade FiW teachers spend more timemeeting with their students on their owntime than U.S. teachers (3.2 hours/week inFiW versus 2.0 hours/week in U.S.).Teachers of 72 percent of FiW eighthgradestudents report that they spend over threehours a week meeting with their students, ascompared to teachers of 29 percent of U.S.students.

To summarize, TIMSS teacher questionnairedata indicate that FiW and U.S. fourthandeighthgrade math students have teacherswho spend similar amounts of time outsidethe classroom on many school-relatedactivities.

33

Differences exist in the amount of time spenton a number of activities, however.

According to the data, FiW fourthandeighthgraders are more likely than U.S.students to have teachers who spend theirspare time preparing lessons.

In addition, eighthgrade FiW students aremore likely to have teachers who spendmore of their spare time preparing orgrading tests and meeting with studentsoutside of their class, perhaps suggestinghigher levels of teacher engagement amongFiW teachers than among U.S. teachers.

45

Exhibit 7: Teachers' Reports on Hours Spent Per Week onActivities Outside the Formal School Day

Activity Hours SpentPer Week

Percent ofFourth Grade

Students

Percent ofEighth Grade


Preparing or grading student tests or examsMore than 3 hours 26 25 72 47Average (in hours) 2.0 2.2 3.4 2.7

Reading and grading other student workMore than 3 hours 76 65 48 46Average (in hours) 3.6 3.1 2.6 2.7

Planning lessons by yourselfMore than 3 hours 67 46 48 34Average (in hours) 3.2 2.5 2.8 2.4

Meeting with students outside of classroomtime

More than 3 hours 6 6 72 29Average (in hours) 1.1 0.9 3.2 2.0

Meeting with parents More than 3 hours 0 0 0

Average (in hours) 0.6 0.7 0.7 0.7Professional reading and developmentactivity


Keeping students' records up to dateMore than 3 hours 19 16 34 16

Average (in hours) 1.9 1.4 2.1 1.6

Administrative tasks including staffmeetings


SOURCE: NCREL analysis of LEA's Third International Mathematics and Science Study (TIMSS) data, 1994-95; FiW Teacher Questionnaire results, NCREL; table 5.5 in Mullis, I.V.S., et al. (1997). MathematicsAchievement in the Primary School Years: lEA's Third International Mathematics and Sciences Study. ChestnutHill, MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston College; table 5.6 inBeaton, A. E., et al. (1996). Mathematics Achievement in the Middle School Year: lEA's Third InternationalMathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, andEducational Policy, Boston College.

4634

Frequency of Teacher Meetings

Another indicator of teacher engagement ishow actively teachers seek feedback onways to improve instruction. Teachermeetings to plan and discuss curriculum andinstructional approaches allow teachers toget feedback from their colleagues on thebest methods to present different types oftopics, or the relative ease or difficulty thatother classes are having covering similarmaterial.

Teacher meetings not only allow teachers tolearn about more effective strategies forteaching their respective subjects, but alsopermit teachers to keep abreast of changes inmajor national, state, and local curricularstandards and assessments. Further, theymay provide important opportunities forteachers to share resources and ideas forinstructing and motivating their students.

As shown in exhibit 8, FiW students aremore likely than U.S. students to haveteachers who report meeting frequently withother teachers in their subject area to discussand plan curriculum or teaching approaches.

In the fourth grade, 26 percent of FiWstudents have teachers who meet with theircolleagues daily to discuss curriculum. Incontrast, 10 percent of U.S. students haveteachers who meet with their colleagues thisfrequently.

This gap between FiW and U.S. is evenmore pronounced for weekly teacherencounters. Eighty-one percent of FiWfourthgrade students have teachers whomeet with other teachers at least once aweek, compared to 59 percent in the U.S.

(These percentages are calculated bycombining the following categories:"almost every day" + "two or three times aweek" + "once a week").

In both FiW and U.S. schools, eighthgrademath teachers meet with their colleaguesless frequently than fourthgrade mathteachers. However, notable differences existbetween FiW and the U.S.

Approximately half (51 percent) of FiWeighthgrade students have teachers whoreport that they meet with other teachers toplan at least once a week, compared to aboutonethird of U.S. students (34 percent).(Again this is calculated by combining thecategories noted above).

At the other end of the spectrum, a notablegap also exists between the FiW and theU.S. Onethird of U.S. eighthgradestudents have teachers who meet with theircolleagues two or fewer occasions (never,once, or twice) over the course of a fullschool year. Three percent of FiW studentsfall into this category.

As with the TIMSS data on the amount ofout-of-school time that teachers devote totheir work, data on the frequency teachermeetings also suggest that FiW fourthandeighthgrade math teachers may be moreengaged than their U.S. counterparts inplanning curriculum and seeking feedbackfrom their colleagues.

Disparities between the number of studentswhose teachers participate in weeklyplanning sessions are most noteworthy, andexist at both the fourthand eighthgradelevels.

354 7

Exhibit 8: Teachers' Reports on the Frequency of Meetings withOther Teachers in Their Subject Area to Discuss and Plan

Curriculum or Teaching Approaches

Frequency

Percent of FourthGrade Students

Percent of EighthGrade Students

First in theWorld

UnitedStates

First in theWorld

UnitedStates

Almost every day 26 10 5

Two or three times a week 17 14 25 9Once a week 38 35 21 16

Once a month 7 18 29 23Every other month 7 5 15 12

Once or twice a year 5 16 1 25Never 0 2 2 8

SOURCE: NCREL analysis of TIMSS data; FiW Teacher Questionnaire results, NCREL.NOTE: Totals may not add to 100 due to rounding.

Teacher Influence Over Key SchoolDecisions

The level of teacher involvement in schoolbudget and curricular decisions can alsoprovide yet another indication of the level ofteacher engagement. Although the degree towhich states and districts allow theirteachers to be involved in these discussionsvaries considerably, actively engagedteachers may exert a lot of influence overthese decisions, while teachers who are lessengaged in improving curriculum,instruction, or the overall schoolenvironment may not.

This section looks at data on the influencethat eighthgrade math teachers in the FiWand the U.S. have over basic schooldecisions. (Data are not available for fourthgraders).

As shown in exhibit 9, eighthgrade mathstudents in the FiW Consortium were morelikely than U.S. students to have mathteachers who report they have a lot ofinfluence over key school decisions.

36

Nearly half (47 percent) of FiW eighthgrade students had teachers who reportedhaving "a lot" of influence over the subjectmatter to be taught.

Ninety-two percent of FiW eighthgrademath students had teachers who reportedthat they had at least "some" (if not "a lot")of influence over the subject matter to betaught.

By contrast, U.S. students tended to haveteachers who reported that they had lesscontrol. Thirty-eight percent of U.S.students had teachers who reported that theyhad "a lot" of control over the subjectmatter, and 73 percent had teachers who feltthey had at least "some" control. Seventy-four percent of FiW eighthgrade mathstudents had math teachers who claimedthey have "some" or "a lot" of control overwhich textbooks are used, compared to only63 percent of U.S. eighthgraders.

This difference may reflect the fact that, insome schools, these choices are made at thestate or district level.

48

FiW eighthgrade students were also morelikely than U.S. students to have mathteachers who reported that they have "a lot"of control over what supplies are purchased(47 percent of FiW teachers compared toonly 23 percent of U.S. teachers).

However, FiW eighthgrade students aremore likely than U.S. students to have mathteachers who felt they had no control overthe amount of money to be spent on supplies(46 percent in FiW compared to 35 percentin the U.S.).

Since not all schools or districts allowteachers to have a say in budgetary matters,it is not clear that these data indicate higher

levels of teacher involvement in the FiWthan in the U.S. However, they do indicatethat FiW teachers have a greater influenceon these decisions than their U.S.counterparts.

In summary, data indicate that eighthgradeFiW students are more likely to be taught byteachers who have control over somecurricular and budget decisions.

Again, these findings reinforce the datareported earlier in the section that FiWteachers may be more engaged than theirU.S. counterparts (to the extent that thesedata capture not only teacher influence, butalso teacher engagement)

Exhibit 9: Teachers' Reports on Their Influence over School Decisions

School DecisionAmount

ofInfluence

Percent of Eighth GradeStudents

FiW U.S.

Subject matter to be taught

A lot 47 38

Some 45 35

Little 6 18

None 2 9

Specific textbooks to be used

A lot 39 27

Some 35 36

Little 22 18

None 4 19

.

The amount of money to be spent on supplies

A lot 13 4

Some 16 27

Little 25 35

None 46 35

What supplies are purchased

A lot 47 23

Some 43 41

Little 10 29

None 0 7

SOURCE: NCREL analysis of TIMSS data, FiW Teacher Questionnaire results.NOTE: Totals may not add to 100 due to rounding.

37 49

Exh

ibit

10: T

each

ers'

Rep

orts

on

The

ir F

amili

arity

with

Key

Cur

ricu

lum

Doc

umen

ts

Doc

umen

tFa

mili

arity

Perc

ent o

f E

ight

hGra

de*

Stud

ents

FiW

U.S

.Ja

pan

Kor

eaS

inga

pore

Nat

iona

l Cur

ricul

um G

uide

for

Mat

hem

atic

s(U

S: N

atio

nal C

ounc

il of

. Tea

cher

s of

Mat

hem

atic

s (N

CT

M)

Pro

fess

iona

l Sta

ndar

dsfo

r T

each

ing

Mat

hem

atic

s)

Ver

y F

amili

ar69

383

475

Fai

rly F

amili

ar31

4871

4123

Not

Fam

iliar

013

1944

2

No

Suc

h D

ocum

ent

00

711

0

Reg

iona

l (S

tate

) C

urric

ulum

Gui

de fo

rM

athe

mat

ics

Ver

y F

amili

ar10

' 27

11

0

Fai

rly F

amili

ar21

3637

260

Not

Fam

iliar

6735

3648

0

No

Suc

h D

ocum

ent

22

2625

100

Nat

iona

l Exa

min

atio

n S

peci

ficat

ions

(US

: The

Nat

iona

l Ass

essm

ent f

orE

duca

tiona

l Pro

gres

s (N

AE

P)

Ass

essm

ent

Fra

mew

orks

/Spe

cific

atio

ns)

Ver

y F

amili

ar0

161

462

Fai

rly F

amili

ar16

2429

2436

Not

Fam

iliar

8258

4148

2

No

Suc

h D

ocum

ent

23

2924

0

Reg

iona

l (S

tate

) E

xam

inat

ion

Spe

cific

atio

ns

Ver

y F

amili

ar11

42

10

Fai

rly F

amili

ar33

2867

280

Not

Fam

iliar

5668

17.4

70

No

Suc

h D

ocum

ent

0.0

1424

100

SO

UR

CE

: NC

RE

L an

alys

is o

f lE

A's

Thi

rd In

tern

atio

nal M

athe

mat

ics

and

Sci

ence

Stu

dy (

TIM

SS

) da

ta, 1

994-

95; F

iW T

each

er Q

uest

ionn

aire

resu

lts, N

CR

EL.

NO

TE

: Tot

als

may

not

add

to 1

00 d

ue to

rou

ndin

g.*E

ight

h gr

ade

in m

ost c

ount

ries.

5051

Teacher Familiarity with Key Curriculumand Assessment Documents

As scientists, mathematicians, andresearchers make advances in math andscience, school textbooks and otherclassroom materials must be periodicallyupdated to reflect new knowledge and waysof thinking. Standards, curriculumguidelines, and student assessmentinstruments also undergo regular revisionsand updates so that they can accuratelyreflect current and emerging research andbest practices.

Thus, key local, state, and nationalstandards, curriculum, and assessmentdocuments typically reflect the most currentmath and science knowledge. Teacherfamiliarity with, and knowledge of, thesedocuments may provide another indicationof how engaged teachers are in keepingabreast of the latest advances in math andscience curriculum and assessment.

This section looks at FiW teacher familiaritywith major curriculum and assessmentdocuments. Comparative FiW and U.S.data, and those from other nations are onlypresented at the eighthgrade level becauseof limited data availability. It should also benoted that there are some definitional issueswith the international comparisons, thereforethese data should be interpreted withcaution.

As illustrated in exhibit 10, FiW eighthgrade students have teachers who showvarying degrees of familiarity with keycurriculum documents. Most show thegreatest familiarity with national standards.Sixty-nine percent of FiW students haveteachers who are "very familiar" and 31percent have teachers who are "fairlyfamiliar" with the National Council of

39

Teachers of Mathematics (NCTM)standards. In contrast, 38 percent of U.S.students have teachers who are "veryfamiliar" and 48 percent who are "fairlyfamiliar" with the NCTM standards.

In high math achievement countries, thepattern varies across countries. InSingapore, teachers of most students were"very familiar" with equivalent documents,while in Japan and Korea, students weremore likely to have teachers who report thatthey are "fairly familiar" but not "veryfamiliar."

The comparison to U.S. teachers' familiaritymay indicate that FiW teachers have agreater involvement with professionalassociations, place more emphasis onprofessional knowledge, or have moreopportunities to pursue outside interests.

The responses of the high performing Asianteachers are much less clear. While teachersin Singapore indicate a similar familiaritywith their national curriculum, those inJapan and Korea do not. Some believe thedata from Japan and Korea reflect anunderstated familiarity, as opposed to a lackof knowledge.

FiW teachers are less familiar with statecurriculum guides than national guidelines.Math teachers of less than half of FiWeighthgrade students are "very familiar" or"fairly familiar" with state curriculumguides. U.S. math teachers are more familiarwith these curriculum guides. Sixty-threepercent of U.S. students have teachers whoreport they are "very familiar" or "fairlyfamiliar" with these guides.

This lack of familiarity could also be afunction of when, and if, states had

52

completed development of their statestandards.'

As for familiarity with exam specificationsfor mathematics, math teachers of most FiWfourth and eighth grade students report thatthey are "not familiar" with the U.S.equivalent of national exam specificationsthe National Assessment of EducationalProgress (NAEP). This is probably becauseIllinois does not participate in the NAEP.

In the U.S., 16 percent of students haveteachers who are "very familiar" withNAEP, and 24 percent have teachers whoare "fairly familiar" with NAEP.

While FiW teachers report more familiaritywith state exam specifications than withNAEP, they do not report particularly highlevels of familiarity with state exams. Infact, teachers of at least half of the FiWeighth grade math students report that theyare "not familiar" with state examspecifications.

This pattern differs slightly for the U.S. Inthe U.S., more eighth grade math studentshave teachers who are "very familiar" or"fairly familiar" with the NAEP than withtheir state exams.

As was the case with state curriculumframeworks or standards, this could be morea function of state policy than teacherfamiliarity.

In summary, TIMSS questionnaire data onteacher familiarity with key curriculumdocuments and exam specificationsreinforce some of the earlier findings onteacher engagement. Although data are notavailable at the fourthgrade level, FiWeighthgrade teachers report high levels offamiliarity with the NCTM Standards,unlike U.S. teachers.

40

Data on state teaching guides and examspecifications show lower levels of teacherfamiliarity, in both the FiW and the U.S.High degrees of familiarity with nationalcurriculum standards are also found in somehigh math achievement countries, but notall. It is unclear, however, whether thispattern reflects actual differences infamiliarity or in understated familiarity(particularly in the case of Japan and Korea).

Summary

In summary, the TIMSS data indicate thatFiW students have teachers that indicatemore engagement than U.S. teachers in abroad array of school-related activities.

These activities include participating inschool-related activities outside the schoolday, meeting with their colleagues,identifying and selecting textbooks, buyingsupplies, and keeping up with newcurriculum and instructional developmentsand techniques.

Other research has posited that one result ofgreater teacher engagement is morestimulating, organized, and/or tailoredinstruction. While certainly not conclusive,these data may suggest that a key componentto delivering better math instruction may beencouraging more active teacherparticipation in classroom planning, schooldecision-making, and keeping abreast of keychanges in curriculum and assessments.

Teaching Environment

In addition to the key factors alreadydiscussed, the environment for teaching mayalso have an important impact oninstruction, and in turn, studentachievement. Teachers who must dealfrequently with non-academic issues withinthe classroom may not have as much time todevote to instruction.

For example, many teachers must routinelyspend class time dealing with disciplineproblems or disruptive students. Externalfactors may also contribute to, or detractfrom, teaching environments in the FiW andthe U.S.

In particular, state-, school district-, orschool-based decisions and policy prioritieswhich affect either the availability ofequipment, the adequacy of physicalfacilities, or the student/teacher ratio mayhave an impact on the teaching environment.

Exhibit 11 presents data on teachers' reportson the factors that limit their ability to teachtheir classes. These reports show similarpatterns exist between FiW and the U.S. andhigh performing countries (Japan, Korea andSingapore). In both the fourthand eighthgrades, across all countries, the three mostfrequently cited factors limiting teachers'abilities to teach by "quite a lot" or "a greatdeal" were:

(1) students with different academicabilities;

(2) high student/teacher ratios; and(3) disruptive students.

While different percentages of teachers fromthese countries reported that these factorsplaced "quite a lot" or "a great deal" oflimitations on their ability to teach, in nearly

all cases, students with different academicabilities were reported to be one of the mostimportant limiting factors.

The only exception was for eighthgrademath students from Singapore, wherestudents were more likely to be taught byteachers who report that high student/teacherratios were the most important limitationplaced on their ability to teach their class.

Across nearly all categories, FiW studentswere less likely than their counterparts in theU.S. and in highachieving countries to haveteachers who report that critical factors limittheir ability to teach their class.

At the other extreme, students in Korea werethe most likely to have teachers whoreported that the various factors placed"quite a lot" or a "great deal" of limitationson them.

Fewer students had teachers who reportedshortages of equipment as limiting factors.In the fourth grade, teachers of four percentof FiW math students reported thatequipment shortages limited their ability toteach by "quite a lot" or "a great deal,"compared to teachers of approximately 25percent of students in the U.S., Japan, andSingapore.

As they did with all factors, Korean teachersreported that equipment shortages placedmore limitations on their ability to teach,with teachers of 54 percent of studentsindicating that this limited their teachingability "quite a lot" or "a great deal."

A similar pattern was found on theequipment shortages in the eighthgradedata from the FiW, U.S. and highachievement countries. The relative wealth

41 5

of FiW districts probably has an impact onthe differences in these results.

In summary, the TIMSS teacher dataindicate that FiW, U.S., and high mathachieving countries report similar patterns inthe factors that affect their ability to teachmath. The limitation at the top of the list in

the FiW, U.S. and all highmath achievingmath countries except Singapore is dealingwith students with a range of academicabilities. Students in Singapore were morelikely to have teachers who reported highstudent/teacher ratios as their mostimportant limitation. Concerns overfacilities and supplies were less importantlimitations for all countries.

Exhibit 11: Teachers' Reports on the Factors that Limit HowThey Teach Mathematics Class

Country

Percent of students whose teachers report each factor limiting how theyteach class as "quite a lot" or "a great deal"

Students withDifferentAcademicAbilities

DisruptiveStudents

Shortage ofEquipment for

Use inDemonstrations

and OtherExercises

HighStudent/Teacher

Ratio

Fourth Grade *FiW 36 15 4 18

United States 41 r 31 r 25 r 38 rJapan 60 -- 28 41

Korea 69 64 54 62Singapore 66 42 25 60

Eighth Grade *FiW 32 21 7 25

United States 44 r 39 r 20 r 29 rJapan 63 -- 12 42Korea 77 60 31 67Singapore 55 44 25 60

SOURCE: Figure 5.4 in Mullis, I.V.S., et al. (1997). Mathematics Achievement in the Primary School Years: lEA'sThird International Mathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing,Evaluation, and Educational Policy, Boston College; figure 5.3 in Mullis, I.V.S., et al. (1998). MathematicsAchievement in Missouri and Oregon in an International Context: 1997 TIMSS Benchmarking. Chestnut Hill, MA:Center for the Study of Testing, Evaluation, and Educational Policy, Boston College; FiW Teacher Questionnaireresults.

*Fourth/Eighth grade in most countries.A double dash (--) indicates data are not available. This question was not included on questionnaires for teachers ofJapanese students.An "r" indicates teacher response data available for 70-84 percent of students.

42 55

Summary

In sum, differences in the contexts forteaching and learning between the FiW andthe U.S. may offer some insight intopossible explanations for the gap inachievement levels between these twogroups.

Accordingly, the preceding sectionsexamined four broad areas that help definethe context for teaching and learning forpossible clues as to the factors that mightdrive these differences: curriculum,instructional practices, teacher engagement,and the teaching environment.

Differences in curriculum may contribute tothe differences in achievement between theFiW and the U.S. These achievement gapsdo not seem to be driven by differences inthe number of topics covered by thetextbooks used by the FiW and U.S.students, as the numbers of topics addressedby U.S. and FiW textbooks are similar.

FiW eighthgrade textbooks, however, tendto focus on algebra and geometry moreheavily than U.S. books do, perhapsreflecting the difference in course-takingbehavior.

Nor do the differences seem to be driven bya more focused coverage of topics in theclassroom, since FiW students spend classtime on just as many topics as their U.S.peers. However, FiW students seem to beintroduced to more advanced topics earlierthan U.S. students. This pattern is found inboth the fourth and eighthgrades, and, asmight be expected, is even more pronouncedin the eighth grade.

Data on instructional practices indicate thatdifferences exist between FiW and U.S.

fourth and eighth grade math classes.According to students, the FiW and the U.S.show similar patterns with respect to thefour most frequently used activities.However, TIMSS data suggest that FiW andU.S. math teachers rely on different methodswhen demonstrating how to do mathproblems.

In the fourth grade, FiW teachers rely on avariety of approaches for teaching; no onemethod dominatesome instructional timeis spent in large groups, some in smallgroups, some working individually.

In the eighth grade, however, the mostfrequently used classroom organizationalmethodin the FiW and in high mathachievement countriesinvolves the mathteacher teaching the whole class; thisapproach is used far more frequently than inU.S. eighth grade math classes. These datasuggest that FiW students may have mathteachers who use direct teaching styles morefrequently than their U.S. counterparts, withthe form of instruction varying according tothe grade level.

In addition, both FiW fourth and eighthgrade students are more likely to be asked todo reasoning tasks than to spend timepracticing computational skills.

The data also indicate important differencesin how homework is assigned and used.FiW students are more likely than U.S.students to have homework assigned everyday and to discuss their completedhomework in class. Classroom discussionsof homework may help to clarify commondifficulties and serve to solve outstandingproblems that their students encounter.

Together, these results suggest that FiWmath teachers are more likely than U.S.

43

5 6

math teachers to challenge their students todemonstrate their mastery of more advancedideas or concepts.

The TIMSS data also suggest that FiWstudents may have teachers that are moreengaged than U.S. teachers in a broad arrayof school-related activities. These includeparticipating in school-related activitiesoutside the regular work day, meeting withtheir colleagues, identifying and selectingtextbooks, buying supplies, and keeping upwith new curriculum and instructionaldevelopments and techniques.

Finally, similar patterns were found inteacher reports on the type of factors thatlimit their teaching abilities. Across theFiW, the U.S., and high achieving mathcountries, teachers reported similar patterns:student factors most limited their ability to

44

teach, while the adequacy of class was lesslimiting.

FiW teachers reported that the adequacy ofsupplies hindered their ability to teach verylittle, no doubt reflecting the relatively highwealth of the districts.

While certainly not conclusive, these datasuggest that key components of delivering aworld-class math education may beencouraging the earlier introduction ofadvanced math topics into the curriculumand spending more time in the classroomconcentrating on instruction.

Efforts to encourage active teacherparticipation in classroom planning, schooldecision-making, and keeping abreast of keychanges in curriculum and assessments arealso likely to be beneficial.

What is the FiW Consortium Doing to Improve Math and Science?

The Consortium is launching numerousactivities as part of its effort to improvemath and science programs and to identifyworld-class standards in instruction,assessment, and curriculum development.

The Consortium also works hard todisseminate its findings to educators,researchers, and policymakers byparticipating in numerous presentations andseminars. Working closely with its partners,it has also taken full advantage of advancesin technology to disseminate materialsdocumenting the FiW's progress to othersvia the World Wide Web.(http://www.ncrel.org /fitw/homepage.htm)

Across the spectrum of activities undertakenby the FiW, educators have maintained theircommitment to including all students in theachievement of math and science.Reflecting this commitment, students withdisabilities were included in the sample ofstudents taking TIMSS, and specialeducation teachers participate in each of theConsortium's Teacher Learning Networks(TLNs).

This section focuses on one of the most fullydeveloped of these activities, the FiW'sefforts to establish TLNs, networks oflearning communities involving educators,parents, and community leaders.

This section also describes the FiW effortsin this area, as well as one TLN's efforts toimprove science instruction by examiningthe TIMSS results in light of current FiWinstructional practices.

455-8

Teacher Learning Networks:Collaborative Learning Communities

The Consortium's TLNs grew out of a cross-district planning effort that involved teachersfrom all districts and all levels of education,Consortium administrators, and outsideadvisors. After the Consortium was formed,planning teams were established toconceptualize a structure for developinglearning communities that would extendbeyond district boundaries.

These learning communities build upon theprofessional development efforts of theindividual districts in order to promotesystemic change. The planning teamsincluded teachers from each of the highschools in the Consortium, as well-asteachers from the elementary schools.

Working in collaboration with theprofessional development specialists fromNorth Central Regional EducationLaboratory, the planning teams developed astructure to engage teachers across four keydomains. These areas are:

Curriculum models. This networkexamines the Consortium's curriculumusing techniques similar to those used bythe International and National TIMSSCenters. It allows teachers to promotecross-district and cross-gradecoordination of curriculum, as well asgreater alignment with national andinternational standards. Networkmembers also explore differences andsimilarities of different districts on thesequence, emphasis, and content coveredin math and science courses.

Instructional practices and models. Thisnetwork focuses on encouraging theimplementation of instructional practicesthat promote engaged learning forstudents. Network members explorevarious instructional methods, includingproblem-based learning, hands-onscience, and activity-centered teaching.

Assessment strategies. This networkassists the Consortium in usingassessments to support decisions andestablish school improvement plans. Itplaces particular emphasis on the use ofperformance-based assessments in placeof traditional testing formats. Membersalso look at the importance of integratingteaching with assessment and methodsof alternative instruction that informinstructional practices.

Technology. This network explores thepotential for using different forms oftechnology to support and augment mathand science education.

As shown in exhibit 12, the structure of theTeacher Learning Networks relies on theorganizational, intellectual, and creative

46

resources of the FiW Consortium, itspartners, and the education community.Approximately 75 teachers are involved inthe learning network activities."

The TLNs are supported by the InstructionalSupport Network (ISN), a group ofcurricular and instructional directors whoprovide technical support to the TLNs. TheISN collects and assists in the interpretationof data and assesses the systemicfunctioning of the networks.

In addition, FiW teachers and staff can drawupon data and expertise available from theDepartment of Education and/or NCREL.They can also make use of the research onthe best practices for teaching math andscience being undertaken by theseorganizations.

Finally, the learning communitiesthemselves also function as a form ofintellectual capital to other teachers andadministrators for improving math andscience achievement.

59

A

Exh

ibit

12: F

irst

in th

e W

orld

Tea

cher

Lea

rnin

g N

etw

orks

Lea

rnin

g C

omm

uniti

es I

nitia

te a

nd S

uppo

rtE

duca

tiona

l Im

prov

emen

t and

Ref

orm

Cur

ricu

lum

Mod

els

Inst

ruct

iona

lPr

actic

e/M

odel

sT

echn

olog

yA

sses

smen

t

Org

aniz

atio

nal

Lea

ders

hip

(Cor

e C

apac

ities

for

Cha

nge)

Vis

ion

Bui

ldin

gIn

quir

yM

aste

ryC

olla

bora

tion

Proc

ess/

Act

iviti

esFo

rmul

ate

Prob

lem

s

lisG

ener

ate

Hyp

othe

ses/

Res

earc

h Q

uest

ions

Cre

ate

Met

hodo

logi

es f

or S

tudi

es

Dis

sem

inat

e Fi

ndin

gs

Prov

ide

Prof

essi

onal

Dev

elop

men

t

Inte

llect

ual

Cap

ital

Dat

aR

esea

rch

Bes

t Pra

ctic

esSt

anda

rds

The

orie

sL

earn

ing

Com

mun

ities

Dis

tric

t/Sch

ool I

mpr

ovem

ent

Eff

orts

Lea

ding

to W

orld

Cla

ss P

erfo

rman

ce a

nd S

choo

lsSO

UR

CE

: Kim

mel

man

, Pau

l., K

roez

e, D

avid

., (A

ugus

t 199

8). P

rese

nted

mat

eria

ls a

nd h

ando

uts,

Sem

inar

on

the

Firs

t in

the

Wor

ld C

onso

rtiu

m. S

pons

ered

by

The

Nat

iona

l Ins

titut

e on

Stu

dent

Ach

ieve

men

t, C

urri

culu

m, a

nd A

sses

smen

t. O

ffic

e of

Edu

catio

nal R

esea

rch

and

Impr

ovem

ent.

Sym

posi

um c

ondu

cted

at t

heU

.S. D

epar

tmen

t of

Edu

catio

n, W

ashi

ngto

n, D

C.

,061

Teacher Network Grants

Teacher network grants are small grantsprovided to Consortium staff interested inpursuing staff development activities,locally initiated action research, orcurriculum development initiatives in mathand science. The FiW leadership identifiedthe two primary objectives of the teachernetwork grants as follows:

To improve the teaching of math andscience through self-identified andConsortium-identified areas of study.The teacher network grants allowteachers to identify areas of study thathave direct links to their abilities toteach math and science.

To allow practitioners to take an activerole in network activities while bothproviding and receiving services. Thesecross-district activities allow teachersand others to learn from and interactwith their colleagues at other schoolswithin their district, as well as theircolleagues in other FiW districts.

To receive a grant, a network member ordistrict must submit a proposal to theConsortium's Grant Review Committee on atopic for study in one of the four networkareas (curriculum, instruction, assessment,or technology).

The proposal must outline the project'sgoals, planned activities, budget, and themethods by which its outcome will beshared with other districts within theConsortium. Grant applications arereviewed by a Grant Review Committee,which is made up of staff and administrativerepresentatives.

BEST COPY AVAILABLE 4

Proposals are assessed according to criteriaestablished by the FiW Board of Directors.Funding is provided for proposals that meetthese four criteria:

(1) congruence with the Consortium'sannual working plan;

(2) direct teacher or staff input;(3) the availability of activities and

outcomes for all Consortium districts;and

(4) focus on math and science.

During the 1997-98 school year,approximately 20 grants were awarded.'Grants were awarded for work on thefollowing general topics in the four networkareas:

Curriculum Models

Curriculum continuity for math andscience from kindergarten through grade12;

Analyzing physics concepts and thesystematic introduction of concepts tostudents; and

Mentoring program for females in mathand science.

Instructional Practices and Models

Implications of brain research in theteaching and learning process;

Staff development to improve instructionfor students with moderate to severelearning disabilities; and

Training staff in problem-based learning.

62

Assessment Strategies

Determining teacher knowledge aboutassessment;

Aligning instruction and assessmentpractices for students; and

Retooling science activities andassessments for students withdisabilities.

Technology

Designing web pages to further math andscience learning experiences.

As they pursue their projects, the TLNs areresponsible for organizing themselves topromote their own learning.

In particular, the networks are intended tofoster exploration of the extent to whichteachers' classroom practices are consistentwith both

(1) their stated beliefs about teaching andlearning, and

(2) current and emerging research and bestpractice.

Furthermore, the networks are exploringways to enhance consistency betweencurrent classroom practice and research onbest practices.

In addition, network members are expectedto lay the groundwork for the expansion ofthese networks into larger, more inclusivelearning communities.

Example of a FiW Teacher LearningNetwork: Analysis of FiW PhysicsAchievement

To provide an illustration of the types ofactivities undertaken by a TLN, this sectionpresents an example of work beingundertaken a group of teachers to use theTIMSS results to improve the Consortium'sscience curriculum and instruction.

With assistance from researchers at NCREL,high school physics teachers decided to takea close look at the performance of FiWstudents on physics-related questions onTIMSS, FiW physics-related curriculum,and instructional practices and beliefs ofFiW physics teachers.

Although much of this report focuses ondifferences in math achievement, thisexample was chosen because it illustrateshow FiW educators are using the TIMSSbenchmark, along with supplemental data, toimprove science achievement.

This section describes the project's goalsand participants, its initial results, and thepotential areas initially identified for futureresearch by the project's participants.

As noted earlier, the FiW results on thetwelfth grade physics exam were lower thananticipated, with twelve nations scoringsignificantly above the FiW, three nationsobtaining scores not significantly differentfrom the FiW, and one nation scoringsignificantly below the FiW.

To gain fuller insight into what it takes tobecome first in the world in physicsachievement, six current and former physicsteachers32 met over the summer of 1998. Atthese meetings, they reviewed the FiWphysics results from TIMSS, as well as their

49 6 3

instructional and curricular practices anddiscussed what they could do to improvephysics achievement.

This cross-district effort built on theteachers' considerable experience andcommitment to improve science instruction.Between them, the six teachers have morethan 100 years of combined physics teachingexperience, and all are extremely active inprofessional development activities andorganizations.

Project Activities

After initial meetings with NCREL staff toreview the TIMSS physics questions anddiscuss the high school results, teammembers identified three project activities.These activities were designed not only togive the network members a betterunderstanding of the FiW students' relativestrengths and weaknesses in solving physicsproblems, but also to give them a betterunderstanding of the context for learningand teaching physics in FiW and the varietyof instructional approaches used across theConsortium.

The project activities are:

Analyzing FiW students' performance onphysics exams using groups of similarTIMSS test questions. This analysis waslimited to the use of released TIMSS testquestions, a relatively small sample ofquestions. To conduct this analysis, thereleased physics questions from theeighth grade science exam and thetwelfth grade physics exam weregrouped according to conceptual modelsin physics (e.g., particles, matter, light,ideal gas, systems, relativistic physics,and force laws). FiW student responsepatterns were then compared to the

response patterns of U.S. students andthe international average.

Extending the TIMSS teacher survey to asample of FiW high school scienceteachers. The TLN administered one ofthe two TIMSS teacher surveys that hadbeen prepared for TIMSS but not usedduring the study. (Unlike in the fourthand eighth grades, TIMSS did not collectdata from twelfthgrade teachers). TheTLN collected data on teacher beliefsand attitudes using one of these surveys.

Creating and administering a pilotteacher survey to collect data on values,style, and "rigor" in physics classrooms.This task involved creating andadministering the Teacher Survey ofRigor, a new survey to a sample ofphysics teachers. The new survey wasintended to collect additional data onhow physics courses are similar ordifferent in style of delivery,expectations for students and whatteachers value as important. Theanticipated survey results also areexpected to complement the results fromthe TIMSS teacher survey.

As these activities demonstrate, the FiWteacher learning networks provide a uniqueopportunity for their members to work with,and learn from, their peers.

Furthermore, they allow FiW teachers to tapinto NCREL analysts and researchers, aswell as their colleagues in other districts, intheir efforts to become first in the world inmath and science.

Initial Results

At the time of this writing, the analysesundertaken by this network team are at

64

different stages of completion. Initial resultsare available from the TIMSS questionnaireanalysis, while the two additional surveysare in the very early stages of analysis.

Nevertheless, the preliminary results havealready pointed to areas where the FiWmight work toward improving its physicsinstruction.

The initial results from the analysis ofTIMSS physics questions done by NCRELpoint to areas of relative strength andweakness in FiW physics achievement. Onearea of interest was in the different topicalareas and different achievement levels.

For example, FiW results for eighthgradescience show strength in the physicalsciences with few exceptions. One of theseexceptions was questions associated with theatomic model, on which FiW studentsshowed their lowest performance.

Twelfth grade FiW students demonstratedstrength on questions related to theNewtonian concept of force. Questions inmodern physics and mechanical wavesshowed the lowest performance levels.

One unexpected result highlighted by theanalysis was that FiW students scored betteron questions dealing with constantgravitational force than constant electric ormagnetic force, despite the fact that thesame general concepts apply to both areas.

The analysis of TIMSS results on the twelfthgrade physics exam also gave the FiWteachers a fuller understanding of the typesof problem-solving skills needed to achieveworld class standards in physics. Forexample, the teachers discovered that fewquestionnaire items could be answered usingrote memory.

In addition, they discovered that all of theTIMSS countries obtained relatively lowperformance levels on the physicsassessment. On average, only 31 percent ofthe items were answered correctly.

Also, FiW physics students performed betterwhen tackling certain types of test questions.In particular, they had higher performanceon multiple choice items (as opposed to freeresponse items) than the internationalsample.

Topics for Further Research Identified bythe Physics Teachers' Learning Network

Based on their preliminary research, theteam has already identified several questionsthat they feel may deserve future attention:

(1) Are similar trends repeated in theTIMSS questions that were not released?

(2) Why do all students (International, U.S.,and FiW) perform poorly on the TIMSSphysics assessment?

(3) Can conceptual models be tracedthrough the fourth, eighth, and twelfthgrades?

(4) How do students perform onexperimental design and scientificprocess items?

To address these questions, the TLN hasidentified some opportunities for expandingtheir investigation that may be particularlyfruitful. In particular, they recommendbroadening the analysis to include all FiWschools, as well as an examination of studentresults on the fourthgrade TIMSS scienceassessment.

They also hope to work on identifyinggroups that might yield richer comparisonson what it means to be first in the world(e.g., identifying an appropriate comparison

51 6 5

group of FiW physics students or group ofnations that might be present for all threetest populations).

Finally, based on the results of the surveyanalysis, they plan to look for appropriatephysics content that might strengthen theircurriculum in areas where FiW studentachievement was not as high.

Summary

The Consortium has begun to embark on ahost of activities to define and clarify worldclass standards and establish learningcommunities. The establishment of TLNs;networks of learning communities involvingeducators, parents, and community leaders,represents one of the most fully developedof these activities.

As a result of a cross-district planning effortinvolving teachers from all education levels,Consortium administrators, and outsideadvisors, the networks engage participants infour key domains:

(1) curriculum models,(2) instructional practices and models,(3) assessment strategies, and(4) technology.

52

These learning communities build upon andcontribute to the professional developmentefforts of individual districts and leveragethe organizational, intellectual, and creativeresources of the FiW Consortium, itspartners, and the education community.

To facilitate the work of the TLNs, smallgrants are available to Consortium facultyinterested in pursuing staff developmentactivities, locally initiated action research, orcurriculum development initiatives in mathand science.

During the 1997-98 school year,approximately 20 grants were awarded.Throughout the upcoming year, thenetworks and grant recipients will exploreways to enhance consistency among currentclassroom practice, current and emergingresearch, and best practice, as illustrated bythe ambitious agenda of activitiesundertaken by a team of physics teachers.

Working with the Consortium's partners,these teachers are successfully using theTIMSS benchmark, along with supplementaldata, to examine physics curriculum andinstruction across districts and grade levels.

66

Summary and Conclusion

In conclusion, the effort begun by a group ofsmall school districts north of Chicago hasalready begun to show some promisingresults. Motivated to take the NationalEducation Goals seriously, this consortiumembarked upon a detailed plan of action to"become first in the world in math andscience by the year 2000."

As'a first step in their plan, Consortiumstudents became the only school districts totake part in TIMSS, the most ambitious,comprehensive, and rigorous internationalassessment of math and science yetundertaken.

In contrast to the U.S., the FiW performedexceptionally well on the Consortium'sinitial benchmark, indicating that they arewell on their way to achieving their goal. Infact, TIMSS results indicate that fourth andeighth grade students performed at, or near,the top of the world in both math andscience.

In the twelfth grade, results were moremixed. Although students taking the generalknowledge assessments achieved worldclass standards, FiW students taking theadvanced math and physics examsperformed near the international average.However, FiW AP students taking theadvanced math and physics exams, perhapsa better group to use for internationalcomparisons, performed at the top of theworld.

Given the Consortium's performance, thisreport explored some of the possible reasonswhy they did so well compared to the U.S.by focusing on math. Initial analyses of therelationship between FiW and U.S. mathachievement and student and family socio-

economic background characteristics foundthat home and family characteristics couldexplain less than half of the difference inscores.

Accordingly, differences in the contexts forteaching and learning math between the FiWand the U.S. were examined as possibleexplanations of the remaining gaps betweenthese two groups.

Four broad areas were explored:

(1) curriculum,(2) instructional practices,(3) teacher engagement, and the(4) teaching environment.

Although similar patterns were reported inall of these areas, important differences didemerge.

The review of curriculum and textbook datafound that the number of topics addressed byU.S. and FiW math textbooks is similar andFiW students spend class time on just asmany topics as their U.S. peers.

Nevertheless, some differences do existbetween FiW and U.S eighth grade mathtextbooks, with FiW eighthgrade textbooksmore focused on algebra and geometry thanU.S. books.

In addition, FiW students seem to beintroduced to more advanced topics earlierthan U.S. students. This pattern is found inboth the fourthand eighthgrades, and, asmight be expected, is even more pronouncedin the eighth grade.

53 67

TIMSS data on instructional practices alsosuggest additional differences between FiWand U.S. fourth and eighth grade mathclasses. Although students report thatsimilar patterns in the four most frequentlyused activities, the data indicate that FiWand U.S. math teachers rely on differentmethods when demonstrating how to domath problems.

In the fourth grade, FiW teachers rely on avariety of approaches for teaching; no onemethod dominates. In the eighth grade, bycontrast, group instruction of the whole classis reported as the most frequent classroomorganizational approach in the FiW and highmathachievement countries.

This approach is used far more frequentlythan in U.S. eighthgrade math classes,suggesting that FiW students may have mathteachers who use direct teaching styles morefrequently than their U.S. counterparts, withthe form of instruction varying according tothe grade level.

The TIMSS data also suggested differencesbetween the types of math activitiesperformed by FiW and U.S. in class. Inparticular, FiW fourthand eighthgrademath students are more likely than U.S.students to be asked to perform reasoningtasks than to spend time practicingcomputational skills.

In addition, important differences exist inhow homework is assigned and used. FiWstudents are more likely than U.S. studentsto have daily homework and to discuss thesecompleted assignments in class. Together,these results suggest that FiW math studentsmay be more challenged than U.S. studentsto show their mastery of more advancedideas or concepts.

54

As for teacher engagement, FiW studentsmay have teachers that are more engagedthan U.S. teachers in a wide assortment ofschool-related activities. Examples includeparticipating in school-related activitiesoutside the regular work day, meeting withtheir colleagues, identifying and selectingtextbooks, buying supplies, and keeping upwith new curriculum and instructionaldevelopments and techniques.

Similar patterns were also found in teacherreports on teaching environments. Acrossthe FiW, the U.S., and high achieving mathcountries, teachers reported similar patterns:student factors most hindered their ability toteach, while the adequacy of class supplieswas less of a limitation. FiW teachersreported that the adequacy of supplieslimited their ability to teach very little, nodoubt reflecting the relatively high wealth ofthe districts.

These data suggest that key components ofdelivering a top notch math education maybe introducing advanced math topics into thecurriculum earlier and spending more timein the classroom concentrating oninstruction.

Efforts to foster active teacher participationin classroom planning, school decision-making, and to allow teachers opportunitiesto learn about key changes in curriculum andassessments are also likely to be positive.

The FiW Consortium knows that theseinternational achievement benchmarks arenot static. While FiW students have donewell in 1996, this success does not guaranteecontinued success since the achievementbenchmark may be set at a different point in1999 and in coming years.

Accordingly, the FiW Consortium has also

68

begun its efforts to define and clarify world-class standards in instruction, assessment,and curriculum. Working with its partnersat the regional and national level, theConsortium is identifying current andemerging research and best practices in allof these areas.

Recognizing that the current context forteaching and learning within the FiWconsortium may also provide some clues asto what it takes to do well in math andscience, the Consortium is also exploring theTIMSS data for suggestions as to whichinstructional, curricular, and assessmentpractices may work well in the U.S.

Finally, the Consortium has worked hard tocreate a structure for developing a cross-district community of learners that would

55

involve educators, parents, and communityleaders.

It has established teacher learning networksin four areas: curriculum, assessment,instruction, and technology; and awardedgrants to groups of teachers pursuingprojects in these areas. These projects,along with the other efforts of learningnetworks, will lay the groundwork for theexpansion of these networks into larger,more inclusive learning communities.

Taken together, these results and activitiesprovide exciting news. They illustrate notonly that U.S. students have the potential tobecome the first in the world in math andscience, but also that districts can work in acollaborative, cooperative manner to strivetowards this goal.

69

Bibliography

Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1996).Mathematics Achievement in the Middle School Years: IEA's Third International Mathematicsand Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, andEducational Policy, Boston College.

Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1996).Science Achievement in the Middle School Years: IEA's Third International Mathematics andScience Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and EducationalPolicy, Boston College.

First in the World Consortium. Achieving Excellence: Initial Findings of Fourth GradePerformance from The Third International Mathematics and Science Study. (Grade 4).

First in the World Consortium. Achieving Excellence: Initial Findings of Twelfth Grade Studentsin The Third International Mathematics and Science Study.

Kroeze, David J., Johnson, Daniel P., and Zalewski, Eugene. Achieving Excellence: A Report ofInitial Findings of Eight Grade Performance form the Third International Mathematics andScience Study. First in the World Consortium.

Martin, Michael 0., Mullis, I.V.S., Beaton, A.E., Gonzales, E.J., Smith, T.A., and Kelly, D.L.,(1997). Science Achievement in the Primary School Years: IEA's Third InternationalMathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation,and Educational Policy, Boston College.

Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1998).Mathematics Achievement in the Primary School Years: IEA's Third International Mathematicsand Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, andEducational Policy, Boston College.

Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1997).Mathematics and Science Achievement in the Final Year of Secondary School: lEA's ThirdInternational Mathematics and Science Study. Chestnut Hill, MA: Center for the Study ofTesting, Evaluation, and Educational Policy, Boston College.

Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1998).Mathematics Achievement in Missouri and Oregon in an International Context: 1997 TIMSSBenchmarking. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and EducationalPolicy, Boston College.

National Center for Education Statistics. (1997). Pursuing Excellence: A Study of U.S. Fourth-Grade Mathematics and Science Achievement in International Context. NCES 97-255, Office of

57 /0

Educational Research and Improvement. U.S. Department of Education. Washington, DC: U.S.Government Printing Office.

National Center for Education Statistics. (1997). Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching, Learning, Curriculum, and Achievement inInternational Context. NCES 97-198, Office of Educational Research and Improvement. U.S.Department of Education. Washington, DC: U.S. Government Printing Office.

National Center for Education Statistics. (1998). Pursuing Excellence: A Study of U.S. Twelfth-Grade Mathematics and Science Achievement in International Context. NCES 98-049, Office ofEducational Research and Improvement. U.S. Department of Education. Washington, DC: U.S.Government Printing Office.

National Research Council. (1996). Mathematics and Science Education Around the World.What Can We Learn? From the Survey of Mathematics and Science Opportunities (SMSO) andthe Third International Mathematics and Science Study (TIMSS). Center for Science,Mathematics, and Engineering Education National Research Council. Washington, DC: NationalAcademy Presi

National Research Council. Third International Mathematics and Science Study (TIMSS) andSurvey of Mathematics and Science Opportunities (SMSO): First in the World. (June 1998)Meeting at Michigan State University.

North Central Regional Educational Laboratory, Materials from the First in the World Web Site:www.ncrel.org.

Robitaille, David F., Schmidt, William H., Raizen, Senta A., McKnight, Curtis, Britton, Edward,Niol, Cynthia. (1993). The Third International Mathematics and Science Study: TIMSSMonograph No. .1 Curriculum Frameworks of Mathematics and Science. Pacific EducationalPress. Vancouver, Canada.

Schmidt, William H., McKnight, Curtis C., Raizen, Senta A. (1996). Splintered Vision: AnInvestigation of U.S. Science and Mathematics Education: (Executive Summary). Boston, MA:Kluwer Academic Publishers.

van der Ploeg, Arie., Hager, Maureen., Kimmelman, Paul., Kroeze, David., Lamaster, Pat., &McNelly, Maggie., (August 1998). Presented materials and handouts, Seminar on the First in theWorld Consortium. Sponsored by The National Institute on Students Achievement, Curriculum,and Assessment. Office of Educational Research and Improvement. Symposium conducted atthe U.S. Department of Education, Washington, DC.

71

58

Endnotes

' The Third International Math and Science Study is the largest, most comprehensive, and mostrigorous international comparison of math and science achievement ever undertaken.

2 Consortium membership has changed over the past several years. This section presents data onthe districts that currently make up the Consortium's membership. Exhibit A-1 in appendix Alists the districts that currently make up the Consortium.

3 Average includes all districts except the Northern Suburban Special Education District.Average expenditures in the Northern Suburban Special Education District range from $8,000 to$25,000 per student, depending on the type of student disability.

Current expenditure per pupil in fall enrollment public and elementary schools. NationalCenter for Education Statistics, Digest of Education Statistics, table 169.

'First in the World Consortium. Low-income students include those who receive public aid, livein institutions for neglected or abandoned children, are supported in foster homes, or are eligibleto receive free or reduced price lunch. This figure does not include data on students from theNorth Suburban Special Education District.

6 See exhibit A-2 in appendix A for more detail.

'National Center for Education Statistics, Digest of Education Statistics, table 69. Data are from1996.

8 North Central Regional Educational Lab, FiW Web Site, Purpose and History. These figuresdo not include data on students or faculty from the Illinois Mathematics and Science Academyand the North Suburban Special Education District.

9 See exhibit A-2 in appendix A for additional data on average teacher salaries in FiW districts.

I° National Center for Education Statistics, Digest of Education Statistics, table 77. Data arefrom 1995-96.

" The topic areas discussed here and presented in exhibit B-3 are those which were determinedby Michigan State University to more closely relate to those included in the TIMSS textbookanalysis and teacher questionnaires. They are different than the topic areas used by theInternational TIMSS Center at Boston College and reported in the international comparisons.

12 The topic areas presented in exhibit B-6 are those which were determined by Michigan StateUniversity to more closely relate to those included in the TIMSS textbook analysis and teacherquestionnaires. They are different than the topic areas used by the International TIMSS Center at

7259

Boston College and reported in the international comparisons.

'3 First in the World Consortium. Achieving Excellence: Initial Findings from Twelfth-GradeStudents in the Third International Mathematics and Science Study. p. 3. Mullis, I.V.S., Martin,M.O., Beaton, A.E., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1997). Mathematics andScience Achievement in the Final Year of Secondary School: LEA's Third InternationalMathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation,and Educational Policy, Boston College. p. 19.

14 First in the World Consortium. Achieving Excellence: Initial Findings from Twelfth-GradeStudents in the Third International Mathematics and Science Study. p. 3. Mullis, I.V.S., Martin,M.O., Beaton, A.E., Gonzales, E.J., Kelly, D.L., and Smith, T.A. (1997). Mathematics andScience Achievement in the Final Year of Secondary School: LEA's Third InternationalMathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation,and Educational Policy, Boston College. p. 19.

15 First in the World Consortium. Achieving Excellence: Initial Findings from Twelfth-GradeStudents in the Third International Mathematics and Science Study. p. 4.

16 First in the World Consortium. Achieving Excellence: Initial Findings from Twelfth-GradeStudents in the Third International Mathematics and Science Study. p. 3.

"Internal NCREL memo from Bill Quinn to Jeri Nowakowski, July 17, 1998.

Is Preliminary analyses undertaken by NCES have found similar effects.

19 TIMSS administered teacher surveys to teachers of fourth- and eighth-grade students but didnot administer teacher surveys to teachers of students in twelfth grade.

20 The authors of this report looked at TIMSS data items that might point to areas that couldexplain achievement differences based on knowledge of FiW districts and educational researchfindings. No claims are being made as to the statistical significance of these findings.

21 Note that the instructional topics shown in exhibit 1 differ slightly from those presented inexhibit B-3. The categories used in exhibit B-3 are those used by the U.S. TIMSS Center atMichigan State University, while those presented in exhibit 1 correspond to the categories usedin the TIMSS teacher survey.

22 Note that the instructional topics shown in exhibit 2 differ slightly from those presented inexhibit B-6. The categories used in exhibit B-6 are those used by the U.S. TIMSS Center, whilethose presented in exhibit 2 correspond to the categories used in the TIMSS teacher survey.

23 The group of U.S. textbooks used for this analysis is described in Schmidt, William H.,McKnight, Curtis C., Raizen, Senta A. (1996). Splintered Vision: An Investigation of U.S.

60

Science and Mathematics Education: (Executive Summary). Boston, MA: Kluwer AcademicPublishers.

24 United States National Research Center, Third International Mathematics and Science Study.First in the World: Curriculum Analysis Final Report. East Lansing: Michigan State University.1998.

'United States National Research Center, Third International Mathematics and Science Study.First in the World: Curriculum Analysis Final Report. East Lansing: Michigan State University.1998, p. 6.

26 United States National Research Center, Third International Mathematics and Science Study.First in the World: Curriculum Analysis Final Report. East Lansing: Michigan State University.1998, p. 6.

"United States National Research Center, Third International Mathematics and Science Study.First in the World: Curriculum Analysis Final Report. East Lansing: Michigan State University.1998, p. 6.

28 United States National Research Center, Third International Mathematics and Science Study.First in the World: Curriculum Analysis Final Report. East Lansing: Michigan State University.1998, p. 9.

29 The U.S. TIMSS questionnaires were completed in spring 1995, at a time when many stateswere still developing their mathematics standards.

" Kroeze, David, and Daniel Johnson. Achieving Excellence: A report of initial findings ofeighth grade performance from the Third International Math and Science Study, p. 2.

31 First in the World. First in the World Consortium Science and Mathematics Grant Program1998-98. Material from FiW Web Site, www.ncreLorg/fitw.

32 The teachers had taught or were currently teaching at Glenbrook North High School,Glenbrook South High School, New Trier High School and Illinois Mathematics and ScienceAcademy.

7 461

Appendix A:Contact Information

and Characteristics ofFirst in the World Consortium Districts

75 A-1

Exhibit A-1: List of First in the World Districts

K-8 School Districts

Avoca School District No. 37Dr. John W. Sloan, Superintendent2921 Illinois Rd.Wilmette, IL 60091847-251-3587Web: www.avoca.k12.il.us

Glenview Community Consolidated No. 34Dr. Thomas Rich, Interim Superintendent1401 Greenwood Ave.Glenview, IL 60025847-998-5000Web: www.ncook.k12.il.us

Lincolnwood School District No. 74Dr. Steve Lake, Superintendent6950 E. Prairie Rd.Lincolnwood, IL 60645847-675-8234

Niles Elementary School District No. 71Dr. Eugene Zalewski, Superintendent6935 W. Touhy AvenueNiles, IL 60714847-647-9752

Northbrook School District No. 28Dr. James Kucienski, Superintendent1475 Maple Ave.Northbrook, IL 60062847-498-7900Web: www.district28.k12.il.us

Sunset Ridge School District No. 29Dr. Howard Bultinck, Superintendent525 Sunset Ridge RoadNorthfield, IL 60093847-446-6383

Wheeling School District No. 21Dr. Lloyd "Bud" DesCarpentrie,Superintendent999 W. Dundee Rd.Wheeling, IL 60090847-537-8270

Frankfort Community Consolidated SchoolDistrict 157-CDr. Pamela Witt, Superintendent10482 West Nebraska St.Frankfort, IL 60423-2235815-469-5922

Golf School District 67Dr. Linda Marks, Superintendent9401 Waukegan Rd.Morton Grove, IL 60053-1353847-966-8200

Mount Prospect District No. 57Dr. Maureen L. Hager, Superintendent701 W. Gregory StreetMt. Prospect, IL 60056-2220847-394-7300Web: www.ncesc.org/dist57

Northbrook School District No. 27Dr. David J. Kroeze, Superintendent1250 Sanders Rd.Northbrook, IL 60062847-498-2610Web: www.northbrook27.k12.il.us

Northbrook/Glenview School District No. 30Dr. Harry Rossi, Superintendent2374 Shermer RoadNorthbrook, IL 60062847-498-4190

West Northfield School District No. 31Dr. Paul L. Kimmelman, Superintendent3131 Techny Rd.Northbrook, IL 60062847-272-6880, ext. 223

A-2

High School Districts

Glenbrook School District No. 225Dr. David Hales, Superintendent1835 Landwehr Rd.Glenview, IL 60025847-998-6100Web: www.glenbrook.k12.il.us

Niles Township School District No. 219Dr. Grif Powell, Superintendent7700 Gross Point Rd.Skokie, IL 60077847-568-3590Web: www.niles-hs.k12.il.us

Special Education District

Northern Suburban Special Education DistrictSEJA-804Mr. David Peterson, Superintendent760 Red Oak Ln.Highland Park, IL 60035847-831-5100

Residential School

Illinois Mathematics & Science AcademyDr. Stephanie Pace Marshall, President1500 W. Sullivan Rd.Aurora, IL 60506630-907-5037Web: mocha.imsa.edu

New Trier School District No. 203Dr. Henry Bangser, Superintendent385 Winnetka Rd.Winnetka, IL 60093847-501-6310Web: nths.newtrier.k12.il.us

SOURCE: http://www.ncreLorefitw/ 1 stpagesfinembers.htm

NOTE: This list is the First in the World Consortium membership as of November 1998.

77A-3

Exh

ibit

A-2

: Fir

st in

The

Wor

ld D

istr

ict C

hara

cter

istic

s

Scho

ol D

istr

icts

# of

Schs

Enr

ollm

ent

Perc

enta

ge (

%)

# of

.

Tch

rs

Tea

cher

Sala

ries

Per

Pupi

l

Exp

endi

ture

Eth

nici

tyL

ow

Inco

me

LE

P

Tea

cher

s

Wh

Afr

Am

His

pA

/PI

B.A

.M

.S.+

K-8

Avo

ca S

choo

l Dis

tric

t #37

265

380

.80.

61.

017

.61.

93.

849

.051

.0. 4

9$

47,0

57$

9,49

1

Fran

kfor

t Sch

ool D

istr

ict #

I57

C3

1,39

396

.00.

81.

02.

10.

10.

080

.219

.868

$35

,575

$4,

492

Gle

nvie

w S

choo

l Dis

tric

t # 3

47

3,59

277

.12.

88.

011

.712

.07.

926

.473

.623

6$

46,8

91$

7,31

3G

olf

Scho

ol D

istr

ict #

67

253

470

.00.

44.

824

.79.

719

.159

.640

.443

$44

,135

$9,

483

Lin

coln

woo

d Sc

hool

Dis

tric

t #74

31,

266

99.1

0.0

0.0

0.9

3.7

0.0

30.5

69.5

106

$53

,188

$8,

769

Mou

nt P

rosp

ect S

choo

l Dis

tric

t # 5

73

1,77

091

.71.

42.

74.

11.

53.

647

.952

.110

1$

46,4

39$

7,89

6N

iles

Ele

men

tary

Sch

ool D

istr

ict #

712

533

70.0

0.9

2.3

26.4

19.1

8.4

29.7

70.3

39$

53,5

86$

9,09

8

Nor

thbr

ook

Scho

ol D

istr

ict #

274

1,44

886

.10.

41.

412

.10.

72.

734

.465

.612

648

,569

$9,

191

Nor

thbr

ook

Scho

ol D

istr

ict #

284

1,70

691

.30.

10.

77.

90.

65.

359

.140

.914

5$

47,2

58$

10,3

34

Nor

thbr

ook/

Gle

nvie

w S

choo

l Dis

tric

t #30

31,

195

81.0

0.1

2.1

16.8

0.6

2.5

28.8

71.2

85$

51,0

25$

8,22

9

Suns

et. R

idge

Sch

ool D

istr

ict #

292

512

91.0

0.0

0.6

8.4

0.4

2.7

41.8

58.2

41$

43,9

42$

9,70

6W

est N

orth

fiel

d Sc

hool

Dis

tric

t # 3

12

939

66.5

1.0

4.2

28.4

3.0

7.4

41.4

58.6

68$

51,1

28S.

9,10

0W

heel

ing

Scho

ol D

istr

ict #

2112

7,15

268

.52.

721

.96.

917

.814

.444

.355

.745

3$

46,6

97$

6,79

2

Sum

4922

,693

1,56

0

Ave

rage

3.77

1,74

679

.41.

69.

19.

88.

87.

942

.257

.812

0$

47,3

39S

7,78

1

Hig

h Sc

hool

Gle

nbro

ok H

igh

Scho

ol D

istr

ict #

225

24,

091

77.0

1.0

3.3

18.6

4.4

3.7

18.0

82.0

284

$65

,000

$12

,100

New

Tri

er H

igh

Scho

ol D

istr

ict #

203

13,

006

85.6

1.0

2.3

11.0

1.0

2.5

27.7

72.3

249

$66

,762

$13

,786

Nile

s T

owns

hip

Hig

h Sc

hool

Dis

tric

t #21

92

4,23

060

.31.

95.

831

.97.

44.

925

.474

.627

0$

64,1

58$

11,6

41

Sum

511

,327

803

Ave

rage

1.67

2,98

973

.01.

34.

021

.54.

63.

823

.576

.526

8$

65,2

63$

12,3

76

Spec

ial E

duca

tion

Nor

ther

n Su

burb

an S

peci

al E

d D

istr

ict

I1,

602

84.4

5.8

3.4

6.8

----

44.0

56.0

--$

42,3

11$

8,00

0-25

,000

Res

iden

tial S

choo

lIl

linoi

s M

athe

mat

ics/

Scie

nce

Aca

dem

y1

628

52.0

9.0

6.0

28.0

8.0

0.0

2.0

98.0

--$

55,1

12$

12,7

00

Stat

e

Stat

e--

--63

.320

.612

.83.

135

.76.

154

.245

.6--

$42

,429

$6,

158

Firs

t in

the

Wor

ld56

36,2

5077

.21.

87.

213

.7*

7.1

*6.

1--

----

--$

*8,

958

SO

UR

CE

: Firs

t in

the

Wor

ld C

onso

rtiu

m.

A d

oubl

e da

sh (

--)

indi

cate

s no

dat

a.

Fig

ure

was

cal

cula

ted

from

K-8

and

Hig

h S

choo

l num

bers

onl

y. S

peci

al E

duca

tion

figur

es w

ere

not i

nclu

ded.

NO

TE

S: I

. Thi

s ta

ble

pres

ents

dat

a on

dis

tric

ts in

the

Firs

t in

the

Wor

ld C

onso

rtiu

m a

s of

win

ter

1999

.2.

Dis

tric

t dat

a fr

om 1

996-

97 s

choo

l sta

te r

epor

t car

d.

3. A

ttend

ance

was

of S

epte

mbe

r 19

96.

4. L

ow in

com

e is

def

ined

as

fam

ilies

elig

ible

to r

ecei

ve fr

ee o

r re

duce

d pr

ice

lunc

h.5.

Ope

ratin

g ex

pend

iture

per

pup

il is

bas

ed o

n 19

95-9

6 da

ta.

BE

ST

CO

PY

AV

AIL

AB

LE7

Appendix B:FiW TIMSS Achievement Results

80

Appendix B: FiW TIMSS Achievement Results

Exhibits B-1 through B-12 present data on the average achievement of each country thatparticipated in the different TIMSS assessments, as well as visual representations of thedistribution of each country's scores.1 The distributions show student achievement at the5th, 25th, 75th and --th percentiles. The dark black band in the middle of each country'sdistribution is the mean plus or minus two standard errors; this band is intended toemphasize the point that each country's average score is only an estimate of the truescore.

The exhibits in this section should be interpreted with caution. Because these data weregenerated using statistical sampling procedures, the average scores are represented withtheir appropriate error bands. Therefore, average scores for countries overlap one anotherin many cases. Thus, it is incorrect to state that the FiW ranked x out of y countries.Rather, countries have been grouped according to whether their scores are significantlyabove, not significantly different from, or significantly below the scores for the FiW.Nevertheless, as the data in these exhibits illustrate, there is considerable variation inscores both within and across countries.

Since the TIMSS assessment was administered, two school districts have not continued toparticipate in FiW activities (Wilmette School District #39 and Glencoe School District#35). The results reported here include students from these districts.

' Note that different groups of countries participated in each of the different assessments. See figure B-1 ofMullis, I.V.S., et al., Mathematics and Science Achievement in the Final Year of Secondary School for asummary of countries that participated in different assessments.

81.B -1

Exhibit B-1: Distributions of Mathematics Achievement in the Fourth Grade

Country Meant Mathematics Achievement Scale Score

Higher than FiW 1

SingaporeSame FiW

625 (5.3)

l

I v...-zzi.,.:,,,,,,,./ 1

asKoreaJapanFirst in the WorldHong Kong(Netherlands)

Lower than FiW

611

597591

587577

(2.1)(2.1)(9.1)(4.3)(3.4)

I i ll:r./41111V,W2A I

..w...-m..4 i1

I v---....,,,,, I

v- ..e.e. 'WA I

I ve zma..- All I

Czech Republic(Austria)(Slovenia)Ireland(Hungary)(Australia)United StatesCanada(Israel)(Latvia {LSS})Scotland^England* A

CyprusNorwayNew ZealandGreece(Thailand)PortugalIcelandIran, Islamic Republic(Kuwait)

567559552550548546545

532531

525520513

502

502499492490475474429400

(3.3)(3.1)(3.2)(3.4)(3.7)(3.1)(3.0)(3.3)(3.5)(4.8)(3.9)(3.2)(3.1)(3.0)(4.3)(4.4)(4.7)(3.5)(2.7)(4.0)(2.8)

t v.., ,an..-...-, ZIA 1

.

I v-zm iz... I

1 zz.,,, ...e. A I

."......(v.",

I IT-m- zezz

w......,,,,-..wwI

V eZZZAWW.F.,.." II

1,/./r ZeM44/1,1

leVie, 'MA I

t v-zziz.,,,,,,,,a 1

I vm..., .......-Ar,...A

L,-...A.,I z.,/ .2 1

I IT.F.r.r., ./Z/Z4,11 I

Llvz.eze1 ,,,A I

II 17Z! 1/Zr A I

I v ........:.w..w I

I ..AS..,./IAI

I WV Zy=e," JI II

I ZA I

i,,,... zz I

Percentiles ofr Performance5th 25th 75th 95th

Mean and Confidence Interval (+/-2SE)

200 250 300 350 400 450 500 550 600 650 700 750 800 850

SOURCE: Student Achievement Data, NCREL; tables 1.1, C.1, and C.3 in Mullis, I.V.S., et al. (1997). Mathematics Achievement in the PrimarySchool Years: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and

Educational Policy, Boston College.

Nations in which more than 10 percent of the population was excluded from testing. Latvia is designated LSS because only Latvian-speaking schoo

were tested, which represents less than 65 percent of the population.

^ Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals

were substituted.

t Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.

NOTE: Nations not meeting international guidelines are shown in parentheses.

B-2

Exhibit B-2: Distributions of Science Achievement in the Fourth Grade

Country Meant Science Achievement Scale Score

FiWSame as

First in the World 611 (9.0)1 1 1 1

Korea 597 (1.9) 1==12=211=21=1Lower than FiW I-

I

Japan 574 (1.8) 1 1/./MM144/-/A I

I I

United States 565 (3.1) 1L.w.m....m.,.....,,,....6,1

1

1

(Austria) 565 (3.3) 1 wz,4,w, " :":".1 I

1 j(Australia) 562 (2.9) 1,e."-/./.4,;(W.F.,,ZAIA 1

I

(Netherlands) 557 (3.1) 1 lel./.4rWIA I

Czech Republic 557 (3.1) 1 1,,,://./M/7"./17 I

England*^ 551 (3.3) 1 WIZ/7 / / /x /11../11 I1

wI .m.erwzirt ICanada 549 (3.0) I

Singapore 547 (5.0) 1 fr."..,,-...,Auvz-v..4

(Slovenia) 546 (3.3) 1 17,/./ZMNIZ/VA i

Ireland 539 (3.3) 1 v .........mrzivz 1

Scotland^ 536 (4.2) ....-.

Hong Kong 533 (3.7) 164....... yv,....,A1

1I

(Hungary) 532 (3.4) II Ill /r' JZZA 1

New Zealand 531 (4.9) 1 1/ /Z 1../M=4"."/"./V I

Norway 530 (3.6) 1 11.4,,,,W/4/7/1 1

I

(Latvia {LSS}) 512 (4.9) $ 1.0W+MMVZ1

(Israel) 505 (3.6) 1 1/7/./W.MeZeMA II I

Iceland 505 (3.3) I IIVZIZZAIWWWZA 1

I

Greece 497 (4.1) I L.WZ/WeV.,/

Portugal 480 (4.0)

11

v/ /l1..,mew.....4

Cyprus 475 (3.3) 1", Z1M/.44.44,11

(Thailand) 473 (4.9) LW AMP ii`A

Iran, Islamic Republic 416 (3.9)I

1/Z.W.AMW ./ 1

(Kuwait) 401 (3.1) 14z.......1.-Arm...4.A

Percentiles ofPerformance

5th 25th 75th 95th

1


200 250 300 350 400 450 500 550 600 650 700 750 800 850

SOURCE: Student Achievement Data, NCREL; tables 1.1, C.1,. and C.3 in Martin, Michael 0., et aL (1997). Science Achievement in the PrimarySchool Years: lEA's Third Internastional Mathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and


* Nations in which more than 10 percent of the population was excluded from testing. Latvia is designated LSS because only Latvian-speaking

schools were tested, which represents less than 65 percent of the population.^ Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals

were substituted.t Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.


BEST COPY AVAILABLE

B-3

Exh

ibit

B-3

: Fou

rth-

Gra

de M

athe

mat

ics

Perf

orm

ance

, by

Subt

opic

Ave

rage

Per

cent

Cor

rect

Mea

ning

of W

hole

Num

bers

Inte

gers

& W

hole

Num

ber

Ope

ratio

nsC

omm

on F

ract

ions

Dec

imal

Fra

ctio

nsE

stim

atin

g Q

uant

ity&

Siz

e

Rou

ndin

g &

Est

imat

ing

Com

puta

tions

Mea

sure

men

t Uni

ts

Hig

her

than

FiW

Sam

e as

FiW

Sam

e as

FiW

Hig

her

than

FiW

Hig

her

than

FiW

Sam

e as

FiW

Hig

her

than

FiW

Kor

ea88

.1

Sam

e as

FiW

Kor

ea82

.5

Japa

n80

.2

Sin

gapo

re79

.2

Firs

t in

the

Wor

ld78

.6

Hon

g K

ong

76.9

Cze

ch R

epub

lic74

.4

Low

er th

an F

iW

Sin

gapo

re71

.8

Firs

t in

the

Wor

ld65

.7

Hon

g K

ong

65.3

Japa

n62

.5

Kor

ea61

.5

Hun

gary

61.2

Net

herla

nds

60.2

Low

er th

an F

iW

Sin

gapo

re83

.2

Kor

ea75

.0

Hon

g K

ong

72.6

Japa

n71

.3

Sam

e as

FiW

Japa

n78

.0

Hon

g K

ong

70.5

Cze

ch R

epub

lic67

.9

Hun

gary

67.3

Kor

ea67

.0

Net

herla

nds

65.9

Sin

gapo

re65

.3

Aus

tral

ia61

.2

Nor

way

60.6

Sam

e as

FiW

Kor

ea76

.8

Japa

n74

.8

Hon

g K

ong

74.8

Sin

gapo

re74

.3

Firs

t in

the

Wor

ld73

.8

Net

herla

nds

68.7

Hun

gary

68.6

Cze

ch R

epub

lic68

.1

Uni

ted

Sta

tes

67.2

Low

er th

an F

iW

Japa

n76

.5

Kor

ea72

.1

Net

herla

nds

72.0

Cze

ch R

epub

lic68

.2

Hon

g K

ong

67.6

Hun

gary

66.3

Sam

e as

FiW

Firs

t in

the

Wor

ld81

.7

Sin

gapo

re81

.6

Japa

n79

.9

Low

er th

an F

iWF

irst i

n th

e W

orld

59.7

Low

er th

an F

iWH

ong

Kon

g76

.4

Hun

gary

74.5

Net

herla

nds

73.3

Cze

ch R

epub

lic72

.9

Isra

el69

.8

Uni

ted

Sta

tes

69.8

Aus

tral

ia68

.9

Can

ada

68.7

INT

ER

NA

TIO

NA

L 65

.9N

orw

ay61

.4

New

Zea

land

59.1

Eng

land

59.0

Tha

iland

57.3

Net

herla

nds

73.2

Hun

gary

71.5

Uni

ted

Sta

tes

67.6

Isra

el66

.3

INT

ER

NA

TIO

NA

L 64

.1C

anad

a61

.5

Aus

tral

ia61

.5

Nor

way

57.6

Eng

land

53.1

Tha

iland

52.7

New

Zea

land

50.8

Sin

gapo

re61

.3

Aus

tral

ia60

.2

Nor

way

60.0

Firs

t in

the

Wor

ld58

.0

INT

ER

NA

TIO

NA

L56

.8

Low

er th

an F

iW

Uni

ted

Sta

tes

53.7

Isra

el53

.3

Aus

tral

ia52

.0

Can

ada

50.4

Cze

ch R

epub

lic49

.7

INT

ER

NA

TIO

NA

L 50

.0E

ngla

nd49

.1

Tha

iland

46.4

New

Zea

land

45.3

Nor

way

'39

.2

Net

herla

nds

47.1

Uni

ted

Sta

tes

46.8

INT

ER

NA

TIO

NA

L 44

.6A

ustr

alia

43.7

Tha

iland

43.6

Can

ada

41.5

Cze

ch R

epub

lic40

.1

Hun

gary

36.4

Isra

el34

.2

Eng

land

32.0

Nor

way

28.9

New

Zea

land

26.3

INT

ER

NA

TIO

NA

L 57

.2F

irst i

n th

e W

orld

52.9

Eng

land

52.2

New

Zea

land

51.9

Can

ada

50.7

Isra

el50

.5

Tha

iland

49.8

Uni

ted

Sta

tes

47.1

.

Can

ada

61.6

Isra

el61

.2

Aus

tral

ia59

.4

INT

ER

NA

TIO

NA

L 55

.2N

ew Z

eala

nd47

.9E

ngla

nd47

.7

Nor

way

46.4

Tha

iland

39.1

Eng

land

52.3

Can

ada

51.7

Isra

el51

.0

New

Zea

land

49.7

Uni

ted

Sta

tes

48.0

Tha

iland

43.2

Per

imet

er, A

rea,

&V

olum

eG

eom

etry

: Pos

ition

&S

hape

s

Sym

met

ry,

Con

grue

nce,

&S

imila

rity

Pro

port

iona

lity

Pat

tern

s, R

elat

ions

, &F

unct

ions

Equ

atio

ns &

For

mul

asD

ata

& S

tatis

tics

Hig

her

than

FiW

Sam

e as

FiW

Sam

e as

FiW

Sam

e as

FiW

Sam

e as

FiW

Hig

her

than

FiW

Sam

e as

FiW

Sin

gapo

re75

.9

Kor

ea73

.5

Sam

e as

FiW

Firs

t in

the

Wor

ld77

Aus

tral

ia71

.9

Low

er th

an F

iW

Sin

gapo

re88

.9

Kor

ea88

.8

Hon

g K

ong

85.4

Firs

t in

the

Wor

ld85

.2

Eng

land

80.9

Low

er th

an F

iW

Sin

gapo

re63

.7

Net

herla

nds

62.7

Firs

t in

the

Wor

ld59

.4K

orea

58.1

Japa

n57

.7

Cze

ch R

epub

lic57

.3

Hun

gary

55.5

Hon

g K

ong

53.3

Low

er th

an F

iW

Kor

ea85

.0

Firs

t in

the

Wor

ld78

.0

Hon

g K

ong

77.8

Sin

gapo

re77

.5

Japa

n77

.2

Low

er th

an F

iW

Kor

ea85

.3

Sam

e as

FiW

Firs

t in

the

Wor

ld86

.4

Low

er th

an F

iWJa

pan

84.3

Sin

gapo

re82

.6

Firs

t in

the

Wor

ld78

.5

Hon

g K

ong

76.9

Hun

gary

72.4

Net

herla

nds

72.2

Low

er th

an F

iW

Sin

gapo

re80

.8

Kor

ea79

.8

Japa

n78

.3

Hon

g K

ong

75.8

Net

herla

nds

75.4

Uni

ted

Sta

tes

73.0

Can

ada

68.0

Cze

ch R

epub

lic67

.1

Aus

tral

ia-

67.1

Eng

land

65.8

Isra

el63

.4

New

Zea

land

61.5

INT

ER

NA

TIO

NA

L61

.8

Hun

gary

60.3

Nor

way

59.5

Tha

iland

55.5

Net

herla

nds

71.0

Hon

g K

ong

70.8

Cze

ch R

epub

lic68

.2

Japa

n67

.5

Firs

t in

the

Wor

ld65

.9

Hun

gary

62.8

Aus

tral

ia62

.2

Low

er th

an F

iW

Eng

land

70.6

Net

herla

nds

70.3

Hon

g K

ong

70.1

Can

ada

69.4

Uni

ted

Sta

tes

67.7

Cze

ch R

epub

lic67

.4

Japa

n66

.7S

inga

pore

65.3

New

Zea

land

65.3

Kor

ea65

.1

Hun

gary

62.3

INT

ER

NA

TIO

NA

L 61

.7Is

rael

59.1

Nor

way

58.3

Tha

iland

47.6

Aus

tral

ia79

.9

Japa

n79

.5

Uni

ted

Sta

tes

78.5

Can

ada

78.5

Cze

ch R

epub

lic77

.8

Net

herla

nds

74.I

Hun

gary

73.3

INT

ER

NA

TIO

NA

L 70

.5N

ew Z

eala

nd69

.1

Isra

el69

.0

Tha

iland

66.5

Nor

way

56.1

Hun

gary

71.4

Net

herla

nds

70.4

Cze

ch R

epub

lic69

.8

Aus

tral

ia69

.6

Uni

ted

Sta

tes

69.1

Can

ada

67.1

INT

ER

NA

TIO

NA

L 63

.2Is

rael

.62

.6

Eng

land

62.0

New

Zea

land

57.5

Nor

way

54.7

Tha

iland

52.4

Aus

tral

ia51

.3

Eng

land

46.6

Isra

el46

.0

INT

ER

NA

TIO

NA

L 45

.6U

nite

d S

tate

s45

.5

Can

ada

43.8

New

Zea

land

43.7

Nor

way

40.8

Tha

iland

34.0

Cze

ch R

epub

lic71

.0Is

rael

.66

.8

Uni

ted

Sta

tes

65.6

INT

ER

NA

TIO

NA

L 62

.0C

anad

a61

.4

Aus

tral

ia60

.0T

haila

nd54

.0

New

Zea

land

50.5

Nor

way

49.5

Eng

land

48.2

Can

ada

59.3

INT

ER

NA

TIO

NA

L 58

.8U

nite

d S

tate

s58

.3

Isra

el57

.1

Eng

land

55.2

Nor

way

54.5

New

Zea

land

49.1

Tha

iland

48.2

SO

UR

CE

: U.S

. Nat

iona

l TIM

SS

Cen

ter,

Mic

higa

n S

tate

Uni

vers

ity.

BE

ST

CO

PY

AV

AIL

AB

LE

Exhibit B-4: Distributions of Mathematics Achievement in the Eighth Grade


- Higher than FiWi

I

Singapore 643 (4.9) ..Same FiW I I I- as

I I I I

Korea 607 (2.4) I"W./ /./.1" /IA

i 1

Japan 605 (1.9) 1v..... ...... I

I

Hong Kong 588 (6.5) .0.0,05/A ' 1

First in the World 587 (11.8) "././../% ifalreAl

Belgium-Flemish^ 565 (5.7) r "I"A11"././.I =MI

Czech Republic 564 (4.9) WI' AVA".1 ../.0.1.7.1%I

Slovak Republic 547 (3.3) . .I- Lower than FiWI

Switzerland^ 545 (2.8) I1 1

(Netherlands) 541 (6.7) I sr.,,rzz J

(Slovenia) 541 (3.1) I

i774777=6.7.).7.11 1

(Bulgaria) 540 (6.3) I v.. .."1

(Austria) 539 (3.0) Iv. ./././M111.1., A I

i 1 1

France 538 (2.9) t II I' t"."/"Mr.."./.4A /

Hungary 537 (3.2) 1...1.....=4/....1 1

1

Russian Federation 535 (5.3) I 11./.17.11WAIA I1 i

(Australia) 530 (4.0) I vw,T,-,.elfzzir.ri 1

Ireland 527 (5.1) I lezzzy.m..,....-A 1

ICanada 527 (2.4) IWZ,1WA,ZZA II I I I

(Belgium-French) 526 (3.4) 1v......m......-A 1

I I I 1

Sweden 519 (3.0) 1 v....T.m-ff.A t

(Thailand) 522 (5.7) I 141., ""o7/1".1 /A II I I

(Israel)* 522 (6.2) 1v.....././....

1

)...-...-/m/.. ..' A I(Germany)" 509 (4.5)1 1

New Zealand 508 (4.5) St,"/".././.1 ' .

England" 506 (2.6) !".01%./../ "4"/W.1"/

Norway 503 (2.2)1

(Denmark) 502 (2.8) -rm.% -i I

United States^ 500 (4.6)(Scotland) 498 (5.5) '01/#.0.0 !AellealreI

MENLatvia {LSS}^ 493 (3.1) Wage/ WAVAA

Spain 487 (2.0) " ../ WA,.111111111Iceland 487 (4.5) SA/AA VW!"

(Greece) 484 (3.1) -..w.... - ,I

(Romania) 482 (4.0) 'Ad' . / d".1. "1 I

Lithuania* 477 (3.5) 1v......m.r."1

Cyprus 474 (1.9) i v..i..m...r..." 1

;

Portugal 454 (2.5) I1/./..."/W/ZA

1

I1

Iran, Islamic Republic 428 (2.2) ".01/./5 rd'a05,1 1

(Kuwait) 392 (2.5) 11/1",..=4,11

i 1

(Columbia) 385 (3.4) I"viW All I

1

(South Africa) 354 (4.4) I v.z.:/./.

Percentiles of

Performance 15th 25th 75th 95thC=ZEZMEMZEM=


200 250 300 350 400 450 500 550 600 650 700 750 800 850

SOURCE: Student Achievement Data, NCREL; tables 1.1, E. I, and E.3 in Beaton, Albert E., et al. (1996). Mathematics Achievement in the Middle

School Years: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and


Nations in which more than 10 percent of the population was excluded from testing. Latvia is designated LSS because only Latvian-speaking

schools were tested, which represents less than 65 percent of the population.


were substituted.



B-5

06

Exhibit B-5: Distributions of Science Achievement in the Eighth Grade

Country Mean+ Science Achievement Scale Score

Same FiWasSingapore 607 (5.5) IrAWIMMIW.W1

First in the World 584 (8.7) I V.,.1.11,/, II i

Czech Republic 574 (4.3) t v....-z,-.."-mAiJapan 571 (1.6) I MN"! ....W1

1 VIII.W/BIZZ/rnelI

Korea 565 (1.9)I vym,-..m....-..-..w..,(Bulgaria) 565 (5.3)

I

(Netherlands) 560 (5.0) zi i

1 vm.,--.=,-zzmi a(Slovenia) 560 (2.5)r vm,-2...-..-rmi(Austria) 558 (3.7)

- Lower than FiWHungary 554 (2.8) I vm.,,m.-zzmi i

i

England" 552 (3.3) I vz..-m.m.-..z..wri a

I VAIZeWZAIBelgium-Flemish^ 550 (4.2)I mz.,..,...wzmzzi(Australia) 545 (3.9)

Slovak Republic 544 (3.2) I, v-..cm=,-.".....1

Russian Federation 538 (4.0) I1

Ireland 538 (4.5) I .iiii's i;

Sweden 535 (3.0) , I ge....m.m...-..-zmi 1

, I

United States^ 534 (4.7) a I

t(Germany)" 531 (4.8) I trz.ww..ACanada 531 (2.6) I vz.w...w..-Awl I

I ilwrzmi INorway 527 (1.9)1

New Zealand 525 (4.4) I 111,ZMAMWMA I!

(Thailand) 525 (3.7) 1.,MWZZIA tl

I 1/"..WZ.W.We'll(Israel)* 524 (5.7)I

I 1,./WIWZIZZZAIHong Kong 522 (4.7)

z.- .Switzerland^ 522 (2.5)I V.W.,;(?Wr.f."1

I(Scotland) 517 (5.1)

VA/WiWZ.r.I

IlSpain 517 (1.7)1

France 498 (2.5).

I vz.w.....zzmi I(Greece) 497 (2.2)

Iceland 494 (4.0) t I.W.1./WZ.WA

(Romania) 486 (4.7) Imz...-,m...zA

Latvia {LSS}^ 485 (2.7)

II

t vz...-..-..m.,..-mi a

I vz.r.-...is,Portugal 480 (2.3)(Denmark) 478 (3.1) I IIVWFM.WZ/Z1 I

Lithuania* 476 (3.4) I 1///,M/ZWAII I

I(Belgium-French) 471 (2.8)t 1....-..m.m.,-1 aIran, Islamic Republic 470 (2.4) tvefcrawcreerJICyprus 463 (1.9)

,

(Kuwait) 430 (3.7) 1 II

, ; 1

(Columbia) 411 (4.1) I vw.f.,...-zmi I

(South Africa) 326 (6.6) I v.....-..z..ezzA1

Percentiles of

Performance 75th 25th 75th 95th

Mean and Confidence Interval (+/-25E)

150 200 250 300 350 400 450 500 550 600 650 700 750 800

SOURCE: Student Achievement Data, NCREL; tables 1.1, E.1, and E.3 in Beaton, Albert E., et al. (1996). Science Achievement in the Middle

School Years: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and


Nations in which more than 10 percent of the population was excluded from testing. Latvia is designated LSS because only Latvian-speaking

schools were tested, which represents less than 65 percent of the population.


were substituted.



BEST COPY AVAILABLE 87 B-6

Exh

ibit

B-6

: Eig

hth-

Gra

de M

athe

mat

ics

Perf

orm

ance

, by

Subt

opic

Ave

rage

Per

cent

Cor

rect

Who

le N

umbe

rsC

omm

on F

ract

ions

Dec

imal

Fra

ctio

ns &

Per

cent

ages

Rel

atio

nshi

ps o

fF

ract

ions

Est

imat

ions

of

Qua

ntity

& S

ize

Rou

ndin

gE

stim

atin

gC

ompu

tatio

nsM

easu

rem

ent

Uni

tsP

erim

eter

, Are

a,V

olum

eM

easu

rem

ent

Est

imat

ions

& E

rror

sH

ighe

r th

an F

iWH

ighe

r th

an F

iWS

ame

as F

iWS

ame

as F

iWS

ame

as F

iWS

ame

as F

iWS

ame

as F

iWH

ighe

r th

an F

iWH

ighe

r th

an F

iWH

ighe

r th

an F

iWS

inga

pore

80.4

Sam

e as

FiW

Sin

gapo

re82

.4

Sam

e as

FiW

Sin

gapo

re77

.9

Firs

t in

the

Wor

ld73

.8

Cze

ch R

epub

lic72

.5

Hon

g K

ong

71.1

Japa

n69

.8

Hun

gary

68.7

Kor

ea68

.4

Low

er th

an F

iW

Sin

gapo

re85

.1

Firs

t in

the

Wor

ld78

.3

Kor

ea77

.6

Japa

n75

.3

Hon

g K

ong

74.1

Bel

gium

(F

I)71

.3

Low

er th

an F

iW

Japa

n77

.9

Sin

gapo

re77

.3

Firs

t in

the

Wor

ld76

.2

Eng

land

72.2

Hon

g K

ong

71.8

Sw

itzer

land

71.8

Cze

ch R

epub

lic71

.7

Net

herla

nds

71.2

Bel

gium

(F

I)70

.7

Kor

ea70

.7

Aus

tral

ia69

.3

Hun

gary

69.3

Sw

eden

69.3

Fra

nce

69.0

Ger

man

y69

.0

Low

er th

an F

iW

Firs

t in

the

Wor

ld89

.1

Cze

ch R

epub

lic86

.9

Sin

gapo

re86

.6

Can

ada

84.7

Kor

ea83

.8

Low

er th

an F

iW

Sin

gapo

re75

.8

Firs

t in

the

Wor

ld74

.5

Cze

ch R

epub

lic69

.8

Japa

n69

.0

Kor

ea66

.7

Low

er th

an F

iW

Sin

gapo

re77

.9

Sam

e as

FiW

Sin

gapo

re84

.5

Japa

n72

.6

Sam

e as

FiW

Cze

ch R

epub

lic82

.9

Sin

gapo

re81

.0

Sw

eden

80.8

Sam

e as

FiW

Japa

n72

.0

Sw

itzer

land

70.2

Bel

gium

(F

1)68

.6K

orea

67.6

Cze

ch R

epub

lic67

.4

Firs

t in

the

Wor

ld67

.0

Hon

g K

ong

65.4

Fra

nce

64.5

Rus

sian

Fed

erat

ion

62.7

Spa

in60

.8

Hun

gary

60.6

Sw

eden

58.9

Isra

el59

.0C

anad

a57

.9

Low

er th

an F

iW

Japa

n72

.2K

orea

69.9

Hon

g K

ong

69.4

Bel

gium

(F

1)68

.7

Firs

t in

the

Wor

ld65

.8

Sw

itzer

land

62.4

Cze

ch R

epub

lic59

.6N

ethe

rland

s58

.9F

ranc

e57

.8

Hun

gary

57.7

Isra

el57

.4

Low

er th

an F

iW

Japa

n74

.8

Cze

ch R

epub

lic73

.2

Bel

gium

(F

I)71

.8

Sw

itzer

land

71.6

Sw

eden

69.5

Fra

nce

69.0

Kor

ea67

.9

Hun

gary

67.5

Net

herla

nds

66.8

Firs

t in

the

Wor

ld66

.4

Nor

way

66.4

Rus

sian

Fed

erat

ion

65.9

Aus

tral

ia65

.0

Hon

g K

ong

65.2

Ger

man

y63

.5C

anad

a63

.0

Isra

el63

.0

Spa

in62

.4

Tha

iland

62.3

INT

ER

NA

TIO

NA

L61

.4

New

Zea

land

61.3

Eng

land

60.5

-

Low

er th

an F

iW

Hon

g K

ong

63.9

Kor

ea61

.8

Firs

t in

the

Wor

ld49

.9

Cze

ch R

epub

lic48

.9

Sw

itzer

land

46.6

Bel

gium

(F

I)45

.6F

ranc

e.

44.1

Hun

gary

42.9

Rus

sian

Fed

erat

ion

44.6

Net

herla

nds

44.0

Aus

tral

ia42

.1

Can

ada

40.4

Tha

iland

40.0

Sw

eden

39.3

INT

ER

NA

TIO

NA

L39

.0

Eng

land

37.1

Isra

el36

.9

Low

er th

an F

iW'

Kor

ea79

.5

Bel

gium

(F

I)78

.9

Hun

gary

78.2

Sw

itzer

land

78.2

Net

herla

nds

76.1

Fra

nce

75.2

Firs

t in

the

Wor

ld75

.0

Aus

tral

ia74

.5

Nor

way

74.5

Ger

man

y72

.9

Hon

g K

ong

72.7

Japa

n72

.5

Eng

land

71.8

New

Zea

land

70.6

Can

ada

69.7

Rus

sian

Fed

erat

ion

696

Low

er th

an F

iW

Bel

gium

(F

1)81

.0

Hun

gary

79.8

Japa

n79

.6U

nite

d S

tate

s79

.2

Net

herla

nds

78.5

Sw

eden

78.5

Aus

tral

ia77

.5

Sw

itzer

land

77.4

Tha

iland

75.4

Fra

nce

74.7

New

Zea

land

74.4

Hon

g K

ong

74.2

Eng

land

74.2

Rus

sian

Fed

erat

ion

73.5

Nor

way

72.3

INT

ER

NA

TIO

NA

L70

.7

Ger

man

y69

.9Is

rael

68.8

Spa

in62

.2

J

Bel

gium

(F

I)64

.1

Sw

itzer

land

62.5

Can

ada

61.3

Sw

eden

60.0

Aus

tral

ia59

.1

Net

herla

nds

58.7

Hon

g K

ong

57.8

New

Zea

land

57.6

Hun

gary

57.3

Nor

way

57.0

Uni

ted

Sta

tes

55.7

Eng

land

54.4

Ger

man

y52

.0Is

rael

51.9

Fra

nce

50.8

INT

ER

NA

TIO

NA

L50

.6

Rus

sian

Fed

erat

ion

50.0

Tha

iland

49.5

Spa

in44

.1

Sw

itzer

land

70.0

Fra

nce

67.2

Can

ada

65.1

Cze

ch R

epub

lic64

.9

Ger

man

y62

.6S

wed

en62

.2

Aus

tral

ia61

.0

Hun

gary

60.7

Nor

way

59.6

Net

herla

nds

62.5

Tha

iland

59.6

Rus

sian

Fed

erat

ion

59.2

Uni

ted

Sta

tes

58.5

Isra

el57

.6

INT

ER

NA

TIO

NA

L56

.9

New

Zea

land

56.8

Eng

land

53.8

Spa

in48

.2

Rus

sian

Fed

erat

ion

65.9

Fra

nce

64.9

Bel

gium

(E

l)64

.6

Sw

itzer

land

61.9

Can

ada

61.0

Sw

eden

59.2

Tha

iland

58.0

Net

herla

nds

57.8

Ger

man

y57

.2Is

rael

56.3

Uni

ted

Sta

tes

55.8

INT

ER

NA

TIO

NA

L55

.4

Nor

way

54.1

Aus

tral

ia53

.5

Spa

in50

.6N

ew Z

eala

nd49

.3

Eng

land

45.9

.

Can

ada

56.2

Aus

tral

ia55

.4R

ussi

an F

eder

atio

n55

.8

Tha

iland

54.2

Sw

eden

53.1

INT

ER

NA

TIO

NA

L52

.1

Ger

man

y51

.7

New

Zea

land

51.5

Nor

way

51.2

Uni

ted

Sta

tes

49.3

Eng

land

49.2

Spa

in49

.2

Can

ada

67.8

Nor

way

66.6

New

Zea

land

65.3

Rus

sian

Fed

erat

ion

63.5

Uni

ted

Sta

tes

62.7

INT

ER

NA

TIO

NA

L61

,0Is

rael

58.4

Tha

iland

57.1

Spa

in54

.8

e

Tha

iland

56.0

INT

ER

NA

TIO

NA

L55

.5

Ger

man

y55

.4

Aus

tral

ia55

.0

Net

herla

nds

53.5

Nor

way

53.4

Uni

ted

Sta

tes

52.3

New

Zea

land

48.6

Eng

land

46.8

Nor

way

36.4

New

Zea

land

35.0

Ger

man

y32

.9

Spa

in29

.9

Uni

ted

Sta

tes

27.8

Tha

iland

66.1

INT

ER

NA

TIO

NA

L64

.7

Isra

el63

.3

Uni

ted

Sta

tes

62.5

Spa

in55

.3U

nite

d S

tate

s53

.0..

2D G

eom

etry

Bas

ics

..

Pol

ygon

s &

Circ

les

3D G

eom

etry

&T

rans

form

atio

nsC

ongr

uenc

e &

Sim

ilarit

yP

ropo

rtio

nalit

yC

once

pts

Pro

port

iona

lity

Pro

blem

sP

atte

rns,

Rel

atio

ns,

& F

unct

ions

Equ

atio

ns &

For

mul

asD

ata

Rep

rese

nt.

& A

naly

sis

Sta

tistic

s &

Pro

babi

lity

Hig

her

than

FiW

Hig

her

than

FiW

Sam

e as

FiW

Hig

her

than

FiW

Hig

her

than

FiW

Hig

her

than

FiW

Sam

e as

FiW

Hig

her

than

FiW

Sam

e as

FiW

Sam

e as

FiW

Japa

n78

.9

Sam

e as

FiW

Kor

ea73

.0

Japa

n70

.8

Sin

gapo

re70

.6

Sam

e as

FiW

s Ja

pan

84.3

Sin

gapo

re78

.6

Firs

t in

the

Wor

ld78

.0

Cze

ch R

epub

lic77

.7

Fra

nce

77.7

Hon

g K

ong

77.6

Sw

itzer

land

75.9

Bel

gium

(F

I)73

.7

Kor

ea73

.3

Hun

gary

71.0

Net

herla

nds

70.6

Rus

sian

Fed

erat

ion

70.2

Can

ada

70.1

Low

er th

an F

iW

Japa

n79

.7K

orea

76.9

Sin

gapo

re76

.7

Sam

e as

FiW

Sin

gapo

re70

.1

Sam

e as

FiW

Sin

gapo

re76

.7

Sam

e as

FiW

Japa

n70

.0

Kor

ea68

.4

Sin

gapo

re67

.2

Firs

t in

the

Wor

ld63

.2

Hon

g K

ong

60.8

Cze

ch R

epub

lic60

.2

Eng

land

60.1

Sw

itzer

land

59.5

Bel

gium

(F

I)59

.2

Hun

gary

59.1

Can

ada

58.2

Aus

tral

ia57

.6

Net

herla

nds

56.9

New

Zea

land

55.7

Fra

nce

55.0

Low

er th

an F

iW

Sin

gapo

re79

.8

Sam

e as

FiW

Japa

n84

.3

Firs

t in

the

Wor

ld83

.2

Kor

ea81

.9

Fra

nce

81.4

Sw

itzer

land

81.3

Sin

gapo

re81

.0

Net

herla

nds

80.6

Cze

ch R

epub

lic79

.6

Sw

eden

79.5

Bel

gium

(F

I)79

.5

Hon

g K

ong

'76

.4

Ger

man

y76

.0

Low

er th

an F

iW

Firs

t in

the

Wor

ld81

.3

Sin

gapo

re78

.5

Kor

ea77

.8

Bel

gium

(F

1)77

.7

Japa

n75

.0

Hon

g K

ong

73.0

Net

herla

nds

72.4

Low

er th

an F

iW

Sin

gapo

re78

.3.

Kor

ea75

.5

Hon

g K

ong

73.5

Firs

t in

the

Wor

ld65

.0

Rus

sian

Fed

erat

ion

63.4

Net

herla

nds

62.9

Tha

iland

62.4

Cze

ch R

epub

lic62

.1

Hun

gary

61.6

Aus

tral

ia58

.8

Sw

itzer

land

57.9

Bel

gium

(F

I)58

.8

Fra

nce

56.1

New

Zea

land

55.8

Eng

land

54.9

Can

ada

54.7

INT

ER

NA

TIO

NA

L53

.8

Isra

el52

.5

Nor

way

52.1

Ger

man

y49

.3

Low

er th

an F

iW

Kor

ea57

.0

Japa

n56

.9F

irst i

n th

e W

orld

50.6

Hon

g K

ong

50.1

Net

herla

nds

41.9

Low

er th

an F

iW

Japa

n67

.8

Kor

ea65

.1

Hon

g K

ong'

64.0

Cze

ch R

epub

lic61

.3

Firs

t in

the

Wor

ld61

.1

Bel

gium

(F

I)59

.4

Sw

itzer

land

57.3

Rus

sian

Fed

erat

ion

56.3

Tha

iland

56.0

Hun

gary

55.9

Net

herla

nds

55.9

Fra

nce

55.7

Low

er th

an F

iW

Japa

n72

.3

Hon

g K

ong

70.5

Kor

ea70

.0

Firs

t in

the

Wor

ld67

.8

Cze

ch R

epub

lic64

.3

Bel

gium

(F

I)62

.1

Rus

sian

Fed

erat

ion

61.4

Hun

gary

60.9

Isra

el58

.3

'Lo

wer

than

FiW

Hon

g K

ong

65.4

Bel

gium

(E

1)64

.1

Fra

nce

61.9

Isra

el59

.7

Cze

ch R

epub

lic57

.4

Rus

sian

Fed

erat

ion

57.2

Firs

t in

the

Wor

ld56

.1

Tha

iland

53.5

Hun

gary

51.6

Can

ada

50.5

Aus

tral

ia50

.4IN

TE

RN

AT

ION

AL

48.9

Sw

itzer

land

48.6

Eng

land

47.0

Net

herla

nds

44.0

Low

er th

an F

iW

Hon

g K

ong

71.0

Fra

nce

69.7

Cze

ch R

epub

lic66

.1

Tha

iland

62.7

Rus

sian

Fed

erat

ion

62.3

Firs

t in

the

Wor

ld62

.2

Bel

gium

(F

I)57

.5

Isra

el54

.8

Hun

gary

53.4

Can

ada

52.8

Net

herla

nds

51.0

Low

er th

an F

iW

Sw

itzer

land

72.4

Can

ada

70.0

Sw

eden

68.7

Aus

tral

ia67

.5

Fra

nce

66.5

Hun

gary

66.2

New

Zea

land

65.9

Eng

land

65.3

Nor

way

64.7

Cze

ch R

epub

lic63

.4

Uni

ted

Sta

tes

63.1

Ger

man

y62

.4

Isra

el62

.3

INT

ER

NA

TIO

NA

L59

.6

Spa

in58

.0R

ussi

an F

eder

atio

n56

.9

Tha

iland

56.5

Can

ada

40.3

Sw

eden

39.2

Bel

gium

(E

l)39

.0

Aus

tral

ia37

.8

Fra

nce

37.3

Sw

itzer

land

36.8

Cze

ch R

epub

lic37

.3

Tha

iland

36.5

Eng

land

34.8

Hun

gary

34.6

Rus

sian

Fed

erat

ion

33.5

INT

ER

NA

TIO

NA

L33

.4

New

Zea

land

33.4

Uni

ted

Sta

tes

31.5

Isra

el31

.4

Nor

way

30.9

Spa

in27

.4

Ger

man

y26

.5

Tha

iland

54.0

Fra

nce

53.6

Aus

tral

ia52

.8

Can

ada

52.4

Spa

in52

.2

Sw

itzer

land

51.9

Net

herla

nds

51.7

INT

ER

NA

TIO

NA

L50

.6

Uni

ted

Sta

tes

49.4

Ger

man

y46

.4N

ew Z

eala

nd46

.6

Nor

way

44.9

Eng

land

44.1

Sw

eden

43.7

Nor

way

75.5

Aus

tral

ia75

.2C

anad

a75

.0

Eng

land

74.8

Tha

iland

74.5

New

Zea

land

74.1

Hun

gary

73.7

Isra

el73

.2

Uni

ted

Sta

tes

72.6

Rus

sian

Fed

erat

ion

72.0

INT

ER

NA

TIO

NA

L70

.9

Spa

in70

.4

New

Zea

land

67.3

Eng

land

66.3

Aus

tral

ia65

.5Is

rael

64.8

Tha

iland

64.5

Ger

man

y63

.7IN

TE

RN

AT

ION

AL

63.6

Nor

way

58.6

Uni

ted

Sta

tes

58.4

Spa

in58

.2

Sw

eden

55.9

..

Aus

tral

ia52

.2 T

haila

nd53

.4C

anad

a51

.6 R

ussi

an F

eder

atio

n53

.2

INT

ER

NA

TIO

NA

L50

.1Is

rael

53.2

Sw

eden

49.2

INT

ER

NA

TIO

NA

L51

.4

Ger

man

y49

.0 U

nite

d S

tate

s51

.1

Eng

land

47.2

Spa

in50

.3

New

Zea

land

47.1

Nor

way

48.7

Isra

el46

.5 G

erm

any

47.6

Nor

way

47.1

Sw

eden

47.2

Uni

ted

Sta

tes

45.2

Spa

in43

.6_

INT

ER

NA

TIO

NA

L52

.6

Sw

itzer

land

51.2

Aus

tral

ia50

.9

Nor

way

46.9

New

Zea

land

45.9

Sw

eden

45.9

Eng

land

45.6

Ger

man

y45

.1

Uni

ted

Sta

tes

44.1

Spa

in43

.6

Ger

man

y44

.3

New

Zea

land

43.2

Sw

eden

42.8

Nor

way

42.4

Spa

in39

.3

Uni

ted

Sta

tes

39.1

II

Spa

in49

.5

Uni

ted

Sta

tes

47.9

Sw

eden

46.1

wM

¢htg

an S

tate

Ivy.

BE

ST

CO

PY

AV

AIL

AB

LE

Exhibit B-7: Distributions of General MathematicsAchievement in the Twelfth Grade


Same as FiW 1

(Netherlands)*Sweden(Denmark)First in the WorldSwitzerland(Iceland)(Norway)(Australia)

Lower FiW

560552

547545

540534528522

(4.7)(4.3)(3.3)(3.5)(5.8)(2.0)(4.1)(9.3)

I I I I

I vm............e..,...-A i

1 1 1

1 ViViWAMMV.46.i9VA I

1 1 1

I vziem.,wm....e.el II I

I VZ46,./....W.W.,,YA I

1 i 1

o1/7....w.,mm.,.w..1 I

1 1

I 11r 4,VAAMPWAIM I

I 1.-zzzr,":"....wi II

II 11:44174,MMEMA/AIA I

I Ithan(France)New Zealand^(Canada)(AUstria)*

(Slovenia)(Germany)"Hungary(Italy)Russian Federation*LithuaniaCzech Republic(United States)Cyprus*(South Africa)

523522519518512495483476471

469466461

446356

(5.1)(4.5)(2.8)(5.3)(8.3)(5.9)(3.2)(5:5)(6.2)(6.1)(12.3)(3.2)(2.5)(8.3) I__rtecmra-7--,,--1

I

I V.,,,,AMMV/WVA I

1 1

I V".44.10:4=0741/1 $

I

1

I WYW/JMNAZAA44.4 I

I

1 1 1

I V/MAMMWMA1 1 1

I 11./AriliZZAi 1 1

we...4,Ammvvzi.....?1

wz.),./..a..ezewn II1

I Vor..WYMMR.:05W.01 I

1 1

$ViVAI.77MMWMA

wr.4)..m.m.....,eyzn

I

III 1

1 1 I

I 1,4".411 ,,AW-40WZA I

1 1 1

I I/M.,,WIZZAIL $

1 i 1


5th 25th 75th 95th

200 250 300 350 400 450 500 550 600 650 700 750 800 850


SOURCE: Student Achievement Data, NCREL; tables 2.1, E.2, and E.7 and figure B.4 in Mullis, I.V.S., et al. (1998). Mathematics and Science

Achievement in the Final Year of Secondary School: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the

Study of Testing, Evaluation, and Educational Policy, Boston College.

* Nations in which more than 10 percent of the population was excluded from testing.^ Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals

were substituted.t Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.


B-8

Exhibit B-8: Distributions of General Science Achievement in the Twelfth Grade


Same as FiW

Sweden .

(Netherlands)*(Iceland)First in the World(Norway)(Canada)New Zealand^(Australia)

Lower FiW

559558549547544532

.529527

(4.4)(5.3)(1.5)(4.8)(4.1)(2.6)(5.2)(9.8)

c

I I I I

/ 1,7M./.1%/././1 I

1 I I I

I 1/.444/....1.1././..1I I

I

1

I lietWeiri./Z,ZA II I I

I 1,7/7":". .1"/./././J I1 1 1

I 14,-...,......,:m...../.41 1

t vz,,,...............,,,,,...-A

I 11/./1 M:M=411.1,ZZAI I 1

I i

thanSwitzerland(Austria)*(Slovenia)(Denmark)(Germany)"Czech Republic(France)Russian Federation*(United States)(Italy)HungaryLithuaniaCyprus*(South Africa)

523

520517509497487487481

480475471

461448349

(5.3)(5.6)(8.2)(3.6)(5.1)(8.8)(5.1)(5.7)(3.3)(5.3)(3.0)(5.7)(3.0)

(10.5)

,I I I

.

1 1/Z/Zei":".447.1 II I 1

1IVIZZ.F.rWZMA I

I I I 1

1 VIZ".../ZZA II I I 1

I trm....v.7....zzzzA 1

WW.F.AMM/M.1131 i 1

WZ.1.1.0iWZAI I 1

I

I LIZZZAMMeZMA II I 1

/ IlZer.Z./././.11 tI I I

I lerWe.1.4M1././MA II 1

I LeZZ/11.,W.e/W1 II I

I 14.1:4460./W..AMI I1 1

ILWAIJf./././VA

I I

1VeZZIAMMVZ/ZA

I 1

I wiz/m.1Percentiles of

. Performance 15th 25th 75th 95thrMean and Confidence Interval (+/-2SE)

200 250 300 .350 400 450 500 550 600 650 700 750 800 850

SOURCE: Student Achievement Data, NCREL; tables 2.2, E.3, and E.8 and figure B.4 in Mullis, I.V.S., et al. (1998). Mathematics and ScienceAchievement in the Final Year of Secondary School: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the


Nations in which more than 10 percent of the population was excluded from testing.^ Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals

were substituted.



BEST COPY AVAILABLE

91B-9

Exhibit B-9: Distributions of Advanced Mathematics Achievementin the Twelfth Grade'


Higher than FiW 1

FranceRussian Federation*Switzerland(Denmark)Cyprus*LithuaniaSweden

Same FiW

557542

533

522518516512

(3.9)(9.2)(5.0)(3.4)(4.3)(2.6)(4.4)

I ! I

I IM./..././ZA Ii1 1

1 1/.41,W. W.7.4/./1 i1 i i 1

I LIZZliZMWW.F.FA I

I 1 1

1 VZ/7./WZIZZA I1 1 1

1 LIZ(/./.7.4/ZilI

i v......)...,,I,A1 1 1

I Erzz.e... .e....../1

Ias

(Australia)Greece^CanadaFirst in the World(Slovenia)(Italy)Czech Republic

Lower than FiW

525513509490475474

469

(11.6)(6.0)(4.3)(4.9)(9.2)(9.6)

(11.2)

I I

1 11:44,7.7.177:11 11 1

1 w...w.m...-...7-,-,,,,A i1 1 1

W.T.44f1W.r./Z/JI1 1 I

I 1

I V/fiJiMMEMW,,,,VA i1 1

1 viii_____ W17 II 1 1v...," ZIA

I I

Germany^(United States)(Austria)*

465442436

(5.6)(5.9)(7.2)

I I I I

I

I1 11%.".77. Z.IZZ A j1 1 1

1 vA,.//ammor.tae;

Per6entiles ofPerformance 1

5th 25th 75th 95th

1Mean and Confidence Interval (+/-2SE)

200 250 300 350 400 450 500 550 600 650 700 750 800 850

SOURCE: Student Achievement Data, NCREL; tables 5.1, E.4, and E.9 and figure B.5 in Mullis, I.V.S., et al. (1998). Mathematics and ScienceAchievement in the Final Year of Secondary School: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the


Approximately 19 percent of the international school leaving age cohort, 14 percent of the U.S. cohort, and 65 percent of the FiW cohort were cover

by the TIMSS sample for this assessment.

* Nations in which more than 10 percent of the population was excluded from testing.^ Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals

were substituted.



B-10

Exhibit B-10: Distributions of Advanced Science (Physics) Achievementin the Twelfth Grade'


Higher than FiWNorway^SwedenRussian Federation*(Denmark)(Slovenia)Germany^(Australia)Cyprus*SwitzerlandGreece^CanadaFrance

Same FiW

581

573

545

534523

522518494488486485466

(6.5)(3.9)

(11.6)(4.2)(15.5)(11.9)(6.2)(5.8)(3.5)(5.6)(3.3)(3.8)

I I I

I ww-,-....,==ezzmA II i

1 1/77.46/7.4/./A II

1FAIZTI,Y4eVer/A II

I

I II/f/ //i0// /A I

I I I

I v........../.....vA1

Ii 1

1 V/ZZIWZJI II I

I vm-,-...v. AA 1

1 V1 /,,,M=e: /MA ITI vz...6,-..,./Aw.......0-A I

1 1

1 VIZZAMIWAeZetel I

I I

v...,,,,ww......,-...-.41 1

I VZIAMEWIA I

I Ias

Latvia (LSS)Czech RepublicFirst in the World(Austria)*

Lower than FiW

488451

446435

(21.5)(6.2)(4.1)(6.4)

Solff 11

I I

I V777.,i417.4 I

II 1.),....w....:ez.,.....A1I 1 1

I 1

1 1

(United States) 423 (3.3)I

I vix....:/zzra


5th 25th 75th 95th

200 250 300 350 400 450 500 550 600 650 700 750 800 850


SOURCE: Student Achievement Data, NCREL; tables 8.1, E.5, and E.10 and figure B.6 in Mullis, I.V.S., et al. (1998). Mathematics and Science



I Approximately.13 percent of the international school leaving age cohort, 15 percent of the U.S. cohort, and 67 percent of the FiW cohort were cover

by the TIMSS sample for this assessment.* Nations in which more than 10 percent of the population was excluded from testing. Latvia is designated LSS because only Latvian-speaking


were substituted.



B-11

Exhibit B-11: Distributions of Advanced Mathematics Achievement forTwelfth-Grade FiW AP Students'

Country . Meant Mathematics Achievement Scale Score

Same FiW APasFirst in the World AP- Lower than FiW AP

587 (4.3) I VZI.M./.4i4IZZ/71 I

I 1

FranceRussian Federation *.Switzerland(Australia) .

(Denmark)Cyprus*LithuaniaGreece^SwedenCanada(Slovenia)(Italy)Czech RepublicGermany^(United States)(Austria)*

557 (3.9)542 (9.2)533 (5.0)525 (11.6)522 (3.4)518 (4.3)516 (2.6)513 (6.0)512 (4.4)509 (4.3)475 (9.2)474 (9.6)

469 (11.2)4655.6)442 (5.9)436 (7.2)

. -

.

F, ..--.4.,1

I

I wir./Aw ...,1

1,-....w ........w...,I

I 145% lffriV.VA

zwz...wmez ..,-...wA1

I WIZ, Ze ,..vi

I ......wz....../. ,......11 1

l v ..,/./.444,11 1,/, ./.17.4 I

I

I ifz," ,-./1

I VW,/ ArZZA

1144,1:4,14444,I I/..../7.44/1


5th 25th 75th 95thUMW/ .01 r


200 250 300 350 400 450 500 550 600 650 700 750 800 850

SOURCE: Student Achievement Data, NCREL; tables 5.1, E.4, and E.9 and, figure B.5 in Mullis, I.V.S., et al. (1998). Mathematics and ScienceAchievement in the Final Year of Secondary School: lEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Center for the


Approximately 19 percent of the international school leaving age cohort, 14 percent of the U.S. cohort, and 28 percent of the FiW cohort were coverby the TIMSS sample for this assessment.

* Nations in which more than 10 percent of the population was excluded from testing.

is Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals

were substituted.



04B-12

Exhibit B-12: Distributions of Advanced Science (Physics) Achievementfor Twelfth-Grade FiW AP Students'


Same FiW APas

Norway^SwedenFirst in the World APRussian Federation*(Slovenia)Germany^

Lower FiW AP

581

573

564

545

523

522

(6.5)(3.9)(5.9)

(11.6)(15.5)(11.9)

I FIZZ///,417.0WA

1VZ/Z/ZinneMZZA

I

l 4%./.07,4,17/11 11

11

1/Z47.44.6,717/A I

1 wz.I z....":44,.-zi 1

1- than(Denmark)(Australia)Cyprus*Latvia (LSS)SwitzerlandGreece"CanadaFranceCzech Republic(Austria)*(United States)

534

518494

488488486485466451

435423

(4.2)(6.2)(5.8)(21.5)(3.5)(5.6)(3.3)(3.8)(6.2)(6.4)(3.3)

ii

1 VZ,./ZIMMIWZrA 1

I

I vz..../..,:mm/m/1il

I 1/VM,7=/...WA

/ 16,7.41./AWZMA I.i t

1 1/M./.,?7./.4071

1 iv,w.,,,:mmi 1

1

1 v...w-Amezz..A1 I

I IIZZ/V1=4/A II

I ILIZZ.4.0%/A I

_..

Percentiles ofE Performance

5th 25th 75th 95th

200 250 300 350 400 450 500 550 600 650 700 750 800 850


SOURCE: Student Achievement Data NCREL; tables 8.1, E.5, and E.10 and figure B.6 in Mullis, I.V.S.,etal. (1998). Mathematics and Science



Approximately 13 percent of the international school leaving age cohort, 15 percent of the U.S. cohort, and 7 percent of the FiW cohort were covere

by the TIMSS sample for this assessment.* Nations in which more than 10 percent of the population was excluded from testing. Latvia is designated LSS because only Latvian-speaking


were substituted.



95.B-13

United StatesDepartment of Education

Washington, DC 20208-5573

Official BusinessPenalty for Private Use, $300

SA: 1999-3011

647

Postage and Fees PadU.S. Department of Education

Permit No. G-17

Standard Mail (A)

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

IC

This document is covered by a signed "Reproduction Release(Blanket) form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

u7 This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Release form(either "Specific Document" or "Blanket").

EFF-089 (9/97)