the effects of teachers qualifications-saudi taiwan

This article was downloaded by: [The University Of Melbourne Libraries]On: 15 May 2015, At: 07:55Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

International Journal of TestingPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hijt20

The Effects of TeachersQualifications, Practices,and Perceptions on StudentAchievement in TIMSSMathematics: A Comparison ofTwo CountriesHamzeh Dodeen a , Faisal Abdelfattah b , SalehShumrani b & Maher Abu Hilal ba Psychology and Counseling Department , UnitedArab Emirates University , United Arab Emiratesb The Excellence Research Center of Science andMathematics Education , King Saud University ,Saudi ArabiaPublished online: 23 Jan 2012.

To cite this article: Hamzeh Dodeen , Faisal Abdelfattah , Saleh Shumrani & MaherAbu Hilal (2012) The Effects of Teachers Qualifications, Practices, and Perceptionson Student Achievement in TIMSS Mathematics: A Comparison of Two Countries,International Journal of Testing, 12:1, 61-77, DOI: 10.1080/15305058.2011.621568

To link to this article: http://dx.doi.org/10.1080/15305058.2011.621568

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the Content) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, and

are not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

International Journal of Testing, 12: 6177, 2012Copyright C Taylor & Francis Group, LLCISSN: 1530-5058 print / 1532-7574 onlineDOI: 10.1080/15305058.2011.621568

The Effects of Teachers Qualifications,Practices, and Perceptions on StudentAchievement in TIMSS Mathematics:

A Comparison of Two Countries

Hamzeh DodeenPsychology and Counseling Department, United Arab Emirates University,

United Arab Emirates

Faisal Abdelfattah, Saleh Shumrani, and Maher Abu HilalThe Excellence Research Center of Science and Mathematics Education,

King Saud University, Saudi Arabia

This study focused on comparing mathematics teachers qualifications, practices,and perceptions between Saudi and Taiwanese schools. Data analyzed in thisstudy were the responses of mathematics teachers to the Teacher BackgroundQuestionnaire8th Grade from the Trends in International Mathematics and Sci-ence Study (TIMSS) in 2007. The Saudi sample consisted of 171 teachers whilethe Taiwanese sample consisted of 152 teachers. The comparison between the twocountries revealed that there were significant differences in teachers preparation forteaching specific mathematics topics, professional development programs, and inteachers perceptions about the effects of school environment on students TIMSSscores. In addition, the two countries results differed in the mathematics topics thathad not been taught to students, in assessment tools commonly used in mathemat-ics, and in the type of questions used in tests. Some teachers qualifications andpractices were found to be related to students scores. Results are discussed andrecommendations for educators and policymakers are proffered.

Keywords: achievement, international testing, mathematics, teachers practices,teachers qualifications, TIMSS

Correspondence should be sent to Hamzeh Dodeen, Psychology and Counseling Department,United Arab Emirates University, PO Box 17771, Al Ain, United Arab Emirates. E-mail: [email protected]

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

62 DODEEN ET AL.

INTRODUCTION

International large-scale assessment has been increasingly used to describe whatstudents know and can do and to provide data for international comparisons incontent domains such as reading, mathematics, and science. The Trends in Inter-national Mathematics and Science Study (TIMSS) is the largest and most ambi-tious international assessment study of educational achievement in mathematicsand science. In addition to achievement and skills, TIMSS collected detailed in-formation on application and implementation of curricula for mathematics andscience. It also collected information about educational systems, school charac-teristics, teacher quality, teachers qualifications and characteristics, instructionalpractices, availability of teaching resources, and the use of technology in teachingand learning activities.

Teacher quality is a key determinant of student learning and achievement(Fallon, 1999; Metzler & Woessmann, 2010). A meta-analysis by Hedges, Laine,and Greenwald (1994) found a positive relationship between teacher qualificationsand practices and student outcomes. Previous studies have also suggested thatthe effect of teachers qualifications on student achievement is positive and real(Greenwald, Hedges, & Laine, 1996; Nye, Konstantopoulos, & Hedges, 2004).According to Goe (2007), teachers qualifications are effective in identifyingteachers who improve their students scores, especially in mathematics.

Teacher qualifications refer to the credentials, knowledge, and experiencesthat a teacher brings to the job. In the context of this study, it includes knowledgein mathematics, mathematics pedagogy, level of preparation, professional devel-opment, and experience in teaching. Quite related to this, is the term Teacherpractices, which refers to classroom practices that the teacher employs such asthe teaching strategies and the assessment activities (Goe & Stickler, 2008).

The effects of such teacher factors on student achievement, especially in mathe-matics have been previously studied and discussed. Several studies have indicatedthat knowledge of mathematics affects students learning at all school levels (Met-zler & Woessmann, 2010; Park & Leung, 2003). Mathematic pedagogy is alsofound to be one of the strongest predictors of student achievement (Hill, Rowan,& Ball, 2005). Quite a few studies have also suggested positive effects of teacherexperience and teacher education on student achievement (Greenwald, Hedges,& Laine, 1996). Monk and King (1994) reported that teachers subject matterpreparation in mathematics and science does have a positive impact on studentachievement in those subjects. Similarly, Goldhaber and Brewer (1997) concludedthat teachers subject-specific training has a significant impact on student testscores in mathematics and science. Teachers professional development is anothercritical factor in school improvement and student learning. Indeed, professionaldevelopment helps teachers acquire knowledge and skills that they bring intoaction in classroom practice (Borger & Tillema, 1993; Cohen & Hill, 2000).

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

TEACHERS QUALIFICATIONS AND STUDENT ACHIEVEMENT IN TIMSS 63

Teachers who are trained and well-prepared are more effective in the classroomand therefore have the greatest impact on student learning (Killion, 1999).

On the other hand, some studies found no beneficial relationship betweenteacher qualifications/practices and student achievement. For example, studies byAndrews, Blakmon, and Mackey (1980) and Haney, Madaus, and Kreitzer (1987)found only a slight relationship between teachers knowledge of mathematics andstudent achievement. Likewise, studies by Harbison and Hanushek (1992) andJacob and Lefgren (2004) found that professional development does not contributeto student achievement. As for teaching experience, there are studies that have notdetected important differences between comparably more or lesser experiencedteachers (e. g., Carr, 2006; Gallagher, 2004).

While research literature on the effects of teacher qualifications and practices onstudent achievement are mixed, teachers quality seems to be a key factor in theteaching-learning process. As shown by Goe and Stickler, (2008), some teacherscontribute more to their students achievement than others. Because teachers aredifferent in their effectiveness, understanding the factors that make some teachersmore effective than others is increasingly interesting to educators and researchers.It has to be noted here that there is a critical lack of research that links teachersqualifications and practices with students achievement (Guarino et al., 2006).

Certainly, student achievement in international assessment tests is influencedby many variables that include but are not limited to students, schools, teachers,parents, socioeconomic status, culture, curriculum, and others (Freguson, 1991;Greenwald, Hedges, & Laine, 1996). Comparing countries, especially those withconsiderable differences in overall performance, helps to assess the influence ofeach variable. In this study, teachers qualifications, practices, and perceptionsand their relationships to students scores on the TIMSS mathematics test wereanalyzed. The analysis was carried out with the data obtained from TIMSS 2007from Saudi Arabia and Taiwan. The national performance of Saudi Arabia in thistest was one of the lowest as compared to other participating countries. In fact,Saudi Arabias average scale score was 329 with a standard deviation of 76 (theaverage of the scale is 500), which placed the countrys ranking as 46th out of48. Analyzing the qualifications and practices of mathematics teachers may wellexplain why the countrys average performance was very low.

To understand how teachers qualifications, practices, and perceptions differbetween very low-achieving and very high-achieving countries, the responsesof Saudi teachers were compared with corresponding responses of Taiwaneseteachers. Saudi Arabia was of particular interest to the authors, given where theywork. Taiwan was selected due to its outstanding national performance in TIMSS2007 mathematics, which was rated as the best in the world. The average scalescore of Taiwan was 598, which placed the country on the first rank. It may alsobe noted that, among the five highest achieving countries (Taiwan (598, SD =106), Republic of Korea (597, SD = 92), Singapore (593, SD = 94), Hong Kong

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

64 DODEEN ET AL.

SAR (572, SD = 94), and Japan (570, SD = 85)), the population of Taiwan at 23million was the closest to the Saudi population of 28million.When comparing twocountries like Saudi Arabia and Taiwan, an important concern that should be givendue consideration is the obvious cultural, educational, and economical differencesbetween the two countries. Evidently, these factors do affect students academicperformance at the individual level as well as at the national level. Therefore, it isnot argued here that the differences in students performance are attributed solelyto teacher qualifications, practices, and perceptions. However, it is still useful tosee how such important factors are different in countries with markedly significantdifferences in their national performance.

The objectives of international studies such as TIMSS have little to do withconstructing a league table of countries. They focus on identifying factors thatcontribute to high achievement and in understanding the practices of other coun-tries for the sake of improving the education in ones own country (Park &Leung, 2003). Teachers play significant roles in the teaching-learning processand in helping students achieve their learning goals and outcomes. Understand-ably, how teachers qualifications and practices affect students performance intests could be useful information for understanding the performance of eachcountry.

Using Saudi and Taiwanese mathematics teachers responses to the TeacherBackground QuestionnaireTIMSS 2007, this study aimed at achieving the fol-lowing objectives:

1. To compare between Saudi and Taiwanese teachers qualifications, prac-tices, and perceptions, in the context of students scores on TIMSS;

2. To statistically test the differences between the two countries scores withregard to teachers preparation for teaching, teachers professional develop-ment, and the correlations between teachers perceptions and their studentsTIMSS scores; and

3. To determine the relationship betweenmathematics teachers qualifications,practices, and perceptions and students TIMSS scores.

METHOD

TIMSS in BriefTIMSS was first administered in 1995 and has been conducted every four yearssince in 1999, 2003, and 2007 by the International Association for the Evaluationof Educational Achievement (IEA). The main purpose of TIMSS is to provideeducational policymakers and educators with reliable and timely data as a basefor better understanding of the performance of their educational systems (Mullis,

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


2003). TIMSS is used to measure over time the mathematics and science knowl-edge and skills of 4th and 8th grade students. The data are used to describe apopulations performance but are not used to assign individual test scores to stu-dents. Forty eight countries participated in the TIMSS 2007, on the 8th grademathematics and science achievement tests. The tests consisted of 12 bookletsdistributed among students in a scheme that ensures that each booklet will be an-swered by comparable random samples of students (Gonzalez, Galia, & Li, 2004).

In TIMSS 2007 each student responded to only a portion of the items and notthe entire assessment item pool. Therefore, the test achieved wide coverage ofcontent representation when responses are aggregated across respondents whilekeeping the response load of individual students to a minimum. To provide anaccurate estimate of achievement and precise measures of trends from previousassessments, TIMSS relied on Item Response Theory (IRT) scaling. TIMSS IRTscaling used what is called Plausible Values methodology to estimate studentsScores. Plausible values are based on student responses to the subset of itemsthey receive as well as on other relevant and available background information(Mislevy, 1991). This methodology increases the accuracy of the estimates of thecharacteristics of whole population and subpopulations of students (Gonzalez,Galia, & Li, 2004; Yamamoto & Kulick, 2000). TIMSS produced five plausiblevalues for each student in mathematics. There was no superiority of any plausiblevalue over the others, but only the first value was used in TIMSS reports (Wang,2001). The scores are reported on a scale from 0 to 1000, with the TIMSS scaleaverage set at 500 and standard deviation set at 100.

Instrument

TIMSS Teacher Background Questionnaire collected information about back-ground, preparation, and professional development of participating teachers. Italso asked about instructional activities and collected very detailed informationabout the classroom context and the mathematical topics taught to students. Foreach sampled school at the 8th grade, a single mathematics class was sampledfor the TIMSS 2007 assessment. Questions pertaining to instructional and assess-ment, content coverage, and teachers views about teaching the subject matterwere tailored for mathematics. The TIMSS 2007 teacher questionnaire was de-signed to take about 45 minutes to complete. The questionnaire was field tested inMarch and April 2006 in 45 participating countries. One of the primary purposesof the field test was to check across participating countries whether the question-naire were appropriate for the measurement purposes for which it was designed.When answering the questionnaire, teachers were asked to refer specifically to theclass of students selected for the TIMSS 2007 assessment (TIMSS and PIRLSInternational Study Center, 2010).

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

66 DODEEN ET AL.

ParticipantsData analyzed in this study were the responses of both Saudi and Taiwanese math-ematics teachers to the TIMSS Teacher Background Questionnaire8th Grade.The Saudi sample consisted of 171 teachers (88 females (52%) and 83 males(48%)) who completed the questionnaire andwhose students took the 2007 TIMSSmathematics achievement test. The average teaching experience of the sample ofteachers was around 11 years. Most of the teachers (73.6%) were aged between2539 years and 5% were older than 50 years. Most of the teachers (96%) hadcompleted university but not a postgraduate degree and only 1% had completed apostgraduate degree.

The Taiwanese sample consisted of 152 teachers (84 females (56.4%) and 65males (43.6%)). Themajority (86%)were aged 2549 years and 12%were 50 yearsor older. Their average teaching experience was approximately 12 years. Most ofthe teachers (72%) had completed university but not a postgraduate degree and22% had completed a postgraduate degree.

RESULTS

The results of this study are presented as follows. First, teachers preparation lev-els for teaching specific topics in mathematics in Saudi Arabia and Taiwan aresummarized and compared. Second, teachers participation in professional devel-opment programs in the past two years is compared. Third, teachers perceptions ofthe effects of school environment on students scores are assessed and compared.Fourth, the mathematical topics that are included in the test and not taught in theclasses which took TIMSS 2007 are identified and compared between the twocountries. Finally, mathematics assessment practices that include number and typeof test (classroom vs. national) and question type (facts, applications, searching forpatterns, and explanation/justification) are compared between the two countries.

Preparation for TeachingTeachers were asked to self-evaluate their preparation level for teaching specificmathematical topics. There were 18 topics listed in the four mathematics areascovered by TIMSS 2007: Numbers, Algebra, Geometry, and Data. Responses oneach topic were listed in three categories: very well-prepared, somewhat prepared,and not well-prepared. The results of this question are presented in Tables 1, 2, 3,and 4, organized in terms of the respective mathematical area. Table 1 summarizeslevels of teachers preparation in teaching five mathematical topics in Numbers.The percentages indicated that teachers generally were well prepared for teachingthe topics listed in the table in both countries. The average percentage of thelevel Very well Prepared for Saudi Arabia and Taiwan were 77.2% and 82.9%,

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


TABLE 1Teachers Level of Preparation for Teaching Numbers

Saudi Arabia Taiwan

Very well Somewhat Not well Very well Somewhat Not wellTopic prepared prepared prepared prepared prepared prepared

Using integers 86.0% 8.4% 0.6% 86.7% 13.3% 0.0%Whole numbers 82.6% 11.8% 0.5% 82.7% 17.3% 0.0%Computation and fractions 79.8% 12.9% 1.1% 85.3% 14.7% 0.0%Decimals and fractions 70.2% 18.5% 1.7% 83.3% 16.7% 0.0%Problem solving 67.4% 23.6% 1.7% 76.7% 23.3% 0.0%Average 77.2% 16.7% 1.1% 82.9% 18.3% 0.0%

respectively. In order to statistically ascertain whether preparation of teachers wassimilar in both countries, a 2 test was conducted using the average percentages ofteachers in the three preparation levels. The result was not statistically significant,(2 (2, 316)= 1.90; p= .39). This means that, on average, preparation of teachersin the area of Numbers is similar in both countries.

As for Algebra (Table 2), results of the two countries were clearly differentin one of the four topics listed above which is Patterns Sequences. Only 37.6%of Saudi teachers stated that they were well prepared for teaching this topic ascompared to 80.7% in Taiwan. The difference was statistically significant: (2 (2,288) = 40.01; p < .01). The average percentage of the level Very well Preparedfor all Algebra topics for Saudi Arabia was 68.5%, which is comparatively muchlesser than that of Taiwan (81.5%). Using the percentages of the averages, a chi-square test was conducted and the result was not statistically significant: (2 (2,301) = 4.19; p = .123).

The results for the area Geometry (Table 3) indicated that Saudi teacherswere less prepared for teaching at least two topics: Relationships among Shapes

TABLE 2Teachers Level of Preparation for Teaching Algebra

Saudi Arabia Taiwan


Simple expressions 80.9% 12.4% 0.6% 85.3% 14.7% 0.0%Equivalent representations 78.7% 12.9% 1.1% 74.7% 25.0% 0.0%Simple equations 72.5% 17.4% 2.8% 85.3% 14.7% 0.0%Patterns sequences 37.6% 37.1% 5.6% 80.7% 19.3% 0.0%Average 68.5% 16.0% 2.5% 81.5% 19.1% 0.0%

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

68 DODEEN ET AL.

TABLE 3Teachers Level of Preparation for Teaching Geometry

Saudi Arabia Taiwan


Congruent figures 83.7% 9.0% 1.1% 78.0% 22.0% 0.0%Geometrical properties 79.8% 14.0% 0.6% 84.0% 16.0% 0.0%Translate and reflect 65.7% 21.3% 4.5% 45.9% 48.0% 0.7%Cartesian plane 60.7% 26.4% 3.4% 69.1% 28.2% 0.7%Measurement formulas 56.7% 27.5% 5.6% 79.7% 19.6% 0.0%Relationships between shapes 31.5% 33.7% 11.2% 61.3% 38.0% 0.7%Average 60.0% 22.0% 4.4% 69.7% 28.6% 0.4%

(31.5%) and Measurement and Formulas (56.7%). The differences between thetwo countries on these topics were statistically significant: (2 (2, 279) = 28.19;p < .01 for Relationships among Shapes and 2 (2, 324) = 17.87; p < .01 forMeasurement and Formulas). In contrast, Taiwanese teachers were well-preparedin all of the six mathematics topics listed. The chi-square test was conducted usingthe percentages of the averages and the result was not statistically significant (2

(2, 299) = 5.79; p = .06).Finally, the results of the fourth mathematical area, Data, are summarized

in Table 4. As can be observed, only 36.5% of Saudi teachers were prepared forteaching Interpretation Data Sets, 43.3% for Judging and Chancing, and 55.6% forRead and Display Data. On the other hand, Taiwanese teachers preparation wasbetter than that of Saudi teachers on each of the three Data topics. This resultmay also be surmised from the differences in the averages of the percentagesbetween the two countries. For example, the average percentage of the level Verywell prepared for Saudi Arabia was 45.1%while it was 63.0% for Taiwan. The 2

test was also conducted here using the average percentages of teachers. The result

TABLE 4Teachers Level of Preparation for Teaching Data

Saudi Arabia Taiwan


Read and display data 55.6% 26.4% 6.2% 74.7% 24.7% 0.7%Judging and chancing 43.3% 34.8% 9.0% 62.0% 35.3% 0.0%Interpretation data sets 36.5% 38.8% 9.6% 52.3% 41.6% 1.3%Average 45.1% 33.3% 8.3% 63.0% 33.9% 0.7%

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


TABLE 5Teachers Participating in Professional Development Programs in the Previous Two Years

Professional Development Topic Taiwan Saudi Arabia Difference

Curriculum 85.4% 19.9% 65.5%Content 84.8% 23.2% 61.6%Information technology in mathematics 74.8% 23.1% 51.7%Pedagogy 79.5% 45.1% 34.4%Assessment 52.3% 24.7% 27.6%Critical thinking and problem solving 41.7% 34.0% 7.7%Average 69.8% 28.3% 41.5%

was statistically significant (2 (2, 297) = 13.58; p < .01) from which we mayinfer that teachers of Saudi Arabia and Taiwan are not similar in their preparationof teaching Data.

Professional DevelopmentTo examine this variable, teachers were asked to determine whether they had par-ticipated in professional development programs in the previous two years. Thetopics of the professional programs included mathematics content, instructions,curriculum, information technology into mathematics, critical thinking and prob-lem solving skills, and mathematics assessment. The results of teachers responsesto this question are displayed in Table 5.

Table 5 shows that most mathematics teachers in Saudi Arabia did not par-ticipate in any professional development programs in the previous two years.Specifically, on average and over the six topics listed, only 28.3% of Saudi teach-ers participated in some professional development programs. In contrast, around70% of Taiwanese teachers had the opportunity to participate in one or more ofthese programs. The difference of percentage participating between the two coun-tries ranged from 7.7% to 65.5%. As mentioned before, teachers professionaldevelopment is associated with improving student TIMSS scores. Students whoseteachers had participated in professional programs, scored better than studentswhose teachers did not participate in such programs. The effect of teachers par-ticipation in professional development programs on students TIMSS scores wasstatistically tested for the two countries. There were six professional develop-ment topics listed on the questionnaire: Content, Pedagogy, Curriculum, IT ontoMathematics, Critical Thinking, and Mathematics Assessment. For each topic, atwo-way ANOVA test was conducted with the factors being country and participa-tion in professional development programs (yes/no) while the dependent variablewas the average TIMSS scores. The results (summarized in Table 6) indicatedthat the differences between the two countries were statistically significant over

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

70 DODEEN ET AL.

TABLE 6Two-Way ANOVA Results: Country by Participating in Professional Development Programs

ProfessionalDevelopment MeanProgram Source DF Square F P 2

Content Country (1, 311) 3321428.21 317.24 .04 .7910469.62

Participating 2444.37 .23 .71 .0010469.62

Country Participating 10469.62 3.71 .06 .002819.60

Pedagogy Country (1, 309) 4189615.54 5297.70 .00 .83790.84

Participating 234.83 .30 .68 .00790.84

Country Participating 790.84 .28 .60 .002866.01

Curriculum Country (1, 308) 3105731.35 197.87 .05 .7715695.62

Participating 3118.19 .20 .73 .0015695.62


IT into Mathematics Country (1, 307) 3852231.99 8123.71 .01 .81474.20

Participating 28.94 .06 .85 .00474.20

Country Participating 474.20 .17 .68 .002854.66

Critical Thinking Country (1, 309) 4833502.50 254.23 .04 .8419012.11

Participating 1385.65 .07 .83 .0019012.11


Assessment Country (1, 309) 4595574.86 1391.43 .02 .843302.77

Participating 7566.36 2.29 .37 .003302.77


all professional development programs except Curriculum (P = .05). Moreover,the effect size values, as measured by eta-squared, indicated that the variablecountry explained most of the variance of the students scores in the listed pro-fessional development programs. Additionally, there was a statistically significant

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


interaction between country and participating in two programs: Curriculum andCritical Thinking. However, the effect size values indicated that this interaction ispractically not significant in explaining the variance of students scores.

The direction of the interaction between country and participating (yes/no)was the same for both Curriculum and Critical Thinking. This direction was asfollows: For Saudi Arabia, the scores of students whose teachers participated inthe professional development programs were higher than the scores of studentswhose teachers did not participate. For Taiwan, the direction of this interactionwas reversed.

Teachers Perceptions About School EnvironmentTeachers perceptions about the effects of school environment, parents, and stu-dents on scores in mathematics were assessed. The list consisted of eight factors,each of which was measured on a scale that ranged from very high to verylow. The results of teachers responses to these factors are displayed in Table 7.

It is clear from these results that both Saudi and Taiwanese teachers were nothappy with several factors that affect student scores in mathematics. From theteachers perspective, parental support for students accomplishments, parentalinvolvement in school activities, students respect for school, and students desire

TABLE 7Teachers Perceptions of School, Parents Role, and Students and Their Correlations with

Students TIMSS Scores

Saudi Arabia Taiwan

High or Correlation High or CorrelationVery with Very with

Factor High Achievement High Achievement

Teachers success in applyingcurriculum

81.1% .12 86.8% .14

Teachers understanding ofcurriculum goals

73.8% .06 74.2% .14

Teachers job satisfaction 60.4% .08 55.3% .04Teachers expectation of students 58.8% .09 72.2% .28Students respect for school 22.0% .02 13.9% .20Parental support for studentaccomplishments

21.9% .21 45.7% .28

Students desire of learning 21.8% .07 35.1% .21Parental involvement in schoolactivities

15.2% .07 25.8% .25

Note. p < .05. p < .01.

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

72 DODEEN ET AL.

of learning were less than what they were expecting in both the countries. Inaddition, the correlation between each factor and students scores in mathematicswas calculated. For Saudi Arabia, parental support for students accomplishmentwas the only factor found to be significantly correlated with the scores (r = .21,p < .01). This correlation, however, could explain only 4% of students scores.Correlations with the other factors were small and statistically not significant.As for Taiwan, the factors: teachers expectations of student, parental support,parental involvement, students respect for school, and students desire of learninghad statistically significant correlations with TIMSS scores. The relative strengthof these correlations ranged from 4% to 8%. Moreover, all these correlations arehigher than their counterparts in Saudi Arabia.

Teachers were also asked to identify the factors that they thought obstructedtheir teaching. Saudi teachers identified the following seven factors: shortage ofsoftware (34.8%), shortage of hardware (32.5%), shortage of computer use sup-port, students interest (28.7%), demonstration equipment (25.6%), instructionalequipment (20.7%), and student disruptiveness (16.0%). These factors can beclassified into two areas: equipment shortage (which includes computer softwareand hardware) and students. According to the results in Table 6, it appears that stu-dents behaviors and attitudes are factors that can seriously and negatively affectteacher performance in classes. As for Taiwanese teachers, only three factors wereidentified as obstacles in their teaching. These were different students academicabilities (47.0%), student interest (43.75%), and student disruptiveness (25.8%).Hence, the obstacles in Taiwanese classes were fewer than those in Saudi Arabia,and there were no equipment problems in Taiwanese classes. In general, mathe-matics teachers from both countries were displeased with some of their studentsattitudes.

Teaching MathematicsGiven that TIMSS is an achievement test, it is assumed that students should betested only on the topics and materials that have already been taught in class. Thecontent domains of the 8th grade TIMSS 2007 mathematics test were Numbers(30%), Algebra (30%), Geometry (20%), and Data and Chance (20%). Teacherswere asked to determine whether the mathematical topics included in the test weretaught in the class who took TIMSS 2007. The question contained 10 topics inNumbers, 8 topics in Algebra, 14 topics in Geometry, and 7 topics in Data. A topicwas considered not taught if more than 50% of teachers said so.

The results in Table 8 indicate that many of the test topics had not been taughtto the Saudi students or they had just been introduced. Specifically, five topicsin Algebra, seven topics in Geometry, and six topics in Data had either not beentaught or had just been introduced. This means that what was not taught in SaudiArabia represented a large portion of the TIMSS 2007 content. In contrast, the

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


TABLE 8Mathematical Topics Not Yet Taught or Just Introduced in Classes

Not yet taught Not yet taughtMathematical or just introduced or just introducedArea Topic (Saudi Arabia) (Taiwan)

Algebra Numeric patterns 79.1% 5.3%Evaluation function 78.7% 15.4%Equivalent representation 74.0% 16.8%Model situation 67.9% 1.3%Linear equation 64.9% 2.7%

Geometry Pythagorean theory 90.3% 1.0%Relationships 85.5% 29.3%Measurement areas 80.5% 48.3%Measurement forum 79.9% 14.8%2D Symmetry 79.2% 31.5%Measurement drawing 67.5% 12.8%Cartesian plan 56.7% 34.5%Similarity 48.5% 74.0%Translation 29.7% 71.6%

Data Using chances 76.5% 98.7%Predict chances 86.0% 97.3%Interpreting data 80.4% 94.7%Characteristics 77.4% 92.0%Organizing data 61.6% 88.7%Reading data 62.0% 88.0%

number of topics that had not been taught in Taiwan was fewer than those in SaudiArabia. As indicated by the majority of teachers, only two topics in Geometry andsix topics in Data were not taught in Taiwanese classes. From the results it appearsthat all Data topics had not been taught in TIMSS classes in both countries. Thisissue should be considered in future TIMSS tests.

Assessing MathematicsThe results from analyzing Saudi teachers responses about their assessment prac-tices revealed that teachers emphasized classroom tests (61.3%) much more thannational or regional tests (26.1%). Similar results were observed with Taiwaneseclasses. Although teachers emphasized the importance of tests in both countries,44% of Saudi teachers conducted only one test per month and 31.7% conducteda test once every two weeks. The last result was very different to that in Taiwanwhere 89.9% of mathematics teachers conducted a test every week.

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

74 DODEEN ET AL.

As for the type of questions used in tests, teachers were asked to determinehow many times they included facts, applications, searching for patterns, and ex-planation/justification in their tests. The results indicated that a high percentageof Saudi teachers included facts and applications in their tests (57.1% and 68.3%,respectively). On the other hand, few teachers indicated that they include search-ing for patterns and explanation/justification in their tests (10.9% and 14.3%,respectively). The results of Taiwanese teachers were facts (25.8%), applications(60.3%), searching for patterns (31.8%), and explanation/justification (17.2%).So, while questions of the facts type were the most used by Saudi teachers thoseinvolving searching for patterns and explanation/justification were the most usedby Taiwanese teachers.

DISCUSSION

Regardless of school level or subject matter, teachers play a significant role instudent achievement. As stated by Metzler andWoessmann (2010), understandingthe effects of teachers on student achievement is important because of its sub-stantial economic impact. Teachers qualifications and practices should, therefore,be considered seriously by educators and policymakers. Many interesting resultshave been revealed from the comparative analysis of Saudi and Taiwanese teach-ers responses. The following discussion will focus only on the results where cleardifferences between the two countries were identified. First, Saudi teachers werenot well prepared to teach several mathematics topics in Algebra, Geometry, andData, while Taiwanese teachers were quite adequately prepared to teach all topics.The differences of the preparedness between the two countries were found to bestatistically significant. Educators and policymakers should consider this resultseriously. Teachers should be very well-prepared to teach all curriculum topicsespecially at this level. Studies have shown that teachers preparation and subjectknowledge have a significant impact on student achievement (Eide, Goldhaber, &Brewer 2004; Greenwald, Hedges, & Laine, 1996; Hanushek & Rivkin, 2006; Nyeet al., 2004).

Second,most Saudimathematics teachers did not participate in any professionaldevelopment programs in the previous two years, while the majority of Taiwaneseteachers had participated in such programs. Results indicated that teachers partici-pation in professional development programs improved students scores in TIMSS.Quality professional development programs can help teachers acquire knowledgeand skills, which can be applied into action in classroom practice (Borger &Tillema, 1993). Studies have indicated that specific types of professional devel-opment programs contribute to teacher quality and student achievement (Goe,2007). Specifically, professional development that is sustained, aligned with thecurriculum, and focused on instruction is shown to be positively associated with

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


student achievement in mathematics in both elementary and high school levels(Wenglinsky, 2002).

Third, many of the mathematics test topics had not been taught to the SaudiTIMSS students or they had just been introduced. The list included five topics inAlgebra, eight topics in Geometry, and all topics in Data. This means that mostof the topics of the test were not learned well by students. In contrast, only a fewtopics were missed out in Taiwanese classes. Evidently, this result is negativelyassociated with student performance. As the TIMSS is an achievement test thatmeasures what students have achieved in the past, clearly, students should not betested in materials or topics that have not firstly been taught.

Fourth, both Saudi andTaiwanese teacherswere not happywith parental supportfor students accomplishments, parental involvement in school activities, studentsrespect for school, and students desire of learning. The correlations between eachof these factors and students scores for Taiwanwere higher than their counterpartsfor Saudi Arabia. Additionally, Saudi teachers identified seven factors that theythought obstructed their teaching while Taiwanese teachers identified only three.Of course, the more obstacles the teachers perceive the less effective they become.

Fifth, results from both countries indicated that teachers emphasized class-room tests much more than national or regional tests. Nevertheless, the majorityof Saudi teachers conducted only one test per month while 89.9% of Taiwanesemathematics teachers conducted a test every week. Given this last result, and thefact that tests are important (in regulation and control, evaluating student progress,motivating students, organizing knowledge and skills, and offering unique learn-ing experiences), it is recommended that Saudi teachers conduct more tests inmathematics classes. However, the effect of this procedure on students achieve-ment in mathematics is not confirmed by the results of this study and needs to beempirically tested.

As for question format, teachers need to include all types of questions such assearching for patterns and explanation/justification. In addition, students shouldhave enough opportunities to work on new procedures and situations rather thanpracticing routine procedures. For example, Grouws and Cebulla (2000) noticedthat students at the 8th grade level in typical Japanese classrooms spent approxi-mately 15% of instructional time applying procedures in new situations and 45%inventing new procedures and analyzing new situations.

Before concluding the discussion of the results of this study, some limitationsshould be addressed. First, the study relied solely on the responses of mathematicsteachers to TIMSS Teacher Background Questionnaire. The self-report natureof the questionnaire may have led to different results than those that may beobtained using other data collectingmethods. Second, curriculummismatch acrosscountries can have a direct impact on the international comparisons of studentachievement (Wang, 2001). This may also give misleading results about studentperformance, abilities, and skills in some countries.

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

76 DODEEN ET AL.

In conclusion, the study showed that quality and practice differences were notedacross the two countries that were ranked at the extremes on the TIMSS achieve-ment scale. Although all these differences could not be quantified accurately, theypartially explain the low performance of some countries in international tests.Educators and policymakers in these countries may be well advised to place spe-cific focus on teachers qualifications, practices, and perceptions to improve theirstudents scores. More specifically, teachers should be prepared to teach all math-ematics topics, they should continuously participate in professional developmentprograms, and they need more positive support and respect from both parents andstudents. As for participating in TIMSS, the topics in the test should be reviewedfirst to make sure that students have been taught all these topics.

REFERENCES

Andrews, J. W., Blackmon, C. R., & Mackey, J. A. (1980). Pre-service performance and the NationalTeacher Examination. Phi Delta Kappan, 61(5), 358359.

Borger, H., & Tillema, H. (1993). Transferring knowledge to classroom teaching: Putting knowledgeinto action. Research on Teacher Thinking: Understanding Professional Development. London, UK:Flamer Press.

Carr, M. (2006). The determinates of student achievement in Ohios public schools (Policy Re-port). Columbus, OH: Buckeye Institute for Public Policy Solutions. Retrieved from http://www.buckeyeinstitue.org/docs/Policy Report Determinats of Student Achievement in Ohio.pdf

Cohen, M., & Hill, H. (2000). Instructional policy and classroom performance: The mathematicsreform in California. Teacher College Record, 102(2), 294343.

Eide, E., Goldhaber, D., & Brewer, D. (2004). The teacher labor market and teacher quality. OxfordReview of Economic Policy, 20(2), 230244.

Fallon,D. (1999).Our grand opportunity: Remarks on teacher education for college and university chiefexecutives. Paper presented at the Presidents Summit on Teacher Quality, University of Maryland.Retrieved from http://www.ed.gov/inits/teachers/conferences/fallon.html

Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money matters.Harvard Journal on Legislation, 44(1), 5563.

Gallagher, H. A. (2004). Vaughn Elementarys innovative teacher evaluation system: Are teacherevaluation scores related to growth in student achievement? Peabody Journal of Education, 79(4),79107.

Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis. Washing-ton, DC: National Comprehensive Center for Teacher Quality. Retrieved from http://www.ncctp.org/publications/

Goe, L., & Stickler, L. (2008). Teacher quality and student achievement: Making the most recentresearch (TQ Research and Policy Brief). Washington, DC: National Comprehensive Center forTeacher Quality. Retrieved from http://www.tqsource.org/publications/March2008Brief.pdf

Goldhaber, D., & Brewer, D. (1997). Evaluating the effect of teacher degree level on educationalperformance. InW. Fowler (Ed.),Developments in School Finance, 1996 (pp. 197210).Washington,DC: NCES.

Gonzalez, E. J., Galia, J., & Li, I. (2004). Scaling methods and procedures for the TIMSS 2003mathematics and science scales. In M. O. Martin, I. V. Mullis, & S. J. Chrostowski (Eds.), TIMSS

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015


2003 Technical Report. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, BostonCollege.

Greenwald, R., Hedges, L. V., & Laine, R. D. (1996). The effect of school resources on studentachievement. Review of Educational Research, 66, 361396.

Grouws, D. A., & Cebulla, K. J. (2000). Improving student achievement in mathematics. Retrievedfrom the ERIC database (ED 463952).

Guarino, C., Hamilton, L., Lockwood, J., Rathbun, A., & Hausken, E. (2006). Teacher qualifications,instructional practices on reading and mathematics gains in kindergartners. U.S. Department ofEducation. Retrieved from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006031

Haney, W., Madaus, G., & Kreitzer, A. (1987). Charms talismanic: Teaching teachers for the improve-ment of American Education. In E. Z. Rothkopf (Ed.), Review of research in education (Vol. 14,pp. 169238). Washington, DC: American Educational Research Association.

Hanushek, E. A., & Rivkin, S. (2006). Teacher quality. In E. A. Hanushek& F.Welch (Eds.),Handbookof the economics of education (2nd ed., pp. 10511078). Amsterdam, The Netherlands: North-Holland.

Harbison, R. W., & Hanushek, E. A. (1992). Educational performance of the poor: Lessons from ruralnortheast Brazil. New York, NY: Oxford University Press.

Hedges, L., Laine, R., & Greenwald, R. (1994). A meta-analysis of the effects of differential schoolinputs on student outcomes. Educational Researcher, 23(3), 514.

Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers mathematical knowledge for teachingon student achievement. American Educational Research Journal, 42(2), 317406.

Jacob, B. A., & Lefgren, L. (2004). The impact of teacher training on student achievement: Quasi-experimental evidence from school reform efforts in Chicago. Journal of Human Resources, 39(1),5079.

Killion, J. (1999). What works in the middle: Results-based staff development. Oxford, OH: NationalStaff Development Council.

Metzler, J., &Woessmann, L. (2010). The impact of teacher subject knowledge on student achievement:Evidence from within-teacher within-subject variation. IZA Discussion Paper No. 4999, Germany.

Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex samples.Psychometrika, 56(2), 177196.

Monk, D. H., &King, J. (1994). Multi-level teacher resource effects on pupil performance in secondarymathematics and science: The role of teacher subject matter preparation. In R. Ehrenberg (Ed.),Contemporary policy issues: Choices and consequences in education (pp. 2958). Ithaca, NY: ILRPress.

Mullis, V. S. (2003). TIMSS 1999 International Mathematics Report. Retrieved from http://timss.bc.edu/timss1999i/publications.html

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? EducationalEvaluation and Policy Analysis, 26(3), 237257.

Park, K. M., & Leung, F. K. S. (2003). Factors contributing to East Asian students high achievementin mathematics: The case of Korea. The Mathematics Educator, 1, 719.

TIMSS and PIRLS International Study Center. (2010). TIMSS 2007 contextual questionnaires. Re-trieved from http://timss.bc.edu/index.html

Wang, J. (2001). TIMSS primary and middle school data: Some technical concerns. EducationalResearchers, 30(6), 1721.

Wenglinsky, H. (2002). The link between teacher classroom practices and students academicperformance. Education Policy Analysis Archives, 10, 12. Retrieved from http://epaa.asu.edu/ojs/article/view/291

Yamamoto, K., & Kulick, E. (2000): Scaling methodology and procedures for the TIMSS mathematicsand science scales. In M. O. Maring, K. D. Gregory, & S. E. Stemler (Eds.), TIMSS 1999 TechnicalReport (pp. 237263). Chestnut Hill, MA: International Study Center, Boston College.

Dow

nloa

ded

by [T

he U

nivers

ity O

f Melb

ourn

e Libr

aries

] at 0

7:55 1

5 May

2015

the effects of teachers qualifications-saudi taiwan

Documents