050-2791 vietnam vol1.qxd 8/24/04 9:24 am page 1

050-2791 vietnam vol1.qxd 8/24/04 9:24 AM P

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

ed

Administrator

29787

Working Paper Series No. 2004-4

VIETNAM READING AND MATHEMATICS ASSESSMENT STUDY VOLUME 1: HOW MEASURING ACHIEVEMENT DISTRIBUTION INFORMS EDUCATION POLICY DEVELOPMENT

Human Development Sector Unit East Asia and the Pacific Region The World Bank

June 2004

The findings, interpretations and conclusions expressed in this paper are entirely those of the authors and should not be attributed in any manner to the World Bank, to its affiliated organizations, to members of its Board of Executive Directors, or to the countries they represent.

Vietnam Reading and Mathematics Achievement Study

CONTENTS

Abstract iii

Acronyms and Other Abbreviations iv

Preface and Acknowledgments v

I. Introduction 1

II. Study Methodology 2

III. Main Study Findings 4

IV. Policy Development 6

A. Teacher Subject Knowledge 6

B. Performance & Resource Standards 9

C. Targeted Programs 11

D. Formula-Based Funding 13

E. Innovation, Autonomy & Accountability 16

F. Measuring Pupils’ Achievement 18

V. Conclusion 21

Appendix 1 Variation Among Schools Within Countries

in Reading Comprehension 23

Appendix 2 Reading Skill Levels and Mathematics Skills Levels 24

ii


ABSTRACT This paper is the first of three volumes presenting the Vietnam Reading and Mathematics Assessment Study conducted in 2001. The present volume sets the study in its country context, describes its genesis, outlines the methodology, and summarizes the results. The main purpose of this volume, however, is to show how the analysis of the study results points to specific policy interventions. The strong link between a teacher’s subject knowledge and the performance of her pupils argues for teacher upgrade programs, while the high variation of pupil performance between schools in the same province calls for local programs that improve school effectiveness. This first volume is followed by a second volume detailing the study results, and a third providing a full description of the study methodology.

iii


ACRONYMS AND OTHER ABBREVIATIONS ASEAN Association of South East Asian Nations AusAID Australian Agency for International Development CIDA Canadian International Development Agency DfID Department for International Development (UK) EFA Education for All EPA Educational Priority Area FSQL Fundamental School Quality Level GDP Gross domestic product HLM Hierarchical Linear Model (http://www.ssicentral.com/hlm/hlm.htm) IEA International Association for the Evaluation of Educational Achievement IIEP International Institute for Educational Planning KEDI Korean Education Development Institute MDG Millennium Development Goals MLn Multilevel Modeling (http://multilevel.ioe.ac.uk/) MoET Ministry of Education and Training NGO Non-governmental organization NIED National Institute of Education Development NIES National Institute of Educational Science NORAD Norwegian Agency for Development Cooperation OECD Organization for Economic Cooperation and Development PEDC Primary Education for Disadvantaged Children PIRLS Progress in Reading Literacy Study SIP School Improvement Program TIMSS Trends in International Mathematics and Science Study UNESCO United Nations Education, Scientific and Cultural Organization UNICEF United Nations Children’s Fund

iv


PREFACE AND ACKNOWLEDGMENTS In 2000, the Vietnam Ministry of Education and Training (MoET) launched a large-scale monitoring study of primary education, the first of its kind in the country. The study would test pupils and their teachers in the last grade of primary education (Grade 5), using a sample cross-sectional survey in two key subject areas, reading comprehension in Vietnamese and mathematics. The MoET set up a committee to define the key research questions that guided the whole survey and the subsequent analysis. In 2001, the year in which the tests were administered, over 90 percent of the relevant age group entered Grade 1 and about 80 percent reached Grade 5. Thus about 20 percent of the relevant age group was not covered by this study. This volume is the first of three. Each volume contains different aspects of the study carried out in Vietnam in 2001. Volume 1 presents the study, summarizes the main issues and proposes policy recommendations under six themes: (a) raising student achievement by improving teacher subject knowledge; (b) improving system effectiveness by applying performance and resource standards; (c) closing the gap between the top and the bottom through compensatory funding; (d) counterbalancing emerging social differences with formula-based funding of education; (e) further improving learning achievement through increased innovation, autonomy and accountability; and (f) measuring the achievement of pupils and schools through regular testing. Volume 2 goes on to present the detailed research results and makes many suggestions for future action, while Volume 3 reports on the full technical aspects of the design and conduct of the study. The issues raised and recommendations suggested in Volume 1 can be used for a debate on the sequencing and construction of a critical path of the different policy options that could be adopted by Vietnam. These policy options will require considerable effort and will also require the creation of a critical mass of national specialists with the technical skills needed to design, conduct, and analyze achievement studies such as this one. They will require strong domestic political leadership and commitment. A number of areas would also benefit from technical assistance and development finance from donors and development organizations. Indeed, many donors – multilateral, bilateral and NGOs1 -- have made significant contributions to the schools sector in Vietnam, and future work should build on those efforts. The study was conducted by the MoET in Vietnam. Many people were involved in the study. Thanks are due to the following persons: the late Vice-Minister Le Vu Hung, who provided the oversight and guidance for this study; Dang Huynh Mai, Vice-Minister responsible for schooling and primary education; senior directors and heads of departments of the MoET; National Education Institutes, which provided specialists and researchers to conduct this study; provincial and district education offices, whose staff collected the data from over 3,600 schools throughout Vietnam in an exemplary fashion; Nguyen Quoc Chi, National Manager of the Vietnam Primary Education Project, who was instrumental in conducting the study and who provided special insights into the problems of primary education; Professor Dang Ba Lam, Director of the National Institute of Educational Development (NIED), who provided a member of the data team; Professor Tran Kieu, the Director of the National Institute of Educational Science (NIES), who 1 Multilateral (e.g. Asian Development Bank, European Commission, International Development Agency, UNESCO, UNICEF); bilateral (e.g. Australia, Canada, France, Japan, Norway, UK); NGOs (e.g. Oxfam).

v


furnished researchers for the questionnaire committee, and also provided Professor Do Dinh Hoan and Dr. Do Cong Vinh to work on the study (several persons on this team were also from the NIES). Professor Hoan and his team were responsible for the test development. Dr. Vinh organized the data editing, entry, cleaning, and analysis. Members of the international support team included Dr. Kenneth Ross, Dr. Mioko Saito and Stephanie Dolata from IIEP/UNESCO; Professor Patrick Griffin of the Assessment Research Centre of Melbourne University; Miyako Ikeda who guided the data entry, cleaning and merging of files for nearly 75,000 records and who conducted many of the subsequent statistical analyses, and finally the principal architect of the study who guided and helped the international advisory team and the national researchers, Emeritus Professor Neville Postlethwaite of Hamburg University. This volume was authored by Christopher Shaw and Neville Postlethwaite, reviewed by Luis Benveniste, Elizabeth King and Christopher Thomas, and edited by Mai Thi Thanh, Jeffrey Waite and Juliana Williams. Christopher Shaw and Mai Thi Thanh World Bank in Vietnam June 2004

vi


I. Introduction Vietnam’s education system is in transition. Once a system that nurtured a high-performing academic elite, it is now aiming to cater for all -- including remote and minority populations, while providing a quality education. This “education transition” mirrors Vietnam’s shift to a market-based, service-oriented economy that requires a broad human capital base. This change in the coverage in education offers critical support to the broader economic transition: international studies (TIMSS 19952 and Barro 20013) have found links between education quantity (increases in the average years of schooling of the population) and education quality (increases in the average student performance on international assessments) on the growth rate of real per capita GDP. At the beginning of the school year 2001-02, Vietnam had almost achieved its national target of universal access to primary education: over 90 percent of the relevant age group is enrolled in primary education. The Government is now moving to address the more difficult issue of enrolling the last out-of-school children from remote areas and disadvantaged circumstances (less than 10 percent of the 6-10 age cohort), and ensuring that the one fifth of students who currently drop out before completing primary school stay to the end of the cycle.

The triple-shift system in primary schools has been eliminated and a school building program is replacing ‘leaf’ or ‘non-permanent’ school buildings in rural areas with durable, permanent facilities. The time dedicated to instruction is being increased with a growing proportion of pupils in full-

day schooling4 and, as resources permit, expansion of full-day schooling is being encouraged in place of unregulated after school tuition. In addition, the school year has recently been lengthened to match the roll-out of a new curriculum for primary school. At the same time as its primary education sector is growing, Vietnam is making rapid progress in universalizing lower secondary education and expanding other levels, in order to meet the target of Education for All (EFA) and meet the Millennium Development Goals (MDG). The most recent population census (1999) showed that the primary school-age population had stabilized and was expected to decline during the decade 2000-2010. The prospect of a fall in demographic pressure, coupled with a projected growth in GDP and an increase in public spending on education during the same period, allowed Vietnam – for the first time ever – to broaden its focus beyond access to schooling, to address the quality of mass basic education. In order to make the transition from increasing access to improving quality, the authorities urgently needed a deeper understanding of the underlying factors that influence the quality of primary education in Vietnam. In addition to its international obligations under the EFA and MDG initiatives, Vietnam was motivated to improve the quality of its education system by a desire to increase its competitiveness and to further integrate its economy with the world. Eager for a more active learning approach, better suited to the demands of globalization, continual rapid change and a market-based economic system, the Government has continued to pursue education reforms on various fronts.

2 International Study Center, 1997, Performance Assessment in IEA’s TIMSS, Boston.

3 Barro, Robert J., 2001. “Human Capital and Growth.” American Economic Review 91 (2): 12-17

4 At present, one-third of pupils are in full-day schooling, and paying for the afternoon session.

1


II. Study Methodology In 2002, new curricula and textbooks for primary schools were introduced and are gradually working their way up the elementary cycle. This curriculum reform aims to modernize the content and methods of teaching and to promote a more dynamic, flexible approach to learning.

Availability of information on education in Vietnam increased during the 1990s. Following a UNESCO sector report5 in the early 1990s, the Government and the World Bank jointly conducted a financing study in the mid-90s,6 which examined the costs and financing of the education and training system and answered questions such as: (i) how much does education cost? (ii) who pays for education? and (iii) do they get good value for money? Further analytic work has been undertaken more recently, including: (a) a review of the primary education curriculum content and instructional methods; (b) an updated review of the costs and financing of primary education undertaken as part of a broader analysis of public expenditures; and (c) an analysis of the 1993 and 1998 household expenditure surveys that linked poverty and education and identified some educational disparities.7 None of these studies however directly addressed education quality and performance or attempts to define the outcomes of the education system in terms of students’ learning achievement.

Further elements of Vietnam’s modernization of education include a program of teacher professional development during 2002-2005 to support the use of the new curriculum and to introduce modern methods of instruction, and a large program of support for primary education of disadvantaged children for the period 2004-2009. To start this process the MoET wished to have a ‘picture’ of key learning outcomes in the last grade of mass education, Grade 5, before such major reforms took place. Having an initial benchmark of learning outcomes would allow a comprehensive assessment of the effectiveness of external support to basic education and the subsequent estimation of value-added of the reforms expressed in terms of increased student learning and achievements. For all these reasons (change of focus in primary education from access to quality, greater impact of quality improvements on economic and social development, need for a baseline measurement of learning performance, etc.,) it was decided that the present study would look in-depth at the quality of the system, taking student achievement in primary education as its focus.

Indeed, before the present study was undertaken, little was known about the overall quality of primary education in Vietnam. Four fifths of those who enter primary schools complete the full cycle of five years and among them a small elite regularly participate with great success in international mathematics Olympiads or similar competitions. But this narrow focus ignored the fact that the remaining group – around one fifth of those who enter a primary school – drop out before the end of

5 UNESCO, 1992, Vietnam Education and Human Resources Sector Analysis, Hanoi. 6 World Bank, 1997, Vietnam Education Financing, Washington DC. 7 National Foundation for Education Research, 2000, Vietnam Primary Curriculum Review, London; World Bank, 2000, Vietnam: Managing Public Resources Better, Washington DC; World Bank, 2002, Trends in the Education Sector 1993-98, Washington DC

2


their schooling. While this study does not measure pupil retention per school, literacy studies in other countries suggest that many adults with less than the full cycle of primary education remain functionally illiterate and innumerate for the rest of their lives. For this reason, the proportion of children not completing primary school education is a matter of concern.

Questionnaires were designed to answer the MoET’s questions. The pupil tests reflected both the old and new curricula; the teacher tests allowed for a wider range of achievement while placing teachers and pupils on the same scale. Once tested and revised, the final set of instruments consisted of: a pupil reading test; a pupil mathematics test; a pupil questionnaire; a teacher reading test; a teacher mathematics test; a teacher questionnaire; and a school head questionnaire.8

From 1998 to 2000, the MoET’s National Institute for Educational Sciences (NIES) conducted a small study of primary pupil achievement at Grades 3 and 5 in fourteen schools in each of five (out of 61) provinces of Vietnam. Given the limitations of the research design and the scale of this work – which was to provide training in research methodology rather than carry out a representative scientific study – it is not possible to generalize from these results to all provinces. The results were nevertheless able to show large disparities in learning achievement both between provinces and between schools within provinces. Grade 3 pupils in the capital, Hanoi, had levels of achievement in mathematics and reading comprehension in Vietnamese that were higher than Grade 5 pupils in the other four, more rural, provinces. These few facts suggested that parts of the primary school system did not yet provide the high quality and equitable learning conditions needed for successful learning achievements by all pupils.

Given the highly decentralized nature of primary education management, the study collected provincial as well as national data; cost and size precluded analyzing data at the district or sub-provincial level. A probability sample of schools (proportional to the enrolment of Grade 5 pupils) was drawn such that the sampling error for any province was between 7.5 percent and 10 percent for a percentage, and about 0.10 to 0.13 of a standard deviation for the estimate of a mean. This gave a sampling error for the national estimates that was very small indeed (between 0.03 and 0.05 for a percentage and 0.003 of a standard deviation for an achievement score); it required testing in 60 schools in each province making a total of 3,660 schools for the nation. A random sample of 20 pupils from all classes in Grade 5 was selected within each school, in order that this first comprehensive achievement study provide a good estimate of the within-school variance.9 The target population (“desired”) was split into two groups. Schools with fewer than 20 Grade 5 pupils were excluded from the study, on the grounds that data collection costs could be reduced substantially without causing major distortions in the study population. Pupils in the remaining schools make up the “defined” population. The numbers of

The MoET opted for a study of Grade 5 achievement in reading and mathematics and its determinants, in order to have systematic and detailed information that could be used for decision making at the national and provincial levels. To ensure its relevance, the MoET provided a list of about one hundred research questions to which it wanted answers. The questions concerned the levels and variation in achievement in the two key subject areas, materials and textbooks in the classrooms, human and material resources in the schools, and selected process variables including school inspection and teaching methods.

8 See Vol. 2, Ch 1, for further details of research design, and Vol. 3, for full details of the test development and calibration. 9 See Vol. 3, Ch. 4, for full details of sample design.

3


III. Main Study Findings schools and pupils in the desired, defined and excluded populations are shown in Table 1.

Table 1: Survey Population (Desired,

Defined and Excluded)

Schools Pupils Desired 14,642 1,812,053 Defined 14,173 1,808,089 Excluded 469 3,964

Overall results indicate that while Vietnam’s average score in reading comprehension is comparable with the results of similar studies conducted in a range of developed and developing countries.11 Figure 1 compares the reading comprehension achievement of Vietnam’s Grade 5 pupils in 2001 to the 1991 results of Grade 3, 4 or 5 pupils in a selection of European and OECD countries.

Testing took place on April 12-13; Grade 5 pupils were randomly selected on the first day and the testing proper was carried out on the second day. The survey response rate for the whole country for schools, pupils, teachers, and school heads is given in Table 2.

Figure 1: Reading Scores

200

250

300

350

400

450

500

East Germany3

Hungary 3 Vietnam 5 in2001

Greece 4 New Zealand 5 USA 4

Country and grade level (all countries except Vietnam refer to 1991)

Scor

e

Table 2: Survey Response Rate

Learning achievement, however, varies widely across the pupil population. Variation in achievement across schools, in the country as a whole and also among schools within provinces, is surprisingly large, as shown in Table 3.12

Planned Number

Response Number

% Response

Schools 3,639 3,635 99.89 Pupils 73,200 72,660 99.26 Teachers 7,278 7,178 98.63 Heads 3,639 3,631 99.78 Table 3: Equity of Achievement Scores

Among Provinces and Schools The response rates in this study were much higher than in similar sample surveys in many other countries. The few schools and pupils that were missing were due to pupils’ absence on the day of testing or schools having been flooded. The following sections present the results of the survey and present recommendations for policy action.10

Proportion of

Variance Available Reading Math Provinces 11% 14% Schools 56% 64%

Tables 4a and 4b shows the percentage of

pupils at different skill levels (increasing

11 To compare learning achievement in Vietnam with that of other countries, reading comprehension scores for Vietnam measured by the present study were calibrated to the scale used in the 1991 IEA Reading Literacy Study. See Vol. 3, Ch. 3, for full details of the international benchmarking of the 2001 Vietnam data with that of the 1991 IEA Reading Literacy study.

10 See Vol. 2, for detailed study results, and Vol. 3, for technical details of the study methodology and analysis. 12 See Vol. 2, Ch. 7.

4


from 1 to 6)13 by school location (isolated, rural or urban).

Table 5b: Proportion of Pupil Variance of Subject Scores (Mathematics)

Table 4a: Percentage of Pupils by Skill

Level (Reading) MLn HLM

Pupil 35% 34% School 65% 66%

Skill Level 1 2 3 4 5 6 I 9.7 21.1 25.5 17.9 17.6 8.2R 4.7 15.4 24.5 20.5 23.8 11.2U 1.5 7.6 17.5 20.7 30.9 21.7T 4.6 14.4 23.1 20.2 24.5 13.1

This inter-school variation is much larger than that observed in most other systems; the 2001 Grade 4/5 IEA PIRLS study indicated a 13 percent variation among schools in Sweden and a 27 percent variation in the USA.15 The policy implications of this level of variation, taken up later in this paper, relate to the need to identify and address quality issues at the school level: teachers, management, learning climate, etc.

I(solated), R(ural), U(rban), T(otal)

Table 4b: Percentage of Pupils by Skill Level (Mathematics)

Skill Level 1 2 3 4 5 6 I 0.4 6.4 17.0 32.6 25.2 18.5R 0.1 3.6 12.4 29.8 29.1 25.0U 0.0 1.3 5.4 20.6 34.2 38.5T 0.2 3.5 11.5 28.2 29.7 27.1

I(solated), R(ural), U(rban), T(otal) As expected, home background is important in predicting pupil learning achievement. However, in several provinces, more variation in pupil’s achievement is due to differences among schools rather than differences between pupils. Tables 5a and 5b show the maximum amount of variance in achievement that can be explained by the pupil level or by the school level.14 Table 5a: Proportion of Pupil Variance of

Subject Scores (Reading)

MLn HLM Pupil 42% 42%

School 58% 58%

Not surprisingly, these disparities in pupils’ learning achievements reflect differences in inputs and process across schools. While overall levels of human and material resources are increasing in schools, these resources are not distributed equitably across schools within provinces.16 One third of the variation in school resources is attributable to differences among provinces, while two thirds of variation is attributable to differences among schools within provinces. There are also disparities in the instructional hours that pupils receive (with most pupils enrolled in half-day schools and only some in full-day schools). Teacher quality varies across schools, with teachers demonstrating different levels of subject knowledge in reading and mathematics. This is important since teachers’ scores on the mathematics test are highly correlated with pupils’ mathematics test scores, at both the national and provincial levels. 17 There tends to be little difference in the kinds of

15 See Vol. 1, App. 1, for the range of variation

among schools found in the 2001 IEA Study. 13 See Vol. 1, App. 3, for definitions of the different skills levels for reading comprehension, and Vol. 2, Ch 2, for further details on the ways in which student achievement is recorded (skill level, benchmark and total score).

16 See Vol. 2, Ch. 7, Tables 7.1 and 7.2. 17 Simple and partial correlations of teacher subject knowledge in mathematics and pupil achievement in mathematics are 0.29 and 0.25 with the effect of home background of pupils removed

14 See Vol. 1, App. 2, and Vol. 2, Ch. 9 and App. 9.2.

5


IV. Policy Development teaching methods that teachers say they employ.18

This section proposes a critical path of policy change, based on the policy messages that emerge from the present study and centered on six main themes:20

Disparities are often associated with isolation or poverty: isolated schools typically have fewer resources, poorer teachers, and insufficient textbooks. It is in the capital province, Hanoi, where the best resources and teachers are available and it is in the remote border and mountainous areas and some Mekong Delta provinces where the major deficiencies are to be found.19

• Raising student achievement -- by improving teacher subject knowledge; • Improving the effectiveness of the system -- by applying performance and resource standards;

• Closing the gap -- through targeted programs; In short, the study paints a picture of a

polarized school system. At one end of the system, the children from well-off families and high socio-economic backgrounds gain access to well-resourced schools and are taught in full-day sessions by well-qualified teachers. In contrast, those from poor families -- often in isolated or remote areas -- may well attend single-shift sessions in semi-permanent buildings and be taught by staff, many of whom are not at the national standard. The standard of education services provided to this second group is of poor quality and as a result the learning outcomes and pupils’ achievement are limited.

• Counterbalancing emerging social differences -- with formula-based funding; • Further improvements in achievement -- through increased innovation, autonomy and accountability; • Measuring achievement of pupils and schools -- through regular testing. Between some of the themes there is some overlap and without careful sequencing of policy changes there is a risk of contradiction if actions are implemented in an unstructured or unplanned manner. Where possible, for each theme, comments are made on: (a) changes in policy and choice of strategy; (b) changes in process; (c) changes in inputs; and (d) changes in the roles and responsibilities of the various actors at different levels in the system.

In summary: • Almost all children of primary school-age are enrolled in school, but about one fifth do not complete the full primary cycle; • On average, pupils in remote rural areas attend for only part of the day in schools with fewer resources, including qualified teachers, and achieve less than pupils in urban schools;

A. Teacher Subject Knowledge The largest predictor of pupil achievement in reading and in mathematics was the relevant teacher subject knowledge.21 Indeed, some Grade 5 pupils performed better than some Grade 5 teachers. The reading comprehension and mathematics score distributions for pupils and teachers are shown in Figures 2 and 3. In each chart the flatter score distribution shows pupil scores, while the steeper distribution shows

• The variation in student achievement is due mainly to differences among schools, and these differences among schools are sharper in some provinces than in others; • Analyses of schools that are more effective may provide guidance for improving school quality and equity in Vietnam.

20 See Vol. 2, Ch. 10, for the full list of policy

recommendations. 18 See Vol. 2, Ch. 4. 19 See Vol. 2, Ch. 2, Table 2.5. 21 See Vol. 2, Ch. 9.

6


teacher scores.22 In reading comprehension, the top 12 percent of pupils scored better than the bottom 30 percent of teachers.

The implication for immediate action is clear: ensure that all teachers’ subject knowledge meet minimum standards. The national policy on teacher qualifications sets standards, but not all primary teachers meet these minimum standards.

Figure 2: Reading Score Distributions (%

of candidate pupils [flatter curve] & teachers [steeper curve])

Figure 4: Relationship between Teacher & Pupil Provincial Mean Reading Scores

0

1

2

3

4

5

6

400

420

440

460

480

500

520

540

560

580

400 420 440 460 480 500 520 540 560 580 600

Province pupil meanPr

ovin

ce te

ache

r mea

nFigure 3: Math Score Distributions

(% of candidate pupils [flatter curve] & teachers [steeper curve]) Figure 5: Relationship between Teacher & Pupil Provincial Mean Mathematics Scores

0

1

2

3

4

5

6

340

390

440

490

540

590

400 420 440 460 480 500 520 540 560 580 600

Province pupil mean

Prov

ince

teac

her m

ean

While the national MoET oversees the teacher training system, primary teacher training colleges are operated by provincial education authorities. The resource base is not comparable across provinces, and different local characteristics and needs lead each provincial education authority to adjust the qualifications offered by their local teacher college. As a result, primary teacher training colleges produce teachers with different levels of subject knowledge. This approach enabled the education system to expand towards universal primary education, especially in remote and rural

The detailed analysis also showed that there were considerable differences among provinces in teacher subject knowledge and this in turn was closely linked to the differences among provinces in pupil achievement. Figures 4 and 5 show the relationship between the teacher (vertical) and the pupil (horizontal) provincial mean scores in reading comprehension and in mathematics.

7

22 See Vol. 2, Ch. 3.


provinces where the intake of trainees was sometimes below the desired level of general education. However, this pragmatic approach has now led to a heterogeneous teaching force with multiple qualifications and different levels of subject knowledge and teaching skill. Given that Vietnam has almost reached its goal of universal primary education and that a decline in the school age population is anticipated over the next few years, the country now has an opportunity to renew and update its teaching force. All teachers should met defined standards of subject knowledge in order to be able to teach in primary schools, at least above the pupils’ top standard level. This requires strict testing of subject knowledge of teachers to ensure that they have reached the standards before being certified as teachers, with perhaps opportunities for re-testing later in their career. The combination of a continuing fall in the size of the primary school-age cohort and the large proportion of under-qualified teachers calls for an increased focus on in-service professional development rather than further expansion of pre-service training. Teacher upgrading and certification should be a priority at both the national and provincial levels. First, there needs to be opportunities to raise all teachers already working in schools to the new standard, through professional development such as upgrading and retraining courses. Second, pre-service training courses must be revised in a way that ensures that all new teachers meet the required standard. For teachers currently in the system, especially those whose subject knowledge is low, professional development opportunities and incentives should be made available. School directors and local education inspectors can assist in identifying candidates. A first step may be to target courses at specific schools or to offer correspondence or distance learning courses in order to increase teachers’ subject

knowledge. Work is already under way in ten pilot provinces, with support from external donors, on the redefinition of standards for primary teachers, the renewal of pre-service courses, the preparation of professional development training for serving teachers, and the renewal and strengthening of provincial primary teacher training facilities. Once the pilot phase of this program is seen to be working, there will be an opportunity to revise and expand the program to the national scale . For new teachers, the teacher training content of the pre-service professional training courses could be modified to take account of minimum standards, and standard national assessment procedures adopted for teacher certification . Once the new courses have been launched, no exemptions should be allowed in the certification of new teachers. A national quality assurance group could construct teacher assessment instruments and link teacher certification to these instruments, thus ensuring that differences across the provincial teacher training colleges are eliminated and that all teacher training colleges meet minimum course standards and deliver comparable qualifications to their trainees. Once certified, teachers need to be deployed to schools in an efficient and equitable manner. The study has shown an uneven distribution of teacher subject knowledge among schools. First, teachers with fewer years of academic education and teaching experience are over-represented in isolated and rural schools. Not surprisingly, these differences show up as differences between provinces and in pupils’ learning performance. The following table shows some of the teachers’ characteristics broken down by school location (isolated, rural or urban). At present, teachers search for their own teaching job and most teach in the province where they did their teacher training. The few teachers that move from their province of origin do so for personal reasons or for promotion to posts of responsibility.

8


Table 6: Selected Teacher Characteristics

School location

Mean age (years) % Female

Ethnic group

% Kinh

Mean years academic education

Mean years professional education

Mean years teaching

experience Isolated 30.5 57 80 11.6 1.9 9.1 Rural 33.2 73 95 11.9 2.1 12.0 Urban 37.9 86 97 12.2 2.2 15.8 Total 33.7 73 93 11.9 2.1 12.2 Newly qualified staff, with little or no experience or lower academic and professional qualifications, rarely obtain posts in urban schools. They tend to be appointed in the more difficult, under-resourced schools in isolated or rural areas of their province. Hence, the current teacher deployment process does not compensate for the disadvantages that already have an impact on the quality of education in such schools. A more rational system of teacher deployment might be obtained if decision-makers – provincial and district officials, as well as head teachers -- had access to better information on the characteristics of applicants for teaching positions. This would require an improved personnel data system, with information available by school. It would also have to be supported by a system of incentives that encouraged an adequate proportion of good teachers to go to and stay in hardship posts. The basic elements of such a system are being developed with the assistance of external donors in ten pilot provinces. If successful, this could be generalized to the rest of the country to obtain a better match between needs and candidates. Teachers with low subject knowledge, should be encouraged to undertake professional development along the lines of the Government’s on-going Primary Teacher Development Project. For new teachers, there should be a national standard for subject knowledge. The deployment of teachers to schools should be conducted on a more rational basis and

attempts made to reduce the disparities in the quality of school staffs among schools. B. Performance & Resource Standards The study shows that some schools produce graduates with very low levels of achievement in reading and mathematics. In countries that have highly developed education systems, the governance and promotion of effective national education delivery systems have three centrally managed components: (i) national goal setting and policy development; (ii) selecting and applying system performance standards; and (iii) coordinating and regulating actions by service delivery units. The second of these components -- selecting and applying system performance standards -- can be implemented along complementary tracks. First, the administration can define minimum operational standards (outcomes) that all locations are required (and resourced) to meet. Second, it can adopt a set of international standards that reflect national aspirations, but are often beyond the reach of poor locations. Selecting and applying performance standards are normally done at the national level in order to ensure close correspondence between international standards, national goals and operational targets. Few provincial or municipal governments have sufficient resources to research and develop comprehensive performance standards as this can be costly. It is critical however that decentralized levels of government in the development of

9


national standards be involved, not only because local participation is in itself desirable, but also because it can ensure that standards are adapted to local constraints. Traditionally in Vietnam, the MoET has set national standards for inputs in absolute terms, and it is only just beginning to define outputs or levels of achievement. The present study defined skill levels of achievement for Grade 5.23 This is a first step in having teachers think in terms of skill levels and conducting regular classroom testing throughout the year, so that the teacher knows which pupil is at which level and can determine the next steps to take. To reinforce this approach the unit responsible for the curriculum should re-examine the curricula and new materials in terms of skill or competency levels, and should feed these concepts into the in-service and pre-service teacher training programs and to the unit responsible for testing. The testing unit could produce classroom tests for regular testing of pupils, and guide teachers in their regular assessment of pupil learning performance throughout the school cycle. During the 1990s, the MoET concentrated on increasing primary school enrollment to meet the Government target of universal education. As Vietnam now edges towards its goal of universal primary schooling, it is time to move beyond the purely quantitative dimension and to pay attention to the quality of educational achievement in primary education. Most central and provincial education administrators have not monitored learning performance using nationally standardized measures. The methodology used in the current study allows policy makers to understand the variables that contribute to learning achievement and the distribution of knowledge across provinces, schools and individuals. Given Vietnam’s economic aspirations, these policy makers may even wish to consider linking national standards for learning outcomes to those of

other East Asian countries (Korea, Thailand or Singapore) or to OECD standards. Since late 2003 the MoET has developed the concept of a ‘minimum service standard’ as a pragmatic transition point on the journey towards the high national standards. The idea of a minimum service standard has also been linked with a regular school audit. The minimum service standard describes not only expected results in terms of learning achievement but also lists of inputs and processes. The present study showed that many classrooms and schools were poorly resourced.24 Resource inputs were of two kinds: material (classroom-specific and school-wide) and human (classroom and head teachers). Isolated schools were the least well-resourced, rural schools somewhat better resourced and urban schools the best resourced. Consequently, provincial school audits or reviews of resources can be used to target programs of supplementary resource allocation. The Government’s program “Primary Education for Disadvantaged Children” (PEDC), started in 2003 with substantial multi-donor support,25 is testing this process. In pilot areas, results from school audits are being discussed by the provincial and national authorities, so that the national authorities can take action to even out differences among provinces where necessary. The ‘minimum service standard’ is now part of the Government’s Fundamental School Quality Level (FSQL) program in primary education. As the contents of the minimum service standard are tested and updated, the Government is applying this approach nationally, accompanied by a core package of resources. The package of inputs and processes described in the minimum service standard will need to be provided to every school so that all are resourced at least at the

24 See Vol. 2, Ch. 4 and 5. 25 AusAID (Australia), CIDA (Canada), DfID (UK), NORAD (Norway), and the World Bank. 23 See Vol. 1, App. 2.

10


minimum level. This should narrow disparities among schools. It is suggested that outcome performance be specified in terms of minimum standards using the skill competency level26 approach adopted for the current study. It is also suggested that the material resources FSQLs be revised. Provincial education authorities should be responsible for reporting on compliance with minimum resource standards, through a regular auditing process. C. Targeted Programs This study shows that the difference between the mean provincial scores in reading comprehension and mathematics in the highest and lowest provinces is 1.5 standard deviations.27 This is equivalent to three years of schooling, a large gap between the top and bottom. The difference between the 25th and 75th percentile of schools is about 1.0 standard deviation in both reading and mathematics.28 One standard deviation, equivalent to two years of schooling, still constitutes a large gap across the middle 50 percent of schools. This confirms the results of an earlier study that showed Grade 3 pupils in the capital outperforming Grade 5 pupils in some other rural provinces. Most education systems throughout the world try to minimize differences among provinces so that children receive a common standard of education no matter where they live. However, the study shows that some children live in areas without a quality school, and that this will likely exacerbate provincial inequality. The main factors associated with large differences among pupils and schools are

listed in Table 7.29 Factors marked with an asterisk show large differences among provinces; reweighting these critical factors across provinces will likely require central MoET intervention. For others factors, it is enough for the provincial authorities to take action.

Table 7: Major Factors Associated with Large Achievement Differences Among

Pupils/Schools Home factors Private corner for study at home Meals per day* Days absent last month Pupil Materials Use own Vietnamese textbooks Use own mathematics textbooks Borrow books School and Staff Factors School resources index* Staff Teacher training* Temporary teachers Total number of pupil problems Total teacher problems Teacher sex Teacher training Classroom furniture* Teacher’s reading score Teacher’s math score Teaching/Learning behaviors Story telling competition Percent of parents met mathematics homework assigned Math homework corrected Reading homework assigned Reading homework corrected

26 See Vol. 1, App. 2.

27 See Vol. 2, Ch. 2. 29 See Vol. 2, Ch. 8 & 9. Factors listed do not include economic and user charges, or specific details on the hours of instruction, as this study did not provide reliable information on these matters.

28 Test score differences were 97 for reading and 108 for mathematics on the standardized scale with a mean of 500 and standard deviation of 100.

11


Factors in the home: A private corner in the home was associated with higher achievement even after taking account of home background. Similarly the regularity of meals and days absent were very different between the highest and lowest achieving schools and pupils. By advising parents about the usefulness of providing a private corner at home as well as the need to minimize their children’s absence from school, the MoET can influence factors in the home. Compensatory measures, such as school feeding programs or conditional cash transfers to poor households, may help in areas where the limited number of meals per day affects pupils’ learning. Pupil materials and books: In a few schools, pupils still did not have a pencil, notebook or other learning materials. It is suggested that each school have at its disposal a number of items such as pencils, erasers and writing supplies to provide to pupils in need. However the important factor that shows up from this study was access to and use of textbooks. The official MoET book list has 29 textbooks and accompanying books that pupils are meant to have. Of these, eight were for the Vietnamese language and five were for mathematics. The study showed that in nearly all cases, pupils had the key books for Vietnamese and mathematics but only 20 to 50 percent of pupils had the other books. In some poor homes, no books were available.

With the renewal of the primary school curriculum, it is time to review whether all of the books are really necessary. As a matter of policy it would seem that a shorter, more realistic list of what pupils must have should be drawn up by the curriculum group. The authorities should then ensure that every child has the books. This will mean subsidizing the books for the very poor families, an action that the MoET is already following for minorities and for disadvantaged children. A book lending scheme has already been piloted in four provinces under a recently completed externally supported primary education

project, and the success of the pilot scheme showed the way forward. Through this pilot, teachers are now familiar with the procedures needed to operate a book lending scheme for all pupils. Many schools have few books either in classroom or school libraries. Where pupils were able to borrow books, schools performed better. As some of the titles from the original list of 29 become optional, they could be made available through the school or classroom library. Other research shows that having access to many books and being able to read them is closely linked to improvement in reading. There have been very successful ‘book flood’ programs in several ASEAN countries; a similar program could be introduced in Vietnam. School staff: The level of teacher training of the entire school staff proved important. This often has to do with the aura of a school and what is sometimes referred to as the school’s culture and work/learning ambience. Having fewer temporary teachers was also a feature of better achieving schools. Teaching and learning behaviors: Higher scores were registered in schools where the head teacher reported fewer behavioral problems amongst the teaching staff.30 Teachers’ behavioral problems were also associated with behavioral problems amongst pupils. In schools where teachers met with a higher percentage of their pupils’ parents, the pupils had higher achievement scores. It was either difficult for the teachers to meet parents in the poorer achieving schools or they were not making a sufficient effort. The need for teachers to meet parents frequently, and not just when test or examination results are announced, is something that should be stressed in teacher training programs. Finally, where homework was given and corrected in both subject areas, schools performed better. 30 See Vol. 2, Ch. 5.

12


How might the Government go about reducing this achievement gap between the top and bottom? The first policy recommendation, to raise everyone’s achievement by improving teacher subject knowledge, will improve achievement of those at the bottom but may not necessarily reduce the gap between the top and bottom. Indeed, as the national average of performance in learning outcomes increases, the amount of variance in learning outcomes is also likely to increase, thus leading to a widening in the achievement gap between the top and the bottom. The second policy recommendation, setting and applying a new minimum standard and ensuring that every school meets or exceeds that minimum standard by equalizing inputs with the provision of a core package of inputs and actions in all schools, will provide a much needed boost to the schools that are at the bottom. However, equalizing inputs and guaranteeing the minimum service standard by all, as the first step on the way to eventually meeting the ideal ‘national standard’ or an agreed international target, may not be enough for schools with large numbers of disadvantaged or minority students, or those with performance difficulties. Thus, the third policy recommendation that is made is for a targeted education program for such schools. The program could include a variety of actions and policies that remove or alleviate the pedagogical obstacles to performance (such as limited duration of instruction, absence of learning materials, or below-standard teachers) as well as non-pedagogic and economic barriers (such as fees for the full-day sessions and other user charges that are often beyond the capacity of poor rural families).

The cumulative impact of the actions recommended above will be considerably greater if, instead of being carried out in an uncoordinated manner, they are included in a coherent targeted education program. To do so would require a senior member of the MoET to take special responsibility for the

co-ordination of the work. One approach to targeted programs that is discussed below is that of a school needs-based funding formula. If differences in school achievement are to be evened out, it will be important first to equalize inputs across schools, and second, to introduce a program of additional actions targeted at schools that are performing below the mean. The program should concentrate on selected inputs over and above the new minimum standards for schools. These measures should include a) having a realistic list of books for all pupils and then ensuring that the books are available to all children; b) providing better teachers to needy schools; c) introducing a book flood program; and d) introducing school intervention programs in selected areas. D. Formula-Based Funding Differences in the pupils’ family backgrounds are often highly related to how well the schools achieve in the different subjects. As pupils leave school and enter society and the world of work, differences in achievement lead to variations in successes and differences in wealth. Over time, and without corrective measures, society itself may gradually become less equal. Since scholastic achievement has such implications for future social and economic success, it is critical that education policy be designed to compensate, at least in part, for the disadvantage that results directly from poverty and isolation. The most tangible asset that poor individuals and families have is their potential human capital, and policies that enhance their human capital have high returns in terms of poverty alleviation. Ensuring that poor and isolated children have access to a good quality basic education will enable them later to participating in the growing economy, and ensure that a lack of educational opportunity does not become a barrier to mobility.

13


Most schools recruit their pupils from the immediate surrounding geographical area. These ‘catchment’ areas are chosen in order to limit the distance or travel time from home to school, and they are also expected to have the effect of encouraging each school to serve its own neighborhood. However, families that live in a particular neighborhood tend to have similar characteristics, and therefore the resulting composition of a school’s pupil body tends to be more homogeneous than would be expected if pupils were randomly allocated among schools. As a result, in an isolated, poor, rural location, the pupils tend to come from families with agricultural and other relatively poor backgrounds. In contrast, in an urban setting, populated mainly by families with relatively wealthy backgrounds, it would be expected that children from similarly wealthy backgrounds would attend the same schools. As a result, individual schools are often relatively homogeneous internally in terms of their social composition, while the differences in social composition of student populations of schools in different locations tend to be quite large. As well as the information on student and teacher learning achievements collected through tests of reading comprehension and mathematics, this study collected information on pupils’ family backgrounds in terms of the number of possessions in the home and other characteristics. Using statistical techniques it has been possible to examine the relationship between pupils’ achievement scores and their home backgrounds, and to see to what extent the variation in pupil achievement was associated with the differences in wealth/poverty differences in family backgrounds; and, whether in a particular province schools had intakes of pupils that were different in terms of their families’ economic backgrounds and that are related to the schools’ achievement in reading. The study shows that in Vietnam the spread in reading scores among schools in a

province was much larger than in other countries where similar research had been conducted. In over one half of the total sixty-one provinces of Vietnam, the variation in reading scores was due more to differences between schools in those particular provinces than to the differences between pupils in the schools. In about fifteen of these provinces the relationship between the school reading scores and the socio-economic composition of the schools’ pupil bodies was strong. Thus, in these fifteen provinces there would seem to be an emerging issue of social segmentation. Furthermore, in ten of these fifteen provinces the provincial average reading scores are below the national average. So in this final group of ten provinces, not only were there large differences and a strong relationship between the reading score and the schools’ socio-economic composition, but the provincial average reading scores were low. In these ten provinces31 it would seem that there may be an emerging issue of social segmentation. “Formula funding” is commonly used to address the problem of inequitable outcomes that are associated with pupils’ socio-economic characteristics. Formula funding is based on the premise that schools catering for socio-economically disadvantaged pupils need more resources to reach the same outcomes as schools serving pupils from wealthier families. In other words, unequal unit inputs can, when allocated equitably, result in equal unit outputs. Under this approach, schools are funded according to a formula that, roughly speaking, multiplies its pupil enrollment by a unit cost that varies in accordance with criteria that reflect the socio-economic characteristics of the community served by the school. Census data are typically used to determine the socio-economic index of the school’s catchment community; the lower the index

31 The ten provinces are : An Giang, Binh Dinh, Ca Mau, Can Tho, Cao Bang, Gai Lai, Hoa Binh, Lang Son, Kon Tum, Yen Bai.

14


(i.e. the poorer the community), the higher the unit cost. The technology for this kind of approach is well developed and has been used successfully in several countries.32

“School improvement programs” (SIP) can be used alongside, or instead of, “formula funding”. SIPs require schools that are identified as catering for disadvantaged pupils to develop proposals to improve learning; these proposals call for resources over and above the base funding that the school normally receives. Like formula funding, the SIP approach ensures that additional resources are directed towards schools with the most needy pupils. SIPs have the advantage, over formula funding, of ensuring that the school management has already had to develop a plan linking the additional resources to improved learning. Formula funding, on the other hand, has the advantage of lower transaction costs, both for the school and for the funding agency, since there are no separate funds to administer.

“Voucher” systems are an alternative to formula funding. Formula funding treats the school as the funding unit, since the per-pupil unit cost is a community average that is typically calculated only once every few years. One pupil entering or leaving the school will not affect the unit cost. In theory, a voucher is assigned to an individual pupil and has a monetary value that varies according to that pupil’s socio-economic characteristics. (In fact, in most cases where this approach is applied, vouchers have a fixed value, but are restricted to disadvantaged pupils.) The pupil is able to “redeem” her voucher at the school of her parents’ choice, with the voucher then being honored by the funding agency. Voucher systems are built on the assumption that, in seeking out the best schools, disadvantaged pupils will not only receive a better education, but will drive

poorly performing schools to emulate the better schools in order to attract voucher-bearing pupils. It is clear that voucher systems are more easily applied in urban settings, where it is reasonable to expect that parents have a choice of schools.

Finally, some public school systems change the socio-economic mix of pupils within schools, as a way of offering disadvantaged pupils the same quality education afforded to pupils from wealthier families (thereby avoiding the poor quality education that is generally associated with “ghetto” neighborhoods). This is done by managing the location of businesses and enterprises in a way that brings different social groups into neighborhoods, or sometimes, by transporting pupils from one school catchment area to another.

Some of these ideas have already been taken up in the recommendation on closing the gap that was made earlier in this volume. Vietnam is already using targeted SIPs with the recently launched primary education program for disadvantaged children. However, as the management of public finance and budget systems is modernized and the linkage between the budget and an education medium-term expenditure framework is strengthened, an approach that includes elements of formula funding could become a promising way of moving further forward. Planning and funding of schools in Vietnam follows an approach that is based on standardized norms. While there are norms that take into account differences in provinces, the approach does not yet fully address all issues such as the financial implications of providing quality basic services to those children with disadvantages or disabilities, or the extra cost of transporting basic supplies and other resources to poorer schools, often located in remote and hard to access places, and thus more costly than supplying large cities. Moving to an approach based upon minimum service standards, providing

32 Ross KN and Levacic R, 1999, Needs-based resource allocation in education via formula funding of schools. Paris: IIEP/UNESCO

15


additional or differential inputs for schools with special characteristics and disadvantaged pupils, and allowing local level flexibility to diverge from standardized national norms could address these issues. At the provincial level there must be flexibility to provide resources to schools so as to reflect the background and needs of the children, where they live, local climatic and communication difficulties etc., and to strengthen the link to outcomes and to meeting the minimum service standards for all pupils. Since differential amounts of resources will be needed especially by isolated schools, data systems can be used to identify shortfalls in the required personnel or those schools in need of extra help. As the system matures and greater responsibility for planning and finance is decentralized, increased local-level autonomy, even down to the level of the school, can be supported through demand-driven funding windows that are targeted to support local innovations. In summary, the Government may wish to include a ‘needs-based’ school funding formula in its budget and public financial resource allocation systems. Such a needs-based school funding formula would inter alia define school resource needs in terms of the socio-economic composition of the school’s pupil population. Through such strategies, if the Government sees the education system as an important instrument to address the effects of family wealth differences, it would be able to redress emerging socio-economic differences. In some provinces it can be seen that there were large differences among schools in reading scores and that they were related to socio-economic differences in the pupil bodies of the schools. This can undermine social cohesion. Where this was the case, provinces were identified and it has been suggested that the results be checked and, where necessary, programs be created that are targeted at specific schools in specific provinces to reduce the relationship between school achievement and school socio-

economic make-up of the pupil body. It is recommended that subsequently a formula funding approach be adopted in order to reduce differences in academic achievement among schools in selected provinces.

E. Innovation, Autonomy & Accountability Can further improvements in achievement be made even after the actions already suggested (improving teachers’ subject knowledge, applying performance and resource standards, reducing the gap between the top and bottom by first equalizing inputs to all schools and subsequently introducing a targeted education program for schools performing below the mean) are carried out? Once all children are in school and the nation’s resources are being distributed to schools equally and efficiently, attention turns away from issues of access and availability to those of quality, performance and results. In the process, the education system needs to adapt and change. A central part of this change is the move from concept of equity of inputs to one based on equity of outcomes or learning achievement. But equalizing achievement often requires different amounts of inputs to schools and thus, this recommendation may appear to be in contradiction with those made earlier which talk of equalizing inputs. This contradiction can be resolved by carefully sequencing the recommended actions. Reference has already been made to the need for equalizing inputs to all schools to ensure that all reach at least the minimum performance standard, and to the idea of a targeted education program for low performing schools. Vietnam has still some way to go before it can say that inputs have been equalized or that it has a well-targeted program that addresses the needs of weakly performing or disadvantaged schools. In this section then, attention is paid to the provision of extra resources to schools, classes and individual pupils in need of increasing their achievement. In order to raise the level of the lowest schools, policy

16


makers may for example need to attract ‘excellent’ teachers and managers from elsewhere in the province to work in those selected poorly achieving schools and districts. Does the Vietnamese education system allow sufficient flexibility to start moving beyond the original concept of equity of inputs? As with many countries, in order to build and to reinforce its national identity, Vietnam harmonized its various curricula and programs and, to ensure uniformity and standardization, it adopted a tightly defined pedagogical approach typified in the ‘model school’. All schools attempt to follow the teaching methods and operating procedures laid down for the fully resourced model school. But the resources needed to operate as a model school are beyond those available to most schools. In addition, the standardized and uniform approach, while it has brought all schools onto a single national program, makes little allowance for local adjustments and gives very limited autonomy for a school to follow a more flexible process that corresponds to local needs. Thus for all except the model school itself, the standardized, uniform, ‘model’ approach that is followed is restrictive and does not foster innovation, allow flexibility, nor does it provide a mechanism to deliver basic service standards to all pupils in all schools. Teaching methods: It is well-known to teachers that pupils are different and that one must vary the methods of instruction for different educational objectives to different pupils. Although frontal teaching is good for some educational objectives, it is small group work or individual work that is better for other objectives. Teaching is not only concerned with raising the cognitive level of achievement but also with producing an interest in learning the subject, in learning in general and a liking for school. In some cases it might even be necessary to guide the children how to behave in school and mix with the other pupils. These tasks require a

very good relationship between the teacher and their pupils. When only one method of teaching has been prescribed, it can militate against the required flexibility and initiatives that teachers may need to take. Given the standardized, pre-defined approach to teaching methods that has been followed in Vietnam prior to the introduction of the new curriculum, it is recommended that the MoET review the need for a policy statement on the desirability of innovation and autonomy, accompanied of course by suitable accountability measures. This can also be reflected in the content of the teacher pre- and in-service training programs. In addition, pupils with low achievement will need a great deal of extra support if they are to catch up with their peers. These pupils are usually from the isolated schools and from disadvantaged regions. They have accrued an accumulated deficit over time. Mobilizing and delivering the extra support that is needed may be done best through local level innovations that adopt a flexible approach to learning. School culture and extra-curricula activities: There is an intangible factor in schools that contributes to their success. It is sometimes called the ‘culture’ of the school and is often attributed to the style and dynamism of the head teacher and district leaders. Inspectors sometimes say that when they walk into a school it is possible to ‘feel’ if the children are busy, committed and eager to learn. One of the questions in the study in part dealt with the kinds of factors that are sometimes associated with such a learning culture. It asked about the children producing their own school magazine and holding story-telling competitions and general knowledge quizzes. It was seen, for example, that only 25 percent of pupils were in schools where there was a school magazine. But one might have asked about other extra-curricula activities such as reading clubs, chess clubs, mathematics clubs, school theatrical plays or having a school orchestra. It is this diversity of

17


activities and the children’s commitments to them that are associated with good schooling. To achieve these kinds of activities a certain amount of autonomy is needed, together with the flexibility to introduce local innovations. Many school directors may need to be encouraged to initiate these kinds of innovative activities. For some of these extra-curricula activities and clubs they may wish to invite village artisans into the schools. They must be free to do this without seeking multiple or bureaucratic permissions, often connected with form-filling. School management autonomy should be matched by greater accountability to parents, to the community and to the administrative hierarchy. One problem in some of the isolated areas is that girls, often from traditional or minority family groups, do not go to school or drop out of school early. School intervention programs have been effective in raising girls’ school attendance. Such programs first encourage parents to change their behaviors toward their children’s schooling. For example, programs where the mothers were encouraged to ask the children to read to them or to tell the story that they have just been reading were found to have a direct and positive effect on improving their children’s reading achievement. Second, these programs encourage mothers to send their daughters to school and to persuade their neighbors to do the same. This kind of mobilization campaign is already familiar to Vietnam and has been used in the past to good effect. The MoET and the provincial offices may consider targeted campaigns of this type in certain zones and link these to other interventions mentioned in the previous section on ‘closing the gap’. It is suggested that the MoET introduce more flexibility in the teaching process rather than having the ‘standard’ approach, more flexibility for provinces to move resources among schools, and more school-level autonomy -- balanced by greater accountability -- and flexibility in the ‘culture of the school’ such that head

teachers can introduce innovations and many school activities that are linked to learning, such as school magazines and reading clubs and the pupils having a small drama club and so on. F. Measuring Pupils' Achievement With the completion of this study the Vietnamese Government and academic community have completed a tremendous task and they deserve to be congratulated. The information collected through this study and analyzed by the national researchers constitutes the first comprehensive, representative, scientific study of education quality in Vietnam. By carrying out this study, Vietnam now has the basic building blocks, methodology, procedures, and a large part of the professional and scientific knowledge necessary for building and operating a national system of monitoring and evaluation at all levels of the education system. Indeed, a repeat on a smaller scale of such a study is already planned later in this decade when the primary curriculum has been fully implemented and the pilot program of professional development programs for serving teachers have been completed. So what should now be the next steps to capitalize upon the knowledge and skills built through this study? Earlier this report reviewed the functions that support the governance and promotion of effective education, such as policy development, selecting and applying performance standards, and coordinating and regulating action. A further essential group of activities needed to ensure effective education delivery systems relates to monitoring, evaluation and accountability. This second group of activities spans three broad areas: measuring performance; analyzing performance (to see divergence between standards and performance); communicating performance (informing stakeholders of the system performance). Measurement and analysis of performance are required to produce valid and reliable

18


results for those who must take decisions about the educational achievement in the schools of the country. The measurement of performance has three strands. The first is frequent measurement of performance throughout the academic year as each module of instruction is completed by a class (often two weekly or monthly modules of work in the class group). This is often done through short tests of pupils administered by teachers for different combinations of learning modules. The second strand of performance measurement is often the annual measurement of performance at the district or more likely the provinces, often undertaken at the end of the academic year. The third stand is less frequent and looks at the overall system performance at the national level where it is probably sufficient to have national monitoring of educational quality conducted every three to five years and based on a sample survey rather than a complete pupil or school census. None of these three strands of performance measurement are “high-stakes” or are used to certify or document individual performance with diplomas or certificates. The analysis of performance will be slightly different for each of the three strands. For the first strand, it is the teachers who must look at the results. They will be looking for the percentage of their class pupils who have mastered each objective and skill level. They will also see the kinds of errors made by the pupils (this assumes that the test developers will construct diagnostic items). Finally, the teachers will be able to see which pupils are ready to move to instruction at the next skill level and where remedial work is needed with some pupils. For the second strand, the provincial officials will be interested in the identification of lowly achieving pupils, schools and districts so that they will know what kinds of extra resources and work will be required. They may be able to identify particular errors being made in groups of schools. For the third strand, national tests

should be produced in key subject areas and the analysis will be aimed at the identification of emerging system-wide problems and broad issues related to curriculum development, related changes in instructional methods, and at whether the education system’s overall goals and objectives, especially in terms of quality, are being met throughout the country. The sector ministry will also wish to look at the relative quality of education across provinces and areas within provinces to identify those with low scores and therefore those in need of help. The use of teacher administered tests will represent a new approach in the classrooms. The overall aim is to furnish information to the teachers for them to know where each of their pupils is at regular intervals throughout the school year. One important point will be at the beginning of the school year. This will allow the teacher to know what he or she has to do in order to have all of the pupils reach the higher levels of achievement. The curriculum group will need to work together with a testing center in order to produce the teacher administered tests. This kind of approach certainly worked very well in South Korea when the Korean Education Development Institute (KEDI) produced these kinds of tests together with remedial and enrichment materials for all key subject areas. It was found that even within classes the pupils were at very different levels. This implies that the teacher may have to use different instructional approaches to have everyone learn to the next level. As was mentioned in the previous section, pupils are different and learn in different ways. The approach of requiring teachers to follow a single standardized way of teaching for all pupils will not necessarily work. Hence, there is a need for both remedial and enrichment materials and tests that allow discrimination between pupils’ abilities. The communication of results will be for the MoET to decide. Teachers will communicate individual results to pupils and

19


Monitoring and evaluation of education performance is often done either by a specialized unit of the education ministry or by an independent education quality assurance agency. The MoET is already moving in this direction through the creation of the testing and accreditation section and the reorganization of NIED and NIES. These changes present a good opportunity to formalize testing and monitoring and evaluation functions and to build a modern professional capacity in this specialization. Most skills are available in Vietnam but are dispersed across different institutions or the staff qualified to work in evaluation and testing are presently working on other topics. The bulk of the technical expertise necessary to conduct thorough evaluations of performance will need to be assembled at the national level within the MoET or eventually in an independent education inspection agency.

aggregate class results to the school head as well as to the parents. Provincial authorities will communicate the results to the district offices as well as to other parts of the provincial administration and the national education ministry. At the national level it will be important to communicate the results to the various divisions of the Ministry, to the education community, to teachers and to the public at large. Special feedback of information will often be passed from the national authorities to the curriculum group. Specialized item analysis, or error analysis of test items, provided to the curriculum teams can assist them to improve the textbooks at the appropriate time. The improvement is usually in terms of sequencing the material as well as about typical errors that have been allowed to exist in the teaching. Where should the function of monitoring and evaluation of education performance be located? This particular study was a one-off, freestanding study, financed through an externally supported project with loans and grants that provided external technical assistance and advisory services. The substantive work was carried out by Vietnamese researchers and educators. Nevertheless, the survey and analysis were not integrated into the regular annual program of the MoET, nor were they anchored to a particular institutional home. The study has been instrumental in building the core technical competencies needed to undertake future studies of monitoring and evaluation of education performance. But in order to mainstream this function, and ensure its sustainability and permanence, further capacity building and linkage to an institutional location need to be made. The skills required are in instrument construction, probability sampling of schools and pupils, data collection, entry, cleaning, and weighting as well as file merging and finally in the statistical analyses of the data in order to provide compact information.

This national quality assurance group would doubtless need some technical help for a three to five year period in test construction, test scaling, and large-scale data collection, as well in data analysis and reporting to education authorities and school communities. It will be important to have personnel at the national and provincial levels who are able to read the research results and understand them and be able to take decisions on remedial action. Such persons will have to be identified and trained and again external partners will be needed to assist in this capacity building exercise. It must be recognized that in information gathering, the above types of studies will not necessarily furnish all of the different kinds of information that are useful to a system of education. For example, it was seen that there were many pupils in schools where the school head perceived behavioral pupil and teacher problems. In such cases, it is often wise to conduct small additional studies focusing only on one aspect of the research, in this case the behavioral problems. Other vexing problems are the impact of extra

20


V. Conclusion tuition on achievement, the role played by user charges and other fees, or the kinds of teaching approaches that pay off with different kinds of pupils. Earlier research suggested that pupils at different levels of skills required different instructional methods;33 further experimental work on this issue would be helpful. To avoid duplication of effort and to ensure a coherent approach that minimizes the burden on teachers and school directors, the MoET could designate a unit responsible for coordinating all of the studies and summarizing all of the results into periodic reports.

To raise the achievement level of disadvantaged students in poorly performing schools is a major challenge. It requires a lot of work and great commitment by all concerned at each level of the system. As more information becomes available through scientific, representative surveys, the MoET is beginning to examine this new empirical data more and more and use such information to guide and instruct their decision-making process. Informed decision-making based on solid facts and figures is always to be preferred to non-informed or intuitive decision-making. The research study reported here set out to bring information to the authorities that was a direct response to the questions raised by senior education managers. The study has provided a comprehensive and scientific baseline measurement of educational quality in primary education not only for the nation as a whole but also for each of the provinces.

The quality assurance capacity of the Vietnamese education system needs to be built up, consolidated and experts assembled and located in one place in a unit that could become the core of a national center. The consolidated unit should construct three types of diagnostic tests (rather than certificative tests). The first will be tests for teachers to use in school during the course of their year’s teaching in order to see what steps to take depending on the skills that their pupils have and have not mastered. The second will be for provinces for the provincial authorities to identify schools and districts where achievement is poor and where remedial action will be required. Finally, it will produce tests to be used in national quality assurance programs that will not only identify weak schools but also provinces where extra help might be needed. The members of the center will need to work in cooperation with the curriculum developers and the group should be nurtured and supported with assistance to develop the necessary skills and grow into a national center.

So, what should be the next step and where should the Government start? First, the study has provided two immediate opportunities; one to disseminate the information and results to provincial education managers and second to use the methodological volume and instruments to strengthen further national capacity and train experts in the field of learning assessment. Beyond this, central and the provincial levels of government will no doubt wish to address some of the issues that have been highlighted and the recommendations made in this report. A list of the major factors that are associated with large achievement differences among pupils and schools has been presented. Individual interventions have been discussed and the need for a combination of these measures into a coherent holistic package has been recommended. The areas with large proportions of non-Kinh and also with large achievement differences between the Kinh and non-Kinh pupils are: Lang Son; Hoa Binh; Lai Chau; Kon Tum; and Tuyen

33 Assessment Research Center, 2003, Instructional methods in Vietnam Pupil Achievement: supplementary analyses, Melbourne: University of Melbourne.

21


Quang. These areas could be designated Educational Priority Areas (EPA), and could be where the first efforts are made. Meanwhile, in parallel with the improvement and compensatory programs

that have been suggested in this paper, measurement and analysis of education quality and performance should be expanded with the application of the techniques used in this study to other levels and types of education and training in Vietnam.

22


Appendix 1: Variation among schools (rhos) within countries in reading comprehension How did the Vietnam intra-class correlations (rhos) compare with the rhos in other countries? In 2001 the IEA undertook a large survey or reading in Grade 4 in many countries. The rhos for those countries have been given below in Table 1.1 Table 1.1: Intra-class correlations for Grade 4/5 in the IEA PIRLS Study (data collection 2001)34

Argentina 0.418 Latvia 0.213 Belize 0.348 Lithuania 0.214 Bulgaria 0.345 Moldova 0.395 Canada35 0.174 Morocco 0.554 Colombia 0.459 Netherlands 0.187 Cyprus 0.105 New Zealand 0.250 Czech Republic 0.157 Norway 0.096 England 0.179 Romania 0.351 France 0.161 Russian Federation 0.447 Germany 0.141 Scotland 0.179 Greece 0.221 Singapore 0.586 Hong Kong (SAR) 0.295 Slovakia 0.249 Hungary 0.222 Slovenia 0.087 Iceland 0.084 Sweden 0.132 Iran 0.382 Turkey 0.271 Israel 0.415 Macedonia 0.424 Italy 0.198 United States 0.271 Kuwait 0.334

It can be seen that the among-school variation ranged from .084 in Iceland to .586 in Singapore. Most were in the region of .10 to .25 and this is what is generally considered normal for primary school. However, this study showed that among-school variation in Vietnam, with a rho of .66, was larger than any of the PIRLS study countries.

34 In many of the IEA countries an intact class was taken thus underestimating the within school variance that in turn results in an overestimation of the between school variance. 35 Canada consisted only of the provinces of Ontario and Québec.

23


Appendix 2: Reading skill levels and mathematics skill levels

Reading Skill Levels Level 1 Matches text at word or sentence level aided by pictures.

Restricted to a limited range of vocabulary linked to pictures Level 2 Locates text expressed in short repetitive sentences and can deal

with text unaided by pictures. Type of text is limited to short sentences and phrases with repetitive patterns.

Level 3 Reads and understands longer passages. Can search backwards or forwards through text to for information. Understands paraphrasing. Expanding vocabulary enables understanding of sentences with some complex structure.

Level 4 Links information from different parts of the text. Selects and connects text to derive and infer different possible meanings.

Level 5 Links inferences and identifies an author's intention from information stated in different ways, in different text types and in documents where the message is not explicit.

Level 6 Combines text with outside knowledge to infer various meanings, including hidden meanings. Identifies an author's purposes, attitudes, values, beliefs, motives, unstated assumptions and arguments.

Mathematics Skill Levels Level 1 Reads, writes and compares natural; numbers, fractions and

decimals. Uses single operations of +, -, x and / on simple whole numbers; works with simple measures such as time; recognizes simple 3D shapes.

Level 2 Converts fractions with denominator of 10 to decimals. Calculates with whole numbers using one operation (x,-,+ or : ) in a one-step word problem; recognizes 2D and 3D shapes.

Level 3 Identifies place value; determines the value of a simple number sentence; understands equivalent fractions; adds and subtracts simple fractions; carries out multiple operations in correct order; converts and estimates common and familiar measurement units in solving problems.

Level 4 Reads, writes and compares larger numbers; solves problems involving calendars and currency, area and volume; uses charts and tables for estimation; solves inequalities; transformations with 3D figures; knowledge of angles in regular figures; understands simple transformations with 2D and 3D shapes.

Level 5 Calculates with multiple and varied operations; recognizes rules and patterns in number sequences; calculates the perimeter and area of irregular shapes; measurement of irregular objects; recognizes transformed figures after reflection; solves problems with multiple operations involving measurement units, percentage and averages.

Level 6 Problem solving with periods of time, length, area and volume; embedded and dependent number patterns; develops formulae; recognizes 3D figures after rotation and reflection and embedded figures and right angles in irregular shapes, data from graphs and tables.

24

Working Paper Series 2004-1 Disability Issues in East Asia: Review and Ways Forward 2004-2 Skills and Growth in Cambodia 2004-3 Evaluating the Performance of SGP and SIP: A Reivew of the Existing

Literature and Beyond 2004-4 Vietnam Reading and Mathematics Assessment Study (Volume 1)

050-2791 vietnam vol1.qxd 8/24/04 9:24 am page 1

Documents