Document of

The World Bank

FOR OFFICIAL USE ONLY

FIEPOY

Report No. 5004

PROJECT COMPLETION REPORT

PHILIPPINES

MASS MEDIA PILOT PROJECT(LOAN S8-PH)

March 23, 1984

Projects DepartmentEast Asia and Pacific Regional Office

This document has a restricted distribution and may be used by recipients only in the performance oftheir official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

PC99373/ED-91/D2111/61

CURRENCY EQUIVALENT

Currency Unit = Peso (P)

US$1.00 = e 11.0 (1983)

ACRONYMS

CET - In-Service Teacher Training (commonly referred to asContinuing Education of Teachers)

DEC - Department of Education and CultureECO - Education Communications OfficeEDPITAF - Education Development Projects Implementing Task ForceMECS - Ministry of Education, Culture and Sports

(successor of DEC)PTC-RD - Philippine Training Center for Rural DevelopmentRATES - Radio-assisted Teaching in Elementary SchoolsSOUTELE - Survey of the Outcomes of Elementary EducationUPIMC - University of the Philippines Institute of Mass

Communications

SCHOOL YEAR

July-April

FOR OFFICIAL USE ONLY

PHILIPPINES

MASS MEDIA PILOT PROJECT (LOAN S8-PH)

PROJECT COMPLETION REPORT

Table of Contents

Page No.

PREFACE . . . . . . .. . . . . . . . . . . . . . . .. . . i

BASIC DATA SHEET . . . . . . . . . . . . . . . . . * * * * ii

HIGHLIGHTS . . . . . . . . . ... . . . . . . . . . . . . . . . . iv

I. -PROJECT BACKGROUND . . . . . . . . . . . . . . . . . ... 1

Sector Setting . . .. . . . . . . . . . . . . . . . . . 1Issues in Elementary Education. . . . . . . . . . . . . . 1

Project Generation . . . . . . . . . . . . . . . . . . . 2

Project Objectives and Strategies . . . . . . . . . . . . 2Project Description . . . . .. . . . . . . . . .. . . 3

II. PROJECT IMPLEMENTATION . . . . . . . . . ... . . . . . 4

Overview ... 0 0...... 4Project Management . . . . . .. . . 0 . * *.*.. 5In-service Teacher Training (CET) . . . . . . . . . . 5Radio-assisted Teaching in Elementary Schools (RATES) . 6

Evaluation . . . . . . . 6Studies. ... . .. . .. 6Engineering Designs .; ... . . . . . . . 7Procurement . . . . . . . . . . . . . . 7Technical Assistance .... . .. . . . . . * ... * * . *. 7Bank Performance. . . . . . . . ... . . . 8

Covenants . . . . .. . . . .. . . .. ... . * 9Implementation Period . . . *. . . . .. . * . g 9

III. PROJECT COSTS . . . . . . . . . 9 . . . . . . . . . . . . . 9

Project Costs by Type of Expenditure . . . . . . . . . . 9Disbursements . . . .. 0.. .. . . . . . . . . . .. . . 10Recurrent Costs per Student . . ... . . . . . . . . . . . 10

This document has a restricted distribution and may be used by recipients only in the performance oftheir official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Page No.

IV. OPERATING OUTCOMES . . . . . . . . . . . .. . . . . . . . 10

Overview . . . . . . . ... . . . . . . . . . . . . . . .10Attainment of Objectives . . . . . . . . . . . . . . . . 11In-service Teacher Training (GET). . . . . . . . . ; . . 12Radio-assisted Teaching in Elementary Schools (RATES). . 12Evaluation of CET and RATES. . . . . a . . .. . . . . . 14Study on Communications Technology for Rural Education . 15Future of the Training Materials . . . . . . . . . . . . 16

ANNEXES

1. Organization of the Education Communications Office (ECO). 172. Local Educational Administration . . . . . . . . . . . . . . 18

ATTACHMENT I

Comments from the Borrower ... . . . . . . . . . . . . . . 19

PHILIPPINES

MASS MEDIA PILOT PROJECT(S8-PH)

PROJECT COMPLETION REPORT

Preface

This is a Project Completion Report (PCR) for the Mass Media PilotProject in the Philippines, for which Loan S8-PH was approved in March 1978.The loan amount was for US$2.0 million; of this amount US$1.15 was disbursedand US$0.85 million cancelled. The Closing Date was December 31, 1981.

The PCR and the Highlights were prepared by the Bank Group's EastAsia and Pacific Regional Office.

The PCR is based on: (a) a draft completion report prepared by theEducation Communications Office (ECO) of the Borrower; (b) informationobtained by a Bank consultant who visited the Philippines in June/July 1982;(c) an evaluation report prepared by the University of the PhilippinesInstitute of Mass Communications (UPIMC); (d) records in the Bank and ECO,including the President's Report, P-2250-PH of March 8, 1978 and the LoanAgreement dated April 21, 1978; and (e) discussions with Bank and ECO staff.

The Operations Evaluation Department decided not to audit theproject and sent the PCR to the Government for comment in November 1983.Comments received from the Government have been taken into account infinalizing the report and are reproduced as Attachment I to the PCR.

a

- ii -

PROJECT COMPLETION REPORT BASIC DATA SHEET

Philippines Fifth Education Project (Loan S8-PH)

KEY PROJECT DATA

Appraisal Actual orItem Estimate Current Estimate

Total Project Cost (US$ million) 3.86 2.86Cost Underrun (X) - -26%Loan/Amount (US$ million) 2.0 1.15Disbursed ) April 14, 1982 1.15Cancelled ) May 3, 1982 0.85Repaid to } December 31, 1982 -

Date Physical Components Completed 12/01/81 12/01/81Proportion Completed by Above Date (X) 100XProportion of Time Overrun (%) nilEconomic Rate of Return (Z) n.a. n.a.Financial Performance n.a. n.a.Institutional Performance Good Satisfactory

OTHER PROJECT DATA

Original Actual orItem Plan Revisions Est. Actual

First Mention in Files 03/01/76Government's Application 12/23/77Negotiations 02/22-23/78Board Approval 03/21/78Loan Agreement Date 04/21/78Effectiveness Date 07/21/78 08/22/78Closing Date 12/31/81 05/03/82Borrower Government of the PhilippinesExecuting Agency Education Communications Office of

EDPITAF Projects OfficeFiscal Year of Borrower January-DecemberFollow-on Project Name Elementary Education Sector LoanLoan Number Loan 2030-PHAmount (US$ million) 100.0Loan Agreement Date 08/25/81

MISSION DATA

Sent No. of No. of Date ofItem by Month, Year days Persons Staff-weeks Report

Appraisal Bank 09/77 26 2 10.40 11/17/77Total 10.40

Supervision I Bank 05/78 4 1 0.80 06/09/78Supervision II Bank 01-02/79 10 2 4.00 03/29/79Supervision III Bank 11/79 6 1 1.20 11/12/79Supervision IV Bank 06/80 6 1 1.20 06/30/80Supervision V Bank 02/81 5 1 1.00 02/20/81Supervision VI Bank 11/81 1 3 0.60 01/07/82Completion Bank 06-07/82 9 1 1.80

Total 10.60

COUNTRY EXCHANGE RATES

Name of Currency (Abbreviation) Philippine Peso (P)Exchange Rates:

Appraisal Year Average US$1 - P 7.4Intervening Years Average US$1 - P 7.5Completion year US$1 - P 8.2

- iii -

PHILIPPINES

MASS MEDIA PILOT PROJECT(S8-PH)

ALLOCATION OF LOAN PROCEEDS (US$)

Original ActualCategory allocation disbursements

Equipment andmaterials 1,060,000 } 954,460.64

Technical assistance 540,000 } 199,530.83

Unallocated 400,000 }

Subtotal 2,000,000 } 1,153,991.47

Cancelled - } 846,008.53

Total 2,000,000 } 2,000,000.00

- iv -

PHILIPPINES

MASS MEDIA PILOT PROJECT (S8-PH)

PROJECT COMPLETION REPORT

Highlights

A loan of US$2.0 million to assist in financing a Mass Media PilotProject in the Philippines, was approved in March 1978 and signed in April1978. The original closing date of the loan was December 31, 1981 and thiswas not formally changed. The loan account was closed on May 3, 1982, whenUS$1.15 million had been disbursed and US$0.85 million cancelled. The finalproject cost was US$2.86 million, 26% lower than the appraisal estimate. Thelower cost was largely due to a reduction in the amount of technicalassistance.

The project was designed to develop and evaluate the cost effective-ness of using educational radio: (a) for in-service teacher training; and (b)as a direct medium for classroom teaching. In broad terms the project has metthese objectives. Evaluation results show that in-service teacher trainingthrough radio lessons combined with provisions of textbooks to children at a1:2 ratio (i.e. one textbook per subject and grade for every two pupils) ismore cost effective than direct transmission to pupils (para. 4.10). The perpupil cost of transmitting programs directly to students was found to be morethan ten times higher than that of the cost of in-service teacher training(para. 4.10). Materials developed under the project are being used on anational scale to retrain teachers to use the new curriculum. In thisrespect the project has had a major impact on national education (para. 4.13).Mass media, however, remains a supplemental activity and does not play a majorrole in education.

One lesson learned from the project is that highly innovativeprojects need close supervision and monitoring. The evaluation design shouldhave been reviewed at various stages of implementation to ensure the necess-ary adjustments were made.

The pilot project was a success in the sense that it tested thefeasibility and effectiveness of using mass media as a means of transmittinginstruction. Evaluation results show that the instructional medium andmaterials were effective but problems were encountered with radiotransmission, which made wide-scale implementation difficult.

Another lesson learned concerns procurement, where low pricedtape-recorders were chosen rather than higher priced machines which used lessamounts of consumable materials (batteries). Life cost evaluation methodologymight have been more appropriate in this instance (para. 2.11).

Experience gained under this project suggests that the Bank is notthe ideal agency to carry out small-scale, innovative projects. Other insti-tutions which have greater technical assistance resources probably have anadvantage in this kind of activity.

I. PROJECT BACKGROUND

Sector Setting

1.01 At the time of project generation (1976/77) the Philippine educationsystem was characterized by high enrollment ratios at all levels and the

formal school system had accomplished universal primary enrollments more thana decade earlier. Secondary enrollment ratios were above 60% and tertiary

ratios around 20% of the relevant age group. More than half of the secondary

students and 90% of tertiary students were in private institutions. Enroll-ments of females equalled or surpassed those of males at all levels.Student:teacher ratios were generally good. Nonformal education and training

had also reached impressive proportions, with around two million personsreceiving instruction annually.

Issues in Elementary Education

1.02 Despite the above-mentioned high participation rate, much needed tobe done in the elementary sector to improve the quality and coverage ofeducation. A basic problem in primary education was the low quality of

instruction, particularly in rural areas. Although the majority of teacherswere college graduates, the quality of preservice training had been weak.Some in-service teacher training programs existed, but they were costly,limited to the summer term and the program content tended to become diffused

as it passed from the national to local level. Only 40% of primary teacherssurveyed in 1976 had received any in-service training. Student achievement

averaged only 50% of the expected standard items; the poorest overall

performances were in reading, languages and mathematics. Substantialvariations in achievement were found between urban and rural areas, with test

scores ranging from an average of 30% in rural areas to 70% in Manila. Ways

needed to be found to improve educational quality and remove the disparitiesbetween urban and rural areas.

1.03 The Role of Mass Media in Education. Media coverage was morelimited than might be expected. Despite high literacy levels, the 60 Philip-pine newspapers and periodicals reached only about a quarter of the adult

population while circulation of printed educational material was but afraction of this. Formal educational broadcasting was undertaken by only one

television station and 23 radio stations in contrast to nonformal education

broadcasts that were made by more than 200 stations. Geographic coverage waslimited and many rural areas had no radio coverage at all, much less

television. Evaluation of direct primary school instruction via radio in theearly 1970s showed that the programs were educationally successful. However,due to limited coverage, unsystematic use, and inadequate programming

capacity, the Department of Education and Culture discontinued formal

educational radio broadcasts in 1975, pending comprehensive reassessment ofthe role of educational radio. The limited coverage of educational broad-

casting meant that five million out-of-school youths were out of reach of theGovernment and the quality of both formal and nonformal education in ruralareas was low.

Project Generation

1.04 The Bank-financed Textbook Project (Loan 1224T-PH) which becameeffective early in 1976, included resources for a pre-investment study oncommunication technology for education. During the first year of implemen-tation of this loan, EDPITAF organized a study team to do the preinvestmentstudy. This team studied areas in which communication technology couldprovide solutions to educational problems. In November 1976 the teampresented the range of possibilities to the Office of the President of thePhilippines in order to obtain directives for focussing the study towardsspecific project proposals. The Government instructed the team to focus onthe use of communication technology for two purposes: (a) quality improve-ments in basic education, particularly through in-service teacher training;and (b) educational support for rural development. The study team's proposalfor a National Educational Communication System reflected these emphases,including provision for development of an educational communicationsinfrastructure and for phased implementation of its recommendations.

1.05 The study team's conclusions led to an understanding in June 1977between EDPITAF and the World Bank that the Bank would support a mass mediapilot project and by August 1977, the EDPITAF study team had prepared a pro-posal for such a project. A Bank mission appraised the proposal in September1977 and reached a general understanding with EDPITAF concerning the magni-tude, composition, and timing of the pilot project. The appraisal missionrecommended that the project be included as an educational radio component ofa proposed Rural Infrastructure Development Project. However, in December1977, Bank management decided to process the mass media pilot project as atechnical assistance loan of US$2.0 million, rather than as a component ofthe proposed Rural Infrastructure Project because it was considered to bemore difficult to manage if it were integrated into this project. ThePhilippine Government endorsed this decision on December 23, 1977.

Project Objectives and Strategies

1.06 The main objective of the pilot project was to establish a basis forpossible nationwide implementation of educational radio as a means to improveeducational quality in the Philippines. Two different approaches to improvingprimary eduation were to be developed and their cost effectiveness evaluated.The first approach was indirect use of radio for in-service teacher training,with the expectation that additional teacher training would improve thequality of instruction received by primary school students. The secondapproach was to broadcast radio lessons directly to students as a component oftheir classroom activities. Improvement in language training was one of thecentral objectives of the mass media program. The proposed project thereforefocussed on two predominantly rural provinces where Pilipino was not the major

- 3 -

language. These provinces were Pangasinan on the island of Luzon, and Leytein the Visayas (only 55% of the population in Pangasinan and about 28% in

Leyte speak Pilipino). A third province, Maguindanao, in the third majorregion of the Philippines (Mindanao) was later added to the project at the

Government's request.

1.07 The pilot project had two other objectives: (a) exploring the use ofradio for rural education as a complement to other activities - the rural andfarmer training centers financed under Loan 1374-PH; and (b) evaluation ofcommunications technologies appropriate to the Philippines.

Project Description

1.08 The project consisted of three parts: (a) In-service TeacherTraining [also referred to as Continued Education of Teachers (CET)];(b) Radio assisted Teaching in Elementary Schools (RATES); and (c) Evaluation,

Studies and Future Project Preparation.

1.09 In-service Training (CET). This component of the project stemmedfrom a need to provide in-service training to teachers within a short periodof time. Instruction was expected to be given via radio broadcasts supple-mented by printed materials and periodic meetings. The effect of thistraining on student performance would then be assessed. The proposed trainingprogram was expected to provide semester-length courses for primary schoolteachers in five subjects: Pilipino, English, social studies, mathematics andscience. The courses were designed to upgrade both teacher mastery of subjectmatter and teaching skills, and to create a favorable attitude towardsproposed educational changes. These distance study courses were to includedaily radio lessons (80 programs, 30 minutes each). The first course(Pilipino) was to be broadcast in the second semester of the 1978/79 schoolyear (July-March). Evaluated and revised broadcasts were then expected toreach a larger audience over expanded geographical areas; concomitantly,courses were to be developed in the other subject areas. The target was toreach 12,600 primary school teachers during implementation, about two-thirdsin Pangasinan province and the remainder in Leyte province. Teachers wouldreceive either in-service or academic credit for the courses.

1.10 Radio-assisted Teaching in Elementary Schools (RATES). This compo-nent of the project was to employ radio for direct classroom instruction. Itwas expected to minimize urban/rural differences in the quality of instruc-tion. Primary school pupils were to receive daily 30-minute radio lessons inPilipino. Pilipino was chosen as the subject area because of its importanceto achievement in over half the school subjects and a pervasive low level ofachievement in communication skills in Pilipino. During the first school yearof the project (1978/79) the radio-assisted teaching series was to be devel-oped for Grade 4 students and field-tested in a small number of classrooms.In the second year (1979/80), Grade 4 Pilipino was to be revised, broadcastmore widely and evaluated; simultaneously, Grade 5 Pilipino was to be

developed. A similar pattern was expected to continue into the third year

- 4 -

(1980/81). During the pilot period, this component was expected to reachabout 2,500 classes with about 75,000 primary pupils.

1.11 Evaluation, Studies and Project Preparation. The principal purposeof the project was to provide information on the cost-effectiveness ofin-service teacher training and radio-assisted teaching in primary schools sothat informed decisions could be made on the use of media (para. 1.09). Theproject, therefore, included a strong evaluation component. The basicevaluation was designed to determine how in-service training through radio andtextbooks, separately or in a variety of combinations, affected studentperformance. Tests were to be administered to students before and after theproject to measure its impact on student achievement.

1.12 A separate activity, included under the "studies" category, wasdesigned to explore and evaluate the use of media-based programs to complementthe activities of the Philippine Training Center for Rural Development(PTC-RD), a component of the Fourth Education Project (Loan 1374-PH). Thestudy was intended to explore the potential of radio to reinforce, complementand supplement effort of the PTC-RD. While categorized as a study duringappraisal, this component was action research on the use of radio in adistance learning system for small farmers. It is referred to below as the"Communication Technology for Rural Education Study (CTRE)".

1.13 Several smaller studies and experiments were planned on the techni-cal aspects of the communication support system. These can be categorizedunder four headings: methods of production of mass media material, distribu-

tion/transmission options, arrangements for reception of mass media communi-cations, and two-way administrative communication for implementation ofnationwide education programs. The purpose of these studies was to provideeducational and technical planners with an up-to-date assessment of the costand performance characteristics of available technological options (includingsatellite communications).

1.14 An educational materials project (including mass media and text-books) was expected to be prepared on the basis of the findings of the above-

mentioned studies. The loan was signed on April 21, 1978 and became effectiveon August 22, 1978.

II. PROJECT IMPLEMENTATION

Overview

2.01 With a couple of exceptions, the project was generally implementedas envisaged at appraisal. The proposed management team was set up on time;one province was added to the project; the various studies and evaluationswere carried out; and the project was completed on schedule. It was a pilotproject based on radio transmission and printed material, and in some areasoutside the regions where evaluation was conducted, tape recorders were used.Cost parameters of the project also underwent a major change. There was a

- 5 -

large under-utilization of technical assistance, which led to a major costsaving but also negatively affected the quality of the summative evaluation.

Project Management

2.02 A separate Education Communications Office (ECO) was established andfinanced within EDPITAF to implement this project. At appraisal ECO wasdesigned with a small staff of ten, but with access to staff resources inEDPITAF and the Textbook Board Secretariat (which is responsible for primaryand secondary textbook development and production). A sizeable amount ofspecialist services (9 man-years) was also envisaged to further strengthenECO's staff (para. 2.13). During implementation ECO grew temporarily into astrong unit of education specialists, media specialists, research and develop-ment staff planners, service staff and field staff. At its peak ECO had65 staff and was much less dependent on outside staff resources than origin-ally envisaged. The lifespan of this organization was, however, limited (onlythree staff remained in July 1982). Most of ECO's staff have been absorbedinto other parts of the Ministry of Education, Culture and Sports (MECS)which, together with EDPITAF, carry out tasks initiated under the aegis ofECO. A large number of the project staff were absorbed by the DevelopmentAcademy of the Philippines which was contracted by the MECS to implement theTeacher Reorientation program under the Elementary Education Sector Loan(Loan 2030-PH). Thus the organization was temporary but its expertise is inuse in other parts of the education system.

In-service Teacher Training (CET)

2.03 The CET program was implemented over a three-year period: 1978/79-1980/81. The first cycle, broadcast in 1978/79 in Pangasinan, was formative.During this cycle, information gathered about the lessons was used to makeprogram improvements. The second cycle transmitted the tested and modifiedmaterial in 1979/80 to a larger audience of about 1,000 teachers. This wasoriginally planned for Pangasinan and Leyte only, but teachers in a provincein the third major region of the Philippines (Maguindanao) learned about thetest phase of the project through the Regional Education Officer, asked toparticipate and were included. The third cycle was disseminative and aimed atabout 2,000 teachers. Initially, all lessons were broadcast, but duringimplementation ECO encountered numerous transmission quality and schedulingproblems at the contracted radio stations. To overcome these problems inareas which were not part of the experimental comparison, ECO resorted to theuse of tape recordings of the lessons that had been prepared for broadcasting.

2.04 The CET program consisted of a series of 45 radio lessons,transmitted to teachers in each of five subject areas, namely Pilipino,Educational Trends, Social Studies, English and Math/Science. Supplementingthe transmissions were a series of three teacher's manuals in each subjectcovering the 45 lessons. The scripts for the broadcasts were written by ECOstaff assisted oy foreign experts and local teachers and supervisors. The

- 6 -

lessons were broadcast so that in the three selected geographical areas,teachers in schools that were issued with radio receivers could participate inthe CET program as well as use the receivers for RATES. To encourage them todo so teachers were awarded certificates after each completed semester andcredit was given towards promotion. With some additional work, the teacherscould also be given three credit units towards an M.A. degree. Teachersparticipating in the MA program had to attend monthly sessions of three hoursduration throughout each semester of study and to pass an examination at theaccrediting institution.

Radio-assisted Teaching in Elementary Schools (RATES)

2.05 This component explored the effective use of radio in school as aprimary support medium to assist teachers in classroom instruction. Itcovered the same three geographical areas as those for the CET programs(para. 2.03). During the first year of implementation (1978/79), formativeevaluation was conducted. The subsequent two years saw a general dissemina-tion to all classes that could be supplied with receivers. The instructionalprogram covered grades 4, 5, and 6. For each grade, 140 lessons were scriptedand broadcast, giving a total of 420 lessons. For each lesson a pupil'sworksheet was prepared and distributed. Achievement tests were designed toevaluate the impact of the broadcast lesson and worksheets. The total numberof classes covered during the project period rose from the initial 30 to2,075, consistent with the number of receivers distributed in the three areas.

Evaluation

2.06 Summative evaluation of both CET and RATES was conducted as a singlestudy carried out by the Institute of Mass Communications of the University ofthe Philippines, Diliman, under contract to EDPITAF. Formative evaluationactivities, for both CET and RATES, were conducted by ECO, as part of thedevelopment of the instructional programs. Systematic plans for formativeevaluation were developed with the assistance of local and foreign consultantsand covered both materials development and implementation activities.Similarly, formative evaluation of CTRE, the distance learning system forsmall farmers, was conducted in-house. In addition, CTRE undertook specialprocess evaluation studies to assess the progress being made in implementingthe teaching programs and to obtain preliminary assessments of effectiveness.These studies were conducted with the help of consultants.

Studies

2.07 Communication Technology for Rural Education was a two-year actionresearch component designed to explore the use of radio as a support medium:(a) to the training given to personnel being trained at the PhilippinesTraining Center for Rural Development (PTC-RD); and (b) in person-to-personcontacts in expanding the knowledge and improving the attitudes of agricul-tural extension workers and local leaders. This study was implemented by theUniversity of the Philippines at Los Banos. The programs were broadcast to

- 7 -

two centers (Bulacan and Cavite) each having both local leader and farmerfamily groups. The activities covered the writing of scripts, their broad-casting, and university staff contact with the groups in weekly dialogues.

The segment aimed at extension workers was never executed because theconstruction of the PTC-RD (aided by the Fourth Education Project) was notcompleted by the time the study was to have occurred.

2.08 The study on options in communication technology evaluated thequality and reliability of existing and proposed communications services for

the MECS.

Engineering Designs

2.09 There were only two minor items of construction in the project,namely that of an earth satellite receiving station at the MECS Regional

Office in Cotabato and a recording studio at headquarters. The Cotabatostation receives and transmits signals from and to the Indonesian earthsatellite Palapa II and links it with an existing installation near Manila and

thence by radio to a terminal in MECS. This station became operational inDecember 1980. The recording studio was intended for recording teachingprograms on tape for the duration of the project and to be of use to the MECS

thereafter. A portion of the leased office space of EDPITAF/ECO headquarterswas converted for this purpose at low cost in December 1978 but was neither

large enough to hold all the recording personnel and equipment, nor suffi-ciently soundproof. It could therefore be satisfactorily used only forreproduction and tape dubbing.

Procurement

2.10 Procurement for the project was carried out in accordance with theprocedures specified in the Loan Agreement. In addition to the earth satel-lite receiving station, expenditures covered about 2,400 radio-cassetterecorders, 171,000 batteries, 26,000 taped lessons, 16.7 million worksheets,

21,000 manuals and one set of studio recording and tape duplicating machinery.

2.11 Procurement problems included lack of local funds which caused someprocurement delays. End-users also complained about the rationale for select-ing a low-priced radio cassette receiver model, which consumes more batteries

than the second lowest-priced model. A life-span costing might have been moreappropriate in this instance to determine the most advantageous bid.

Technical Assistance

2.12 The project envisaged technical assistance and fellowship trainingand made provision for both. The purpose of each was to help in: (a)preparation of teaching materials; and (b) evaluation of the various projectelements. Recruitment of the technical assistance took two forms. One wasthe direct employment of experts by ECO. The other was the hiring by contractof the University of the Philippines Institute of Mass Communications (UPIMC),

- 8 -

and the University of the Philippines at Los Banos, each of which provided thestaff who had the required technical knowledge.

2.13 About nine man-years of foreign technical assistance was envisagedand two man-years for fellowships. Only 16 man-months of short-term foreigntechnical assistance was financed under this project because project manage-ment considered that most of the required expertise could be hired locally./1This was reflected in the staffing pattern of ECO, which had a much largerstaff than that envisaged at appraisal. Of the two man-years planned forfellowships, 18 man-months was used by 21 participants. The planned andactual utilization was as below:

Table 2.1: TECHNICAL ASSISTANCE - PLANNED & ACTUAL

Man-months Participants (Number)Planned Actual Actual

Foreigntechnical 108 16 14assistance

Fellowships 24 18 21

Of the 14 foreign experts for technical assistance, five were concerned withassisting the production and use of the teaching media and the other nine werespecialists in management or evaluation. Five were selected directly by ECOand the rest via UNESCO or other agencies. The two institutions of the

University of the Philippines provided the services of their own specialiststaff.

Bank Performance

2.14 The project originated as a study in the Bank-financed TextbookProject (Loan 1224T-PH) which became effective early in 1976. Appraisal wasdone during a 26 man-day appraisal mission in September 1977. This two-manmission reviewed the facets of Loan 1224T-PH that became the underpinnings ofLoan S8-PH. As this was considered a technical assistance project a fullappraisal report was not required. A President's Report and legal documentwere prepared. There have since been six supervision missions all undertakenby the Bank and involving 53 man-days of mission time. Because of thespecialized nature of the project, practically all these missions wereundertaken by mass-media and evaluation specialists. The heavy reliance oneducation specialists from other parts of the Bank to supervise the project

/1 A similar amount of foreign specialist services for this project wasfinanced from the proceeds of the Textbook Project (Loan 1224T-PH).

led to a less than desirable monitoring of the overall project design andobjectives by the education division (AEPED).

Covenants

2.15 There were two special conditions regarding implementation, both ofwhich were complied with. These were: (a) that ECO would arrange withspecified local agencies to carry out project studies; and (b) that theGovernment would maintain ECO during implementation with a satisfactoryorganization and staffing.

Implementation Period

2.16 The project was implemented substantially in accordance with theoriginally envisioned time frame. The originally established Closing Date wasDecember 31, 1981 and the loan account was closed on May 3, 1982.

III. PROJECT COSTS

Project Costs by Type of Expenditure

3.01 Estimates at appraisal were that the project would cost US$3.86 mil-lion with a foreign exchange component of US2.0 million or 52% of the total.Actual total cost of the project at loan closing amounted to only US$2.86 mil-lion. Table 3.1 presents a comparison between appraisal estimates and actualcosts of the project by type of outlay:

Table 3.1: ESTIMATED AND ACTUAL PROJECT COSTS BY OUTLAY(US$ million)

Actual costsas % of

Appraisal appraisalestimates Actual costs estimate

Local Foreign Total Local Foreign Total (+ or -)

Equipment 0.10 0.69 0.79 0.04 0.48 0.52 -34%Program materials 0.11 0.37 0.48 0.26 0.47 0.73 +52%Technical assistance 0.52 0.54 1.06 0.75 0.20 0.95 -10%Administration 0.76 - 0.76 0.66 - 0.66 -13%

Subtotal 1.49 1.60 3.09 1.71 1.15 2.86 -11%

Contingencies 0.37 0.40 0.77 - - - -

Total 1.86 2.00 3.86 1.71 1.15 2.86 -26%

- 10 -

3.02 Reasons for the cost under-run included: (a) reduction in theamount of foreign technical assistance estimated at about US$500,000; (b)decision to use the radio recorders from the RATES program on a double shiftbasis for the CET program at a saving of 2,900 recorders (US$200,000); and (c)reduction in the procurement of vehicles at a saving of US$50,000.

Disbursements

3.03 The rate of disbursements compared with that planned was:

Table 3.2: ESTIMATED AND ACTUAL CUMULATIVE DISBURSEMENTS

Disbursements% (actual

Year Estimated Actual compared with--- US$ million --- estimate)

1979 0.6 0.1 171980 1.3 0.6 461981 1.8 1.0 561982 2.0 1.1 55

The undisbursed amount of the loan account - US$0.9 million - was cancelledwhen the loan account was closed in May 1982.

Recurrent Costs Per Student

3.04 Annual unit costs were computed by the UPIMC as shown in Table 3.3.Recurrent cost per primary school pupil in the Philippines was about US$42 in1980./l Table 3.3 shows that using any of the combinations that involveddirect radio transmissions to the pupils (RATES) added about US$10-11 to thissum - i.e., a 25% increase in recurrent costs. Such an increase on a largescale would obviously not be feasible.

/1 Source: Basic Data, IBRD SAR Sector Program for Elementary EducationReport No. 3423a-PH, May 29, 1981.

- 11 -

Table 3.3: ANNUAL COST PER PRIMARY LEVEL STUDENT BY PROGRAM AND TREATMENT

Range of unit costsTreatment Peso US$

combination/area Low High Low High

A. PangasinanWith RATES, without CET,

2:1 textbook ratio 76 81 10.13 10.80With RATES, with CET, without

textbooks 78 83 10.40 11.06With RATES, without CET,without textbooks 74 79 9.86 10.53

Without RATES, with CET2:1 textbook ratio 5 6 0.74 0.82

Without RATES, with CET4:1 textbook ratio 5 5 0.64 0.71

Without RATES, without CETwithout textbooks - - - -

B. LeyteWith RATES, without CET

without textbooks 74 79 9.86 10.53Without RATES, without CETwithout textbooks - - - -

Source: UPIMC Final Report on Summative Evaluation. June 1982. Table 15,page 131.

IV. OPERATING OUTCOMES

Overview

4.01 The project was completed on time and at a lower total investmentcost than originally estimated. It reached the targetted audiences of elementary teachers and pupils and it introduced an earth satellite communicationsystem which the MECS is now operating. A considerable amount of data on theCET/RATES elements has also been accumulated. Project results indicate thatthe use of mass media improves pupil performance in varying degrees dependingon the audience and the concurrent use of other materials. There areindications that the combination of: (a) providing one textbook to a pair

- 12 -

of pupils; and (b) training teachers through mass media was the most costeffective delivery method. Although the project organization was disbanded,the core of the project staff continues to work on development of educationmaterials in other agencies.

Attainment of Objectives

4.02 The four-year pilot project sought to develop the use of communica-tion technology to improve educational quality. The project succeeded inachieving this objective. However, the use of radio has not been widelyadopted by the Government because of transmission problems.

4.03 The original design of the project called for using radio as themain medium of instruction. Problems with this delivery method included:(a) poor quality transmissions from the contracted broadcasting station; (b)excessive amount of power cuts; and (c) lack of advance information forteachers and group leaders on the content of the sessions. Project managementdecided therefore to produce tape recordings for most project items whichthey used outside the experimental areas rather than rely on directtransmission. Present use of the programs are through tapes which are sentto project schools. These had obvious advantages over using radio: (a) thequality of sound was superior; (b) the program was fully reliable; (c) theaudio material could be used at any convenient time; (d) the tapes could beplayed over in advance by the leader or teachers who then knew what was ineach session; and (e) tapes could be stopped at any point for repetitionand/or clarification.

In-service Teacher Training (CET)

4.04 This project component was designed to: (a) develop a systemcapable of writing, broadcasting and modifying course materials for the fivesubject areas; and (b) use the materials developed, together with a learningmanual and face-to-face contacts in regular group meetings in an instructionprogram for as many teachers as possible in three selected areas.

4.05 Table 4.1 gives an overall picture of the training operation bysemesters. By the end of the project, some 7,700 sessions in Pilipino hadbeen provided but only 100 in Science/Mathematics. A full year's course wasavailable in the other subject areas. Over 17,000 teachers participated, or92% of the target. All the course material has been recorded on cassette tapefor further use and much is in use already under Loan 2030-PH.

P - PLANNED

A - ACTUALTable 4.1: IN-SERVICE TEACHER TRAINING

PLANNED AND ACTUAL TEACHER AUDIENCE REACHED

PANGASINAN | LEYTE | MAGUINDANAO

Subject Year 1 Year 2 Year 3 | Year 2 Year 3 | Year 2 Year 3 lArea School 1st 2nd lst 2nd x lst 2nd Ist 2nd i 1st 2nd 1st 2nd | TOTAL

year 78 79 79-80 79-80 80-81 80-81 | 79-80 79-80 80-81 80-81 79-80 79-80 80-81 80-81

Educational Trends P-O P-O P-O P-O P-0 P-0 I P-O P-O P-O P-0 IP- P-0 P-0 P-0 P-OA-162 A-250 A-671 A-789 A-100 A-1032 | A-384 A-472 A-659 A-598 A-150 A-300 A-300 A-300 | A-6167

Pilipino P-500 P-500 P-500 P-1000 P-1000 P-1000 P-500 P-1000 P-1000 P-1000 P-0 P-0 P-O P-O 0 P-8000A-222 A-504 A-749 A-846 A-979 A-1122 I A-413 A-504 A-706 A-636 A-150 A-300 A-300 A-300 | A-7731

Social Studies - - P-500 P-500 P-1000 P-1000 - P-500 P-1000 P-1000 P-c P-cl P-3500A-0 A-100 A-500 A-834 0- A- A-273 A-495 _ - A-200 A-200 | A-2602

English - - - P-500 P-500 P-1000 I - - P-500 P-1000 I - - - P-0 I P-3500A-0 A-100 A-490 - A-c A-307 A-200 A-1097

Science/Math - - - P-500 P-500 | - _ - P-500 | - _ _ - | P-1500A-c A-100 I _ A-0 | | A-100

Planned 10,500 | Planned 8,000 | Planned - Nil | 18,500Actual 9,550 | Actual 5,447 | Actual 2,700 | 17,697

Grand Total Planned - 18,500

Grand Total Actual - 17,679

- 14 -

Radio-assisted Teaching in Elementary Schools (RATES)

4.06 During its first year of operation this project component developedradio-assisted teaching of Grade 4 Pilipino in Pangasinan, which combinedradio lessons with the use of pupils' worksheets. After formative evaluation,the program was continued for two years in subsequent grades 5 and 6 followingthe same pupils. After the initial testing year, other pupils were added, astested teaching material became available. The radio lessons were supple-mented by the use of pupils' worksheets throughout the three years. Thenumber of pupils who participated was governed by the number of availableoperational radio-cassette players. The pattern is shown in Table 4.2.

Table 4.2: RATES IMPLEMENTATION SCHEDULE BY CLASSES

School YearGrade 1978-79 1979-80 1980-81

Target Actual Target Actual Target Actual

{Classes 30 30 100 120 n.a. 1,7184 {Pupils 1,200 1,200 4,000 4,800 n.a. 68,720

{Classes 30 30 100 3305 {Pupils 1,100 1,110 3,700 12,210

{Classes 300 276 {Pupils 1,050 945

Total Classes 30 30 130 150 n.a. 2,075

Total Pupils 1,200 1,200 5,100 5,910 n.a. 81,875

Evaluation of CET and RATES

4.07 The evaluation of the effects of radio on pupils' performance was amajor objective of the project. To test the impact at the end of the projectperiod, nine treatment combinations were established. These were:

(1) RATES alone;(2) CET assisted teachers, without other inputs;(3) RATES and CET combined;(4) RATES and one textbook to two pupils;(5) CET and one textbook to two pupils;(6) A 1:2 textbook/pupil ratio without other inputs;(7) A 1:1 textbook/pupil ratio without other inputs;(8) RATES, CET and 1:2 textbook/pupil ratio; and(9) CET and a 1:1 textbook/pupil ratio.

- 15 -

Students were given pre-tests and post-tests in Pilipino, covering reading andlistening skills. Although other outcomes of CET were assessed, the major

attempt at measuring the impact of lessons for teachers was made in theexperimental design, which assessed student achievement in Pilipino. Analysesof the test data were conducted in 1980 by the University of the Philippines,Institute of Mass Communications (UPIMC) which issued its final report in June

1982. Unit costs of the various treatments rate are shown in Table 4.3.

4.08 The findings indicate that in the two main testing regions,Pangasinan and Leyte, the strongest improvements in pupil performance resulted

from the treatment combinations shown in Table 4.3.

Table 4.3: EFFECTIVE MEANS OF IMPROVING PUPIL PERFORMANCE

Cost rangeTreatment per pupil Treatment number

US$ /a

RATES with CET trained teachers 10.40-11.06 (3)

RATES with a 1:2 textbook/pupil ratio, 10.10-10.80 (4)

RATES alone 9.86-10.53 (1)

CET and a 1:2 textbook/pupil ratio 0.74-0.82 (5)

/a The costs are UPIMC's figures. Caution should be used in applyingthese costs in absolute terms because of reasons stated in paras.4.02-4.03.

4.09 While all these methods produced significantly increased pupilperformance, it is interesting to note the cost differentials. The

combination of providing one textbook to a pair of students and training the

teachers through mass media (treatment #(5)) produced a result which was lessthan 1/10th as costly on a per pupil basis as any of the other combinations.

Study on Communications Technology for Rural Education

4.10 Quantitative analysis of this study is difficult because there is noinformation on how many persons listened to the broadcasts. Some people

listened but did not attend group meetings, others attended but did not listenoften, and continuity was sporadic. The UPIMC findings were that:

- 16 -

(a) radio (in spite of its limitations), is a stimulus whichcan interest farmers who are not very literate both tolisten and learn and can also promote group discussion;

(b) radio is a good supplement to existing rural community andagricultural services but it is difficult to form new groups touse it; and

(c) timing of radio broadcasts must be flexible and discontinuedin certain seasons as they will not be listened to whenfarmers are in their fields.

4.11 The radio programs did have another impact. The Ministry ofAgriculture asked to have copies of the scripts and in 1981 the Ministry ofPublic Instruction broadcast them in toto to the whole of the Philippines onits national 50 kW station.

Future of the Training Materials

4.12 The pilot project accumulated a considerable amount of knowledgeand the means of extending it to more pupils and teachers. This consists of:

(a) the pedagogic material in the pupils' worksheets (RATES)and the teachers' manuals (CET); and

(b) the equipment to make use of the above and to replicate morematerial.

Project results are being used in the Elementary Education Sector Program(Loan 2030 of FY81) where teachers are being trained in new curricula andteaching methodology with the aid of taped material. Textbooks are beingdistributed to schools at the rate of one textbook per two pupils. This is inaccordance with earlier practices, but the results of Loan S8-PH show that itis an effective practice in terms of learning achievements.

PHILIPP INES

MASS MEDIA PILOT PROJECT

ORGANIZATION OF THE EDUCATION COMMUNICATIONS OFFICE (ECO)

Pit'tate-r. ft Diectora, MCEx-acutive Director. cOIar eiamsLXaOfft eas (1, VE&I & UE)

|Internationa 1 oo"oratinsL' gte : |e

IPeticipattg Natfoaat AgencieN ECO pro rc Director 1m tCO istent Project DLr eto _ry

| Suvnwtie Ev;l.ztten

Curricu.No DevelopmentAnd

NEC C.rrt cuIu rou

l _ tE~~~~~~~~~Aanl trAv

too Currculw DOw esoP-at Media ftanning ftoductivo Rsaearch and Dw lopsent Project Planomog Untl Progra * rvie, Unit Field Dpratle n 1ett Field 'tilieatioa__ .tirit untt Unit |

r- Curricutunp D"lopmest e l dis PlananmeoJroductton Research and D.vlop.eot Project FlnontV Head Lay-Out Deeignotl Tield tratltas ?PaaestarocOffi!er Officer Officer Copy Editor Officer a) I Rrgioaai Coot

tAtIS Cucrriaum 1p hedta Vesgg Officer Field Reuearch Officer Assistaut projeet Lay-Out Assist eot/eIt 14 lull-time field lejon ISpecislist lAns ScriptWitetr PATES Evlusttioe Officer ?leantni Officer StothL Officer staff ti pa"astem bI I Di.. Ftetd

Cl I part-tbm l_ 1aa print Editor Ciz Formative svluatteen tAIS fto5rae fl?a yrint Tachnticn))ls.- 2 full-time Felid CtilikatLon'' writers, Production AeeiutAnt Offtcer cl Officer ograph Operator Staff to Leyte Coeisatoct. (I'cld ypfat Typits, " T t Planenrng Ivstrialian Officer fie fellies Field i. a tatu rsi

ocfiet toffnlerr Staff io Modulea e) a tge Sra%i"Audi* Terhtcinelo/Wr,- deijo. lAoders (factch

ducefon Traffict.trc fr-

officear P ng..lnac. Scat

a) I Ragaonal CoolA.Flon Ill

b) I flivitito VteiVtflitatiooC.~,d lmarstrfhiuperintend*nq

c) 7 Fejn= Seg idL.eader% (FAcMals4iojor of Dcvi,Word lnii. &flistrict Sipenson La Leyte)

1 1 legiocal Cocilef ion %II

lb) 2 t'ivisicv Ft.e

(Ed. S~PswvlseEc) 3 For... Stsait

Leaders lid.supervisors.

- 18 -

ANNEX 2

PHILIPPINES

MASS MEDIA PILOT PROJECT

LOCAL EDUCATIONAL ADMINISTRATION

(Ministry of Education (Generaland Culture Administration)

Director Region Region

Superintendent ; IAssistant Superintendent Educational ProvinceGeneral Education DivisionSupervisors

Subject Supervisor District Municipality

Schools District(Barangay)

COMMENTS FROM THE BORROWER - 19 - ATTACHMENT I

(p. 1 of 2)

REPUBLIKA NG PILIPINAS(Republic of the Philippines)

MINISTRI NG EDUKASYON AT KULTURA(MINISTRY OF EDUCATION AND CULTURE)

M A Y N I L A(MANILA)

rANGGAPAN NG MIHISTRIOFFICE OF THE MINISTER

15 January 1984

Shiv S. KapurDirectorOperations Evaluation DepartmentWorld Bank1818 Street NoW.Washington D.C. 20433U. S. A.

Dear Mr. Kapur:

Thank you very much for your letter asking for comments onthe S008-Ph Project Completion Report prepared by the OperationsEvaluation Department.

I would like to draw your attention to the need for revisionof the Highlights Section, paragraph 3; 2.01; 2.03;4.01; and 4.02of the report. In particular, there is need to correct said sections:which describe the transmission of radio lessons in the project sites.

The actual transmission activities were:

1. For Year One (Schoolyear 1978-79) which was the shakedown phase,we piloted the Project in 30 Grade IV Pilipino classes in Pangasinanand 162 teacher participants of the Educational Trends Course of CET.In accordance with the loan covenant, we used taped lessons as itwould be rather expensive to use radio air time for a very limitednumber of project participants.

2. For Year Two (Schoolyear 1979-80), we used radio broadcastlessons to reach 120 Grade IV Pilipino classes and 5,582 teacherparticipants in Educational Trends and Pilipino Courses of CET, inthe two (2) pilot sites of Pangasinan and Leyte. These were themain project sites included in the summative evaluation.

Again, as per loan agreement, tapes were used in 30 classes ofGrade V Pilipino and the cET courses (Social Studies and English)which were pre-tested to a limited number of teacher participants(p.4-20 Project Completion Report).

- 20 -ATTACHMENT I(p. 2 of 2)

In Maguindanao, radio transmission was used by RATES andtaped lessons were used by CET. Take note, however, thatMaguindanao was not Dart of the testing sites of UP-IMC. Weextended the project to this area primarily because of thestrong request from the MECS Regional Office to extend ourservicea -to wr 8-ao.

3. For Year Three (Schoolyear 1980-81), which was the expansionphase of the projeat, radio lessons were transmitted to RATESGrade IV and V Pilipino and the CET courses in the 2 projectsites (Pangasinan end Leyte).

Taped lessons Wrere used for the pre-testing of RATES GradeVI Pilipino and OFT courses (science/Math).

In addition, we conducted two experiments using tape-assistedinstruction (pp. 6-46 up to 6-54 of the Project Completion Report).These studies were in addition to, not a switch from, the radiolessons aired in Pangasinan and Leyte.

In sum, although taped lessons were used all throughout thelife span of the Project, they were just pre-test activities conductedin accordance with the loan covenant. Special Studies on tapedinstructions were carried out also in accordance with Schedule 2 B 2of the loan covenant. Radio transmission was adhered to in the twomajor project sites from 1978 to 1981.

We are sending you a telex version of these comments. We arealso sending, through WB Education Consultant John Middleton, a copyof the Project Completion Report which we prepared for your reference.

Best regards.

Very truly yours,

ZENAIDA T. DOMINGO