M&E DRAFT REPORT

1. Executive Summary

2.0 Background

The Goal of BESPOR is the achievement of education for all within a balanced national poverty strategy. Its purpose is to enhance the capacity of DoSE to implement the strategy in and through the educator sector. A key priority to support its implementation is strengthening monitoring and evaluation (M&E) systems and coordinating structures with DoSE. These will, in turn, support the establishment of a strong education sector programme and facilitate the monitoring BESPOR’s progress towards expected outcomes (purpose level) as defined in the logical framework analysis (LFA).

2.1 Government/Sectoral Policy

Monitoring and evaluation is guided by the draft 2004-2015 Education Policy which gives priority to basic education with a focus on increasing access, gender parity and quality of education. The policy in Section 11.5 gives the responsible for monitoring, support and supervision of teaching and learning at the school level to the Standards Quality Assurance Directorate (SQAD). Head Teachers are responsible for improving the performance output of both teachers and students with schools setting internal objectives, targets and strategies for quality improvement. Student assessment is based on learning achievement targets. This is discussed further in section 2.3 under situational analysis.

At the regional and central levels, there are no clear policies on monitoring and evaluation of administration, finance, and knowledge management. At the institutional level, section 14.2.1 states that SQAD is responsible for “monitoring compliance of the Education Policy and its associated acts and regulations with all institutions operating in The Gambia”. In reality, reviews have taken place on an ad hoc and informal basis with Gambia College and seldom with other institutions.

2.2 Stakeholders and Beneficiaries

A wide range of stakeholders are impacted on and/or by monitoring and evaluation processes and activities in the educator sector. These range from national institutions like WAEC, Gambia College, and DoSE with its Regional Directorates to schools, teachers, parents, children and external providers; e.g. donors.

Table 1 provides an overall of their roles and how each stakeholder can contribute to the process and what the benefits could be derived in terms of BESPOR interventions.

Table 1: Stakeholder Analysis for M&E

Stakeholder Role Interest Contribute ReceiveSenior Management Team

Policy Direction and Coordination

Accountability Policy Directives, control and resources

A clearer picture how of how M&E works in DoSE

Coordination Committee Meetings

Policy Implementation and Problem-Solving

Accountability Advise on how to strengthen coordination of M&E with DoSE

A clear picture of implementation issues related to M&E

PlanningDirectorate/ EMIS

Control of Planning and EMIS

Accountability Updating data for planning and tracking EFA targets

Improved EMIS system to be used as a tool for planning & monitoring at center and regional levels

Standards and Quality Assurance

Monitoring of quality of teaching and learning in

Improved QA monitoring systems

Inputs into baseline on teaching/learning

Development of a more coordinated and

Stakeholder Role Interest Contribute ReceiveDirectorate schools to address EFA in

the six regionsconditions; Supervision of NATs, develop decentralized monitoring to the cluster level

coherent system for monitoring quality of teaching and learning across projects and programmes

HRD Directorate Management of Staff Performance Appraisal data and processing information for rewards and sanctions

To institutionalize SPAS

Issues related to existing SPAS and practical ways to move forward

Establishment of a standardized system for performance assessment

Region 5 Education Directorate

Coordination of M&E activities

Improved delivery of education quality

Knowledge of situation in Region 5; lead changes to management systems, inputs into baseline studies

Coordinated M&E System across existing projects with Region 5;better baseline data on quality of education in the region

Cluster Monitors Monitoring Schools within designated cluster

Assessing whether schools are achieving their objectives/targets

Training needs of teachers and gaps in M&E System

Improved monitoring with agreed upon forms and reporting formats and flow of information

Schools Monitoring Whole School Development including curriculum. resources, teachers, PTAs, etc.

How to effectively coordinate and monitor school targets

School Development PlansAdvise on what is going well and what is not at the school level and

Improved M&E Systems involving teachers and community at large

Community/Parents

Local decision-makers Improved education serve

Advise about the situation on the ground

Better understanding of how well their school and children are performing

Children Pupil’s perception of what works and does not work in schools

Improved quality of education

Advise on how to improve school performance

Better classroom instruction and better school management

EFA Unit Coordination EFA Activities in six Regions and conduct annual impact assessment and reports of progress towards targets

To use the Catalytic fund to achieve EFA Goals

Plan for EFA activities in Region 5 to avoid duplication

Increased Coordination and cooperation in Region 5 where BESPOR is piloting a whole school development model

West African Examination Council

Development and administration of National Assessment Tests for Grades 3 & 5 an d9

Assessment of how well the education system is performing

Expertise to develop tests, do data analysis and reporting to see how well the system is performing in 2005

Increased awareness of developing NATs that test a range of competencies and increased knowledge of the situation in the six regions

Gambia College Lead a Longitudinal Study on WSD in 15 to 20 schools in Region 5

Linking research to teaching training

Expertise to conduct school-based case studies

Ways to strengthen pre-service and in-service training

Donors Financing and Monitoring

Influence Advice and Resources

Goodwill and influence

National Consultants for BESPOR

Professional advice Increase one’s professional knowledge/skills

Advice Income

2.3 Situational Analysis

2.3.1 Monitoring Learning Achievements

The scale for gauging the effectiveness of the education system in The Gambia rests on pupils’ performance on examinations. With the phase out of the Grade 6 examination, there was an identified need to develop learning achievement targets (LATs) as benchmarks for improving the quality of education. These LATs were developed for lower basic education in 2000 and were revised in 2005 to align with the current syllabus and textbooks in use. How closely this alignment has been achieved requires further investigation. The revised materials are now being distributed to Regional Offices, which are responsible for familiarizing teachers with the LATs. At the time of this report, Regions 5 and 6 had not received the materials or training. For upper basic, LATs have not been developed although there is a plan to do this in the future.

The system for monitoring learning achievements (MLAs) is still evolving with a two prong approach that includes continuous assessment and National Achievement Tests (NATs). Continuous assessment by lower basic teachers remains problematic as many of them have do not have a good understanding of the LATs nor do they have the skills to develop performance monitoring tests (PMTs). SQAD plans to take corrective measures through in-service training of teachers to address this. For upper basic, teachers have been trained in continuous assessments and are doing this without any LATs as benchmarks.

Within the Education Policy, a National Achievement Test (NAT) should be conducted every two years using 25% of all pupils in Grades 3 and 5 to inform the system on pupils’ achievement at the lower basic education level. This has not happened. Delays are attributed to lack of resources to do this on a continuous basis.

Data on the quality of basic education is very limited. The MLA study on Grade 4 learning achievements in 2000 indicated that less than ten percent of the children attained 70% or higher on learning achievements set for the four core subjects (English Language, Mathematics, Science and Social/Environmental Studies). Test items were based on LATs with no baseline data on what was actually being taught in the classroom. As such, many pupils were tested on items that they had not learned. This was a contributing factor to high failure rates as were the high level of difficulty of the items. The study showed also that pupils in private schools performed better than their colleagues in Mission and Government Schools and that there were wide disparities across regions and between urban and rural schools. Similar findings were found in 2002 when WAEC carried out a small study of learning achievements of Grade 3 and 5 in 2002. Since then, no national assessment tests have been carried out.

It is important to note that the study of Grade 4 pupils in 1998 looked at conditions of teaching and learning that may have influenced children’s learning achievements. Findings indicated that a host of factors impacted on pupil’s performance. They ranged from the level of support from parents or caregivers, availability of resources, qualifications of teachers as well as their degree of confidence and motivation to the level of support from Head Teachers to name a few. There was unfortunately insufficient empirical data to determine which underlying factors had had a major negative impact. Also the study paid insufficient attention to gender issues. For instance, for many survey questions there was no gender disaggregation of the answers, representing a missed opportunity to build understanding of how pupils’ gender affects learning conditions and hence achievement.

One of the outcomes of the study was that monitoring, supervision and inspection processes needed to be revisited with the view to increasing the frequency of visits and providing professional support to teachers. SQAD’s recent efforts to decentralize monitoring to the cluster level with the appointment and training of Cluster Monitors in all regions is seen as a positive step towards strengthening the monitoring system. In reality, however, there are few Cluster Monitors on the ground in Region 5 where BESPOR will pilot whole school development (see section 2.3.2).

2.3.2 M&E Systems within DoSE

Within DoSE, there is no coordinated monitoring system across the Directorates at the national and regional levels. Existing M&E systems tend to operate as separate entities with little flow or sharing of information. Some of these systems are fairly well-developed (e.g. EMIS); others are in the process of being developed (SQAD’s new Participatory Monitoring System) and some are inactive (Staff Performance Appraisal System). Details follow.

EMIS

The EMIS was developed as a multi-segment database to provide management reports for planning purposes. It includes data on schools, personnel, financial and geographical information. Data across all segments is not up-to-date. Efforts to validate school data between 2002 and 2004 were plagued with problems related to poor data collection procedures and limited capacity of EMIS Staff. Efforts are underway to strengthen EMIS staff skills with the goal to complete validation of 2004 data by the end of May 2005. School data for 2005 was being collected at the time of this report. Data entry is scheduled to start in late May and take two months. If there are no major problems, data for 2005 should be ready to validate in September 2005 as a first step towards getting baseline data for BESPOR.

Data analysis remains a problem due to weak capacity of EMIS staff and high attrition rates. No in-depth data analysis has been done to support planning and monitoring since the EFA Assessment Report in 2000. What little has been produced has been done by a Peace Corp volunteer, who will leave in June, thus further aggravating the capacity problems. Furthermore, the 1993 Population Census data was used to calculate the net enrolment (NER) and will need to be updated with the 2003 Population data in order to provide a more accurate picture. To address these issues, the BESPOR Consultant for ITC is putting together a short-term plans to hire an interim consultant to support data analysis and long-term plans to build sustainability with the EMIS Unit (see Philip’s report). Such steps will enable BESPOR to support a 2005 EFA Assessment as part of its baseline studies. (see section 5.2.2 for details).

Staff Appraisal Performance System (SPAS)

In 2001, The Department of State for Education introduced a nationwide staff performance appraisal system (SPAS) to enable policy makers and managers to make well-informed decisions on how to effectively deploy and upgrade the civil service personnel. The system was mainly been used to assess the performance of teachers and head teachers, and on occasion middle and senior managers. Information was then entered to a database which has not been kept up to date since the Peace Core Volunteer left in 2003.

At the time of this report, it was not clear to what extent SPAS was in use at the school and regional levels, if at all. At the central level, SPAS information was still used for short-listing and for awards or promotion purposes. Informal reports, however, indicated that teachers were being penalized if their records were not up-to-date or if they had never been assessed. The PMO is one of the key stakeholders and has the mandate to do the system wide assessment but this has not happened.

Problems of SPAS have been related to a lack of interest and ownership by DoSE to maintain and improve the system. Assessment has been top down with Regional officers assessing head teachers and head teachers assessing teachers. There is no mechanism for bottom-up assessment of head teachers by teachers, parents and pupils. Related to this is the lack of a standardized system for assessment with competency levels for teachers, head teachers, managers to attain. Lines of reporting are unclear and feedback systems are not well developed to support the professional development of personnel.

Standards, Quality Assurance Directorate (SQAD)

SQAD is leading the drive to improve quality of teaching and learning in basic education by 2015. Based on the Ghanian model, a decentralized approach to monitoring and supervision is being developed whereby cluster monitors (CM) are appointed for each main cluster within the six regions. TORs for cluster monitors are multi-faceted and include responsibility for training needs analysis, training, monitoring resources, supervising

teachers, and developing cluster action plans to name a few. This shift from the traditional role of the inspector as the authority to one of facilitation is aimed at fostering a participatory approach to monitoring where all stakeholders are working as a team to improve teaching and learning conditions in schools.

One important aspect of this new system is Participatory Performance Monitoring (PPM) where by schools and communities work together to set their own school targets for EFA, develop and implement schools plans and then monitor and assess progress towards these targets. This approach entails teachers setting performance monitoring tests (PMTs) each year and schools holding annual school performance appraisal meeting (SPAM) with community stakeholders to review the test and other results before setting new or revised targets. Community leaders and parents would be trained in school development and monitoring so that they would be able to make valuable contributions to the process as well as monitor how well their children are doing.

The system in many ways reflects a shift to a holistic model of whole school development in which the quality of teaching and learning must be improved if ultimately pupil achievement is to improve. This means that administration, management, curriculum, infrastructure, resources, etc should aim to positively impact upon teaching and learning and stakeholders need to adopt values, attitudes and beliefs to support this focus.

In reality, SQAD’s model is in its infancy stage with few cluster monitors on the job in Region 5, and little infrastructure and training in place to operationalise the system. At all levels, advocacy for innovation and change is still being carried out to build a sense of ownership among stakeholders. Lines of communication and reporting are being developed and feedback mechanisms established to allow continuing refinement and improvement as the system evolves. BESPOR is well-timed to model good practices and facilitate the development of this new monitoring system in Region 5.

2.5 Other Interventions

2.5.1 EFA Initiatives Education For All National Action Plan 2004-2015 has been developed and funding was recently procured to the tune of $US 4 million from the Catalyic Fund to support EFA initiatives in 2005. Funding for 2006 will depend, to a large extent, on mechanisms put in place to monitor and evaluation progress against EFA Fast Track Initiative (FTI) targets that have been set by DoSE.

These M&E mechanisms are not well defined in documents reviewed except to say that PPARBD and REDs will jointly collect school data annually. SQAD will monitor learning achievements and school performance based on its proposed cluster model. HRD will continue to carry out staff appraisal schemes to inform the training and promotion prospects of teachers. The SMT and CCM will backstop and provide policy guidance and directives and that sector reviews with donors would be conducted annually. What remains unknown is how these mechanisms are working at school, cluster and regional levels, what indicators they are measuring and means of verification, who is responsible for data collection and reporting, and what mechanisms are in place to feedback to SMT and donor institutions.

Discussions with SQAD and the RTO from Region 5 indicated that the major thrust has been on implementing training activities and procurement equipment (motorcycles, vehicles, generators) since its inception in February 2005. Details were sketchy on the kind and quality of the training and training schedules and what follow-up monitoring was being done after training.

The management process is being coordinated by the DoSE/EFA Unit with PCU. Implementation has been divided into four components with coordinators as focal points as per Table 2. These coordinators would also be responsible for monitoring.

Table 2

Component CoordinatorAccess and Equity PCU – Jaware GayeParticipation & Retention of Girls Basic Ed - Mr. SannehQuality of Teaching & Learning SQAD: Momodou TourayInstitutional Mgt and Capacity at Regional and Center Levels

Planning: Mohammad Jallow

This has implications for BESPOR in that the Project shares a common goal with EFA initiatives, shares a focus on improving teaching and learning conditions, targets many of the same stakeholders and requires support of many of the same DoSE staff at the national and regional levels. In light of this, an important step for the Project is review existing monitoring systems used for EFA initiatives and where possible align and/or integrate BESPOR monitoring into these systems while strengthening these systems at the national level and in Region 5. This alignment should be extend to reviewing monitoring systems used by other donors (e.g. UNICEF/FAWEGAM for its girl friendly schools’ initiative) and identifying ways to coordinate monitoring efforts towards a common goal.

2.6 Documentation Available

With regards to M&E, there are few documents that provide a good overview of the monitoring processes, how benefits and impact are measured, and how information flows and feedbacks back to improving the education system. A list of documents related to education policy, action plans and research studies is in Annex 4.

3.0 Proposed Intervention

BESPOR is unusual in that it is a project that seeks to develop a programme as one of its outputs. As such, a two pronged approach to M&E will be undertaken which not only puts in place systems to monitor BESPOR’s project outputs, but also develops and strengthens DoSE’s own M&E systems to support a sector wide education programme. The first system would track progress and impact of the Project over a three year span. The second would track progress and impact of the Programme over a ten year span. Both would entail support to baseline data at the project and programme levels and increased coordination and collaboration with all stakeholders.

3.1 Overall Development Objective

The overall objective of Monitoring and Evaluation is to contribute towards monitoring how well initiatives in the education sector are contributing to poverty reduction. M&E will track whether education programmes and projects are having a positive impact on poverty alleviation while ensuring accountability and transparency in the allocation of resources to all stakeholders.

3.1 Project Purpose The project purpose is to strengthen M&E systems to monitor progress towards DoSE’s strategic goals and objectives. This is critical given that DoSE is committed to developing a sector wide programme and is currently constrained by inadequate coordination and human resources to do the job well. It is envisioned that over the three-year project that BESPOR

would work jointly with DoSE to put in place an enhanced M&E systems that are coordinated and streamlined to improve monitoring at all levels and across programmes and existing projects in order to address the shortfalls described in 2.3. At the purpose level, strengthening M&E systems is seen as essential in the pursuit of improved policies, planning and resource allocation, tracking programme and project effectiveness, and ensuring a flow of information between beneficiaries and providers and policymakers to improve the education system as a whole.

3.2 Project Outputs

To achieve the project purpose, three objectives and outputs have been identified over the three-year Project. (See Table 3). They reflect the two pronged approach mentioned above.

Table 3

Objective Outputs 1. To put in place internal monitoring, evaluation and reporting systems to track BESPOR expected results across all project components

1. An M&E Framework developed and operationalised to track BESPOR outputs and outcomes

2. To strengthen M&E systems at DoSE Central level to improve efficiency and coordination

2. Enhanced M&E Systems at the central level to support the implementation of the Gambia Education Sector Development Programme

3. To strengthen M&E systems in Region 5 to support whole school development and other EFA initiatives

3. M&E Systems in Region 5 strengthened to monitor all EFA activities including BESPOR’s.

The first output is an operational result and links directly to Project requirements as outlined by DFID and the Government of The Gambia. The second and third outputs are developmental results and link to the need for capacity building in M&E at the central and regional level as being important to support the establishment of a sector wide education programme.

3.4 Project Activities

BESPOR is a multi-component project to be implemented by several stakeholders. Components include teacher development with Gambia College, whole school development in Region 5, distance education cutting across the education sector, ICT at the national level and in Region 5, and the development of the sector wide education programme. All these components need to be linked to the purpose level through agreed on expected outputs and activities.

3.4.1 BESPOR M&E Framework

For BESPOR, the development of a logframe (LFA) with expected outputs, outcomes and performance indicators is the first and most important step in developing a M&E Framework for Output 1. The LFA will drive the M&E process and the indicators will be used as benchmarks to monitor progress towards expected outputs and outcomes. This step will require a wide participation of DoSE stakeholders and partners in the development of an LFA and in annual reviews of its indicators and critical assumptions in order to ensure a common understanding of what BESPOR is trying to achieve. The first LFA would be produced as part of the Inception process with the aim of building a broad understanding and ownership of the LFA as a M&E management tool. Subsequent reviews of the LFA would be conducted annually in May or June as the BESPOR evolves.

The second and often parallel step is to identify baseline data to be collected. For BESPOR, four key baseline studies that would support both the project and sector programme have been identified but will require further discussion with DoSE stakeholders. These include

quantitative data (e.g. EMIS, student performance tests) to qualitative data (e.g. conditions on teaching and learning) and are discussed in further detail in Section 5.2 under implementation.

The third step is to put in place internal monitoring and reporting systems that feed into the Project management reports once implementation is underway. Details would need to be worked out in collaboration with the component implementation teams and training provided to support the M&E process and development of monitoring tools.

It is these activities that will form the core activities for Output 1 and are summarized in Table 4 along with activities for Outputs 2 and 3.

Table 4

Project Results

Output 1 Output 2 Output 3

An M&E Framework developed and operationalised for BESPOR

M&E Systems at the Central level enhanced to support implementation of the Gambia Education Sector Development Programme (GESDP).

M&E Systems at the Regional and School levels strengthened to monitor whole school development.

Key Project Activities

1.1 Finalize Project Scope and LFA with stakeholders

2.1 Establish an M&E Team at the National Level

3.1 Establish an M&E Review Team for Region 5

1.2 Facilitate Baseline Data Collection, Analysis and Reporting

2.2 Review existing M&E Systems in DoSE, reporting lines, flow of information, and human resource capacity

3.2 Review existing M&E Structures and Systems at the Regional, Cluster and School Levels, flow of information and human resource capacity

1.3 Identify key components of M&E and establish Internal Monitoring and Reporting Systems with Project Implementation Teams

2.3 Develop capacity at the Central level to coordinate and strengthen internal M&E systems across the Directorates

3.3 Identify training needs and support training and resources to strengthen M&E systems and processes

1.4 Identify M&E training needs of implementation teams and provide training where required

2.4 Identify and support baseline studies and reviews under BESPOR that would feed into the development of the GESDP

3.4 Develop a coordinated monitoring system for Region 5 that integrates BESPOR and other EFA activities

1.5 Prepare M&E Activity schedule and budget with timelines for annual reviews, mid-term and end-of-project evaluations and as well as internal project monitoring

2.5 Facilitate the development of LFA indicators for GESDP Programme at the macro level and within sub-groups

3.5 Revise or develop M&E Instruments to support M&E where required

1.6 Integrate BESPOR M&E processes into existing DoSE systems and structures wherever possible

2.6 Facilitate the development an M&E Framework with guidelines for annual joint reviews and common reporting formats to be used by all stakeholders

3.6 Establish a Reporting System to feed findings into Quarterly and Annual Reports for DoSE and Donors

3.4.2 Capacity Building of M&E at the National Level

For Output 2, the initial focus would be on capacity development at the national level in preparation of GESDP and to support the monitoring of BESPOR. What form these activities will actually take depends on the several factors. The first is the scope of BESPOR and resources available. The second is the willingness of DoSE Directorates to address the

shortfalls identified in the situational analysis section. The third is a commitment to embrace ‘change’.

Proposed activities that will be defined in more detail include:

1. a joint review of existing M&E structures and systems, reporting lines, and human resource capacity across the Directorates and affiliated institutions (e.g. Gambia College) with a plan of action to move forward;

2. strengthening of M&E systems through audits, on-the-job training, workshops, and forums to enhance coordinating structures and systems and flow of information;

3. identifying training needs and providing training to senior and middle management staff who are responsible monitoring and evaluation; and

4. creating mechanisms to disseminate information through formal reporting systems, consultative meetings, EMIS and research reports, newsletters, etc.

With BESPOR, there is a unique opportunity to ensure that the M&E Framework for the GESDP is strongly linked to programme outputs. This would entail support at two levels. The first is to facilitate the development of logframes and relevant indicators to support each sub-group’s strategic plans. The second is to facilitate joint sessions of Government and donors to reach a consensus on how joint reviews should be conducted and to agree on reporting formats to be used by all stakeholders. This will require an enormous coordinated effort and intensive discussions. A possible starting point is to facilitate document reviews and study tours to other countries which are implementing SWAPs to gain knowledge of how these countries developed their M&E framework and what lessons they learned during the developmental process. Such shared experiences among Gambian stakeholders would facilitate the process of drafting of a programme framework for The Gambia.

3.4.3 Capacity Building for M&E in Region 5

The M&E Systems in Region 5 are not well-defined or coordinated. They depend heavily on availability and mobility of staff and directives from SQAD headquarters. Output 3 would address these concerns through building the capacity at the regional, cluster and schools levels to manage and implement a coordinated approach to monitoring and reporting across all EFA initiatives. The nature of the activities will depend on how the whole school development model evolves, how well SQAD’s new monitoring system is working, and the willingness of education officers and teachers to embrace change and innovation.

Proposed activities would include:

1. a review of existing M&E structures and systems, reporting lines, and human resource capacity at the cluster and school levels;

2. strengthening of M&E systems through joint planning, on-the-job training, standardizing M&E tools and reporting formats, establishing feedback mechanisms across all projects, and holding forums to identify ways to integrate and streamline all project monitoring;

3. identifying training needs and providing training to key staff responsible monitoring and evaluation activities at different levels. This could include study tours to other regions in Africa that have good models of whole school development; and

4. creating mechanisms to disseminate information through regional and cluster reports, consultative meetings, EMIS reports, baseline studies, school-case studies, etc.

4.0 Assumptions

See LFA in Technical Annex 1 for summary

4.1 Assumptions at Different Levels (to be updated)

Purpose Level: There is continued commitment to systems monitoring and to improving management

systems within and across DoSE Directorates and education partners.

Output 1: DoSE implementation teams remain relatively stable with minimal attrition and

reassignment of staff critical to M&E operations. Lines of responsibility and reporting on BESPOR are clearly defined, approved and

adhered to. There are qualified national consultants to support baseline studies

Output 2: Government remains committed to reform within the education sector and to

continuing the decentralization process to the regional level Donors and other external partners are willing to be participations in the development

of an sector wide education programme DoSE Directorates are willing to coordinate their efforts to ensure a more efficient

monitoring system

Output 3: Cluster Monitors are in place in Region 5 DoSE/Regional level keeps staff turnover to a minimum Infrastructure and resources are in place to allow routine monitoring Local communities are willing to participate in school development and monitoring

4.2 Risks and Flexibility (this need to be flushed out)

The FTI initiatives are demanding much attention from senior and middle management to the detriment of their routine responsibilities. This could have a negative impact on BESPOR in that staff may not be stretched too thin and have little time to support BESPOR interventions.

To be completed once analysis is done.

5.0 Implementation

5.1 Physical and non-physical means insufficient information to complete this section

5.2 Organisation and Implementation Procedures

5.2.1 The M&E Teams

The implementation of the M&E Component rests on a strong partnership with DoSE, the Regional Education Directorate in Central Rivers Division and Consultants from TAMC. A critical starting point is the formulation of M&E Implementation Teams at the national level and in Region 5 to guide and support the strengthening of M&E structures and processes. These teams should be in place as soon as possible in order to move forward with the development of concrete plans for M&E.

The central level team should be comprised of key stakeholders across the Directorates (e.g. Planning SQAD, HRD, Basic Education) who have an invested interest in strengthening M&E. This team would be responsible for the technical preparation of different elements

(e.g. baseline studies) and taking the lead to strengthen coordination across DoSE Directorates and programme and project components.

The regional team should be comprised of the Director of Region 5 and core staff at the regional and cluster levels who responsible for coordination of M&E at the cluster and school levels. They would be responsible for putting together annual plans for monitoring EFA activities across all projects and programmes in the region and for data collection, analysis and reporting.

5.2.2 Proposed Baseline Studies

As a model for WSD evolves and the development of GESDP gets underway, reliable gender-disaggregated baseline data on the current state of affairs is required. This is best done through utilizing and supporting existing systems and in doing so build the local capacity wherever possible. Discussions with personnel in Planning/ EMIS, SQAD and WAEC identified three strands for data collection and analysis to support baseline data. All three strands would be repeated in Years 3 or 4 to measure ‘change’. To this, we recommended adding a fourth strand and that is a longitudinal qualitative case study in selected schools in Region 5 where the WSD model is being piloted. These four strands are summarized in the table below.

Strand Baseline Studies ResponsibleStrand 1 An National Assessment of EFA in 2005 using quantitative

indicators such as enrolment rates by gender, repetition rates, dropout rates, attendance rates, physical facilities, etc which can be directly derived from and monitored using EMIS data. A separate report on Region 5 would be done to support the whole school development pilot.

DoSE Planning/EMIS Director Lead: Mr. Yunus Hydara with Mr. Alpha of EMIS

Strand 2 An national assessment of pupils’ performance in Grades 3 and 5 cross the four core subjects to support DoSE efforts to establish national education standards and assess how well the system is performing. The sample size would be 30%.

Additions to the baseline study would include administering the Grade 5 tests across the four core subjects to the pupils’ teachers in order to measure their level of content knowledge.

To support the research, pertinent background data on teachers and pupils would be collected at the same time.

Test Development and Analysis: West African Examination Council (WAEC) Test Administration in close collaboration with DoSE . Lead: Mr. Abraham Joof of WAEC

Strand 3 A pre- and post survey of conditions of teaching and learning in lower and upper basic schools in Region 5 to identify factors that impact on student performance and the establishment of an effective education system. Sample size would be 30%, approximately 30 to 32 schools of which 16 would be those schools selected to pilot the whole school development model under BESPOR. .

Lead: to be determined (either DoSE/SQAd or Gambia College)Interim Lead: Mr. Malamin Sonko, National M&E Consultant

Strand 4 A longitudinal case study of 10 to 20 lower and upper basic schools in Region 5 to track qualitative indicators on school and teacher performance and document the process of change as the BESPOR whole school development model is implemented.

DoSE, Tertiary /Higher Education and Research Directorate with University of GambiaLead: Mr. Musa Sowe

To support and build the capacity of DoSE, Gambia College and WAEC, local and international consultants would be contracted for short-term inputs to facilitate the baseline studies. BESPOR M&E Consultants would build the capacity of the teams, monitor quality control and ensure that baseline data and reports are produced in a timely and efficient manner. Further details for each strand follow.

Strand 1: EFA Assessment Report

The EFA Assessment would utilize existing EMIS data from 2000 to 2004 and the 2005 school data that is currently being collected by the Directorate of Planning. Data analysis would be done by EMIS staff wherever possible. The International consultant who designed the EMIS database would be hired to build the capacity of the EMIS staff in data analysis, support complex data analysis, and troubleshoot system problems. The EFA Secretariat would provide financial and human resources to analyze the data and prepare the report in a similar format to the EFA Assessment Report of 2000. Critical to the analysis is the capturing of the CSD’s 2003 population data and collection of missing data on Madrassahs, ECD, Adult Literacy, etc. The output would be a National 2005 EFA Assessment Report that looked at education trends over the past five years and would feed into the development of a sector-wide education programme. The Reports could be updated annually or bi-annually as new data becomes available.

To support WSD in Region 5, a separate EFA report would be produced with EMIS data and with the support of M&E Consultants. Capacity would be built within Region 5 to verify the data in all lower and upper basic schools and interpret the findings as part of the capacity development process. The approach would serve two purposes. It would build awareness among school heads and cluster monitors of the potential uses of EMIS as a management tool. It would also build competence within the staff to handle and verify statistics in order to produce reasonably accurate assessment reports for monitoring and planning purposes.

Strand 2: National Assessment Tests for Grades 3 and 5

A National Assessment Test for pupils in lower basic education is critical to assessing how well the system and pupils are performing across the four core subjects (English Language, Mathematics, Science, Social/Environmental Studies). The West African Examination Council (WAEC) is experienced and well-positioned to carry out NATs and prepare national and region reports. In moving forward, a review of how LATs are linked to the curriculum and textbooks in use is an important first step. The second step is field visits to each region by WAEC/SQAD to better understand whether teachers are following the curriculum and how they are assessing pupils’ performance in the classroom. Such preparatory steps will prepare the way for training a core team to development test items across the core subjects and develop appropriate questionnaires for baseline data.

The sample size would be approximately 11,000 Grade 3 pupils and 9,000 Grade 5 pupils. In addition, it is recommended that Grade 3 and 5 teachers be given the Grade 5 subjects tests to measure their level of content knowledge. Pertinent background information on pupils and teachers would be collected at the same time to allow a triangulation of possible variables that may affect pupils’ performance.

To support the process, it is recommended that an International Consultant with expertise and experience in national assessment testing in Africa be hired to support the development of test items in order to avoid the pitfalls of the earlier studies. Outputs would be a National Report on pupils’ performance to support the development of a sector wide education programme. A separate sub-report on Region 5 would be produced to support BESPOR’s pilot initiatives in WSD. These NATs would be held again in Year 3 or 4 of the Project to measure how well the system is performing after BESPOR and other donor interventions.

Strand 3: Survey of Teaching and Learning Conditions

The baseline study on conditions of teaching and learning would build on the 2000 MLA study by DoSE and the 2003 study that looked at factors (attendance, textbooks, homework) that contributed to student achievement. It is envisioned that pre- and post questionnaires and structured interviews would be developed for pupils, teachers, schools and parents in

order to assess factors that might impact on pupils’ learning achievements within the scope of whole school development. The kind of data to be collected would be directly linked to the six cross-cutting themes of WSD and to gender issues that impact on teaching and learning. The target would be 30% of the schools in Region 5, e.g. 30 and 35 lower and upper basic schools. Twenty of the schools would be those targeted to pilot the WSD model with BESPOR Project and the remaining would be non-project schools.

A partnership of DoSE/SQAD and Gambia College should coordinate and lead the initiative with a baseline team made up of members from Region 5 and the national level. Capacity of the teams would be built to develop appropriate instruments, and collect and analyze the data. The draft questionnaires would be trialed and reviewed by WSD stakeholders before being finalized. It is recommended that a national consultant be hired to design the database, coordinate data entry and do the data analysis. BESPOR M&E Consultants would facilitate the capacity building process and monitor progress.

Strand 4: Qualitative School-based Case Studies Whole School Development cuts across many areas (e.g. curriculum, teacher support systems, school management, community participation, gender equity) and involves many stakeholders. It is complex by design making it impossible to capture the benefits and impact of interventions as discrete entities or through quantitative data only. As such, it is recommended that a longitudinal study that tracks whole school development in a selected number of pilot BESPOR schools (e.g. 10 to 20) should be carried out over the life of the Project. This would produce qualitative data to see the inter and intra relationships and factors that contribute, either positively or negatively, to school and teacher performance. In addition, a selected number of student, teacher and head teacher cohorts within these same schools would be followed more closely on a year by year basis to monitor individual changes as well as processes that are contributing and impacting on the wider school and community.

The study would build on the data collected in project schools in Strands 2 and 3 and where possible the data correlated to assess the variables that impact on change. The design of the study and data collection would be done after the WSD model is conceptualized and Strand 3 is completed. Data would be collected at regular intervals and the findings would be fed back to stakeholders responsible for various areas; e.g. Gambia college for pre-service teacher training, CREDIT and REO 5 for professional development of teachers, DoSE/SQAD for quality assurance across all areas, etc.

In such an evaluation study, a preference is to identify external sources to lead the study in order to ensure an objective view. The DoSE Directorate of Tertiary/Higher Education and Research has been identified as the implementer with support from the University of Gambia. It is envisioned that the case study team would be made up of a cross-section of stakeholders with the Directorate of Tertiary/Higher Education taking the lead and being responsible for data analysis and reporting. The data would provide baseline to guide the development of the education sector-wide programme as well as serve as a model of good practices at the school, cluster and regional levels as the WSD model evolves.

Given the complex nature of the study which will involve school and community stakeholders, the capacity of the case study team will need to be built in the areas of social and education research. This will require short-term inputs of a Consultant with particular expertise in social and education research. Technical Annex 4 provides major steps for each study with proposed timelines. The timelines reflect the need to do baseline studies in Year 1. A National Workshop to share the findings across the four strands would be convened in April 2006. In Year 3 or 4, the

data would be collected across all four strands again to measure change as a result of BESPOR interventions.

5.3 Time Schedule

A three-year timeline for M&E activities is impossible to predict without further analysis and input from DoSE and its M&E Teams. This report recommends that the workshop to develop the initial logframe be held as soon as possible to allow key BESPOR Consultants to participate in the process. Based on the outcomes of that workshop, the LFA would be finalized. The implementation teams would then identify or refine project activities to align them with the agreed on outputs. Proposed timelines are in Technical Annex 4.

Baseline studies would be implemented as soon as possible with preparatory activities happening in July through September 2005, and data collection and data analysis between November 2005 and July 2006. These proposed timelines require further discussions with respective institutions, and alignment with the school calendar and BESPOR budget.

5.4 Cost Estimate and Financing Plan

6.0 Factors Ensuring Sustainability

6.1 Policy Support and Coordination

6.2 Environmental Protection Not applicable 6.3 Socio-cultural aspects/gender issues

6.4 Institutional Management Capacity, public and Private

7.0 Monitoring and Evaluation

7.1 Monitoring Indicators

Technical Appendices

T1: Logical Framework Planning Matrix of the Proposed Programme DesignT2: Detailed BudgetT3: ToRs for any TA requiredT4. Other technical information and data, as required

Administrative Appendices:A1: Consultancy Methodology/WorkplanA2: ItineraryA3: List of persons/organizations consultedA4: List of Documentation Consulted