an14851 questionnaire report final version...2007/03/15 · total 13 74 responses and 60 projects...

Evaluation NPT/NFP

Results of the Questionnaire Survey

Final report

Client: Dutch Ministry of Foreign Affairs

ECORYS Nederland BV

Mart Nugteren

Rotterdam, 14 May 2007

SKH/AN14851/Questionnaire

ECORYS Nederland BV

P.O. Box 4175

3006 AD Rotterdam

Watermanweg 44

3067 GG Rotterdam

The Netherlands

T +31 (0)10 453 88 00

F +31 (0)10 453 07 68

E [email protected]

W www.ecorys.com

Registration no. 24316726

ECORYS Macro & Sector Policies

T +31 (0)31 (0)10 453 87 53

F +31 (0)10 452 36 60


Table of contents

1 Results of the Questionnaire Survey 7

1.1 The Survey 7

2 The NFP Programme 9

2.1 Process description 9

2.1.1 The different modalities 9

2.1.2 Year of participation 10

2.1.3 Type of beneficiaries 10

2.1.4 Involvement of the Embassies 11

2.1.5 Initiating and outlining the MYAs 20

2.1.6 Initiating and outlining the Tailor Made Training 22

2.2 Relevance of NFP 23

2.2.1 Compliance with national development priorities and Dutch

development co-operation priorities 23

2.2.2 The demand driven character of NFP for the beneficiaries. 25

2.2.3 Potential duplication with other donor programmes 27

2.2.4 The role of local and regional training suppliers 28

2.3 Effectiveness of the NFP 31

2.3.1 Objectives of the NFP 31

2.3.2 Contribution to objectives 32

2.3.3 Quality of TMT 33

2.3.4 Effectiveness of NFP in a more general sense 34

2.3.5 Sustainability 36

2.3.6 Other remarks on effectiveness 39

2.4 Efficiency 40

2.4.1 Transparency of the procedures 40

2.4.2 Management by Nuffic 41

2.4.3 The Tender procedure 42

2.4.4 Administrative burden 43

2.4.5 Implications of tying the programme 44

2.5 Improvement compared to previous fellowship programme 45

3 NPT 47

3.1 Process Description 47

3.1.1 Type of partners involved 47

3.1.2 Year of participation and duration 48

3.1.3 Type of projects 48

3.1.4 Involvement of embassies 49

3.1.5 Initiating and outlining NPT 52


3.2 Relevance of the programme 53

3.2.1 Compliance with national and Netherlands international cooperation

development policies 53

3.2.2 The demand driven character of NPT 56

3.2.3 Potential duplication with other donor programmes 59

3.2.4 Role of local and regional training suppliers 60

3.2.5 Relevance during implementation 62

3.2.6 Other remarks on relevance 64

3.3 Effectiveness of the NPT 65

3.3.1 The objectives of NPT 65

3.3.2 Contribution to objectives 67

3.3.3 Quality elements 69

3.3.4 Monitoring 71

3.3.5 Sustainability 72

3.3.6 Other remarks on effectiveness 73

3.4 Efficiency of the NPT 74

3.4.1 Price – quality relation 74

3.4.2 Transparency of procedures 74

3.4.3 Role of Nuffic 77

3.4.4 Availability of resources 79

3.4.5 Administrative burden 80

3.4.6 The limitations of tying 81

3.4.7 The tender procedure 81

3.4.8 Suggestions from different parties on improvement of efficiency. 83

3.5 Improvement compared to previous programme 85

4 Synergy 87

4.1 Synergy with NPT, seen from the angle of NFP beneficiaries 87

4.2 Synergy with NFP, seen from the angle of NPT beneficiaries 87

Results of the Questionnaire Survey 7

1 Results of the Questionnaire Survey

1.1 The Survey

As part of the evaluation a questionnaire survey was organised among embassies

participating in the NFP or the NPT, and among beneficiary institutions of NFP and NPT.

The questionnaires (in English, French or Spanish) were distributed through the Internet

or by e-mail. The following table shows the number of organisations targeted (the

‘panel’) and the number of responses received:

Table 1 Survey data

Size Panel Responses % Response Incomplete Resp.

NPT-Embassies 15 13 86.7% 0

NPT-Beneficiary

institutions

167 74 44.3% 13

NFP-Embassies 49 36 73.5% 3

NFP-Beneficiary

institutions

40 14 35.0% 0

The response rate of the beneficiary institutions of the NFP programme was

disappointing; in all other cases the response was sufficient to allow for statistically

reliable conclusions.

The following countries were represented in the survey:

Table 2 Countries represented in the survey (numbers in cells show number of respondents)

NPT-Embassies NPT-Beneficiary

Institutions

NFP-Embassies NFP-Beneficiary

Institutions

Albania 1

Armenia 1 (Emb. In Georgia)

Bangladesh 1

Benin 1 5 (3 projects)

Bhutan 1 (Consulate)

Bolivia 1

Bosnia-Herc. 1

Brazil 1

China 1 1

Colombia 1 3 (3 projects) 1

Costa Rica 1


NPT-Embassies NPT-Beneficiary

Institutions

NFP-Embassies NFP-Beneficiary

Institutions

Cuba 1

Ecuador 1 1

Egypt 1

Eritrea 1

Ethiopia 1 8 (5 projects) 1

Georgia 1

Ghana 1 2 (2 projects) 1

Guatemala 1 9 (5 projects) 1 1

India 1 1

Indonesia 1 4 (3 projects)

Iran 1

Kenya 1

Macedonia

(FYRoM)

1

Mali 1

Moldova 1 (Emb. In Ukraine)

Mongolia 1 (Emb. In PR China) 1

Mozambique 1 6 (4

projects)

1

Nepal 1

Nicaragua 1

Palestinian A.T. 1

Peru 1 2

Philippines 1

Rwanda 7 (6 projects) 1

South Africa 1 3 (3 projects) 1 1

Surinam 1

Tanzania 1 9 (9 projects) 1 2

Thailand 1

Uganda 1 6 (6 projects) 1

Vietnam 1 3 (3 projects) 1

Yemen 1 3 (3 projects) 1

Zambia 1 6 (5 projects) 1

Zimbabwe 1 1

Total 13 74 responses and 60

projects

38 (in 36 responses) 14

In principle all beneficiary institutes of the NPT received a questionnaire (12 ‘bounced’,

i.e. the e-mail with the Internet Link and the Questionnaire attached returned). Of the

NFP beneficiaries only those with a MYA or those which had participated in a TMT were

addressed.


2 The NFP Programme

2.1 Process description

2.1.1 The different modalities

In the questionnaires sent out concerning the Netherlands Fellowship Programme (NFP) a

distinction is made between the different modalities:

• Ph.D. courses;

• Masters Programmes;

• Short Courses;

• Tailor-Made Training Courses;

• Refresher Courses; and

• Multi-Year Agreements as an awarding instrument for the NFP.

This is not a very clear distinction, i.e. a MYA may contain TMTs (in fact, it often does)

and all other modalities. In most cases however respondents did not react very differently

for the different modalities, i.e. they tend to see the programme as an integral aspect. In

this paper findings concerning specific modalities will only be mentioned of these

findings differ significantly from those of other modalities.

Ten of the respondent institutions (71 percent) have concluded a MYA with NUFFIC,

which definitely has influenced the reaction on the questionnaire. In contrast, among the

countries represented by the responding embassies only a minority (39 percent) was host

to one or more MYAs. In one case, four MYAs had been signed, the average in the

countries of the responding embassies amounted to one and a half.

The different respondents have experience with all other modalities, most with refresher

courses (ten institutes, 71 percent), least with PhD fellowships (four institutes, 29

percent). The embassies mention that the refresher courses in their countries are the least

represented (13 countries, 36 percent), and individual fellowships the most represented

(35 countries, 97 percent). This apparent contradiction may be caused by the fact that

institutions that responded all have a structural relation with the NFP. This latter fact is

also illustrated by the number of (ex-) NFP participants in each institute, which ranges

between 12 and 270. The total number of participants represented by the respondent

institutes is about 560 (some respondents could not give an exact number), and the

average number per institute 40. If a correction is made for the extreme observation of

270 NFP participants, the average of NFP participants in the respondent institutions is 22.

TMT appear to be the most important modality if expressed by the number of

participants: out of the mentioned 560 participants over 480 participated in a TMT.


2.1.2 Year of participation

The following figure shows the years in which the NFP beneficiaries participated in NFP:

Table 3 Years of participation (multiple responses were possible)

Start-year participation Number of NFP beneficiaries

2002 2

2003 2

2004 6

2005 7

2006 8

2.1.3 Type of beneficiaries

Most of the NFP beneficiaries are public bodies, as illustrated in the next table:

Table 4 Legal status beneficiaries

Number %

Public or governmental

organisation

8 57%

NGO 4 29%

Self-financing part of University 1 14%

Unclear 1 14%

The following chart shows the sectors represented by the respondent institutes:


Figure 1 Sectors in which NFP institutions are active (multiple responses were possible)

Agriculture; 21%Teacher training; 7%

Education; 50%

Research and

development; 36%

Public administration; 7%

Health/Medicine; 7%

Justice & Human Rights;

29%

Natural Resources &

Environment; 21%

Policy development &

management; 43%

Social Economic

Development ; 7%

Water & Sanitation; 7%

Other, namely; 14%

It is interesting to note that 50 percent of the respondents classify themselves as involved

in education, even 93 percent if Teacher Training and R&D are included.1 It is not clear

why these institutes are involved in a MYA and/or a TMT, and not in NTP. In fact,

several of the sources interviewed considered the difference between the NPT and a

MYA vague, and this observation confirms this view.

2.1.4 Involvement of the Embassies

The following modes of involvement of the RNEs in the NFP were checked:

• promotion of the programme;

• identification of candidates;

• screening of candidates;

• screening of requests for MYAs;

• screening of requests for TMTs;

• assistance to (potential) candidates;

• preparation of the outlines for TMT and MYA.

The following graph shows the involvement of the respondent embassies in the

promotion of the NFP.

1 Since multiple answers could be given, these institutes consider themselves also active in other sectors, e,g, they can be

agricultural training institutes.


Table 5 Extent of involvement of embassies in promotion of NFP; 1=no involvement; 5=intensive involvement

Category Number of respondents %

Not involved 7 20%

Hardly involved 12 34%

Neutral 8 23%

Involved 6 17%

Intensive Involvement 2 6%

35 100%

The modal group2 is the group ‘hardly involved’. A majority of the approached embassies

(19 out of 35, category 1-2) see themselves therefore not or hardly involved in the

promotion of the programme. Interestingly, eight respondents consider themselves

involved and even intensively involved in the promotion of the programme. There

appears to be therefore a large variety in extent of involvement in promotion, a variety

which will also be observed in other forms of embassies’ involvement.

Box 1 Examples of Embassies’ involvement in the NFP

By providing NFP brochure together with a standard letter (with detailed information) to local authority

(government), Corporations, NGO and private sectors. Providing information through e-mails and

telephone calls.

The NFP programme is managed and administered from the embassy. The federal ministries based in

the capital and ministries of the state governments based across the country are to be reached to inform

about the NFP. This is done by direct mailing, group and individual interaction with decision makers. Of

course the embassy website is an effective tool as well. Non governmental organisation need to be

targeted particularly in the rural heartland where availability of information itself is a problem let alone

internet access. Good NGOs, universities and other institutions need to be identified and reached with

the NFP possibilities and these are done through pre-research and consequent follow up with printed

materials. Being a continental sized country all corners of the country are reached from the embassy in

the capital.

I take an initiative role and send some brochures to 2 or 3 universities, as the embassy here is not

interested in promoting the NFP program.

1. Through the national news paper. NFP information and deadlines are published in the national

newspaper. It gets distributed to all 20 Districts of the country. 2. The NFP brochures are distributed to

all government Ministries, Departments, Corporations and to all private sectors and NGOs in the

country. 3. Individual copies are also given to the candidates from remote areas where there are no

good communication facilities etc. 4. Upon request of the interested organizations, we did NFP

presentations during their annual conference/workshops. 5. NFP procedures are also printed and

distributed to every one whoever is interested in the NFP programme. 6. Visiting hours for NFP has also

been introduced to help the applicants properly. 7. We also organize gatherings and orientation

programme to the NFP selected candidates every year and we invite the media Broadcasting Service

and the National Newspaper. They provide a good coverage in the TV and news.

2 The group represented by the largest group of respondents.


The actively distribute information regarding the NFP-programme to organisations, amongst which local

authorities. We promote local authorities to apply, not only for short courses and masters, but also for

tailor-made training. There is even one government agency under the MYA. We have a window on

Monday and Wednesday and respond to enquiries made. We also receive a high number of information

requests by phone. If necessary, we meet with potential candidates (individuals or organisations) to

explain the possibilities presented by the programme.

We do not promote the NFP very much with local authorities. We would create "false hopes" and in

reality we do not get that many fellowships.

Almost all embassies distribute NFP brochures, but some embassies actively address

organisations which they consider relevant for participation in NFP. Some embassies

organise information meetings for interested employing organisations. The box above

contain some of the direct answers on the open question ‘how is the embassy involved in

promotion?’ illustrating the variety of approaches.

The following graph illustrates the involvement of the embassies in the identification of

participants for a masters programme, a PhD study or a short course.

Table 6 Extent of involvement of embassies in participant identification; 1=no involvement; 5=intensive involvement


Not involved 3 9%


Neutral 5 14%

Involved 8 23%


35 100%

Also here a variety can be observed, although the bias is now clearly towards

involvement, the modal group being ‘intensive involvement’.

A different picture emerges for the identification of potential southern partners for a

MYA or beneficiary institutes for TMT or refresher courses.


Figure 2 Extent of involvement of embassies in identification of NFP partner institutes; 1=no involvement; 5=intensive

involvement (in absolute numbers)

0

5

10

15

20

25

1 2 3 4 5

MYA TMT Refr.

For MYA and TMT there is a certain variety, i.e. although the majority of the embassies

consider themselves to be not or not heavily involved, a large minority (eight, 23 percent)

feel that they are involved or even heavily involved. This picture correlates with the

earlier observation concerning the involvement in promotion (see Table 5)It is clear that

the embassies see no role for themselves in the identification of candidates for refresher

courses, which is not surprising: the initiative here is somewhere between the Dutch

providers and their previous participants.

In the role of the embassies in the screening of the candidates a distinction was made

between screening on:

• eligibility of the individual candidate;

• completeness of the application;

• quality of the application.


Figure 3 Extent of involvement of embassies in screening of candidates for masters’, PhD or short courses; 1=no

involvement; 5=intensive involvement (in absolute numbers)

0

5

10

15

20

25

1 2 3 4 5

Elig. Compl. Qual.

This picture clearly correlates with the involvement of embassies in identifying

participants. All embassies see at least some role in verifying whether the applications are

filled in correctly (group 1 for ‘completeness’ is empty), but for all three types of

screening the modal group is ‘heavily involved’. In addition, notice should be made of the

following answer on an open question concerning the role of the embassy in trouble

shooting:

Box 2 Illustration of the role of the embassy in the screening process.

A Governmental Organisation does the first screening of fellowship applications for Short Courses,

Master and PhD Programmes, which explains my previous answers when I said that I am partially

involved in screening, checking the documentation, etc. Applications that do not fulfil the NFP

requirements do not reach the Embassy. However, once the applications are sent by the Embassy to

Nuffic with a recommendation, I do support the whole process, like over viewing the visa process with

my colleague at the Consulate Section of this Embassy, solving any problems concerning their tickets,

etc.

Involvement of the embassies in screening potential partner institutes for MYA, TMT and

refresher courses is shown in figure 4. Also here the lack of the role of the embassy in the

screening of candidates for refresher courses stands out (only completeness of the

application is verified).


Figure 4 Extent of involvement of embassies in screening of NFP partner institutes; 1=no involvement; 5=intensive

involvement (in absolute numbers)

0

5

10

15

20

25

30

1 2 3 4 5

MYA TMT Refr.

Embassies play a clear role in assisting candidates, as shown in the following figure,

where group 1, ‘no involvement in assistance to candidates’, is empty.

Table 7 Extent of involvement of embassies in assisting candidates; 1=no involvement; 5=intensive involvement


Not involved o 0%


Neutral 5 14%

Involved 10 29%


35 100%

When asked about the type of assistance, the following options where possible:

• obtaining of visa;

• obtaining of tickets;

• other.


Figure 5 Type of assistance provided by embassies to candidates (in absolute numbers)

0

2

4

6

8

10

12

14

1 2 3 4 5

visa ticket other

Under the category ‘others’ the respondent embassies mentioned

• verification of the knowledge of English

• providing general information on (living in) the Netherlands

• legalisation of birth documents

• assistance in filling in the application forms (three respondents)

• liaison with the Dutch training institute, resolving problems (three respondents).

One embassy had embarked on the organisation of preparatory meetings and the

distribution of lists of names of previous participants.

Finally the embassies were asked whether they were involved in drawing up the outline

of MYA and/or TMT.

Table 8 Extent of involvement of embassies in drawing up outlines; 1 = no involvement; 5 = intensive involvement


Not involved 15 48%


Neutral 6 19%

Involved 4 13%


35 100%

It is safe, on the basis of the above, to draw the following conclusions on the role of the

embassies in the pre-training stage:

• All embassies consider themselves involved, and a majority significantly involved, in

screening and assisting individual participants of masters, PhD or short courses;

• Most embassies consider themselves involved in identifying individual participants of

masters, PhD or short courses;

• A minority of embassies, however, consider themselves involved in promotion of

NFP;


• A minority of embassies see an involvement for themselves in identification or

preparation of MYAs or TMTs;

• There is hardly any involvement of the embassies in the refresher courses;

• In the role of the embassies there appears to be therefore a bias towards a passive

role. The picture is however one of variety, there are a number of pro-active

embassies.

This is confirmed by the role of the embassies in the process of actual conduct of the

different NFP activities. When asked about then role of the embassy in trouble shooting

and monitoring and evaluation, the following picture emerges:

Figure 6 Extent of involvement of embassies in trouble shooting and M&E; 1 = no involvement; 5 = intensive involvement

(in absolute numbers)

0

2

4

6

8

10

12

1 2 3 4 5

trouble sh. M&E

In the answers to the open questions on the type of trouble shooting it appears that most

of these related to travel and official documents problems (birth certificates, different

spellings of names in passports etc.). Some remarks stand out:

Box 3 Examples of trouble shooting

Verifying that certificates from Dutch Universities are authentic

Verifying accusations and counter accusations from a local company awarded a MYA which seemed to

have submitted wrong information

I suggested to Nuffic that it would be a great service if candidates could apply on-line for a fellowship (it

would mean less work for the RNE fellowship officer).

Problems in communication with Dutch Universities: some informed about the result of fellowship

awards to the applicants directly without informing the Embassy, some requested Fellowship Officer to

contact persons who did not apply through the Embassy (wrong procedures)

Some Dutch Universities keep on sending award packages to the Embassy instead of the Consulate

General in …, even though its Fellowship Officer's name is mentioned in every application form.

The answers to the open questions concerning monitoring and evaluations related mainly

to monitoring the screening process, creating the impression that the involvement of


embassies in actual monitoring is even less than demonstrated in the figure above,

However, the following answers show that several embassies have a keen interest in

knowing the effect of the NFP and the relevance of the NFP:

Box 4 Selected embassies’ opinions on the monitoring function

We try to monitor and evaluate the programme, but it is hard to estimate for example the impact of the

programme. However, we hope that through the recently established Netherlands Alumni Association,

we will get a better picture on how alumni use their trainings.

We do not get feedback about the results of the scholarship holder in the Netherlands. Sometimes we

have contacts with the fellowship holder afterwards. The alumni association does not work.

The embassy always requests candidates who have been granted a fellowship to inform the embassy

on their progress. Some do and some do not. Useful outcomes from these reactions are integrated in

the service of the embassy if applicable.

Assessing the number of participants from year to year and trying to find out the shortcomings.

I did ask the candidates to come back upon completion of their stay in the Netherlands for feed back in

2007

Through the Netherlands Alumni Association, we try to find out how and if the course was effective. We

have not been able to follow up with the organisations but that is something we could plan to do in the

coming future.

The Fellowship Officer makes annual report on the NFP's implementation to the Ambassador and the

Officer advises Nuffic on the actual national situation and proposed at one instance a different form of

MYA. This recommends to award fellowships to persons/institutions in the prioritised sectors in

development cooperation between the NL and VN.

Fellowship Officers are very active in evaluating candidates. Frustration is that their advice is not always

followed, even though they know what they're talking about for their respective countries/candidates.

We have printed in bold those remarks concerning the frustration of some embassies that

their advice on the selection of candidates, which apparently would have improved the

cohesion between NFP and the general Dutch bilateral assistance, was not followed up by

the Dutch training institute.

The insight of the embassies in the results of the NFP varies from modality to modality,

but generally is very limited. In contrast, a large majority of the embassies would

appreciate to have a better insight: 85 percent in the case of TMT and PhD, 82 percent for

Refresher Courses and 91 percent for short and Masters courses.


Figure 7 Insight of embassies in results of NFP (in absolute numbers)

0

2

4

6

8

10

12

14

16

18

20

Not at all Some Good Very good

TMT PhD Refr. Short Masters

The next box presents some activities performed by the RNEs which could not be

categorised in one of the activities discussed above.

Box 5 Other RNE activities

For tailor made course assisting the applicant organization to finalize the proposal. Following up with

both NUFFIC and applicant organization over the proposal. Helping Netherlands Alumni Association in

organizing orientation program to the new participants every year. Counselling students who wishes to

study in NL in self finance for undergraduate courses and helping them find suitable courses/institutes.

Attending opening ceremony of refresher courses, helping them with visa if necessary and attending

workshops and meetings that concerns NFP.

With some NFP Fellowship Officer colleagues in other embassies in the world, we try to identify

common problems etc. to do suggestions towards NUFFIC, in order to try to improve the programme.

Involvement in setting-up or managing alumni associations (three response)

Participation in diploma awards events

2.1.5 Initiating and outlining the MYAs

As stated above, of the fourteen southern organisations which responded to the

questionnaire, ten (71 percent) had entered into a MYA with NUFFIC. This percentage is

however not representative, since only organisations who had entered into a MYA and

organisations which had benefited from a TMT had been addressed. The four

organisations which had not yet entered into a MYA unanimously declared that they

would be willing to do so. In one of these organisations discussions with NUFFIC about

such an agreement had already started.

A variety of actors appear to play a role in initiating a MYA: in three cases the Dutch

Embassy, in two cases a local consultant, and in one case NUFFIC. In the other cases the


institute itself reportedly took the initiative and contacted NUFFIC or the Embassy. None

of the MYAs was initiated by a national (parent-) agency or Ministry.

The following graph illustrates the involvement of possible organisations in the

formulation of training needs in the MYA:

• Embassy;

• Local consultant;

• NUFFIC;

• parent-Ministry or other National Agency;

• the Organisation itself;

• the Dutch Education Institute.

The questionnaire here did not ask which organisation was most involved, it is therefore

possible that different organisations play parallely an important role. It appears that the

organisation itself plays an important role, although one organisation sees no role for

itself in this process. Local ministries are hardly involved in the process. Involvement of

local consultants is either absent, or considered (six cases) involved or strongly involved,

this most probably dependent on the fact whether or not a local consultant had been

appointed. Also the involvement of NUFFIC (seven cases involved or strongly involved)

plays a role. Although in the majority of cases the Dutch training institute is not involved,

in two cases mention was made of involvement or strong involvement.

Figure 8 Importance of different organisations in formulating training needs for MYA; 1 = no involvement, 5 = intensive

involvement. (in absolute numbers)

0

1

2

3

4

5

6

7

8

9

1 2 3 4 5

Embassy Cons. Nuffic Ministry Own org. Dutch ed. inst.

The conclusion can therefore be made that the process appears to be demand driven, with

a strong involvement from the side of the southern institute. There are however cases in

which the needs formulation is heavily influenced by NUFFIC or the Dutch training

institute.

This will be further elaborated in the section dealing with relevance.


2.1.6 Initiating and outlining the Tailor Made Training

Out of the fourteen responding southern institutes, twelve had been recipient of one or

more TMTs. This implies that in eight to ten of these cases, the TMTs were probably

organised in the framework of a MYA. In all cases the TMTs aimed at capacity building

through human resources development, either in the technical field (e.g. experts in water

resources management), general management skills or human right issues. The large

majority of these organisations felt that they had sufficient knowledge of the training

needs of their staff (eleven) and sufficient time (ten) to formulate a concrete training

outline.

Involvement in the establishment of the training outline was investigated for the

following parties:

• the institute itself;

• sector ministries;

• relevant NGOs;

• embassy;

• local consultant;

• Dutch training institute;

• NUFFIC.

Figure 9 Importance of different organisations in formulating outline for TMT; 1 = no involvement, 5 = intensive

involvement. (in absolute numbers)

0

1

2

3

4

5

6

7

8

9

10

1 2 3 4 5

Own org. Min. NGOs Embassy Cons. Dutch ed. inst. Nuffic

There appears to be a certain correlation with the findings concerning the MYA which is

not surprising considering the fact that at least eight of the respondents are the same.

These findings can therefore be seen as a confirmation of the findings on the MYA. The

process is demand-driven, in as far as the involvement of the southern institute itself is

involved. The role of national policy makers is however almost negligible. Instead,

NUFFIC and to a more limited extent the Dutch training institute also play in a number of

cases an important role in the process. Also the RNE is involved or heavily involved in


ten of the twelve cases, which is interesting since this perception is not confirmed by the

opinion of the respondent RNEs.

The respondent institutes were unanimously satisfied with the roles played by the

different parties in this process.

Further reference is made to the section on relevance.

2.2 Relevance of NFP

2.2.1 Compliance with national development priorities and Dutch development co-operation

priorities

The RNEs, asked about whether the NFP is compliant with national development

policies, provided the following answers:

Table 9 RNEs opinion on compliance of NFP with national development priorities


No compliance 4 12%

3 9%

Neutral 9 26%

6 18%

Full compliance 12 35%

35 100%

It appears that, although the majority of respondents agree with the statement that the

NFP is compliant with national priorities, an important group (over 20 percent) disagree

with this statement. Almost half of respondents disagree or has no opinion on this

(‘neutral’). Since, according to the embassy, the initiative for the application is almost

fully with the individual candidate (PhD, Masters, Short Courses and Refresher courses)

or with a local institute (MYA, TMT) there is in most countries no mechanism which

could ensure a full compliance with national development priorities. This is consistent

with the findings on the role of national policy makers in identifying training needs or

training outlines for TMTs or MYAs (see section 2.1.5 and 2.1.6), which according to the

southern institutes is almost negligible.

The southern institutes involved in a MYA felt however that the programme was well in

line with the national strategy for the sector in which the institute was active: eight

respondents (80 percent) felt that this was fully the case, one partially and one had no

opinion on this.

Two questions related to compliance with Dutch bilateral development policies were

addressed to the embassies:

• Is the NFP responsive to the needs in sectors identified for Dutch Development

Assistance;

• Is the NFP harmonised with other activities in sectors identified for Dutch

Development Assistance.


The following graph illustrates the findings.

Figure 10 RNEs opinion on compliance of NFP with Dutch development priorities (in absolute numbers)

0

2

4

6

8

10

12

Disagree Neutral Agree

responsive harmonised

Also here, a mechanism for compliance is lacking, which is logical, considering the fact

that NFP is not limited to the sector policy. A majority of respondents feels that the NFP

should indeed not be limited to the Dutch priority sectors (85 percent, the other 15

percent feeling that it should be limited).

Despite of the above, the majority of respondents feel that NFP has an Added Value to

the bilateral cooperation programme (may be exactly because it is not limited to the same

sectors).

Table 10 NFP has Value Added for Dutch Bil. Policy, acc. to RNEs.


Disagree 1 3%

1 3%

Neutral 6 18%

8 24%

Agree 17 52%

35 100%

Obviously, among the respondents who agree there are those who represent the

Netherlands in countries which have no bilateral cooperation programme with the

Netherlands. Of the other respondents, the following remarks were selected which are

representative for all remarks made.


Box 6 Open remarks on Added Value of NFP

The NFP programme itself stand alone as one of the best tools of assistance. It has an added value of

fostering cultural and people to people contact between … and the Netherlands as NFP fellows are the

best ambassadors for the Netherlands.

NFP programme besides being the open fellowship, has played a very important role in the

development of the candidate, organizations and to the country. The government of …. does not have

sufficient funds for the capacity building of the civil servants, and neither to strengthen the private

sectors, although the government puts the private sector development in the centre. The NFP

programme has played a very important role in helping the government and the private sectors in

strengthening the capacities of the potential employees in the development of their organizations.

It could be linked better to the government priorities for development in the country.

It has added value in the sense that providing training, expertise and know how helps the root of many

organisation problems. Without expertise, some organisations do not have the capacity to administer

programmes (from the bilateral cooperation) successfully.

2.2.2 The demand driven character of NFP for the beneficiaries.

The number of applications is an indicator of demand. RNEs were asked whether the

number of applications indicate that there is a demand for the different modalities of the

NFP.

Figure 11 Number of applcations indicate demand for NFP, according to embassies (in absolute numbers)

0

5

10

15

20

25



The high score for short and masters courses may be influenced by the large number of

applications, rather than the extent to which this proves a demand (after all, short courses

are in number of participants still the largest modality of the NFP).


Generally, the different respondent embassies agreed that the NFP is demand-driven and

flexible, although the modal group has no opinion on this:

Figure 12 RNEs opinion on the demand driven character and flexibility of NFP (in absolute numbers)

0

2

4

6

8

10

12

14


demand-dr. flexible

The southern institutes generally felt that the MYA was well in line with the strategic

working plan of their organisation, as well as the relevant sectoral priorities: eight

respondents felt that this was fully the case, one partially and one had no opinion on this.

Among the open answers only the following critical remark was made:

Box 7 RNEs opinion on the demand driven character and flexibility of NFP (in absolute numbers)

The selection of participants for the Masters programmes did not take into account the status of the

applicants. Nuffic seems not have a good understanding of the structures of administration set up we

have. It appears that they consider titles in their own rights. For example, they give advantage to Sub

County Chief over an Assistant Chief Administrative Officer when the later is at a higher level. In other

words, Nuffic tends to consider officers with " definitive titles" eg Chief Administrative Officer, Medical

Superintendent, Sub County Chief, Police Commander, Community Development Officer etc. In the

process, the selections do not reflect the traditional process of administrative command in the service.

Virtually all respondent southern institutes felt that their organisation had sufficient

possibilities to express their ideas on the training outline: eleven out of twelve TMT

beneficiaries and nine out of ten MYA partners answered ‘yes’, in both groups one

respondent answered ‘to a limited degree’.

Southern institutes were asked whether the training outline of the TMT reflected the

training needs of the participants. Seven out of twelve respondents felt that they reflected

the needs very closely, the remaining five answered ‘well enough’.

The conclusion therefore is that there is a demand for NFP and that in the design and

implementation of MYA and TMT the demand of the beneficiary is taken into account.


The NFP tender procedure has implications for efficiency, but also for relevance, since it

was expected that through opening up the demand to more potential suppliers (i.e. Dutch

training institutes), the actual supply would be better geared to the demand. The above

findings seem to confirm that in most cases indeed the supply matches the demand, but in

order to verify whether this can indeed be attributed to the ‘modus operandi’, the

following question was asked to the southern institutes:

• Do you, during implementation, experience differences between actual

implementation and contents of the Dutch provider’s proposal?

Table 11 Experience of differences between proposal and implementation


Hardly any 5 42%

Little 2 17%

Neutral 1 8%

Large difference 3 25%

Very large 0 0%

Do not know 1 8%

12 100%

The following box groups the open answers on the question concerning the differences

between implementation and plan.

Box 8 Differences between plan and implementation

Small differences mainly in the speed of implementation

Materials for the first two modules were not specifically made on our request, but were ‘off-the-shelf’

material which had been used in other trainings in Ecuador. Obviously, the material was not useless,

but tailor-made material geared to our specific needs would have been better. On the other hand, the

‘distance-learning part’ should be implemented by ‘distance-trainers’ for which it would be desirable if

this part were to be implemented under the responsibility of Dutch universities. However, according to

the Consortium, the low budget makes it necessary to have this part implemented by local

professionals, who however do not dispose of the proper experience.

2.2.3 Potential duplication with other donor programmes

Among the fourteen respondent southern institutes, three were also beneficiaries of other

bilateral HRD programmes (Belgian, Spanish and Finnish), and all three rated the co-

operation with the NFP as ‘much better’ than the other programme. Interestingly, all three

noted that the structural character and the variety of activities open under a MYA made

the Dutch programme score higher than the other.

A wider perspective was offered by the respondent embassies. In 213 of the 34 countries

represented by the sample, similar donor programmes were being implemented. In the

answers to the open question reference was made to fellowship programmes of the USA

3 Possibly more, ten respondents did not know.


(Muskie Fellowship Programme, Ford Foundation, Fulbright), UK (Cheveningen, Charles

Wallace), Germany (DAAD), Japan Fellowship Programme, NORAD, SIDA, France

(Centre Dumas), Spain, Australia and others. Only two of the respondents rated these

‘very similar’, the majority rated them as ‘somewhat similar’. One of the respondents

remarked: although there is a wide supply, the demand outruns the supply.

2.2.4 The role of local and regional training suppliers

Seven of the twelve respondent institutes involved in a TMT were of the opinion that

regional training providers are certainly (two) or probably (five) available which could

have potentially provided similar services as those received from NFP. Two answered

‘probably not’, and two did not know. The majority again felt that inclusion of such

providers would be desirable in the training of their staff: three felt that this was desirable

in most cases, and six in all cases. One respondent felt this to be definitely not desirable.

The Dutch Training Institutes obviously make use of local or regional training providers

in a MYA or TMT, and seven out of the twelve institutes which responded to this

question felt that this was done sufficiently (four respondents had no opinion, one felt that

this was not sufficiently practised). Through the open questions it appeared that at least in

one case inclusion of local training providers was a result of the advice of the TEC to the

Dutch provider. Also co-operation in the organisation of a TMT with other Dutch

programmes in the field (SNV) is mentioned.

Also the embassies were asked whether local or regional training capacity was available,

comparable to the quality as delivered under NFP. The following graph groups the

answers. Unfortunately, the category ‘do not know’ is quite large, but it appears that:

• According to the opinion of a majority of respondents, especially for short and

master’s courses there is alternative local or regional capacity available (here also the

category ‘don’t know’ is smallest);

• There is a group of countries or regions, be it a minority, in which NFP potentially

duplicates locally available training capacity.

Obviously, refresher courses are a special case.


Figure 13 Availability of local / regional training capacity, according to Embassies (in absolute numbers)

0

2

4

6

8

10

12

14

16

Cons. Extent Certain Extent Small Extent Not at all Don't know


Embassies were also asked whether they considered the inclusion of regional capacity

desirable.

Figure 14 Inclusion local / regional capacity desirable according to Embassies (in absolute numbers)

0

2

4

6

8

10

12

14

16

18

In all cases In some cases Problematic Don't know

TMT PhD st. Refr. Short Masters

It is obvious that a majority of respondents see a role for local providers, especially for

short courses. When asked whether the Dutch Training Institutes, in the design of their

programmes make sufficiently use of local or regional capacity, the following picture

emerged.


Figure 15 Sufficiency of inclusion local training capacity, according to RNEs. (in absolute numbers)

0

5

10

15

20

25

Yes No Don't know

TMT PhD st. Refr. Short Masters

Of the group who has an opinion on this, the majority therefore feels that there is space

for more involvement of local players. The following selected but representative open

remarks shed more light on this.

Box 9 Opinion of embassies on involvement local / regional training capacity

I believe the logistical and other related issues could be the constraint the Dutch institutions find in their

effort to offer courses regionally. There are a few attempts as far as I am aware but in general the

engagement and effort need to be intensified.

As most applicants for short courses and master degree programmes are individual applicants, Dutch

institutions involved are not making much use of the regionally available expertise, unlike cases with

Tailor-made training courses.

We may not forget that the NFP brings the fellowships holder in contact with the Netherlands and the

Dutch culture. Most of the fellowships holders come back with a very positive impression of the

Netherlands and with some of the Dutch values. And that's a very very important aspect of the

Fellowships Programme. Besides, neighbouring countries send students to … as the ….. education is

normally better than in those countries. We don't need the NFP for regional expertise! We need it for

Dutch expertise.

In two years time I have never received a proposal/suggestion from Dutch side to use a regional expert.


2.3 Effectiveness of the NFP

2.3.1 Objectives of the NFP

The Southern institutes were asked which of the following objectives rates as most

important:

• general improvement of quality of staff;

• improvement of competences in specific areas;

• reducing manpower shortages in specific divisions;

• reducing manpower shortages in specific positions;

• contributing to career perspectives of mid-career staff;

• contributing to training policy of the organisation;

• attracting new talented staff members;

• improving quality of the organisation;

• improving the management of the organisation.

Obviously the questions were not mutually exclusive. The following graphs shows the

objectives seen as most important:

Figure 16 Which NFP objectives rates as most important among benefiaries (in absolute numbers)

0

1

2

3

4

5

6

7

8

9

attracting new talented staff members contributing to training policy of the organisation

reducing manpower shortages in specific divisions contributing to career perspectives of mid-career staff

improving the management of the organisation improving quality of the organisation

reducing manpower shortages in specific positions general improvement of quality of staff

improvement of competences in specific areas

Most respondents rated staff improvement (either general or specific) as the most

important objective – attracting new staff as the least important.


2.3.2 Contribution to objectives

Next, respondents were asked for which of the above objectives the NFP TMT is most

effective, i.e. has contributed most.

Figure 17 Objectives for which TMT is most objective (in absolute numbers)

0

1

2

3

4

5

6

7

8

9

10

attracting new talented staff members contributing to training policy of the organisation

reducing manpow er shortages in specific divisions contributing to career perspectives of mid-career staff

improving the management of the organisation improving quality of the organisation

reducing manpow er shortages in specific positions general improvement of quality of staff

improvement of competences in specif ic areas

Respondents see a strong rank-correlation: there were the objectives were seen as

important, also the contribution was assessed as strongest. Only three objectives ranked

differently:

• improving competences in general: nine respondents rated this as ‘most important’,

whereas ten respondents rated this as ‘strong contribution’ (highest possible score);

• reducing manpower shortages in specific positions: eight respondents rated this as

‘most important’, whereas six respondents rated this as ‘strong contribution’;

• improving management of the organisation: seven respondents rated this as ‘most

important’, whereas six respondents rated this as ‘strong contribution’.

Specific obstacles for meeting the objectives were the limited absorption capacity of the

programme (‘small number of participants, only new topics taught under NFP’), the fact

that sector management (‘policy level’) was not influenced and brain-drain. This latter

factor, normally an important factor threatening the sustainability of any development

programme, was also mentioned once among the open answers.

It is obvious that NFP is seen as a programme aiming at capacity building through

strengthening of staff and strengthening of the organisation.


2.3.3 Quality of TMT

Ten of the twelve ‘Southern respondents’ felt that they were sufficiently informed,

through training evaluations and training reports, to form an opinion on the usefulness of

the NFP for the individual staff members. The following is based on this judgment.

Respondents could give an opinion on:

quality of the Dutch trainers;

• quality of local or regional trainers;

• content of the courses;

• responsiveness to specific training needs of participants;

• match with the knowledge level of participants;

• duration of the courses;

• size of the groups.

Figure 18 Appreciation of the quality of NFP training by Southern Institutes (in absolute numbers)

0

1

2

3

4

5

6

7

8

9

Highly insuff. Insufficient Neutral Good Very good

Dutch Trainers Local Trainers Content Match needs Match level Duration Group Size

The following box contains some suggestions from the respondents to further improve the

programme.


Box 10 Suggestions for improvement

Introduce a proper / better mix of long-term (PhD, Masters) with short, ore applied courses.

NFP should include some bachelors programme which are in short supply in Africa, like health

programmes at bachelor level - there are very few regional institutes offering these.

Increase the number of training AND the number of participants

Dutch providers MUST be required to: 1. use data from local context, 2. include local experts as part of

their training teams 3. be more familiar with the local context and issues.

Local experts on issues pertaining to local environment be engaged. Method of identifying trainers be

more transparent

2.3.4 Effectiveness of NFP in a more general sense

It should be noted that the Southern Institutes which responded are organisations which

have benefited from a MYA or a TMT (in most cases even both), which may bias the

findings. The embassies are in a better position to provide a view on the entire NFP, i.e.

including the Masters, PhD, Short and Refresher courses which are organised outside the

framework of a MYA.

The following graph shows the extent to which the different modalities, according to the

RNEs, cater to short-term training needs and capacity building in a broad range of

organisations.

Figure 19 Does NFP cater to S.T. training needs and cap. building, according to RNEs (in absolute numbers)

0

2

4

6

8

10

12

14

16

18

20

Very weak Weak Neutral Strong Very strong Don't know

TMT PhD Refr Short Masters

The modality which stands out here are the refresher courses, but mainly because the

category ‘do not know’ is extremely large. Also earlier it was noted that the RNEs have

no insight in this modality. Also the PhD modality scores lower, but only slightly. In


general, the RNEs which have an opinion on this feel that NFP caters to training needs

and capacity building.

Box 11 Opinions of RNEs on potential effectiveness of NFP for capacity building

For medium sized organisations, a dose of Dutch training can radically change the way organisations

operate as most of these organisations lack the capacity to train their personnel. Therefore one person

who got an NFP can transfer the knowledge to other and effect a shift in the confidence level and help

them find innovative ways of delivering their services and products. The spin-off effect is tremendous.

Our candidates are mainly civil servants and people of NGO's, researchers, university teachers and so

on. The embassy always looks especially if the concession of the scholarship will help the organisation,

will be positive for the development and will result in the spreading of knowledge and expertise.

The impression exists that though there is a need for short term courses and for capacity development

in general, much is left to individual initiatives or needs despite the embassy’s efforts to integrate this in

the organisation needs. Some progress has been made in this regard but is not satisfactory.

Tailor made ranks strong because, we have had only one organisation training under this course, we

have not yet evaluated the outcome of the course. Refresher courses are also not very commonly

attended, but for Masters and PhD's they are in great demand because the job market is very

competitive and organisations are looking for not only the best people but also competent, Netherlands

is known to be one of the countries with very good universities and hence is becoming very competitive.

It appears that people studying in the Netherlands through NFP have been very effective in

implementing their knowledge. They have high positions as advisers to PM or even Ministers

The large majority of respondent embassies feel that the design of the NFP programme is

appropriate to alleviate qualitative and quantitative shortages of skilled manpower: 58

percent feels that it is appropriate and 24 percent even ‘very appropriate’ (six percent

does not know). The modalities considered most appropriate are MYA and TMT, least

appropriate are grouped in the following box.

Box 12 Elements in the NFP considered least appropriate by the RNEs

Refresher courses

PhD courses

Generally courses with a high academic content

Short courses

Courses not geared to national priorities

Courses like mathematics, tourism, theology, fine arts, humanities, ‘soaps and society’.

The MYA is considered to be by 48 percent of the RNEs of major importance for the

achievement of the objectives of the NFP, by 33 percent of some importance. Those who

rated it as not or less important, as appears from the open question, have no experience

yet with a MYA. Especially the possibility to assess the needs and to focus supply on the

needs is appreciated.

Of the ten organisations with whom a MYA had been signed, six were satisfied with its

implementation, one very satisfied, but three highly (!) unsatisfied. Reasons for the latter

are especially the disappointing speed of implementation, although also some criticism

was given on the lack of diversity and focus of the training.


2.3.5 Sustainability

An important criterion for effectiveness and sustainability of the effects of training

programmes is the extent to which the activities are directed at the proper participants.

Embassies were asked whether the selection criteria sufficiently guarantee that the

appropriate candidates are selected.

Figure 20 Criteria guarantee that appropriate candidates are selected, according to embassies. (in absolute numbers)

0

2

4

6

8

10

12

14

16

18



Earlier the involvement of embassies in the process was described, from which it may

appear that they may not be the most objective source of information.

Both embassies and Southern institutes were asked about the likeliness that participants

of different training activities could be retained by their employers, i.e. whether no brain-

drain would occur. The following chart shows the opinion of the embassies on this.


Figure 21 Retaining of participants by their organisation, according to embassies (in absolute numbers)

0

5

10

15

20

25

30

Yes No Don't know


The category ‘do not know’ is large, which is not surprising, given the fact that embassies

reported that they had only a very limited insight in the effects of NFP, but nevertheless it

appears that there are no strong suspicions of brain-drain. The question was extended to

the possibility of international brain-drain:

Figure 22 Do ex-participants stay within their country, according to Embassies. (in absolute numbers)

0

5

10

15

20

25

Yes No Don't know


Embassies see no problem resulting from brain-drain, therefore.

The same questions were put to the Southern institutes.


Figure 23 Retaining of participants by their organisation, according to respondent Southern institutes (in absolute

numbers)

0

1

2

3

4

5

6

7

8

Certainly Probably Probably not Certainly not Don't know


Figure 24 Do ex-participants stay within their country, according to respondent Southern institutes (in absolute numbers)

0

1

2

3

4

5

6

7

8

9

10

Certainly Probably Probably not Certainly not Don't know


Both embassies and institutes therefore do not foresee that massive brain-drain will limit

the effectiveness of the NFP.


2.3.6 Other remarks on effectiveness

The following box groups the remarks made by the respondent embassies on

effectiveness, which coul not be classified in any of the categories above. Note the

number of remarks on the final selection of candidates by the Dutch training institutes.

The selection criteria of the universities. The fellowship officer knows better the needs of the country

and the entity the candidates work for. - The use of an entity of the country that evaluate and approve

the candidate before the application is send to Nuffic has proven very useful here. It assures that all

candidates are really involved in development projects where the knowledge is required and used.

Recommendation part of the concerned RNE should be taken seriously as they are the one who really

can evaluate which training is suitable for which organization or applicant of their country of

representation. Institution especially newly added to the NFP list should be more flexible in informing the

Fellowship Officers of the awards and rejection as some are doing directly to the applicants of which

FOs are not aware.

NUFFIC should be more strict in publishing NFP booklets and not changing course subjects or dates or

adding institutes under NFP after the brochure has been distributed to all RNE and from there to local

authorities.

For tailor made courses FOs from all countries should be well informed of the progress of the outline

proposal. Institutes should not admit applicant for short courses of 1.5 weeks under one module and

advice them to apply for NFP as we FOs forward the application to NUFFIC who rejects since it is not

under NFP list. Institutes under NFP should update their websites and make the application forms

available in time so that applicants can apply in time and get the admission letter in time to meet the

deadline of NFP application.

Recommendations made by the embassy should be taken better into account by the universities, who

do the final selection. Also, universities should more seriously apply their initial selection (for the

application letters), because the embassy gets the impression that they try to hand out as many

applications as possible to secure a sufficient number of fellowships. ALL universities should inform the

embassy also the candidates selected.

As regards the Middle East, either eliminate the program or expand it considerable. If expanded,

appoint Fellowship Officers who can devote 100% of their time to the program. As matters stand now,

there is only so much we can do.

Don't let schools and universities select the fellowship holders!

There is a need for tailor made training, but people do not know how to write a proposal and therefore

let go of the opportunity.

Brief information in Portuguese would be helpful to introduce the full info brochure to the organisations.

This programme should be incorporated within the embassy as one of the bilateral programmes

because as is the case at the moment, nobody recognises what the programme is because the results

are almost invisible and cannot be measured.

Dutch universities should respect and take into account the objectives of the NFP as well as

recommendations made by Fellowship Officers.


2.4 Efficiency

2.4.1 Transparency of the procedures

RNEs were asked whether, according to them, the NFP procedures were sufficiently clear

t to the following parties:

• the embassy;

• candidates;

• employers of the candidates;

• consultant (responsible for outline of MYA).

Figure 25 Are NFP procedures clear, according to embassies (in absolute numbers)

0

5

10

15

20

25


embassy candidates employers consultant

Although the majority feels that procedures are sufficiently transparent, there is a distinct

minority which feels that this is not the case, especially for the candidates and their

employers.

The survey among the Southern institutes confirm the above: seven out of fourteen agree

with the statement that the guidelines for NFP TMT courses are transparent and

understandable, another five fully agree. Two are neutral on this subject.

Over 80 percent of the embassies feel that the division of tasks and responsibilities

between Nuffic and the Embassy are reasonably or well defined. Of the Southern

institutes, 72 percent (ten) are of the opinion that the tasks between Nuffic, the Dutch

training institute and themselves are reasonably or well defined, 21 percent (three) are

neutral on this topic. Of the same institutes, 86 percent are of the opinion that the

communication with Nuffic is clear and sufficient. The following box contains the

remarks from the side of the embassies on this topic.


Box 13 Comments from embassies on procedures and communication with Nufffic

The guidelines for fellowships officers at Embassies are very detailed and clear.

I've never had any problem with Nuffic. Whenever I have requested their assistance (for different

matters) it has always been provided quickly. I think that as far as a lot of Fellowship Officers are

concerned, more decisions should be made by Nuffic than presently is the case.

The provisional acceptance by the institute is still creating some confusion as participants like to think

they are accepted. The qualitative criteria for judging tailor-made proposals are not all that clear. The

NFP guidelines brochure would be more easy if structured according to subject area rather than

according to course modality.

There is no frequent contact between Nuffic and the embassy.

Have more communication with students

More collaboration with local partners emphasized.

2.4.2 Management by Nuffic

Embassies provided the following assessment of the quality of the management of NFP

by Nuffic.

Figure 26 Appreciation of quality Nuffic management by embassies (in absolute numbers)

0

5

10

15

20

25

30

Very poor Poor Neutral Good Very good


On a scale from one to ten (one lowest, ten highest appreciation) the embassies gave

Nuffic a score of 7.52 for managing the programme. The lowest mark given was ‘four’

(two respondents), the highest ‘ten’ (also two respondents), the modal score (thirteen

respondents) was ‘eight’. The Southern institutes were more positive, the average score

here being 8.50, the lowest ‘seven’ and the highest ‘ten’ (each with four respondents).

The overall score ‘sufficient to good’ could therefore be given. In the open remarks

however some criticism was given.


Box 14 Critical remarks on management by Nuffic

Tailor Made course processing should be faster to remain relevant for the applicant. At least a yes-or-no

decision need to be given within a reasonable time frame. Multi year agreement criteria need to be

communicated more effectively.

Improving the consultation and communication with the embassies. ex. when selecting the candidates

list of the multi year agreements

Nuffic should decentralise the programme in such a way that we have regional officers because in this

way we will be able to know the needs of different countries and structure the programmes/courses

accordingly. Also, if we have regional officers training in the region can be easily faciltated thus making

it cheaper (because it will reduce the travel and accommodation costs).

Nuffic should play a stronger role in the whole process of NFP implementation, i.e. to make sure the

objectives of the NFP are respected and realized by the Dutch universities. With regards to Tailor-made

training programmes, several times it took too long to give feedback on the proposals which were

already processed and sent earlier to them by the Embassy.

Tailor Made course processing at Nuffic need to be faster as the relevance of the programme is lost

after a very long delay in giving a decision.

Regarding tailor-made training, the time taken for processing within Nuffic is too long and should be

shortened. We have had cases of a tailor-made application taking 1 ½ year before we got an answer

from Nuffic.

The connections within the Embassy depend on who is in charge. One could consider how to uniformise

the services. Quality differs from person to person.

More flexibility and transparency required.

2.4.3 The Tender procedure

The following graphs show the appreciation of the Tender procedure for TMT as well as

the role of the Tender Evaluation Committee by the Southern Institutes. The appreciation

is clear.

Figure 27 Appreciation of Tender procedure for TMT (in absolute numbers)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Very negative Neutral Very positive

Procedure TEC


The following box contains some comments from the Southern institutes on the Tender

procedure.

Box 15 Comments on tender procedure

The process is open and allows the organisation that receive the training to participate.

The Process of tendering and evaluation all takes place in Netherlands. So one can assess how the

evaluation is done.

We were clearly informed on the purpose and procedure of the tenders and given adequate time and

support to ensure that the tender process gets done properly.

We were not availed of the other contestant for the training. Dutch institute was unilateraly seconded to

us to conduct the training. However with few reservations all went well and we are grateful.

As last time we were told to agree on the tender committees opinion and not invited to participate on the

proceedings we feel there is a limitation. In the future we look forward to be invited.

During the evaluation of the tender one might invite the contact person if it is to be transparent

The following question was finally asked to the southern institutes: ‘would you have

preferred more options in the choice of training institutes for a TMT?’

Table 12 Would the Institute have liked more options in selection training provider


Yes, more competition is better quality 6 50%

Yes, more competition is lower price 0 0%

Yes, more competition is better price / quality ratio 3 25%

Yes, more competition is better demand orientation among

suppliers

1 8%

No, this would harm relations with Dutch institute 1 8%

No, other reason 1 8%

35 100%

Although reasons differ, out of the twelve respondents ten would have preferred more

options. This is also related to another aspect which will be discussed below, i.e. the

effects of the tying of the NFP to Dutch providers.

2.4.4 Administrative burden

From the open question put to the embassies it appears that normally the management of

NFP within the embassies takes less than a FTE on annual base. The extreme low is one

week per year, four hours per month and two hours per week. Other embassies report

however 1.5 to two FTEs on an annual basis. This difference is most probably caused by

the presence in the country of MYAs and TMTs.

Out of the 33 respondents, 21 felt that this was sufficient. From those, who feel that this

was not sufficient, the following response was received.


Box 16 Comments by embassy staff on NFP administrative burden

First of all, the embassy does not recognise fellowships as very important part of development

cooperation, therefore they tend to dismiss the activities of NFP programme

2.4.5 Implications of tying the programme

Since the programme is tied to Dutch providers, the embassies were asked whether the

Dutch institutes active in the respondents’ countries have sufficient capacity to respond to

the demand in a cost-effective way.

Figure 28 Do Dutch Institutes have the capacity to respond efficiently to the demand (in absolute numbers)

0

5

10

15

20

25

In all cases In most cases No Don't know, n.r.


The following remarks were made on the above.

Box 17 Efficiency of providing services by Dutch providers

The embassies have not enough information on the Dutch institutions involved in NFP activities, i.e.

embassies cannot make judgements on the Dutch institutions whether they have sufficient capacity to

respond to the demand in a cost-effective and efficient way.

Yes, but regional providers can be good as well and traveling and staying in the region would require

less costs and hassle than going to the Netherlands for only a few weeks.

Study in the Netherlands is very expensive, especially when housing, food, insurance, tickets etc. are

included. Therefore, I do not consider Dutch institutions to be cost-effective.

Out of the respondent embassies, twenty (61 percent) feel that the total costs of the NFP

are mostly or always proportionate to its benefits. Twelve (36 percent) does not have a

judgment on this, only one respondent feels that this is in general problematic.


Box 18 Concluding remarks from embassies on efficiency of NFP

There should be more communication between fellowship officers and Dutch institutions with NUFFIC

as the co-ordinator so that the 3 parties are all aware of the existing problems in the NFP programmes

and can therefore work together to imporve the situation. A kind of on-line forum where people from the

3 parties who are involoved in the NFP programmes can voice their opinions might be of help.

There should be more communication between fellowship officers and Dutch institutions with NUFFIC

as the co-ordinator so that the 3 parties are all aware of the existing problems in the NFP programmes

and can therefore work together to imporve the situation. A kind of on-line forum where people from the

3 parties who are involoved in the NFP programmes can voice their opinions might be of help.

would still stress on Regional Representatives/Officers for proper needs assesment and quality training.

I have no insight in the costs involved. But I think it could be done cheaper if it would also be executed

in the region, e.g. linked to NPT program or for short courses by regional institutes.

I don`t have insight in the costs. I see the NFP as an important factor for development. But, I don`t see

the costs for the Netherlands. I think its more like an investment which many times will pay itself back in

goodwill towards de Netherlands. Fellowship holders are among the best and most talented people in

their countries and many of them are going to play an important role in their societies.

2.5 Improvement compared to previous fellowship programme

Here a comparison is made with the previous, pre-2002, programmes, in which no MYA

were available and tendering was not used as a method to select the supplier.

Of those respondent embassies who can judge, most consider the NFP an improvement

over the previous Fellowship programme: eleven out of sixteen agree, and three strongly

agree, two are neutral. Of the five Southern institutes who can compare, three fully and

one simply agrees with the statement that the new NFP is an improvement. Two

comments were singled out:

Box 19 Comments on improvement compared to previous programme

Actually it is only the procedure that changed (with the inclusion of the admission letter) and some

requirements (no photo, no medical) but the workload is the same.

The new programme has been drawn up after the Fellowship Workshop attended by all FOs, NUFFIC,

DCO and few of the Institutes. The problems were discussed and relevant solutions agreed upon by all

the involved. So it is much better and has improved a lot compared to past.


3 NPT

3.1 Process Description

3.1.1 Type of partners involved

The 74 respondent Southern Institutes who responded to the Questionnaire represent 58

different NPT projects. In total, about 1925 persons from these organisations have

directly participated in the NPT activities, mainly in training. The minimum number of

participants per respondent was one, the maximum 250.

The following table shows the legal status of the respondents:

Table 13 Type of Southern organisation (multiple responses possible)


Ministry 8 11%

National Agency 7 7%

Polytechnic 0 0%

Specialised Training Institution 12 15%

University 45 60%

Higher Vocational Education Institute 5 6%

Private Enterprise 0 0%

International Organisation 2 3%%

74

This tallies to a certain extent with the structure of the Dutch partners:

Table 14 Type of organisation of the Dutch lead partner


University 42 59%

Higher Vocational Education Institute 4 6%

International Education Institute 12 17%

Private Company / NGO 6 8%

Other 5 7%

Do not know 2 3%

71 100%


3.1.2 Year of participation and duration

The following table shows the year of participation.

Table 15 First year of participation in NPT


2002 3 4%

2003 3 4%

2004 21 30%

2005 18 39%

2006 16 23%

71

The duration of the projects (as envisaged) is shown in the next table:

Table 16 Envisaged duration of NPT projects


1 year 3 4%

2 years 4 6%

3 years 15 21%

4 years 44 62%

5 years 4 6%

Do not know 1 1%

71

The large majority of the NPT projects therefore appear to have an extended character,

i.e. some 90 percent are expected to last three years or longer, the modal group being four

years.

In addition, 36 percent of respondents mentioned that their institution participated in

several NPT projects: 25 respondent mentioned that their institute was on average

involved in more than three NPT projects, minimum two and maximum six.

3.1.3 Type of projects

The following pie chart shows the sectors to which the projects are supposed to provide a

contribution.


Figure 29 Sector to which NPT project is supposed to contribute

Informatics/Information

Systems 17%


management 23%

Natural Resources &

Environment 23%

Justice & Human Rights 7%Health/Medicine 6%

Gender 14%

Public administration 19%

Entrepreneurship 12%

Teacher training 39%

Agriculture 20%

Engineering/technology 16%

Quality assurance 16%

Social Economic Development

16%

Water & Sanitation 14%

Other, namely; 10%

NPT is, unlike NFP, not limited to training, also other instruments of capacity building

may be applied. It appears however from the following table that training is a major

component.

Table 17 Importance of training as project component


Training major component (>75%) 39 56%

Training represents about half of the project input (<75%, > 25%) 26 37%

Training a minor component of project (<25%) 5 7%

70

3.1.4 Involvement of embassies

The 13 embassies who replied to the questionnaire, when asked whether they were

involved in promotion of the NPT, answered as follows:

Table 18 Involvement of RNEs in promotion of NPT


Not involved 3 23%


Neutral 1 8%

Involved 6 46%


13


This is a different picture as in the case of NFP: under NPT over half of the RNEs is

involved or even intensively involved in promotion of the programme. The following

block contains some examples on how this promotion is implemented.

Box 20 Examples of promotion of NPT by Embassies

Close consultation with Ministry of Higher Education at relevant times in the year (e.g. with the

presentation of the Policy on Higher Education). Meetings with the Ministry of Vocational and Technical

Education during the World Bank appraisal mission for the second TVET project.

We were intensively involved in the selection phase (linking Nuffic with the government, identifying key

organisations / persons to invite for the stakeholder workshops), as well as in the monitoring. We are in

regular contact (phone, visits, meetings) with the institutes that receive NPT-support as well as with the

providers.

As I have a good knowledge of the programme I have introduced the progrmme to the Ministry of

Education, helped in the first plan of outlines and whenever necessary, bioth Nuffic, and the programme

partners in Ethiopia, as well as Dutch institutions can contact me for support.

RNE ENCOURAGES SECTORS IN WHICH WE ARE ENGAGED TO MAKE MAXIMUM USE OF THE

NPT FACILITY. THE FELLOWSHIP OFFICER HAS PUBLICISED AVAILABILITY OF NPT AT

DISTRICT LEVEL AND HAS SET UP A NETWORK OF CONTACTS PERSONS TO REACH THE END

USERS.

The RNEs appear also involved in demand identification, but are less involved in

assisting the Southern institutes in demand articulation, e.g. through support in making an

outline.

Figure 30 Involvement of RNEs in demand identification or articulation through assistance (in absolute numbers)

0

1

2

3

4

5

6

No

involvement

Intensive

Involvement

Demand Identification Demand articulation


RNEs report no involvement in the matching of demand and supply, i.e. the tendering

(two RNEs mention to be ‘hardly involved’; the others are reportedly ‘not involved’).

In order to check the involvement of the RNEs during the implementation phase, they

were asked about their involvement in trouble shooting and monitoring and evaluation.

Figure 31 Involvement of embassies in Trouble Shooting and M&E (in absolute numbers)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Fully disagree Disagree Neutral Agree Fully agree

Demand driven Flexible

Box 21 Examples of RNEs’ involvement during NPT implementation

Project managers meet embassy staff on missions to learn about recent developments; support and

advice is asked from embassy staff with regard to critical and difficult situations. However, this does not

apply to all NPT projects.

In some case the RNE has intervened in the problems (in coordination with Nuffic).

RNE has an advisory role when it comes to the selection of NPT projects, once a pre-selection has

been made by the Ministry of Planning and Investment (MPI). In case problems arise with regards to

Vietnamese authorities, RNE may play an intermediary role. Generally speaking though, RNE has a

very low profile in the NPT programme.

AT THE MOMENT WE ARE NOT, HOWEVER WE ARE AWARE OF A GAP IN THIS AREA.

Though the Embassy is not involved in actual implementation, it plays an active role in ensuring

coherency and consistency between the Embassy's programme and NPT (as much as possible).

NUFFIC and the Embassy have established a regular line of communication and NUFFIC calls upon the

Embassy if necessary.

Monitoring and evaluation are Nuffic tasks, but they have intensive dialogue with the embassy that

facilitates their M&E tasks.

We monitor progress by meeting with the institutes, the providers as well as the government. We read

the inception and progress reports and where needed, try to support the institutes with potential

problems they face in implementing the projects. In short: we try to facilitate the proces.

Sometimes we receive visitors from NL institutes and sometimes we are being sent progress reports.

During the annual Nuffic monitoring mission the Nuffic coordinator of the Embassy might join in parts.


Some embassies report other forms of involvement, e.g. participation in opening events,

providing feedback to NUFFIC, trying to facilitate coordination with other development

activities etc.

When asked whether the embassies had an insight in the (interim) results of the NPT

projects, different answers were received.

Table 19 Insight RNEs in results of the NPT programme


Very little insight 5 38%

Little insight 1 8%

Neutral 3 23%

Good insight 4 31%

Very good insight 0 0%

13

Seven of the thirteen respondents mentioned that they would appreciate to have more

insight in the (interim) results, the other six however answered this question with a clear

‘no’.

3.1.5 Initiating and outlining NPT

The RNEs see NUFFIC as the party taking the initiative for an NPT project. The role of

the embassy itself is also seen as important.

Table 20 Which party takes initiative for NPT, according to RNEs (multiple responses possible)


Local institutes 4 31%

Embassy 6 46%

Nuffic 9 69%

DGIS 1 8%

Dutch education institutes 3 23%

National Ministries / Agencies 3 23%

The Southern Institutes were asked in what manner they got involved in the programme.


Table 21 How got Southern Institutes involved


Approached by National Agency / Ministry 18 24%

Approached by local consultant 10 13%

Approached by national partner institute 1 1%

Approached by the RNE 20 28%

Approached by Dutch education institute 1 1%

Approached by NUFFIC 11 15%

Southern Institute contacted NUFFIC 5 7%

Southern Institute contacted National Agency / Ministry 2 3%

Do not know 3 3%

This seems to tally largely with the information provided by the RNEs.

3.2 Relevance of the programme

3.2.1 Compliance with national and Netherlands international cooperation development policies

The focal sectors for Dutch development assistance in the same thirteen countries are the

following:

Figure 32 Focal Sectors for Netherlands Development assistance

Agriculture 23%Civil society 31%

Education 69%

Entrepreneurship 15%

Good governance 62%

Gender 38%Health 54%

Justice & Human Rights

31%

Natural Resources &

Environment 46%


management 8%

Private sector development

62%

Water & Sanitation 54%

Other, namely 8%


For the above sectors, in eleven of the thirteen countries (in one case it was not known)

the national authorities had developed a national sector strategy or action plan. When

asked whether higher education played a role in this national strategic plan, the following

picture emerged:

Table 22 Does higher education play a role in the development strategies for the focal sectors

Number of respondents %

Yes 7 64%

No 4 35%

11

Although eight of the thirteen countries received budget support from the Netherlands,

only in four cases this support was extended to one or more of the sectors selected under

the NPT programme. There appears to be therefore no automatic or guaranteed coherence

between budget support and NPT. Asked whether the respondent embassies fear potential

tension between budget support and NPT in the same sector, it appeared that about half of

the respondents see potential tensions.

Table 23 Tensions between budget support and NTP, according to RNEs


Potential tensions 2 15%

Risk of potential tensions 2 15%

Neutral 4 31%

Co-existence with little tensions 1 8%

Co-existence without problems 4 31%

13

It should be remarked that nine of the thirteen respondents could not speak from own

direct experience. The following box provides some remarks by respondents.

Box 22 Relation between NPT and budget suport

NPT projects selected complement the bilateral agreements between the national department of

education and the Netherlands Government. For agriculture, there is no bilateral agreement but the

projects complement a national strategy for the Department of Agriculture. Co existence of the two is

very positive for institutional strengthening of the two sectors

NPT conflicts with the policy environment as currently experienced, since it is tied aid (linked to Dutch

providers) and the institutes are selected in close collaboration with Nuffic. We tried to overcome the

second bottleneck by giving a prominent place to the national government in the selection and

implementation, i.e. the President Office - Regional Administration and Local Government for

decentralisation, the Ministry of Industry and Trade for business environment and - of course - the

Ministry of Higher Education. [The country] is currently working on a Joint Assistant Strategy, which will

further shape the collaboration between development partners and the national government. In the JAS

it is clearly indicated that general budget support is the preferred mode of support, and if not, than

sector budget support, it specifies that no tied aid will be provided, and that a TA-pool will be created.

NPT projects run the risk of being less harmonised and aligned, thereby increasing transaction costs for

local partners


In addition to the above issue of alignment with budget support, the RNEs were also

asked whether they agree with the statements that:

• NPT is coherent with Dutch focal sectors

• NPT is harmonised with other Netherlands funded activities

Figure 33 Coherence of NPT with Netherlands development cooperation, according to RNEs (in absolute numbers)

0

1

2

3

4

5

6

7

Fully disagree Disagree Neutral Agree Fully agree

focal sect. coh. harmonised

The following box provides some open remarks on the above, especially on the limited

harmonisation.

Box 23 RNEs opinions on coherence and harmonisation

NPT does not necessary need to be coherent with focal sectors of an Embassy. Particularly when

considering the education sector, one needs to separate between developing the overall sector (this is

done through the bilateral programme) and building capacity across sectors - which could include the

education sector but not only (here NPT could play a role). The main challenge is to ensure that the

NPT provide enough flexibility to be truly demand-driven. Although NPT is rather well integrated and

responding to demands, the programme might have looked different if it was a bit more flexible in its

set-up.

It is coherent with Dutch sectors only to the extend of NPT projects with universities. We also have a

large number of NPTs with polytechnics taking place in various sectors like fashion, automobile

engineering etc. It is not much harmonised with other activities in the sense that it is tied aid and mostly

supply-driven. The same can be said for ORET export subsidies on infrastructure by the way.

The NPT programme is completely coherent with the Dutch focal sectors but there is some critic on how

the stakeholders and demand analysis is been performed (the RNE influenced considerable this

analysis). Conceptually the NPT activities are harmonised with other activities but in practise

coordination is still difficult. In some cases and in some themes there are difficulties in confirming the

additional value of the Dutch counterparts in the NPT projects

In ten out of the thirteen countries (77 percent) in which the respondent RNEs are located,

a national strategy for higher education had been developed and endorsed by the key

stakeholders.


Asked about consistency with national policies, RNEs responded as follows:

Table 24 Opinion of RNEs on consistency NPT with national development policies


Fully disagree (that NPT is consistent) 1 8%

Disagree 2 15%

Neutral 4 31%

Agree 5 38%

Fully agree 1 8%

13 100%

3.2.2 The demand driven character of NPT

RNEs, when asked whether the NPT is demand driven and flexible, replied as follows.

Figure 34 RNEs opinion on demand drivenness and flexibility (in absolute numbers)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Fully

disagree

Disagree Neutral Agree Fully agree

demand driven flexible

There appear to be therefore a relatively large number of respondents who have doubt on

the flexibility of the NPT.

According to the Southern institutes, the needs in the sector were mostly defined or

clearly defined at the time the organisation was considering to join the NPT.

Table 25 Were sector needs defined?


Not defined at all 3 4%

Only very limited 4 6%

Partly defined 22 32%

Clearly defined 40 58%

69 100%


The large majority of respondents (88 percent, nine percent disagreed) felt furthermore

that all relevant stakeholders were consulted in the need identification. Also the

appropriateness of the need identification was assessed by 91 percent as good.

Participation of the institute itself in the demand identification was classified by the

largest group as ‘highly involved’.

Table 26 Participation by Southern Institute in Demand Identification


Not involved at all 3 5%

Little involved 0 0%

Neutral 12 18%

Involved 14 21%

Highly involved 37 56%

66 100%

Asked whether the Institutes had sufficient time and information to formulate the project

outline, the following information was obtained.

Figure 35 Sufficiency of time and information for defining outline (in absolute numbers)

0

10

20

30

40

50

60

Yes, enough partially Insufficient

information time

Of 64 respondents, 57 (89 percent) felt that the institutes had sufficient possibilities to

express their own ideas about the needs. Another seven (eleven percent) felt that this was

only to a limited extent true.

Southern institutes were asked about the involvement of several actors in developing and

formulating the outline:

• the institute itself;

• Ministry of Education;

• Sector Ministries;

• Prime Minister’s or President’s office;


• NGOs;

• RNE;

• local consultant;

• Dutch partner institute;

• Nuffic.

Figure 36 Involvement in creating outlines (in absolute numbers)

0

10

20

30

40

50

60

Not at all Very limited Somewhat Involved Strongly inv.

Inst. Itself Min.Educ. Sect.Min. RNE Loc.Cons. Dutch inst. Nuffic

It appears that the institute itself has the strongest involvement, but also Nuffic and the

Dutch partner institute have in many cases a strong involvement. The importance of the

local consultant may be influenced by the simple fact whether he played a role at all:

where he did, he was seen as involved or even strongly involved, but in some (older)

NPTs there was no local consultant involved.

Involvement of national policy makers, either in the Ministry of education or a Sector

Ministry, appears limited although there are cases where the involvement is strong. The

following chart demonstrates that the southern institutes are generally satisfied with the

roles played by the different parties. One critical remark made by an institute stood out:

Box 24 Critical remark on establishment of outline

The Dutch institutions have their own priorities which may greatly differ from those of the southern

partners. So it would be unwise to involve in the outline formulation process. In most cases the Dutch

embassies may have their own interest and end up frustrating a good project by denying it approval; the

same thing applies to sector ministries. I think in the initial stages you need input from the Southern

partner, local Consultant if need be, and Nuffic. However, the project being formulated must be

supported by an independent Needs Assessment report


Figure 37 Satisfaction of Southern institutes with the different roles (in absolute numbers)

0

5

10

15

20

25

30

35

40

45

50

Not at all Very limited Somewhat Involved Strongly inv.

Inst. Itself Min.Educ. Sect.Min. RNE Loc.Cons. Dutch inst. Nuffic

General satisfaction among the institutes with demand identification and demand

articulation is distributed as shown in the next graph.

Figure 38 Appreciation of institutes of demand identification and articulation (in absolute numbers)

0

5

10

15

20

25

30

Very

negative

neutral very

positive

identification articulation

3.2.3 Potential duplication with other donor programmes

A significant part of the Southern Institutes, 39 percent, is also involved in similar

programmes from other donors (ten percent does not know). Mentioned were

programmes from GTZ, SIDA, Finida, DANIDA, NORAD, World Bank and France

(ARHES).

Asked about whether these programmes are similar, the following picture emerged:


Table 27 Similarity of other donor programmes according to Southern institutes


Very similar 3 9%

Somewhat similar 22 69%

Slightly or not similar 7 22%

The embassies were asked whether there a in the different countries similar programmes

as NTP.

Table 28 Existence of similar programmes according to RNEs


Yes 6 46%

No 4 31%

Do not know 3 23%

13 100%

Mention was made of Australia, ADB, USAID, GTZ , NORAD, France, Belgium (VLIR)

and DfID. The similarity however is again limited.

Table 29 Similarity of other donor programmes according to RNEs


Very similar 3 9%

Somewhat similar 22 69%

Slightly or not similar 7 22%

3.2.4 Role of local and regional training suppliers

The embassies, when asked whether there is regional or local capacity available which

may provide support comparable to that received under NPT, provided the following

answers.

Table 30 Availability of local / regional capacity according to RNEs


Certainly 5 38%

Probably 3 23%

Probably not 3 23%

Certainly not 1 8%

Do not know 1 8%

13 100%

Furthermore, embassies were asked whether according to them involvement of local or

regional capacity is desirable.


Table 31 Desirability of involving local expertise


In all cases 4 31%

In some cases 7 54%

Generally problematic 0 0%

Do not know 2 15%

Apparently, consortia managing the different NPT projects make use of local supply.

Embassies were asked whether this was done sufficiently, and the common opinion was

that this was not done or insufficiently.

Box 25 Opinions of RNEs on involvement local expertise

Not in all projects. There is a tendency to rely on expertise available from Dutch institutions

Of the projects we have, there is no regional involvement as yet

Dutch institutions could further improve the use of regionally available expertise

Only during the phase of consultation.

Some do, others don't but generally I have the feeling that the linkages are not well-explored.

NO, MOST DUTCH INSTITUTIONS RELY ON OWN EXPERTISE.

The Southern Institutes were asked the same questions.

Table 32 Availability of local / regional capacity according to Southern Institutes


Certainly 10 16%

Probably 19 30%

Probably not 17 27%

Certainly not 11 17%

Do not know 6 10%

63 100%

Table 33 – Desirability of involving local expertise according to Southern Institutes


In all cases 24 38%

In some cases 34 54%

Generally problematic 1 2%

Do not know 4 6%

63 100%

In addition, institutes were asked to what extent regional actors were used in NPT.


Table 34 Involvement of local / regional capacity in NPT projects


Not involved at all 18 29%

Little involvement 22 35%

Involved 20 32%

Strongly involved 3 5%

63 100%

Of these respondents, 71 percent felt that involvement of local or regional actors was

sufficient, the other 29 percent clearly not.

3.2.5 Relevance during implementation

Embassies were asked whether they considered the overall design of NPT appropriate to

achieve the aim to strengthen capacity in a sustainable manner.

Table 35 Appropriateness of NPT to realise capacity building, according to RNEs

Nmbr of respondents %

Not at all appropriate 0 0%

Not appropriate 3 23%

Neutral 3 23%

Appropriate 7 54%

Very appropriate 0 0%

13 100%

The remarks made by the RNEs on above mentioned issue related strongly to different

elements earlier discussed, i.e. coherence with Dutch or national development strategies

and involvement or local actors.

Box 26 Remarks by Embassies on appropriateness of design NPT

Methodology fits badly with SWAP approaches and the harmonisation/alignment agenda. Capacity

building too often perceived as organising workshops.

Linked to local institutions contributes to sustainability. To some extent the projects are not adequately

linked to policy reforms, with the exception of a specially designed project to support institutional reform

in the Ministry of Higher Education.

It is appropriate with to important comments: - there should exist more attention for the stakeholder and

demand analysis.

NEEDS TO BE MORE COGNIZANT OF EXISTING SECTOR POLICY AND IMPLEMENTATION

FRAMEWORKS.

With only one exception, in all NPT projects an inception phase had proceeded the actual

implementation. The importance of this phase was assessed by the institutes as follows:


Table 36 Importance of Inception Phase for success of project


Not at all important 3 5%

Of little importance 2 3%

Neutral 3 5%

Important 19 30%

Very important 36 57%

63 100%

Apparently, the Inception Phase gives rise to changes in the design of the project, but not

dramatically.

Table 37 Differences between Inception Report and project outline


No or minor differences 23 37%

Some differences 33 52%

Large differences 6 10%

Do not know 1 2%

63 100%

The following open remark is illustrative:

Box 27 Difference between proposal and Inception Report

Basically, the inception phase completes the proposal. Some additional activities were added. But this

addition does not significantly change the proposal. For example, during the inception phase it was

found that an English course is needed to make training more successful. To draw bigger attention to

society, we also run a national seminar to signify the cooperation between our institution and the Dutch

consortium.

Institutes were also asked whether they experience differences between the proposal of

the winning Consortium and actual implementation.

Table 38 Difference between proposal and implementation


Hardly any difference 27 42%

Little differences 19 30%

Neutral 12 19%

Clear differences 5 8%

Very large differences 1 2%

64 100%

Differences between proposal and implementation are therefore not considered

significant. One critical remark was noted:


Box 28 Remark on difference between proposal and implementation

In most cases the people indicated in the tender document are not available to provide the missions.

In most cases Dutch experts are interchanged, and at times the replacements are not as experienced

as those that were put in the tender document.

A final question related to the extent that the services provided reflect the needs as

indicated in the outline.

Table 39 Match between services and needs


Highly insufficient match 1 2%

Insufficient 4 6%

Sufficient 30 48%

Highly sufficient 28 44%

63 100%

3.2.6 Other remarks on relevance

Of the respondent in the RNEs (thirteen), only two had personally been involved in the

decision to embark on the NPT programme. However, nine would have made the same

positive decision at this moment (but four definitely not). One of the respondents replied:

Box 29 Reason for or against NPT

Yes and No: Yes: the programme provides opportunities for capacity building in post-secondary

education. Capacity is a great concern to all and no one quick fit strategy exists to address this. NPT

has managed to adapt well to national requirements and the projects are as much as possible aligned

with the Higher Education Sector Strategy. No: the NPT procedures with project formulation, tendering

etc are rather cumbersome. There should be ways to make the programme more flexible.

In the following box some general remarks made by RNEs on relevance are contained.

Box 30 Other remarks made by RNEs on relevance

NPT can be policy relevant, however, the method (projects using mostly inputs from Dutch institutions)

fit not very well with a SWAP and an agenda of donor harmonisation, alignment and untied aid.

The NPT-programme should be reshaped to fit policy contexts like in Tanzania and Zambia for that

matter. Perhaps the Netherlands government should promote Dutch institutes in these countries

(instead of imposing them) so that they can become strong competitors in TA-pools. Providing tied aid

can no longer be provided. On the other hand, for some countries, the current NPT-programme might

still be very relevant.

Seperate cash contributions from TA. NPT primarily focussing on TA, training etc. TA then needs to be

set up flexible - aimed at the highest possible quality - not necessarily linked to a particular institution.

NUFFIC to play a more active role in quality control, brookering expertise, rather than managing

projects.


Should be more complementary to the sector programmes of the Embassy and the responsibility for

implementation should be more coordinated in the country (by better involving the sector specialists and

the Embassy during implementation).

GREATER INTER-SECTORAL LINKAGES COULD BE PROMOTED.

3.3 Effectiveness of the NPT

3.3.1 The objectives of NPT

The following chart shows the objectives as rated by the respondent Southern institutes as

being most important. The following objectives could be selected:

• improved educational facilities;

• new (non-educational) equipment;

• new or adjusted organisational structure;

• new or renewed curriculum or work programme;

• improved management skills of staff;

• improved training skills among students;

• improved technical skills among students;

• improved subject knowledge;

• improved subject awareness among students.

Improved training skills among students (participants) and a new curriculum rate highest,

an improved organisational structure as lowest. It is interesting to note that ‘improved

technical skills’ and ‘improved subject knowledge’, both objectives which indeed may be

better served under NFP, score significantly lower. NPT is also not seen as an equipment

provider, i.e. ‘new equipment’ scores one but last (although ‘new facilities’ are still

considered by 27 respondents to be the most important objective). NPT is also not seen as

a pure management tool: ‘new organisational structure’ scores lowest, and ‘improved

management skills’ at the lower end of the medium range. Interesting is the fact that the

‘training the trainer’ concept is recognised by most respondents as a central element of

NPT.

In the category ‘others’ mention was made of the development of a research programme

(twice), development of a multi-cultural mentality (once), gender awareness (one

embassy) and improving the status of the Institute.


Figure 39 – Importance of objectives according to institutes (in absolute numbers)

0

5

10

15

20

25

30

35

40

45

Impr. org. structure New equipment impr. mgt skills

impr. subj. awareness Impr. facilities impr. subj. knowledge

impr. techn. skills new curriculum impr. training skills

This question was also raised to the RNEs. The following graph shows the answers.

Figure 40 Importance of objectives according to RNEs (in absolute numbers)

0

1

2

3

4

5

6

7




Grosso modo it appears that the RNEs share the expectations of the NPT with the

Southern Institutes. Only improved management skills are considered to be more

important by the RNEs, improved facilities as significantly less important.


Institutes were asked to give an opinion on the usefulness of training received under NPT

for the individual participants. Eleven out of the 61 respondent institutes declared

specifically that they had no information on this, or were not interested, the others

received this information through formal evaluation or through informal feedback.

Table 40 Usefulness of NPT training for individual participants


Not at all useful 0 0%

Limited 0 0%

Useful 6 12%

Very useful 44 88%

50 100%

3.3.2 Contribution to objectives

The following graph shows the extent to which, according to the institutes, NPT actually

contributes to these objectives. Since the ranking was the same as in the graph above, it

can be easily observed that:

• the contribution to those objectives which were rated as the most important is indeed

considered significant; and

• the contribution to objectives as rated as less important, i.e. improved subject

knowledge and awareness and improved technical skills, is rated as well high.

Figure 41 Contribution by NPT to objectives, according to institutes (in absolute numbers)

0

5

10

15

20

25

30





The embassies were asked to what extent the above objectives had been achieved till date.

For all categories the modal answer (nine to eleven of the thirteen respondents) is ‘partly

achieved’. Only for the objective ‘new non-educational equipment’ the modal answer is

‘hardly achieved’. This objective was however also rated as being less important by all

respondents (see Importance of objectives according to RNEs)

Both Embassies and RNEs were asked whether NP contributes to reducing quantitative

and qualitative shortage of professional staff in the different sector. The following graphs

show the answers.

Figure 42 Contribution to reducing shortage of staff, according to RNEs (in absolute numbers)

0

2

4

6

8

Very

weak

Weak

contr.

Neutral Strong

contr.

Very

strong

Quant. RNE Qual. RNE

Figure 43 Contribution to reducing shortage of staff, according to institutes (in absolute numbers)

0

10

20

30

40

Very

weak

Weak

contr.

Neutral Strong

contr.

Very

strong

Quant. Inst. Qual. Inst.

Institutes are therefore more positive, or optimistic, on the effectiveness of NPT than the

RNEs.

The following box contains the main hurdles for the realisation of objectives, as

mentioned by the respondents.


Box 31 Hurdles for realisation of objectives

Setting up projects objectives is done on a theoretical basis. Achieving them practically meets

challenges much of which may not have been foreseen. Secondly, realisation of the project objectives is

managed by different institutions and people, these different personalities may not have the same

capacity to attain the projected out put from the project.

motivation of the participants and the good cooperation between the Indonesian and Dutch teams

Communication with staff of some of the regional centres involved, which are diverse ion nature

Delay in fund release and procurement procedures.

The low funds available for the project activities.

If mission experts are changed in the process of the project implementation period

In our case, the institution is new and when the project was established there were many issues to

clarify on how it was to function. The NPT Project helped the institute in this because created condition

to go on in what was planned for this project.

Embassies were asked in which sectors the NPT is likely to have the largest versus the

smallest impact.

Table 41 Expected impact of NPT

Largest impact in sector Smallest impact in sector

Education, a/o Universities, Polytechnics, Teacher

Training (four responses)

Education (one response)

Private Business Sector Public administration (lack of commitment)

Justice, a/o criminality and security Gender

Environment, water Civil society

Agriculture, horti (flori-) culture HIV / AIDS (possibly)

3.3.3 Quality elements

With reference to the earlier question, concerning the involvement of the respondents in

projects comparable to NPT but financed by other donors (see section 3.2.3), the

institutes were also asked how they compare NPT to these other programmes.

Table 42 Comparison quality of NPT with other donors’ programmes


Much better 8 30%

Better 9 33%

Equally good 10 37%

Worse 0 0%

Much worse 0 0%

27 100%

The following box contains some praise, but also critics on NPT. The latter mainly

concerns lack of flexibility in adjustment of the budget.


Box 32 Explanation on comparison NPT with other programmes

It focuses on capacity building which is most important element in sustainable development and poverty

reduction. It gives freedom in targeting the crucial deficit in human resource.

More involvement of recipient organization, with much more emphasis on building of local capacity and

long term sustainability

The project and its objectives look very good on paper, but the assigned budgets are insufficient and

lead to a delay of actions. There should be a way to reshuffle the budget (three comparable responses)

The NPT is comprehensive and targets the capacity building of our own staff.

There is more involvement of the beneficiary institution(s) both at management and decision making.

The beneficiaries do own the project.

Both programmes focus on strengthening institutional capacity and have tailored their programmes to

the expressed needs of the institution.

There is good and direct co-operation between the Northern and Southern Partners. Partners are

responsible for their joint workplans and budgets.

A general appreciation of the quality of NPT, from the side of the Southern Institutes, is

favourable. Although it appears that training is more appreciated than the other support,

the difference is marginal.

Figure 44 Quality of NPT services according to institutes (in absolute numbers)

0

5

10

15

20

25

30

35

Very

negative

Neutral Very

positive

Quality of training Quality of other support

The following chart gives a break-down of the appreciation of the performance of the

different partners:

• Dutch lead institution;

• Dutch consortium partners;

• Local or regional Consortium partners;

• Nuffic;

• RNE;

• local consultants;

• government institutions.


Figure 45 Quality appreciation of different NPT actors by institutes (in absolute numbers)

0

5

10

15

20

25

30

35

40

45

Very

negative

Neutral Very

positive

Not appl.

Dutch lead inst. Dutch partners Local partners Nuffic

RNE local cons. gvt. inst.

Generally the appreciation is positive, and no significant differences may be observed for

any of the partners, although especially the Lead Partner and Nuffic receive much praise

(possibly apart from the high score on ‘not applicable’ for the Dutch consortium

partners).

The following box contains some of the open remarks.

Box 33 Remarks on roles of different actors by institutes

Local Partner not playing any active role!

Responses From the Ministry of Education is not fast as required

Actually, I have problem judging something that is not done yet such as the performance of Dutch lead

institution and Performance of the Dutch consortium partner(s).

I am very pleased of the progress so far, all parties give a lot of support, without which the project would

not run smoothly.

For reason not known to us, the Netherlands Embassy has not been directly involved. It seems to be

involved more with the NFP projects. As of now there are no regional institutions involved. This project

received very low backstopping support during conception and preparation. Time provided to prepare

the project outline was very short. NUFFIC provided useful support preparation of the project outline

and has been providing useful guidelines (e.g. financial) on running the project.

The implementation of the project is very much integrated within the operations of the Ministry so there

is full integration of system by government.

This project has had three Dutch programme managers. The current manager is performing very well.

3.3.4 Monitoring

This topic is dealt with here under effectiveness, assuming that a proper M&E mechanism

may facilitate the achievement of objectives.


The large majority of institutes, 51 out of 61, were positive (28) or very positive about the

monitoring of the project.

Appreciation by institutes and embassies on Nuffic’s organisation of the monitoring

function is hown in the graph below.

Figure 46 Nuffic’s role in monitorino (in percentages)

0

5

10

15

20

25

30

35

40

45

very poor poor neutral good very good don't

know

Emb. Inst.

Some open remarks are given in the box below.

Box 34 Some views on monitoring by the beneficiary institutes

a talk of one or two hours is really not sufficient

As a result of Nuffic monitoring we have been able to bring to attention concerns about lack of funding

for local staff to be involved in research and the failure at some stage of The Dutch Institute to provide

staff as had been planned. These shortfalls were corrected through the intervention of Nuffic

The monitoring missions are good but in some cases they come in when its too late. There is need to

set up a monitoring office within each country, this office could comprise of one person to monitor

projects over the year.

3.3.5 Sustainability

Both RNEs and institutes were asked about the likeliness of brain-drain, i.e. the chance

that participants of training will leave the organisation or even the country.


Figure 47 Are participants likely to stay in their organisation or their country, according to RNEs and institutes (in

percentages)

0

10

20

30

40

50

60

70

80

90

100

Yes No

RNE-organisation RNE-country Inst-organis Inst-country

Figures were converted to indices, to make data for embassies and institutes comparable.

A correction was made for a small category which did not know, but it appears that the

chance of brain-drain is not perceived as large by the different respondents. This is

somewhat surprising, since brain-drain is historically one of the main threats to

sustainability.

3.3.6 Other remarks on effectiveness

The following box contains some more remarks from the side of the embassies on ways

to improve the efficiency of the programme.

Box 35 Suggestions to improve effectiveness of the programme

NPT-programme should be seen in relation to comparable programmes financed by EC and World

Bank. Harmonisation and alignment are insufficiently considered in the framework of support to higher

education. Here to my opinion there is a potential for improvement. On the longer term, not the supply

from the side of the Netherlands will be the issue, but rather European Higher Education in general.

more involvement of local and regional actors (training institutes, consultancy firms) - better

understanding of the harmonisation/alignment agenda - critical reflection on methodologies for capacity

building (which is too often limited to workshops or study tours in the Netherlands)

Working more closely with the Ministry of Education to facilitate and support better linkages with basic

education, secondary education and higher, i.e. TVT education. In other words, there should be more

awareness of the importance of a sector wide approach, rather than a sub-sector wide approach.

Lesser prominence of NUFFIC, and more embedded in local structures (although difficult in the case of

Yemen).

See earlier: main challenge is to identify the areas that would require institutional capacity building

(together with the Ministry and Embassy) and formulate how this capacity could be organised: flexible

methods: bringing in capacity, longer-term TA, training workshops on the spot, cash transfers to eg

quality funds. Another important aspect is that it should be easier to change the direction during


implementation, bringing in new elements, stopping projects that are not really effective, providing

topping-ups, extensions.

Generally I think that NPT provides a complementary function to our bilateral programme that is highly

focussed on systematic interventions. The NPT provides some opportunities to be more creative,

addressing specific concerns with capacity and it should remain so. One should try to develop NPT

more flexible and have NUFFIC playing a stronger role in terms of quality control.

IN GENERAL NPT IS A QUICK REACTION INITIATIVE CAPABLE OF DELIVERING QUALITY

PRODUCTS IN TIMELY FASHION. THE INITIATIVE HOWEVER IS NOT IN LINE WITH THE BUDGET

SUPPORT MODALITY WHICH REQUIRES ALL ODA TO BE PLANNED FOR IN THE GOVERNMENT

MEDIUM EXPENDITURE FRAMEWORK. DISCUSSION ARE ON GOING TO CONSIDER HOW NPT

COULD BE ANCHORED WITHIN THE SWAP MODALITY.

3.4 Efficiency of the NPT

3.4.1 Price – quality relation

Of the thirteen respondent embassies, seven were of the opinion that generally the costs

of NPT the projects are proportionate to its benefits. Two however felt that this was not

the case (four had no opinion on this).

The institutes had the following opinion on the price / quality relation of NPT.


Very poor 0 0%

Poor 3 5%

Neutral 18 30%

Good 28 46%

Very good 12 20%

In the following the opinions of the different parties on the efficiency of NPT will be

elaborated upon.

3.4.2 Transparency of procedures

The opinion of RNEs on the transparency of NPT procedures for the different actors, i.e.

• RNE itself;

• local authorities;

• potential beneficiaries.

is shown in the following graph.


Figure 48 Transparency of procedures to different actors, according to RNEs (in absolute numbers)

0

1

2

3

4

5

6

7

8

not at all not clear neutral clear very clear

RNE loc.auth. pot.ben.

As to the division pf tasks between Nuffic and the embassies, the following table groups

the opinion of the RNEs as to the extent to which these are clear and well defined.

Table 43 Division tasks between Nuffic and embassy clear, according to RNEs


Fully disagree 1 8%

Disagree 3 23%

Neutral 0 0%

Agree 7 54%

Fully agree 2 15%

13

Although the majority therefore assesses the task description clear, there is some

discontent, as is alo apparent from the next box.

Box 36 Opinions RNEs on task division with Nuffic

Nuffic indicated several times to us that they didn't expect us to be as involved as we are. The embassy

sees NPT as part of its development cooperation and therefore wants to be fully involved. We had quite

some discussions with Nuffic before we could agree on a proper division of tasks.

To this embassy the NPT activities are not part of the programme. We don't decide about cooperation

and sometimes we even don't know what is going on. We don't have sufficient capacity for it!

I communicate wel with Nuffic, and Nuffic communicates well to me, this is important for the division of

tasks.

THIS IS A LARGELY AN AD HOC ARRANGEMENT AND THERE IS NO CLEAR DELINEATION OF

TASKS BETWEEN RNE AND NUFFIC.

Institutes were asked what they felt on:

• transparency of guidelines;

• division of tasks and responsibilities between Nuffic, consortium leader and own

institute;

• communication with Nuffic.


Figure 49 Opinions of institutes about transparency, division of tasks and communication (in absolute numbers)

0

5

10

15

20

25

30

very poor poor neutral good very good

transp. guidelines division tasks communication

Institutes were also asked whether they considered the division between Nuffic and

Embassy clear and efficient. Most institutes apparently had no opinion on this.

Figure 50 Is Task division Nuffic and Embassy clear and efficient, acc. to institutes (in absolute numbers)

0

5

10

15

20

25

30

35

Yes No Dont know

Clear Eff.

Several of the Dutch institutes interviewed mentioned that the type of contractual

arrangement between NUFFIC and the beneficiary created problems for them. The

respondents among the Southern institutes mentioned the following kind of contractual

arrangement.


Table 44 Type of contractual arrangement


Two-party between Southern Institute and NUFFIC 5 7%

Two-party between Dutch Institute and NUFFIC 11 16%

Three- party between Southern and Dutch Institute and NUFFIC 52 74%

Do not know 2 3%

70

3.4.3 Role of Nuffic

Both embassies and institutes assessed the input of Nuffic on a number of aspects.

Institutes commented on:

• communication on the implementation of NFP

• support to preparation of project outline

• support to implementing organisations

• general programme management

• budget control

• quality of information on Nuffic website

• accessibility of Nuffic staff

• expertise of Nuffic staff

Figure 51 Appreciation different aspects Nuffic’s work by institutes (in absolute numbers)

0

5

10

15

20

25

30

35

very poor poor neutral good very good don't know

communication prep.outline support to impl.org. progr. mgt

budget control Nuffic website accessibility expertise

On a scale of 1 – 10, (1 being the lowest, 10 the highest) the same institutes gave Nuffic a

score of 8.38 for the management of NPT. In comparison, the Dutch consortium received

a score of 8.16.

Embassies reported on the same, but instead of ‘support to preparation of outline’, an

assessment was given on Nuffic’s supervision of the local consultant.


Figure 52 Appreciation different aspects Nuffic’s work by embassies (in absolute numbers)

0

1

2

3

4

5

6

7

very poor poor neutral good very good don't know

communication supervision l.c. support to impl.org. progr. mgt

budget control Nuffic website accessibility expertise

On a scale of 1-10, the embassies gave Nufiic a score of 6.38 for the management of the

programme. Embassies therefore are more critical about Nuffic’s role than the beneficiary

institutes but also appear (considering the high score in the ‘do not know’ category) less

informed.

Nevertheless, 28 out of 63 respondents (44 percent) were of the opinion that more help

from the side of Nuffic would be useful.

Box 37 Areas in which more assistance from Nuffic would be appreciated

This is my first experience working with NUFFIC. I have to learn many new things, especially financial

matters. Nuffic financial rules seem very different from that of the Indonesian government. Honestly, I

often feel afraid of making mistakes in spending money from Nuffic. I know that I can hire someone who

can handle this. But it should be the one who understands English. And this is quite difficult.

not much choice between yes and no, if the Nuffic desk officer should spend more time in the country

and with the project, his remarks would have been better funded

Assistance regarding the training project coordinator, the accountants and in filing in the spreadsheets

of purchased items. for better project management.

especially in designing the project outline in line with the goals of the NUFFIC

Linking our institution to projects of a similar nature in the region (if any)

A visit once every 6 months, including attendance of a inter-institutional meeting would be valuable for

understanding

The following box provides some open remarks on ways to improve Nuffic’s role by the

embassies.


Box 38 Ways to improve management by Nuffic

The quality of the different programme managers is variable, which has to do with the demands that

Nuffic places on the professional expertise of her staff. It is internal human resource management, and I

do feel that Nuffic should demand more of her programme managers.

Better understanding of SWAP, harmonisation and alignment would facilitate comprehension of the

specific concerns of embassies.

At times more pro-active in terms of coordination between embassy and NUFFIC/NPT, and between the

various projects (learning from each other).

The set-up needs to be more flexible. NUFFIC should provide more quality control of the contribution of

the Netherlands Institutions. Focus needs to be on institutional capacity building. Systemic change in

the education sector should be left to be dealt with by the Embassy.

The next box contain similar remarks from the side of the Southern institutes, which at

times, considering the generally positive performance rating, are surprisingly critical.

Box 39 Ways to improve the programme

The Dutch organisation through its International office is handling too many projects. If they are to give

due regard to improve and work with the Southern Institutes, they either have to have more staff or they

apply for a number they can accommodate and give sufficient time.

Improve procurement.

Build skills in Monitoring and Evaluation of the projects.

May be cross cultural communication needs further strengthening.

The should show more respect to Southern partners and give them room to implement what they think it

is the best approach to address project activities.

There is the need to keep to agreed dates for missions. Frequent changes in dates disrupt our other

programmes

The use of external consultants from the region will help improve the quality of services

3.4.4 Availability of resources

Institutes were asked whether the available resources were sufficient for a proper project

implementation.

Table 45 Sufficiency of funds


Sufficient 34 56%

Partly 19 31%

Barely 3 5%

Insufficient 5 8%

61 100%

According to the institutes, generally less than half of the financial budget is available for

their institute: for 20 respondents (33 percent, the modal group) this is less than 20

percent, for 54 percent it is less than 40 percent. Nevertheless, the majority of institutes is

satisfied (36 percent) or even highly satisfied (18 percent) with the distribution. Eightteen


institutes (30 percent) are neutral on this, and only 16 percent dissatisfied. Properly the

latter provided the following responses.

Box 40 Some opinions on division of budget

High percentage of budget allocated to cover salaries and travel costs of Dutch partners - they're quite

expensive, and specialise in international projects. They therefore make adequate budgetary provision

for a good salary. In one case, the pressure was immense from Dutch partner side to allow one of their

project member to relocate to recipient county. With favourable exchange rate, living in an African

country and being paid in Euro could be very profitable.

The Dutch partner is taking too much money. Much of which is put on a vote of "Expert Missions".

Missions in the end are now seen as an opportunity to visit a foreign country.

The principle of Nuffic funding available for Dutch expertise & support is a bit limiting, as the logistical

and managerial costs of having an inter-institutional collaboration is relatively high.

The south partner is a mere observer.

The following graph shows the appreciation by the institutes of the availability of the

following resources:

• own local qualified staff;

• partner’s qualified staff;

• financial resources;

• educational resources at own institution;

• educational resources at partners’ institutions.

Figure 53 Satisfaction among institutes with availability of resources (in absolute numbers)

0

5

10

15

20

25

30

Not

satisfied at

all

Neutral Very

satisfied

local qual.staff partner’s qual.staff fin.res.

ed.res.own inst. ed.res.partners

3.4.5 Administrative burden

On average, two staff members in each embassy are engaged in the NPT, but the

workload normally represents between 0.025 and 0.1 FTE. The modal group spends two

to three days a month on the programme. Eight of the embassies consider this as

sufficient, two barely and three insufficient. The latter are however not of the opinion that


the RNEs should have more time to manage the programme, but that Nuffic should take

more work from their shoulders.

Institutes, asked whether the available time to implement the project was sufficient,

provided the following answers.

Table 46 Is sufficient time available to implement the project?


Not at all 2 3%

Not really 2 3%

Neutral 10 16%

Sufficient 28 46%

More than sufficient 19 31%

61 100%

3.4.6 The limitations of tying

Eleven of the thirteen embassies felt that in general Dutch institutions were capable to

respond to the demand in an effective and efficient way. One respondent however

disagreed. The following box contains the most striking open remarks.

Box 41 Open remarks on the issue of tying

The question is too broad. The Dutch institutions clearly have sufficient capacity, but the question

should rather be whether using that capacity is the most cost-effective and efficient way to solve

capacity issues in Zambian institutions.

Obviously, the tied character of the programme is not in line with the efforts of untying aid. However,

untying would only be acceptable to the programme if other international cooperation programmes in

this field would be untied. There is also the aspect of giving opportunities to Dutch Higher Education

institutions to intensify their international experience, which is good for the development of expertise in

Dutch institutions.

feedback from the visiting evaluator showed that in significant number of cases NL expertise is seen as

essential in comparison with alternatives.

Yes, but some are too far away from the programme (both in distance as well as in culture) to

sufficiently respond to real issues

3.4.7 The tender procedure

Satisfaction with the tender procedure, as well as with the functioning of the TEC among

institutes is distributed as follows:


Figure 54 Satisfaction among institutes with Tender procedure and TEC (in absolute numbers)

0

5

10

15

20

25

30

35

40

Very negative neutral very positive

Tender Proc. TEC

The high appreciation is obvious, which is consistent with the Institutes’ and Embassies’

appreciation of Nuffic’s management of the Tender procedure. Only the embassies appear

to be poorly involved (62 percent does not know).

Figure 55 Appreciation of role Nuffic in Tender procedure (in percentages)

0

10

20

30

40

50

60

70

very poor poor neutral good very good don't

know

Emb. Inst.

The following box contains some critical comments from Southern Institutes on the

tender procedure.

The following question was finally asked to the southern institutes: ‘would you have

preferred more options in the choice of training institutes for a TMT?’. The answer has

implications as well for the tied character of NPT.


Table 47 Would the Institute have liked more options in selection training provider


Yes, more competition is better quality 26 41%

Yes, more competition is lower price 1 2%

Yes, more competition is better price / quality ratio 18 29%

Yes, more competition is better demand orientation among

suppliers

6 10%

No, this would harm relations with Dutch institute 1 2%

No, other reason 11 17%

63

The following box contains some final comments on the tender procedure

Box 42 Comments on Tender Procedure

We were not involved in shortlisting of tendering institutes.

I think the tender only invites partners which are in Dutch. this limit the access of better partners to run

the project.

Procedures are okay, however, there is a need to consider at least three bidders before evaluation, that

is more time should be given to attract competition including re-advertisement.

The tender procedure works very well as it allows the evaluation team to be very objective as its

members initially work independently and later on compares and agree on the final scores.

The tender procedure is fine. However, a lot of points should be given to how the Northern partners

implemented a previous or another ongoing project. For example it is common for Northern partners to

put in the tender document very good CVs of Professors who will never be available to execute project

activities. This is very common. Unfortunately this is what in most cases Southern partners base on

when awarding points.

3.4.8 Suggestions from different parties on improvement of efficiency.

The embassies were asked whether they could identify any obstacles which may hamper

efficiency of the programme, to which eight replied.


Box 43 Possible obstacles to efficiency, according to embassies

Difficult to integrate effectively the NPT projects in the larger, more strategic policy dialogue, plans and

implementation. No adequate linkages with the Ministry of Education. lack of awareness and knowledge

of Dutch institutes (and consultants) on swap approach and harmonisation. Lack of involvement of staff

in overall strategic issues (again with the exception of the project that is specifically aimed to work at the

ministry of higher education on these issues).

The rather lengthy procedures of tendering and maybe also the costs of long term Technical assistants.

I feel that where long term experts would be appropriate, they are not placed in a programme because

of the high costs involved.

First two years of the NPT programme were needed to tailor the procedures and regulations to the local

situation. so the programme had a very late actual start - More flexibility needed for adjustment along

the way - Better focus on results, outputs and outcomes and make them more operational - Improve

measurability against well established baseline (Partly based on debriefing evaluator)

Costs related to the missions of Dutch institutions rather high and do not always benefit the outcome: is

this the most efficient way to use TA? - timetable is often linked to Dutch Institutions - not enough

flexibility to adapt to the speed of developments/annual calender in Mozambique - cumbersome tender

procedures caused delay in starting up of some activities

The following suggestions for improvement were given (which however might just as

well have been mentioned under effectiveness).


Preferably support to higher education should be through the SWAP with local partners having

ownership over procurement of TA. If however NPT-type approaches are continued, better

understanding of what harmonisation means is required. Focus should be more on local level

institutions (training institutes, universities, faculties) than central ministries.

Be aware of the limitations of the project approach with e.g. nice project offices amidst offices of local

staff which have much lesser facilities. Optimise the presence of the NPT coordinator in the country to

work closely with counterparts for capacity building purposes.

Of 61 respondents among the Southern institutes, 20 felt that there were still obstacles for

optimal efficiency.

Box 45 Obstacles for efficiency according to institutes

The agreement should be tripartite

Performance management aspects should be well detailed and southern organizations trained

adequately on these aspects.

The need to have a Dutch partner makes project management quite expensive. More could be saved

for other project activities if local (in this case Uganda) institutions could co-implement the project with

the beneficiary institution(s).

The NPT needs to look beyond 4 years. In four years you can only have a project but not a collaboration

with the Northern partners. For example PhD students take four years; so how would they be facilitated

to engage in a collaboration after training? In some cases the project needs more funds for

infrastructure. So NPT needs to have some of flexibility on the current 20% limit.

Budget transfers are slow. we are yet to receive our 6% after almost eight months of being in the

project.


The following box mentions some suggestions for improvement of efficiency, from the

side of the institutes.


Consider extension at mid term of implementation to ensure sustainability. Consider additional needs

arising during implementation, not considered during designing.

There is need to link projects in the region that are working on similar project. This will enable project to

learn from each other and the future sustainability of projects if the funding ceases.

Probably the support (especially when it comes to Ph.D. and masters training) should not be tied to

Dutch institutions alone.

One of our projects could not find a partner institution in the NL. it has delayed the implementation of the

project. It has necessitated also the modification of the objectives of the programme. In such cases,

there shall be an openness to look around outside the NL.

The managing organization like Nuffic need to set up a National Coordination and Monitoring Office in

each participating country that can over projects in that country. Funds for all projects in that country

can be transferred to one account under this body and the different projects can access funds on the

basis of need. NORAD has such a model in Uganda. For example if one project failed to take off then

funds would be reallocated to another project that is successful. But under the current arrange, each

project has a direct linkage with Nuffic. This would also allow Nuffic to get one Financial Report

(consolidated) from each country.

3.5 Improvement compared to previous programme

Only eleven of the respondents in the Southern institutes are familiar with the predecessor

of NPT (MHO). Out of these, seven consider NPT an improvement, five of which even a

large improvement. One does not agree on this, three have no opinion.

Box 47 Opinions of Institutes on difference with MHO

To the best of my knowledge the earlier version were focussed to an institution rather than to a sector.

In this way it likely to impact more on the development of the sector as a whole. There is also a desire to

involve the relevant government institutions in a more comprehensive manner

The project identification and elaborate inception phases render full clarity of the expressed need by the

Southern Partner and allows the institution to take full ownership of the project.

neutral because positive is the greater influence of the southern partner (demand driven) negative is the

lack of partership as against consultancy

The objectives of the Southern Partner are fully integrated into the project. Improved involvement of the

Southern Partner, particularly at the Tener Evaluation stage.

Of the embassies, nine respondents are familiar with MHO, out of which five consider

NPT to be an improvement, two feel that this is definitely not the case.


Box 48 Opinions of RNEs on difference with MHO

It is a different programme. The MHO focused on Institution building within universities in the South, the

new programme focuses on improving post secondary training capacity in a country. As the

programmes are different, one cannot say one is better than the other.

NPT is much more coherent with the embassy programme in terms of content, but fits badly with the

SWAP methodology. MHO programmes were insufficiently linked to embassy priorities, but were strong

in building long-term relationships with faculties.

The focus on one university, which was the MHO programme realised institutional change. Also it made

cooperation with several Dutch universities possible. There was more sustainability and there was a

longer period for the implementation.


4 Synergy

4.1 Synergy with NPT, seen from the angle of NFP beneficiaries

Only two of the respondent Southern institutes were familiar with NPT. From the answers

on the open questions it is apparent that there is little or no synergy observed, one

respondent reports rather confusion.

4.2 Synergy with NFP, seen from the angle of NPT beneficiaries

Of the respondent Southern Institutes twelve were familiar with NFP. Out of these, nine

considered the distinction between NPT and NFP clear, two unclear and two had no

opinion on this. The same reaction was received on the question as to whether the

distinction is efficient. It appeared from the open remarks that the institutes saw as the

main difference the fact that ‘NPT is targeted at institutions, NFP is targeted at

individuals’. In other words, the capacity building character of NFP is not recognised by

the institutes. One respondent felt strongly that the two programmes should be integrated.

The embassies were asked whether they saw a synergy with NFP.

Table 48 Synergy between NFP and NPT


None at all 2 15%

Hardly any 4 31%

Some 6 46%

Extensive 1 8%

13 100%

The following box contains the ideas of the embassies on ways (of any) to improve

synergy.


Box 49 Suggestions on improving synergy

Hardly possible given the difficulty to find suitable candidates for the NFP because of language issues,

academic background, cultural and gender issues). We are trying hard nevertheless!

The procedures to enter a MY programme with NFP are too complicated for organisations. A synergy

could only exist if NPT uses NFP inside their project. I would propose more synergy among the different

institutes as there are some cooperation between institutions within the embassy programme with the

NPT institutes.

That would only be possible if decisions on selection of candidates does not lie with the Dutch

institutions. Nobody has an influence on that. Another question is whether one would want this synergy.

Training components can be built into the NPT programme, so NFP would not be necessary to

complement NPT, unless we have a funding problem.

don't think it is necessary to have synergy as the NFP is much more geared towards individuals. The

tailor made programme, however, does provide opportunities to make the NPT more flexible and have

more tailor-made training in country.

Fellowships for the courses developed under NPT - More short NFP courses conducted (partly) in the

region by NL institutes with local educational institutes in order to enhance local capacity. This is now

being tried already by ITC and WI and the Embassy is very positive about it.

THERE NEEDS TO BE CENTRAL COHERENCE AND OPERATIONAL INTERFACE BETWEEN THE

TWO INITIATIVES.

an14851 questionnaire report final version...2007/03/15 · total 13 74 responses and 60 projects...

Documents