the episouth plus project - episouthnetwork.org · faculty of medicine, ... the episouth plus...

THE EPISOUTH PLUS PROJECT

MMiiddtteerrmm EEvvaalluuaattiioonn RReeppoorrtt Evaluation Period: 15 October 2010 – 15 September 2012

This report was prepared by a subcontractor as requested by the EpiSouth

Plus partnership and in accordance with a Terms of Reference (ToR).

Subcontractor: University of Thessaly, School of Health Sciences,

Faculty of Medicine,

Laboratory of Hygiene and Epidemiology,

Larissa, Greece

Evaluator:

Assoc. Prof. Christos Hadjichristodoulou,

SHIPSAN ACT Joint Action Coordinator

[email protected]

Evaluation assistant researcher:

Elina Kostara, MSc., Researcher

The EpiSouth Plus project is cofunded by the European Union DG-SANCO/EAHC and DEVCO/EuropeAid together with the participating national partner Institutions.

The financial support of the Italian Ministry of Health and ECDC is also acknowledged.

EpiSouth Plus Midterm Evaluation of 74

Contents

Executive Summary ......................................................................................................................... 4

1 Scope and objective ................................................................................................................ 7

22 MMeetthhooddoollooggyy ............................................................................................................................ 8

2.1 Evaluation Tools .......................................................................................................................... 8

2.2 Questionnaire Dissemination ..................................................................................................... 9

2.3 Questionnaire analysis.............................................................................................................. 10

2.4 SWOT analysis ........................................................................................................................... 10

33 EEvvaalluuaattiioonn RReessuullttss ................................................................................................................. 11

3.1 Response Rate .......................................................................................................................... 11

3.2 Evaluation of respect of scheduled milestones and deliverables according to the project Work

Packages (WPs) .................................................................................................................................. 12

3.2.1 Interview results ........................................................................................................................ 15

3.3 Evaluation of achievement of project indicators, both for quantitative and qualitative aspects16

3.4 Project Management Evaluation .............................................................................................. 18

3.4.1 Evaluation questionnaire analysis ............................................................................................. 18

3.4.2 Comments by the questionnaire responders ............................................................................. 19

3.4.3 Interview results ........................................................................................................................ 20

3.5 Dissemination Activities Evaluation .......................................................................................... 21

3.5.1 Previous Evaluation of the WP2 Website and Electronic Bulletin ............................................. 22


3.5.3 Comments by the questionnaire responders ............................................................................. 26

3.6 Training Activities Evaluation ................................................................................................... 28

3.6.1 Previous Evaluation Activities ................................................................................................... 28


3.6.3 Comments by the responders for WP4 training ........................................................................ 31

3.6.4 Comments by the responders for WP5 training ........................................................................ 31

3.7 Documents and deliverables produced Evaluation .................................................................. 32


3.8 Meetings/Working Groups Evaluation ..................................................................................... 36

3.8.1 Previous Evaluation of the 1st WP7 Steering Team meeting .................................................... 36


3.8.3 Comments by the responders: ................................................................................................... 37

3.9 Network Consolidation Evaluation ........................................................................................... 38

3.9.1 Previous evaluation report ........................................................................................................ 38

3.9.2 Evaluation of the EpiSouth Network ......................................................................................... 39

*p-value: 0.1 ................................................................................................................................................. 40

3.9.3 Evaluation of the EpiSouth Platform ......................................................................................... 40


3.9.4 Comments by the responders regarding different questions presented in Tables 15 and 16 ... 42

44 EEppiiSSoouutthh PPlluuss SSWWOOTT aannaallyyssiiss .............................................................................................. 44

55 SSuussttaaiinnaabbiilliittyy .......................................................................................................................... 46

66 AAddddeedd VVaalluuee ooff tthhee EEppiiSSoouutthh PPrroojjeecctt ................................................................................ 48

6.1 Added value of the collaboration between the EpiSouth Network and WHO in the framework of

the EpiSouth Plus WP7 ...................................................................................................................... 48

77 LLiimmiittaattiioonnss .............................................................................................................................. 50

88 CCoonncclluussiioonnss ............................................................................................................................. 51

Annexes ........................................................................................................................................... 54

Annex 1- Questionnaire 1 - Completed by the Advisory Board of the EpiSouth Network ............................... 54

Annex 2- Questionnaire 1 - Completed by all Focal Points of Institution partners and Countries involved in the

EpiSouth Network ............................................................................................................................................. 58

Annex 3- Questionnaire 1 – Completed by the Steering Committee of the EpiSouth Network ....................... 63

Annex 4- Questionnaire 1 - Completed by the WP4 Steering Team ................................................................ 66




Annex 8- Questionnaire 2 - Completed by all Focal Points of Institution partners and Countries involved in the

EpiSouth Network ............................................................................................................................................. 73

Annex 9- Questionnaire 3 – Interview Checklist ............................................................................................... 73

Table Contents

Table 1: Response rate of the evaluation questionnaires ...................................................................................... 11

Table 2: Deliverables evaluation ............................................................................................................................. 13

Table 3: Milestones Evaluation ............................................................................................................................... 14

Table 4: Indicators Evaluation ................................................................................................................................. 16

Table 5: Evaluation Questionnaire 1: Management and timeframe evaluation..................................................... 19

Table 6: Website statistics ...................................................................................................................................... 21

Table 7: Number of Presentation/ Participations to conference and Meetings by Work Package ........................ 22

Table 8: Evaluation Questionnaire 1: Website evaluation ...................................................................................... 23

Table 9: Evaluation Questionnaire 1: Quarterly EpiSouth Electronic Bulletin evaluation ...................................... 25

Table 10: Evaluation Questionnaire 1: EpiSouth Weekly Epi Bulletin (e-WEB) ...................................................... 26

Table 11: Evaluation Questionnaire 1: WP4 Training Activities .............................................................................. 29

Table 12: Evaluation Questionnaire 1: WP5Training Activities ............................................................................... 30

Table 13: Evaluation Questionnaire 1: Documents and deliverables produced ..................................................... 32

Table 14: Evaluation Questionnaire 1: Meetings and Workshops .......................................................................... 36

Table 15: Evaluation Questionnaire 2: EpiSouth Network ...................................................................................... 40

Table 16: Evaluation Questionnaire 2: EpiSouth Platform ...................................................................................... 41


Executive Summary

Objectives

This evaluation highlights the successful results, outcomes and critical aspects following the

implementation of the four specific objectives of the project, together with the assessment

of correct timing and adequacy of planned deliverables. The evaluation was conducted with

the participation of the Episouth Plus Network partners, the EU Institutions (EAHC, DG

SANCO, DG DEVCO, ECDC) and International Institutions such as WHO.

Methodology

The methods used regarding the evaluation of both process and outcome evaluations

included an analysis of the project’s documentation and an analysis and verification of the

project’s indicators. Moreover, a checklist for interviews and two questionnaires to collect

information on the project management, core activities and network consolidation were

prepared and disseminated to eight target groups. A descriptive analysis of the collected

questionnaires was conducted as well as a statistical analysis of the results related to the EU

and Non EU participant responses.

Results

A total of 10 out of 18 (55.5% response rate) interviews were conducted with members of

the Advisory Board and Focal Points and a total of 178 questionnaires (EU: 90/178, 51% and

Non EU: 88/178, 49%) were disseminated to 92 stakeholders. A total of 109 out of 178

(61%) questionnaires were collected. A lower response rate was recorded in Non EU

partners (36/88, 41%) in comparison with the EU countries (73/90, 81%) (p-value: <0.001). As

a result, the Non EU countries’ views and opinions are not equally represented. Seventy

three out of the 109 (68%) collected questionnaires were from EU countries, whereas 36 out

of 109 (33%) were from Non EU countries.

The evaluation of milestones and indicators has shown that minor and major delays occurred

in all work packages. The reason for the delays as identified by the responders includes,

amongst others, the political instability in certain countries and the long-term negotiations

between EpiSouth Plus and the European Commission (EAHC, DEVCO) regarding the Epidemic

Intelligent platform (WP6). The overall belief of the majority of the responders is that the

coordination structure of the project and the problem–solving process was effective in


resolving the delays, however, responders from the EU and International organisations

commented that the project would have benefited from a more proactive approach

regarding coordination activities. The Website, the Quarterly Electronic Bulletin and the

Weekly Epi Bulletin were rated highly by the majority of the responders and identified their

importance in the dissemination of results and in providing necessary information to the

visitor. More than 80% of the responders identified that the project has great potential and

that it has achieved reliable and collaborative relationships among public health

professionals. It is characterised as a valuable network that has fully facilitated the exchange

of alerts and health information (>80%, EU: 18/22, Non EU: 8/10), enhancing the

coordinated response to public health events in the Mediterranean Area (60%, EU: 81.8%,

Non-EU: 80%). Responders also identified that the network is not used to its full potential

and there are opportunities for improvement, especially in the capacity of the Network to

strengthen participants’ laboratory capacity in terms of diagnoses response and in using the

participant organisation’s strengths and expertise. However, it should be noted that this

facilitation will be better demonstrated following implementation of activities in the

following year. It is worth mentioning that some partners from the Mediterranean countries

reported as a shortcoming the limited access to European data, however, this is due to EU

legislation constraints. A fear of duplication of information shared through the Epidemic

Intelligence was expressed as a threat as well as uncertainty over the quality of data shared.

The lack of long-term concrete vision for the sustainability of the project was identified by

the key stakeholders (EU and International organisations) as a limitation and a threat to the

project, while the identification and dissemination of the added value of the project was

indicated as an immediate need.

Conclusions

The majority of the partners rated both the process and the outcome of the project as

satisfactory or excellent, while the key stakeholders from the EU and International

organisations raised serious issues both on the timeliness and the quality of the deliverables.

Differences of EU and Non-EU countries in the evaluation results should be considered

together with the low participation of the Non-EU countries in the evaluation process. We

believe that the coordination of the project should take into consideration all the

recommendations made to improve the quality and the timeliness of the deliverables.

Moreover, the added value of the network and the platform should be better defined and

disseminated to the key stakeholders (EU and International organisations) and the public.


Many ideas were put forward by all participants on the sustainability of the activities on

which the partnership has to elaborate further.


1 Scope and objective

This report is Milestone No 4 of the Work Package 3 – Evaluation and is the internal

evaluation of the project covering a 24-month period of the project implementation (15

October 2010- 15 September 2012). The evaluation was conducted with the participation of

the Episouth Plus Network partners, the EU institutions (EAHC, DG SANCO, DG DEVCO, ECDC)

and International Institutions such as WHO.

Objectives

This evaluation highlights the successful results, outcomes and critical aspects following the

implementation of the four specific objectives of the project, together with the assessment

of correct timing and adequacy of planned deliverables. The following aspects will be

described and discussed:

� respect of scheduled milestones and deliverables according to the project Work

Packages (WPs);

� achievement of project indicators, both for quantitative and qualitative aspects;

� active participation and networking of both associated and collaborating countries in

the project activities;

� measure the benefits / added value gained on health and security cooperation among

the associated and collaborating countries;

� relevant recommendations for sustainability to support the finalization of the

sustainability plan of the project and for possible further developments (with particular

reference to the framework of the EC DEVCO Instrument for Stability (IfS)/ Centre of

Excellence (CoE) Programme.

The report consists of the following main sections: In Section 1 the scope and objective of the

evaluation are outlined. Section 2 describes the methodology followed. Section 3 presents

the evaluation results. Section 4 presents the EpiSouth Plus SWOT Analysis. Section 5

outlines the sustainability options identified by the partnership. Section 6 presents the added

value of the EpiSouth project, Section 7 outlines the limitations of the study and Section 8

outlines the conclusions of the midterm evaluation.


22 MMeetthhooddoollooggyy To implement the evaluation strategy including all actions as described in the Terms of

Reference (ToR), evaluation tools were designed. The evaluation, both process and outcome

evaluation includes the following activities:

� Analysis of the project’s documentation

� Analysis and verification of the project’s indicators

� Preparation of tools (questionnaires and interview’s check list) for collecting

information form the institution partners of the Project and Focal Points of the

Countries involved in the EpiSouth Network aimed at evaluating project management

and core activities and network consolidation

� Collection of information from the institution partners of the Project and Points of the

countries involved in the EpiSouth Network

� Communications to be conducted through telephone calls and if there is a need, site

visits to be conducted to key stakeholders (EAHC, DG SANCO, ECDC etc). This will be

optional and only with the agreement of the project leader.

� Preparation of an evaluation report in draft

� Finalisation of the Mid-Term evaluation report

The evaluation process assessed the relativeness, effectiveness, efficiency, sustainability

utility, coherence, completeness, complementarily, coordination and additionally the

processes and outcomes.

2.1 Evaluation Tools

The tools used for the evaluation included the project’s available resources listed below and

the following:

� an evaluation questionnaire (1) for collecting information form the institution

partners of the Project and Focal Points of the Countries involved in the EpiSouth

Network aimed at evaluating project management and core activities (Annex 1, Annex

2, Annex 3, Annex 4, Annex 5, Annex 6, Annex 7)

� an evaluation questionnaire (2) to assess network consolidation (Annex 8)

� a checklist for interviews (Annex 9),


The available resources used were those specified in the ToR including :

i) Documents produced by the Project (Reports, Meeting Minutes, Presentations

and abstracts, Bulletins, Draft of Sustainability Plan etc.)

ii) Information on the use of the Members area of EpiSouth web site (content,

frequency of use, frequency of downloads, etc).

iii) The monitoring sheets filled every six months by each WP leader which were

integral part of the bi-annual reports to DG DEVCO and interim report to DG

SANCO (Four rounds WPs monitoring sheets are available)

iv) The report from October 2011 where the EpiSouth Project was evaluated by DG

DEVCO with a Result Oriented Monitoring (ROM) Mission

v) The report from April 2012 where the EpiSouth Project was evaluated by DG

DEVCO with an evaluation mission for the Instrument for Stability Programme

vi) Preparation of tools (questionnaires and interview’s check list) for collecting

information form the institution partners of the Project and Focal Points of the

Countries involved in the EpiSouth Network aimed at evaluating project

management and core activities

2.2 Questionnaire Dissemination

Questionnaire 1 was divided to eight different questionnaires in accordance to the target

group that was going to be disseminated. Hence each group was able to evaluate documents

and activities that were relevant to the group they belong to. The questionnaires were

disseminated to the following target groups:

� Advisory Board (18 members)

� Focal Points (75 members)

� Steering Committee (20 members)

� WP4 Steering Team (20 members)





� Head of Laboratories members of the Mediterranean Regional Laboratory Network

(MRLN) (23 members)

During the collection period, it was communicated by certain Focal Points that they would

respond collectively by sending one questionnaire. Hence the original list of the members of

the target groups was reviewed accordingly.

2.3 Questionnaire analysis

The questionnaires were entered into an Excel spread sheet and analysed using Epi Info

software. A descriptive analysis of the collected questionnaires is presented in the sections

below. The common variables of the 7 questionnaires were analysed together and

consolidated to produce one common result. Statistical analysis of the results was conducted

exploring differences on the response rate among the EU and Non EU participants using the

fisher exact or chi-square test.

2.4 SWOT analysis

The evaluation strategy used to implement all actions specified in the ToR was based on a

SWOT analysis. SWOT analysis is a strategic planning method used to evaluate the Strengths,

Weaknesses/Limitations, Opportunities, and Threats involved in a project. It involves

specifying the objectives of the project and identifying the internal and external factors that

are favourable and unfavourable to achieve the objectives. The SWOT analysis brings a

clearer common purpose and understanding of factors for success and provides linearity to

the decision making process allowing complex ideas to be presented systematically. The

SWOT analysis not only provides the basis for completing the evaluation tasks and to assess

core capacities and competencies but to also lead to an action plan that will identify the most

distinctive actions that must be addressed immediately, actions that need to be researched

further and actions to be planned for the future.


33 EEvvaalluuaattiioonn RReessuullttss In the following sections the results of the evaluation questionnaires are presented as well as

the results from the evaluation of all available resources provided by the project leadership

(documents, previous evaluations etc) as these are specified in the Methodology section.

3.1 Response Rate

A total of 18 key stakeholders (Advisory Board and Focal Points) were invited to give an

interview on the EpiSouth management and timeframe, the strengths/

weaknesses/opportunities/threats and on the sustainability of the project. A total of 10

interviews were conducted.

A total of 178 questionnaires (EU: 90/178, 51% and Non EU: 88/178, 49%) were

disseminated to 92 stakeholders. The questionnaire collection period was from 5/11/12 until

15/12/12. A total of 109 out of 178 (61%) questionnaires were collected (EU: 73/90, 81%

and Non EU: 36/88, 41%) (p-value: <0.001). Seventy four out of the 109 (68%)

questionnaires collected were from EU countries whereas 36 out of 109 (33%) were from

Non EU countries. In Table 1, the number of questionnaires disseminated (total, EU and Non

EU) as well as the number of questionnaires collected and the response rate by target group

are presented.

Table 1: Response rate of the evaluation questionnaires

TOTAL EU countries Non EU countries

Target

Group Disseminated Collected

Response

Rate Disseminated Collected

Response

Rate Disseminated Collected

Response

Rate

Advisory

Board 17 8 47% 12 71% 5 42% 5 29% 3 60%

Focal

Points -

Quest. 1 47 28 60% 22 47% 18 82% 25 53% 10 40%

Focal

Points -

Quest. 2 50 32 64% 24 48% 22 92% 26 52% 10 38%

Steering

Committee 8 7 88% 5 63% 6 120% 3 38% 1 33%

WP4 ST 24 19 79% 11 46% 9 82% 13 54% 10 77%

WP5 ST 5 3 60% 4 80% 3 75% 1 20% 0 0%

WP6 ST 15 6 40% 5 33% 5 100% 10 67% 1 10%

WP7 ST 12 6 50% 7 58% 5 71% 5 42% 1 20%

Total 178 109 61% 90 51% 73 81%* 88 49% 36 41%*

*p-value: <0.001


3.2 Evaluation of respect of scheduled milestones and deliverables according

to the project Work Packages (WPs)

The EpiSouth Plus partnership, in accordance with the Technical Annex and the annual work

plans, had to produce a total of 10 Deliverables (D) out of which five have been delivered, the

other four are currently in the process of being prepared and one has not started. Table 2

presents the list of deliverables and their status and Table 3 presents the evaluation of the

timely achievement of milestones.

Delays were noted in all WPs. Internal as well as external reasons such as the political

instability in certain countries of the partnership influenced the implementation of the

project. In WP1 the deliverables were produced within an acceptable timeframe but delays

were noted in the organisation of the 1st

Project Meeting that was postponed to a later date

due to the freezing of WP6. Regarding the activities of WP2 no major delays were noted

except in the finalisation of the website. The activities of the WP3 (Evaluation) and especially

the Midterm evaluation was also delayed since it was going to be presented in the 1st

Project

meeting and that meeting was postponed to M26. The Evaluation Plan although it was timely

delivered, in terms of its completeness is should be noted that it was mainly devoted in

listing the time schedule. The delays noted in WP4 were minor and mainly due to the political

instability causing a low response from the partners and due to legal constrains that had to

be taken into consideration regarding the training in the Institute Pasteur in France.

However, the training in the Institute of Pasteur was conducted with only one month of

delay. Activities of WP5 were also delayed but again this was due to the low participation by

members of the steering team due to the geopolitical situation and to the freezing of the

collaboration agreement with the work package co-leader. The decision to freeze WP6

activities that were related to the Epis based secured platform from July 2011 to January

2012 has influenced the timely preparation of milestones and deliverables. Regarding WP7,

the political instability has also delayed the implementation of activities. Moreover, regarding

WP7 activities it should be noted that the methodology and strategy for WP7 has been

revised. This has been done with a stepwise approach started with request of agreement to

the Advisory Board during the Meeting in Brussels in May 2012, sharing of needs at the

WHO-Lyon Meeting in July 2012 and consolidated in the 1st Project Meeting in Rome with

the WP7 steering team. A detailed explanation of the reasons for the delays is outlined in the

Interim Technical report (D1). It should be noted that the project leadership has requested an


amendment in order to reschedule certain activities and to extend at no extra cost the

contract for another 8 months.

Table 2: Deliverables evaluation

Deliverable

Number

Deliverable Title Description As

planned

Status

Until Month (M) 24

1. Interim and final technical

implementation reports,

including financial reports

(WP1)

The Inception (M3) Interim (M17) and

final(M33) technical and financial reports

will be prepared in accordance with the

grant's requirements

M3 –

Inception

M17 –

Interim

Delivered M4

Delivered M17

2. Reports of the Project

Meetings (WP1)

Reports will be done for the AB, SC and

Project Meetings (M4, M17, M30)

M4, M17,

M30

Delivered M5

3. EpiSouth quarterly

electronic Bulletin (WP2)

A quarterly bulletin will be produced and

disseminated with project's news and

focus topics

Quarterly 1st Issue delivered in

M 9 (June 2011)

6 Issues prepared until

September 2012

4. Final Evaluation Report

(WP3)

The final evaluation will rely on indicators

verification, on-line questionnaires and

interviews. The results will be reported in

the final report.

M33 Not started

5. Laboratory Training

packages and

Recommendation for

training and capacity

building program (WP4)

Training packages will be developed on

the basis of the lab needs assessment

(M20) and Recommendation for training

and capacity building program will be

developed on the basis of all the inputs

and lessons learned coming from the

implementation (M29)

M20

M29

In progress

1 training package

completed on Dengue

and Biosafety in the

Laboratory

6. Directory of Regional

Laboratories involved in

surveillance/early

warning system in South

Europe and

Mediterranean area

(WP4)

A Directory of Mediterranean

laboratories will be created in

accordance with needs of geographic and

pathogen coverage

M29 M24

7. Preparedness plans & Risk

management capacity

building materials for

Workshops and Sessions

and Simulation Exercise.

(WP5)

Packages will be developed for all the

capacity building activities planned (M9,

M12, M21, M24, M28)

M9, M12,

M21,

M24, M28

In progress :

One Workshop and

training package

completed on cluster

analysis

Simulation exercise in

progress

8. Strategic document on

Capacity building

development regarding

Preparedness plans & Risk

management procedures

(WP5)

A detailed report will be prepared

describing the strategic approaches to be

utilized (needs assessment, multi-country

workshops and cross border exercises

and related teaching materials)

M29 In progress

9. EPIS based secured

platform interop. with EU

& other EWSs(subject to

ECDC delivery & countries

agreement)& Epi Bulletins

(WP6)

Epi Weekly Bulletin (e-WEB) since the

project's starting (M1) and launching of

the EPIS based secured platform interop.

with EU & other EWSs (subject to ECDC

delivery & countries agreement)

M1 Delivered M1- A total

of 101 E-web bulletins

published from

October 2010-

September 2012.

Delays in launching of

EPIS platform due to

freezing

10. Strategic document on

national/international

issues which may

interact/ interfere with

IHR implementation

(WP7)

Strategic document including guidelines

will be written on the basis of the project

surveys and assessments results.

M29 In progress


Table 3: Milestones Evaluation

Work

Package

Milestones Month Actual Comments

WP1 1st Steering Committee in Luxembourg M3 M4

Sustainability plan M 4 M6

Annual Project Work Plans M4 & M16 M4 & M 16 M5 & M19

2nd Steering Committee & 1st Project Meeting in

Paris

M 16 M24

2nd Project Meeting (with SC) & Final Conference in

Rome

M 29 N/A

WP2 Dissemination plan M 4 M4

Promotional leaflet and outline M 4 M4

EpiSouth-plus website updating M 6 - Continuous – Delays were noted

on the delivering the final

version of the website

Presentations at conferences M 12 M12 A total of 27 presentations See

Section 3.5 - Table 7 for a

detailed list

WP3 Evaluation plan M 4 M4

WPs activities evaluation questionnaires M 14 M24 See Annex 1-7

Networking Evaluation Questionnaires M 15 M24 See Annex 8

Mid-term Evaluation Report M 18 M26 Delays due to delays of 1st

Project Meeting

Presentation of the Mid-term Evaluation results to

the Review Meeting with EC Experts Committees

M 19 M26 Delay in organizing the Project

Meeting

WP4 Meeting of expert committee to set minimum

requirements needed

M 7 M15

Mapping of existing biological expertise M 8 M13-M14

Meeting with heads of Laboratories interested in the

approach and corresponding to the minimum

requirements

M 13 M17

Identification of needs of laboratories for diagnoses

for priority diseases in the region: human, technical

and organizational

M 15 M21

Delays in collecting the

questionnaire due to political

instability

One-week Lab training at IP Paris M 21 M22 A total of 17 out of the 21

selected laboratories

participated

WP5 Report on in-depth core-capacity needs assessment M 8 M21

Workshop 1 and Capacity building Session 1 in

Madrid for 30 people

M 12 M16

Workshop 2 and Capacity building Session 2 in

Madrid for 30 people

M 24 N/A

2 one week training in Algeria (M9 and M22) M 22 Not

completed

Due to internal country level

obstacles both NFP dismissed.

Updated of the Directory of Training and Fellowships M 30 N/A

WP6 Management of the Mediterranean Cross-border

secured platform

M 1 M13-M18

Delays due to WP freezing

Review of functional specification for EPIS M 6 M13-M24

2 One-week stage at InVS of non-EU participant

(M10 - M 22)

M 22 Not

completed

Testing phase for EW Platform in EPIS and

assessment process (M11-M16)

M 16 M13-M24

Launching of EpiSouth/EPIS (if consensus met) after

1st Project Meeting

M 20 N/A

WP7 On-line questionnaire for data collection on

countries needs and constraints in enhancing their

surveillance systems

M 10 N/A New approach (See Interim

Report for further information)

Results of on-line questionnaire and preliminary

report

M 15 M20

On-line questionnaire on national procedural and

legislative aspects relevant to IHR implementation



Results of on-line questionnaire and preliminary

report



Preliminary strategic document report M 27 N/A


3.2.1 Interview results

In the question if there were there any delays in the project plan of the EpiSouth Plus

project the interviewees responded the following:

� Five interviewees responded that the general meeting should have been organised much

earlier and closer to the middle of the project.

� Four interviewees identified the freezing of WP6 as the most important delay in the

project.

� Three interviewees identified that delays occurred in WP7 and commended that now the

scope has changed.

� Two interviewees identified that WP5 delayed due to lack of agreement with the co-

leader.

� Two interviewees responded that there were no major delays.

� One interviewee responded that the training activities were delayed for 3-6 months

� One interviewee responded that the delays are not the trend in EpiSouth Plus project

though could happen sometimes and basically the cause is dispute on financial affairs.

� Due to limited communication only through the website one responder was uncertain

what delays occurred and for what reason.

In the question on what are the reasons that resulted in delays in the project’s work plan

the interviewees responded the following:

� Financial funding.

� Arab Spring.

� Delays in epidemic intelligence work occurred because of as fear of duplication with ECDC

and the Electronic alert bulletin stopped issuing for three or six months and the different

views and vision between DG SANCO and EPISOUTH. Certain important aspects leading to

duplications were not identified at the beginning of the project.

� The delay in the training activities occurred because of the subject chosen, West Nile

Virus. The training was going to take place in France. However, due to national legislation

issues they could not conduct the training hence it will now be completed in Italy.

� Lack of discussion between the teams sometimes.


� Lack of flexibility.

In the question on whether according to their opinion the issues were resolved within an

acceptable timeframe the interviewees responded the following:

� Five interviewees believed that the delays were not resolved within an acceptable

timeframe

� Not an acceptable timeframe but it was due to the decision of DG SANCO and ECDC (2

interviewees).

� The partnership had a lot of adjustments to make and there are still delays due to the

freezing of WP6 (2 interviewees).

3.3 Evaluation of achievement of project indicators, both for quantitative

and qualitative aspects

In order to achieve the projects objectives a set of process, output and outcome indicators

have been established to measure the level and quality of the achievement.

The indicators were revised by the subcontractor in collaboration with WP1 and WP3 Work

Package Leaders. The revised indicators were included in the amendment requested by the

project leadership and are summarised in Table 4.

Table 4: Indicators Evaluation

INDICATORS Level of Achievement

Specific Objective 1: Establishment of a Mediterranean Regional Laboratories

network (WP4)

Process indicators

Number of participants attending the Meeting organised by the Pasteur Institute for

Laboratories Representatives out of the total number of those selected

Completed - 12 participants out of

18 MRLN analysis team

The list of the laboratories from the EUROMED region eligible to participate in the

network is developed

Completed

Capacities assessment of the laboratories participating in the network is completed Completed

Training material is developed on time Completed (M22)

Output indicators

Number of trainees attending the training courses (>=60%) and the number of

participants attending the meetings organised by the Pasteur Institute (>=60%)

Nineteen laboratories (over the 21

of the MRLN) have proposed a

candidate for this training, but only

17 could participate (7 EU and 10

non-EU) = 80.95%

12 participants out of 18 MRLN

analysis team (66.6%)

>80% satisfied trainees from the training courses

Trainees were very satisfied of the

training and 94% have declared that

the training met their expectations

(16/17)

Number of Laboratories which accept to participate in the Network out of the total

number of those whose standards were adequate to the project’s objectives (50%)

30 laboratories from 21 countries

responded out of 27. A total number

of 26 Human labs and 3 Veterinary

Labs identified. 20 human labs and 3


Veterinary Labs identified fulfilled

the selection criteria (76.6%).

Outcome indicators

Reference procedures available in the recommendation for training and capacity

building program and considered by each country for the collaboration between PH

Institutions and laboratories of the EpiSouth Network

N/A – to be evaluated in the final

evaluation of the project

Laboratory needs in the Mediterranean Area identified and shared with stakeholders

and European and international Institutions

Completed

Number of laboratory exchange of information carried out between the network of

laboratories from EU MS and Non EU countries and include the number of laboratory

generated EWRS/IHR notifications of cross border events



Specific Objective 2: Promotion of common procedures in interoperable Generic

Preparedness and Risk management among the countries involved in the Network

(WP5)

Process indicators

In-depth capacity needs assessment carried out, based on information collected from

a sample of participating countries

Completed.

21 out of 27 (77.7%) countries

participated in the assessment.

Two Workshops carried out One workshop (Month 16)

2nd

scheduled for M24

Training material and simulation exercise is prepared Training material is prepared

Simulation exercise in progress

Training courses are organised One training course completed

Output indicators

At least 70% of EpiSouth countries participate in the simulation exercise N/A – to be evaluated in the final


At least 70% satisfied participants in the simulation exercise N/A – to be evaluated in the final


At least 70% of EpiSouth countries attending each Workshop out of the 27 countries

in the Network

47 participants

22 out of 27 ( 81.5%) countries -

institutions

At least 70% satisfied participants from the workshop 7/9 (77.8%) responders rated the

course content as excellent

Outcome indicators

At least 50% of EpiSouth countries consider the usage of the Strategic document on

Capacity building development regarding Preparedness plans & Risk management

procedures



Increase of collaboration between partners from EU and Non EU countries for the

detection of threats and risk management during the last 6 months of the project



Specific Objective 3: Enhance Mediterranean Early Warning Systems (EWS) allowing

alerts and Epidemic intelligence information sharing among EpiSouth countries and

developing interoperability with other European Early Warning platforms (WP6)

Process indicators

Epis based cross-border secured platform developed Completed

Reference procedures available and endorsed by countries for information sharing

within EpiSouth secured platform and among it and other EW systems

Completed (October 2011)

Development of eweb bulletins (EpiSouth Weekly Epidemic Intelligence) Completed

Output indicators

>70% satisfied with the platform’s functionality (accessibility, user-friendly,

timeliness)

11/23 (47.8%) responders rated the

functionality of the platform as

Excellent whereas 11/23 (47.8%)

rated it as satisfactorily (See Section

3.9)

Number of eweb bulletins (EpiSouth Weekly Epidemic Intelligence) 101 E-web bulletins published from

October 2010-September 2012

Number of registration in the e-web bulletin mailing list 486 members are registered in the e-

web list (until 28th

November 2012) Outcome indicators

Increase the number of alerts published and their distribution among the Partners

during the last 6 months of the project



Increase the number of episodes of collaboration within the network between EU and

non-EU countries and between the Network and other EW platforms during the last 6

months of the project



Willingness of countries to continue the usage of the platform and appreciation of the

platform’s usefulness and need measured through core packages evaluation




questionnaires

Specific Objective 4: Production of a strategic document, with guidelines based on

assessments and surveys, aimed at facilitating IHR implementation (WP7)

Process indicators

Workshop conducted Completed - 2 workshops in Rome

and Lyon

Literature review report on IHR implementation on the EpiSouth Region is prepared

and areas of priority are identified

Completed

A plan/protocol for the pilot implementation of the guidelines on coordination of

surveillance and response between points of entry and national surveillance systems

is developed



Output indicators

Number of participants in the workshop Meeting in Lyon: 13/14 participants

Meeting in Rome: 18/18 participants

>70% satisfied with the pilot implementation of the guidelines



Final report of in depth analysis of priority area (literature review and analysis of

existing monitoring frameworks)

Completed

>70% satisfied with the pilot implementation report N/A – to be evaluated in the final


Outcome indicators

Number of countries that consider to adapt the WHO Guidance in their countries on

the basis of the pilot results >=50% measured through core packages evaluation

questionnaires



Number of EpiSouth countries which have been helped by the EpiSouth WP7

documentation in the identification of priority areas and the IHR implementation (

>=50% measured through core packages evaluation questionnaires)

In the Midterm Evaluation

Questionnaire the “Usefulness” of

the literature review was rated as

Excellent by 7 out of 14 participants

(50%) and 6 out of 14 (42.9%) rated

its usefulness as satisfactory.

3.4 Project Management Evaluation

3.4.1 Evaluation questionnaire analysis

As shown in Table 5, the majority of the responders (>79%) rated as fully effective the project

management and the steering committee as for achieving the project objectives. About 63%

(24/38) believed that the Steering Teams fully assisted in the achievement of objectives and

facilitate the collaboration whereas 37% (14/38) believed that this was only partially

achieved. Almost 68% (26/38) rated as fully effective the communication means and

frequency whereas almost 32% (12/38) rated as partially effective the communication means

and frequency. More than 55% of the responders rated as fully effective the problem –

solving process and the monitoring and evaluation of activities in achieving the projects

objectives and in keeping on track in terms of quality and timeliness.

There were no significant differences between the responses from the EU and Non EU

countries except for the effectiveness of the problem solving process. The Non EU partners

that rated the problem solving process as partially effective were 10 out of 14 (71.4%)


whereas a significant lower percentage of the EU responders (6/24, 25%) rated as partially

effective the problem-solving process (p-value: 0.028).

Table 5: Evaluation Questionnaire 1: Management and timeframe evaluation

Fully Partially Not at all

1. Do you think that the management

(planning and organising) of the project is

transparent and effective for achieving the

project objectives?

31/39 (79.5%) EU: 21/25 (84%)

Non-EU: 10/14 (28.6%)

8/39 (20.5%) EU: 4/25 (16%)

Non-EU:4/14 (28.6%) 0/39 (0%)

2. Do you think that the Steering Committee

is effective for achieving the projects

objectives?

27/34 (79.4%) EU: 18/22 (81.8%)

Non-EU: 9/12 (75%)

7/34 (20.6%) EU: 4/22 (18.2%)

Non-EU: 3/12 (25%) 0/34 (0%)

3. Do you think that the Steering Teams

established for each work package (WP)

are effective in achieving the specific

objective of the WP and in facilitating

collaboration?

24/38 (63.2%) EU: 16/25 (64%)

Non-EU: 8/13 (61.5%)

14/38 (36.8%) EU: 9/25 (36%)

Non-EU: 5/13 (38.5%) 0/38 (0%)

4. Do you think the communication means

and frequency is an effective process?

26/38 (68.4%) EU: 17/25 (68%)

Non-EU: 9/13 (69.2%)

12/38 (31.6%) EU: 8/25 (32%)

Non-EU: 4/13 (30.8%) 0/38 (0%)

5. Do you think that the problem – solving

process was effective for achieving the

project objectives?

21/38 (55.3%) EU: 17/24 (70.8%) *

Non-EU: 4/14 (28.6%) *

16/38 (42.1%) EU: 6/24 (25%)

Non-EU: 10/14 (71.4%)

1/38 (2.6%) EU: 1/24 (4.2%)

Non-EU: 0/14

(0%)

6. Do you think that the monitoring and

evaluation of activities are adequate and

assist in keeping on track in terms of

quality and timeliness?

24/39 (61.5%) EU: 15/25 (60%)

Non-EU: 9/14 (64.3%)

14/39 (35.9%) EU: 9/25 (36%)

Non-EU: 5/14 (35.7%)

1/39 (2.6%) EU: 1/25 (4%)

Non-EU: 0/14

(0%)

* p-value: 0.028

3.4.2 Comments by the questionnaire responders

� The partner web tool is not effective for project management, because there is open

sharing of information and documents, as there is too much segmentation in the

individual WP steering teams and only final documents are posted for the knowledge of

all partners. The WP leader’s coordination can be improved, allowing more sharing and

consultation. The monitoring and evaluation has presented major delays, and so far no

assessment of the quality of the outputs is verifiable, only timeliness.

� We need to have more interactive problem solving process especially with funding

organisations.

� EpiSouth is a complex and ambitious network that has proven its usefulness, its

pertinence and that actually contribute to reinforce the health security in the area. Its

composition (1/3 EU countries 2/3 non-EU countries and the sources of funding i.e.

mainly EU commission) has generated difficulties that has been difficult to overcome.

EpiSouth is to often consider by donors as purely EU network, though it is not the case.

It is the mixing of EU and Non EU countries that does make the rational for such a

network and this should be acknowledged and implemented in the request from


donors. The lack of long term perspective (i.e. sustainability) is also an obstacle to

adequate problem solving in this domain. Nevertheless, despite difficulties EpiSouth is a

very successful network that have been able to cope with very complex issues including

(political instability eg Arab Spring)

� Communication means and frequency - there is a need for frequent update of provided

information regarding the list of participating countries and directories

� Major problems for project and activities management aroused following the Arab

Spring that affected several EpiSouth countries

� The announcement of the coming meetings is too short. As government employee, we

are required to ask for attendance permission via a special committee 3m prior to the

meeting

3.4.3 Interview results

In the question regarding their opinion regarding the management of the EpiSouth Plus the

interviewees responded the following:

� Five interviewees identified that it is a large network difficult to manage it with many

different countries from different political, geographical and cultural background having

different needs.

� The negative points identified in the management are that it is too bureaucratic in terms of

financial allocations. Difficulties were identified in the collaboration between EPISOUTH

and DG SANCO. EPISOUTH and DG SANCO did not have the same view and this caused

many delays especially for WP1 and WP6 (3 interviewees).

� There is enough coordination, stakeholders talk together and management is doing their

best (2 interviewee).

� Two interviewees commended that the management is weak, they should have a stronger

role and are not fully committed and dedicated and that a full time person should have

been employed to overlook activities.

� The management framework of the EpiSouth Plus project was rated by four interviewees as

satisfactory and identified as positive points in this management style the coordination and

efficiency of communications.


� One interviewee identified that the management are facing problems but are dealing with

them well. It might take time but they succeed to manage all issues.

� One interviewee responded that the management is committed and motivated but they do

not have a clear vision in where the project should focus. The management is not reactive

enough but expects others to react. There seems to be confusion between the objectives

and the methods. The network is the objective and is very important but need to build at

this network and to identify the means of its mobilisation (1 interviewee).

� One interviewee commended that they have a clear vision but they do not know how to

achieve it

� One interviewee expressed that the management should complete all deliverables and the

no-cost extension will help this.

� One interviewee commended that the management are not knowledgeable about Middle

East and international work and this affects the efficiency of the management.

� One interviewee commended that the quality of the reports are poor as they are very

descriptive.

3.5 Dissemination Activities Evaluation

The dissemination activities evaluated in this section are the website, the Quarterly EpiSouth

Electronic Bulletin, the EpiSouth Weekly Epi Bulletin (e-WEB) and other dissemination

activities (e.g. presentations and posters presented to conferences and workshops). The

timeliness and level of achievement of milestones of the dissemination activities have been

evaluated in the previous sections.

As shown in Table 6, a total of 7,792 number of visits were recorded from the period of May

2011- June2012. In the same table, the detailed number of visits per six months is presented.

From the results presented, almost a 25% decrease of the number of visits was recorded

from May 2011 to June 2012. However, the geographical coverage has increased from 83

countries in the first six months of the website’s operation to 93 in the last six months.

Table 6: Website statistics

Period Number of Visits

May-September 2011 2,283 visits came from 83 countries/territories

October – December 2011 1,993 visits came from 89 countries/territories

January 2012 – March 2012 1,783 visits came from 104 countries/territories

April 2012 - June 2012 1,733 visits came from 93 countries/territories

Total 7792 total visits


The EpiSouth Plus partnership has achieved a good level of representation in international

conferences promoting the awareness level of the project’s objectives and activities. The

partnership has presented and participated in a total of 27 conferences/meetings. Table 7

presents in detail the number of presentations/ participations per work package.

Table 7: Number of Presentation/ Participations to conference and Meetings by Work Package

WP No Number of Presentations/ Participations to Conferences/ Meetings

WP1/WP2

Coordination

/ Networking

1. Speech at the Cypriot Presidency Conference on Cross Border Health Threats in the EU and

Neighbouring Countries, Nicosia, Cyprus 5 July 2012 (with mention to EpiSouth)

2. Presentation at the Cypriot Presidency Conference on Cross Border Health Threats in the EU and

Neighbouring Countries, Nicosia, Cyprus 5 July 2012 (with mention to EpiSouth)

3. Poster presented at the High Level Conference "EU Health Programmes: results and perspectives",

Brussels, Belgium 3 May 2012

4. Presentation at Inter-Agency Meeting on environmental health in Southern and Eastern

Mediterranean , Marseille, France 14 March 2012

5. Poster presented at the 14th European Health Forum - Workshop on Health Security, Gastein,

Austria 5 October 2011

6. Presentation at the Union for the Mediterranean-Health Forum, Brussels, Belgium, 30 June 2011

7. Presentation at the 4th Eurasia Congress of Infectious Diseases, Sarajevo, Bosnia & Herzegovina, 1-

5 June 2011

8. Presentation at the Forum Mediterraneo in Sanità 2011, Palermo, Italy 24-26 May 2011

9. Presentation at the Subregional Meeting on Shared Solutions to Common Threats - Vaccination

and Vulnerable Populations in South East Europe, Tirana, Albania 28-29 April 2011

10. Presentation at the International Meeting on Emerging Diseases and Surveillance - IMED 2011,

Vienna, Austria 4-7 February 2011

11. Presentation at Mecids Executive Boarding Meeting, Jerusalem, Israel 26-27 November 2010

12. Presentation at the annual meeting of Arbo-zoonet, Rabat, Morroco 22-24 November 2010

13. Presentation to Delegation of Egyptian Ministry of Health , Rome, Italy 19 October 2010

WP4 14. Poster presented at the 15th Annual Conference of the European BioSafety Association (EBSA),

Manchester, UK 11-13 June 2012

15. Poster presented at the 4th Eurasia Congress of Infectious Diseases, Sarajevo, Bosnia &

Herzegovina 1-5 June 2011

16. Presentation at the ENVID Meeting, Antalya, Turkey 12-14 May 2011

17. Presentation at the Final Meeting of the EU Project EQADeBa, Brussels, Belgium 11-12 April 2011

WP6 18. First poster presented at the 15th International Congress on Infectious Diseases, Bangkok, Thailand

13-16 June 2012

19. Second poster presented at the 15th International Congress on Infectious Diseases, Bangkok,

Thailand 13-16 June 2012

20. Presentation at EWRS Meeting, Luxembourg, 24 January 2012

21. Presentation at the 4th Eurasia Congress of Infectious Diseases, Sarajevo, Bosnia & Herzegovina 1-

5 June 2011

22. Presentation at the Advanced Research Workshop on Internet-based Intelligence for Public Health

Emergencies and Disease Outbreak, Haifa, Israel 13-15 March 2011

23. Presentation at the International Meeting on Emerging Diseases and Surveillance - IMED 2011,

Vienna, Austria 4-7 February 2011

24. Presentation at EWRS Meeting, Luxembourg 1-2 December 2010

25. Presentation at the annual meeting of Arbo-zoonet, Rabat, Morroco 22-24 November 2010

WP7 26. Presentation at the Intercountry Meeting on Strengthening of Surveillance and Response

Capacities under IHR 2005, Beirut, Lebanon 26-28 March 2012

27. Presentation at XXIV Meeting of the South-eastern Europe Health Network, Tirana, Albania 10-11

November 2010

3.5.1 Previous Evaluation of the WP2 Website and Electronic Bulletin

An evaluation of the website and quarterly electronic bulletin was conducted by the WP2

Work Package leader. An online questionnaire was created to evaluate the public and

members area of the website. However, a small response rate was achieved with a total of


22 (25.6%) countries’ focal points responding to the questionnaire. Results indicate a good

level of satisfaction and utilization for the project’s website and quarterly electronic bulletin

among the countries’ focal points who participated to the survey. The results of this

evaluation identified certain improvements and issues needed to be resolved especially in

relation to the Network Working Area (NWA). The geographical distribution of the

responding focal points over-represents the European countries included in the network and

under-represents countries of north-Africa, south-east Europe and middle-east (see report

titled “Evaluation of the Network website and quarterly electronic bulletin”, Annex XIII in the

Interim Technical Implementation report).


In Table 8 the details of the analysis of the evaluation questionnaires in relation to the

website are presented. The majority of the responders seem to be using the NWA and more

than 80% believe that the website fully contributes to the dissemination of results and

provides the necessary information to the visitor. Almost 78% (28/36) of the responders

rated that the website fully provides the visitor with all the necessary information concerning

the project while 22% rated it as partially. It should be noted that a higher percentage of the

Non EU responders (5/12, 42%) rated that the website partially provides the necessary

information in comparison to EU countries (3/24, 12.5%) (p-value:0.08). The public area of

the website is visited often only by the 36.8% (14/38) of the responders. From the different

sections of the website more than 60% of the responders rated as satisfactory the sections

of the Events, Meeting and Events in the NWA, and the News section whereas more than

50% rated as satisfactory the Network and Bulletin sections. A significant percentage (9 out

of 38, 23.7%) commended that they have faced problems while visiting the website.

Regarding the NWAs the majority of the responders commended they the use them but the

statement that one fourth of the responders faced problems while visiting the site should be

further investigated and resolved by the coordination of the project.

Table 8: Evaluation Questionnaire 1: Website evaluation

Website Fully Partially Not at all

7. Do you think that the website contributes to

the dissemination of the project results?

31/38 (81.6%) EU: 20/25 (80%)

Non-EU: 11/13 (84.6%)

7/38 (18.4%) EU: 5/25 (20%)

Non-EU: 2/13 (15.4%)

0/38 (0%)

8. Do you think that the website provides the

visitor with all the necessary information

concerning the project?

28/36 (77.8%) EU: 21/24 (87.5%)

Non-EU: 7/12 (58.3%)

8/36 (22.2%) EU: 3/24 (12.5%)*

Non-EU: 5/12 (41.7%)*

0/36 (0%)

Often Sometimes Seldom

9. How often do you visit the public area of the

website?

14/38 (36.8%) EU: 9/25 (36%)

Non-EU: 5/13 (38.5%)

18/38 (47.4%) EU: 10/25 (40%)

Non-EU: 8/13 (61.5%)

6/38 (15.8%) EU: 6/25 (24%)

Non-EU: 0/13 (0%)


10. How often do you visit the Network Working

Area (NWA) of the website?

10/35 (28.6%) EU: 6/22 (27.3%)

Non-EU: 4/13 (30.8%)

21/35 (60%) EU: 12/22 (54.5%)

Non-EU: 9/13 (69.2%)

4/35 (11.4%) EU: 4/22 (18.2%)

Non-EU: 0/13 (0%)

Excellent Satisfactory Relatively

Satisfactory

Requires

Improvement

11. How do you rate the

Meeting and Events Area in

the NWA?

10/37 (27%) EU: 7/25 (28%)

Non-EU:3/12

(25%)

24/37 (64.9%) EU: 15/25 (60%)

Non-EU: 9/12 (75%)

2/37 (5.4%) EU: 2/25 (8%)

Non-EU: 0/12 (0%)

1/37 (2.7%) EU: 1/25 (4%)

Non-EU: 0/12 (0%)


Document Area in the

NWA?

17/38 (44.7%) EU: 12/25 (48%)

Non-EU: 5/13

(38.5%)

17/38 (44.7%) EU: 10/25 (40%)

Non-EU: 7/13 (53.8%)

3/38 (7.9%) EU: 2/25 (8%)

Non-EU: 1/13 (7.7%)

1/38 (2.6%) EU: 1/25 (4%)

Non-EU: 0/13 (0%)


Discussion Forum in the

NWA?

3/35 (8.6%) EU: 3/22 (13.6%)

19/35 (54.3%) EU: 10/22 (45.5%)

Non-EU: 9/13 (69.2%)

7/35 (20%) EU: 5/22 (22.7%)

Non-EU: 2/13 (15.4%)

6/35 (17.1%) EU: 4/22 (18.2%)

Non-EU: 2/13

(15.4%)


Activities of the Network

sections in the public area

of the website?

9/37 (24.3%) EU: 7/24 (29.2%)

Non-EU: 2/13

(15.4%)

20/37 (54.1%) EU: 12/24 (50%)

Non-EU: 8/13 (61.5%)

7/37 (18.9%) EU: 4/24 (16.7%)

Non-EU: 3/13 (23.1%)

1/37 (2.7%) EU: 1/24 (4.2%)

Non-EU: 0/13 (0%)

15. How do you rate the Events

section in the public area of

the website?

10/38 (26.3%) EU: 6/25 (24%)

Non-EU: 4/13

(30.8%)

24/38 (63.2%) EU: 16/25 (64%)

Non-EU: 8/13 (61.5%)

4/38 (10.5%) EU: 3/25 (12%)

Non-EU: 1/13 (7.7%)

0/38 (0%)


Bulletin section in the

public area of the website?

17/38 (44.7%) EU: 11/25 (44%)

Non-EU:6/13

(46.2%)

21/38 (55.3%) EU: 14/25 (56%)

Non-EU: 7/13 (53.8%)

0/38 (0%)

0/38 (0%)


Directories section in the

public area of the website?

7/37 (18.9%) EU: 5/24 (20.8%)

Non-EU: 2/13

(15.4%)

23/37 (62.2%) EU: 14/24 (58.3%)

Non-EU: 9/13 (69.2%)

6/37(16.2%) EU: 5/24 (20.8%)

Non-EU: 1/13 (7.7%)

1/37 (2.7 %) EU: 0/24 (0%)

Non-EU: 1/13

(7.7%)

18. How do you rate the News

section in the public area of

the website?

11/38 (28.9%) EU: 8/25 (32%)

Non-EU: 3/13

(23.1%)

24/38 (63.2%) EU: 14/25 (56%)

Non-EU: 10/13 (76.9%)

3/38 (7.9%) EU: 3/25 (12%)

Non-EU: 0/13 (0%)

0/38 (0%)

Yes No

19. Have you faced any

problems while visiting the

website?

9/38 (23.7%) EU: 6/25 (24%)

Non-EU: 3/13 (23.1%)

29/38 (76.3%) EU: 19/25 (76%)

Non-EU: 10/13 (76.9%)

20. Did you use the WP4 NWA?

7/8 (87.5%) EU: 4/4 (100%)

Non-EU: 3/4 (75%)

1/8 (12.5%) Non-EU: 1/4(25%)

21. Did you use the WP5 NWA? 1/2 (50%)

EU: 1/2 (50%)

1/2 (50%) EU: 1/2 (50%)


3/5 (60%) EU: 3/4 (75%)

2/5 (40%) EU: 1/4 (25%)

Non-EU: 1/1 (100%)


5/5 (100%) EU: 4/4 (100%)

Non-EU: 1/1 (100%) 0/5 (0%)

* p-value: 0.08

In Table 9, the details of the analysis of the evaluation questionnaires in relation to the

EpiSouth Electronic Bulletin are presented. More than 67% believe that the EpiSouth

Electronic Bulletin fully contributes to the dissemination of results and provides the

necessary information to the visitor. Twelve out of 37 responders (all EU) believe that the

electronic bulletin partially contributes to the dissemination of the project results in

comparison with zero from Non-EU countries (p-value: 0.003). The layout was rated as


excellent and satisfactory by all responders. About 71% (27 out of 38) of the responders

rated the range of content as satisfactory whereas 26.3% (10 out of 38) as excellent. It

should be noted that a higher percentage of the Non EU responders rated the range of

content as satisfactory (12/13, 92.3%) in comparison to EU responders (15/25, 60%) (p-

value: 0.1). Around 26% rated as excellent the range of content. A significant higher

percentage (9/25, 36%) of EU responders rated as excellent the range of content in

comparison with the Non-EU (1/13, 7.7%) (p-value: 0.1). The entire Bulletin is read by 34.2%

(13/38) of the responders (EU: 10/25, 40%, Non-EU: 3/13, 23.1%), 44.7% (17/38) read most

of it (EU: 9/25, 44.7%, Non-EU: 8/13, 61.5%), whereas 18.6% (7/38) read almost half.

Table 9: Evaluation Questionnaire 1: Quarterly EpiSouth Electronic Bulletin evaluation

Quarterly EpiSouth Electronic Bulletin Fully Partially Not at all

24. Do you think that the Quarterly EpiSouth

Electronic Bulletin contributes to the

dissemination of the project results?

25/37 (67.6%) EU: 13/25 (52%)

Non-EU: 12/12 (100%)

12/37 (32.4%) EU: 12/25 (48%)

Non-EU: 0/12 (0%)

0/37 (0%)

25. Do you think that the Quarterly EpiSouth

Electronic Bulletin provides the reader with all

the necessary information concerning the

project?

25/37 (67.6%) EU: 15/24 (62.5%)

Non-EU: 10/13 (76.9%)

12/37 (32.4%) EU: 9/24 (37.5%) *

Non-EU: 3/13 (23.1%)* 0/37 (0%)

Always Often Sometimes Seldom

26. How often do you

read the Quarterly

EpiSouth Electronic

Bulletin?

17/38(44.7%) EU: 11/25 (44%)

Non-EU: 6/13 (46.2%)

13/38(34.2%) EU: 7/25 (28%)

Non-EU: 6/13 (46.2%)

8/38 (21.1%) EU: 7/25 (28%)

Non-EU: 1/13 (7.7%)

0/38 (0%)

All Most Almost half Little

27. How much of the

Quarterly EpiSouth

Electronic Bulletin

do you read?

13/38(34.2%) EU: 10/25 (40%)

Non-EU: 3/13 (23.1%)

17/38(44.7%) EU: 9/25 (36%)

Non-EU: 8/13 (61.5%)

7/38 (18.4%) EU: 5/25 (20%)

Non-EU: 2/13 (15.4%)

1/38 (2.6%) EU: 1/25 (4%)

Non-EU: 0/13

(0%)

Excellent Satisfactory

Relatively

Satisfactory

Requires

Improvement

28. How do you like the

layout and

presentation of the

Quarterly EpiSouth

Electronic Bulletin?

16/38 (42.1%) EU: 13/25 (52%)

Non-EU: 3/13 (23.1%)

21/38 (55.3%) EU: 12/25 (48%)

Non-EU: 9/13 (69.2%)

1/38(2.6%) EU: 0/25 (0%)

Non-EU: 1/13 (7.7%)

0/38 (0%)

29. What do you think

of the range of

content?

10/38 (26.3%) EU: 9/25 (36%)**

Non-EU: 1/13 (7.7%)**

27/38 (71.1%) EU: 15/25 (60%)**

Non-EU: 12/13 (92.3%)**

1/38 (2.6%) EU: 1/25 (4%)

Non-EU: 0/13 (0%)

0/38 (0%)

* p-value: 0.003 ** p-value:0.1

In Table 10, the details of the analysis of the evaluation questionnaires in relation to the

EpiSouth Weekly EPI Bulletin are presented. Almost 79% (30/38) believe that the EpiSouth

Weekly EPI Bulletin fully contributes to the dissemination of new health events occurring in

both Non-EpiSouth and EpiSouth countries. The EpiSouth Weekly EPI Bulletin is always read

by almost 58% (22/38) of the responders whereas 34.2% (13/38) often read it. Around 51.4%

(19/37) of the responders believe that all the information of the e-web bulleting is useful to


their job whereas 43.2% (16/37) believe that most of the information is useful. The layout

and range of content was rated by the majority of the responders as excellent and

satisfactory (>97%).

Table 10: Evaluation Questionnaire 1: EpiSouth Weekly Epi Bulletin (e-WEB)

EpiSouth Weekly Epi Bulletin (e-WEB) Fully Partially Not at all

30. Do you think that the EpiSouth Weekly Epi

Bulletin (e-WEB) contributes to the dissemination

of the new health events occurring in both non-

EpiSouth and EpiSouth countries

30/38 (78.9%) EU: 19/25 (76%)

Non-EU: 11/13 (84.6%)

8/38 (21.1%) EU: 6/25 (24%)

Non-EU: 2/13 (15.4%)

0/38 (0%)

Always Often Sometimes Seldom

31. How often do you read

the EpiSouth Weekly Epi

Bulletin (e-WEB)?

22/38 (57.9%) EU: 12/25 (48%)

Non-EU: 10/13 (76.9%)

13/38 (34.2%) EU: 10/25 (40%)

Non-EU: 3/13 (23.1%)

3/38 (7.9%) EU: 3/25 (12%)

Non-EU: 0/13 (0%)

0/38 (0%)

All Most Almost half Little

32. Do you find the info of

the e-web bulletin useful

for your institute/MoH?

19/37 (51.4%) EU: 10/24 (41.7%)

Non-EU: 9/13 (69.2%)

16/37 (43.2%) EU: 12/24 (50%)

Non-EU: 4/13 (30.8%)

1/37 (2.7%) EU: 1/24 (4.2%)

Non-EU: 0/13 (0%)

1/37(2.7%) EU: 1/24 (4.2%)

Excellent Satisfactory

Relatively

Satisfactory

Requires

Improvement

33. How do you like the

layout and presentation

of the EpiSouth Weekly

Epi Bulletin (e-WEB)?

21/38(55.3%) EU: 14/25 (56%)

Non-EU: 7/13 (53.8%)

17/38 (44.7%) EU: 11/25 (44%)

Non-EU: 6/13 (46.2%) 0/38 (0%) 0/38 (0%)

34. What do you think of the

range of content?

19/38 (50%) EU: 13/25 (52%)

Non-EU: 6/13 (46.2%)

18/38 (47.4%) EU: 11/25 (44%)

Non-EU: 7/13 (53.8%)

1/38 (2.6%) EU: 1/25 (4%)

Non-EU: 0/13 (0%)

0/38 (0%)

3.5.3 Comments by the questionnaire responders

Ideas that could improve the website:

� Many times access to the network area is difficult because of the need of change

constantly the password. In addition, when there is a need of downloading a

document, it is not clear where they are and many clicks are needed to have access to

it. Simplify the route or give some shortcut. It would be useful to simplify the

platform the site for communication purposes and specific training sessions should be

run. For instance, the forum is a vital function in EpiSouth but the utilization is less

than needed or expected in such a network. A specific area for joint preparation of

documents (wiki) would be useful

� We need to activate the forum as for many times no topics are implanted or

discussed losing its weight.

� To have a section on news, frequent updates, partners’ news and activities, links with

other initiatives, etc.


� Sometimes the entry in the platform takes time, maybe a technical problem or errors

of connection

� The publication only in English language. Consider multi-language interface.

� It is not so clear and requires a bit of time to understand the organisation/structure of

the website

� The box "Activities of the Network" very often does not allow to correctly read the

contents of the sections in the Public Area of the website. We do not know whether

this is a problem linked to our information system

� Not able to find easily some information

� It takes sometime each time to understand the system

� Too fragmented. Location of information difficult to identify. Password procedure

should be revised.

Ideas that could improve the Quarterly EpiSouth Electronic Bulletin:

� Do not write dark bold text on dark background.

� I am a passive recipient of the project information and so far the information

available was not useful for my day-to-day work.

� It must be kept short and informative as it is

� Not all information can be included otherwise it will discourage readers

Ideas that could improve the EpiSouth Weekly Epi Bulletin:

� In case of alerts it is warranted to present basic global guidelines on the alert

(disease) management

� E-Web is almost always read in the email version sent by the WP Leader

� I can confirm that info of the e-web bulletin are indeed considered very useful by

both my institute and The MoH - I would expect the same type and quality of

information from other network and to the best of my knowledge very few

international networks produce weekly bulletin with such frequency (weekly) and

adapted to my needs


Further comments

� The WP6 SOP’s have been modified to comply with the numerous EC requests (EAHC,

SANCO..) and therefore does not correspond by what it was intended to be at the

beginning and therefore this should be considered while analysing the results of this

current survey. I can confirm that info of the e-web bulletin is indeed considered very

useful by both my institute and the MoH.

� Used NWA mainly to look up for updated documents - The rest is managed by my

team

� Take into consideration the fact that due to problems of cash flow part of activities

were on in the case were lagging behind.

� In the Public Health Institution List - No Data for R. Macedonia, WP8-Directory of

Human Public Health and Veterinary Public Health Officials for Zoonoses - some data

are not correct with actual situation in the country and need to be updated

� Not applicable the discussion forum is not really used not because it is not well done

but because focal point prefer other mode of communication

3.6 Training Activities Evaluation

3.6.1 Previous Evaluation Activities

An evaluation of the training course conducted under WP4 on “Dengue and biosafety in the

laboratory” which was held in the Institute Pasteur, France from the 2nd

to 6th

of July 2012

was conducted at the end of the training course. In the evaluation both theoretical and

practical parts were evaluated as well as the trainee’s expectations for the second training.

According to this evaluation the trainees were very satisfied with the training and 94%

(16/17) of the trainees declared that the training met their expectations. However, trainees

expressed a request for more practical exercises and laboratory training. The trainees

identified topics to be addressed in the second training including complementary topics to

those addressed during the first training and training on viral culture in BSL3 conditions.


In Table 11 the results of the evaluation for the WP4 training activities are presented in

detail. More than 60% of the responders rated all aspects of the training course as Excellent.

Almost 67% of the responders rated as excellent the administrative support, comfort to


meeting venue description of objectives, trainers performance, teaching methods, length of

course and relevance to job. More than 80% rated the information regarding the

participation and the details on the venue as excellent. No major differences were noted

between the responders from the EU and Non EU countries.

Table 11: Evaluation Questionnaire 1: WP4 Training Activities

Poor Average Good Excellent

Coordination and administration

35. Contact with

EPISOUTH prior to the

training courses

1/16(6.3%) Non-EU:

1/8 (12.5%)

1/16 (6.3%) EU: 1/8 (12.5%)

Non-EU: 0/8 (0%)

4/16 (25%) EU: 3/8 (37.5%)

Non-EU:

1/8 (12.5%)

10/16 (62.5%) EU: 4/8 (50%)

Non-EU: 6/8 (75%)

36. Information regarding

your participation

0/16 (0%)

2/16(12.5%) EU: 1/8 (12.5%)

Non-EU: 1/8 (12.5%)

1/16 (6.3%) EU: 1/8 (12.5%)

Non-EU: 0/8 (0%)

13/16 (81.3%) EU: 6/8 (75%)

Non-EU: 7/8 (87.5%)

37. Details on the venue 0/15(0%)

0/15 (0%)

3/15 (20%) EU: 3/8 (37.5%)

Non-EU: 0/7 (0%)

12/15 (80%) EU: 5/8 (62.5%)

Non-EU: 7/7 (100%)

38. Instructions for the

course

0/15 (0%)

1/15 (6.7%) EU: 1/8 (12.5%)

Non-EU: 0/7 (0%)

5/15 (33.3%) EU: 3/8 (37.5%)

Non-EU: 2/7 (28.6%)

9/15 (60%) EU: 4/8 (50%)

Non-EU:5/7 (71.4%)

39. Administrative support 0/15 (0%)

0/15 (0%)

5/15 (33.3%) EU: 2/8 (25%)

Non-EU: 3/7 (42.9%)

10/15 (66.7%) EU: 6/8 (75%)

Non-EU:4/7 (57.1%)

40. Comfort of meeting

venue

0/15 (0%)

0/15 (0%)

5/15 (33.3%) EU: 2/8 (25%)

Non-EU: 3/7 (42.9%)

10/15 (66.7%) EU: 6/8 (75%)

Non-EU: 4/7 (57.1%)

41. Trainee’s selection

requirements

0/15 (0%)

0/15 (0%)

7/15 (46.7%) EU: 4/8 (50%)

Non-EU: 3/7 (42.9%)

8/15 (53.3%) EU: 4/8 (50%)

Non-EU: 4/7 (57.1%)

42. Publicity of the course

to target audience

0/15 (0%)

0/15 (0%)

6/15 (40%) EU: 3/8 (37.5%)

Non-EU: 3/7 (42.9%)

9/15 (60%) EU: 5/8 (62.5%)

Non-EU: 4/7 (57.1%)

WORK PACKAGE4: TRAINING COURSE ON DENGUE AND BIOSAFETY IN THE LABORATORY

43. Description of the

objectives

0/15 (0%)

0/15 (0%)

5/15 (33.3%) EU: 3/8 (37.5%)

Non-EU: 2/7 (28.6%)

10/15 (66.7%) EU: 5/8 (62.5%)

Non-EU: 5/7 (71.4%)

44. Course content 0/15 (0%)

1/15 (0%) EU: 1/8 (12.5%)

Non-EU: 0/7 (0%)

4/15 (26.7%) EU: 3/8 (37.5%)

Non-EU: 1/7 (14.3%)

10/15 (66.7%) EU: 4/8 (50%)

Non-EU: 6/7 (85.7%)

45. Course materials 0/15 (0%)

0/15 (0%)

4/15 (26.7%) EU: 3/8 (37.5%)

Non-EU: 1/7 (14.3%)

11/15 (73.3%) EU: 5/8 (62.5%)

Non-EU: 6/7 (85.7%)

46. Participants (number

and adequacy)

0/15 (0%)

1/15 (6.7%) EU: 1/8 (12.5%)

Non-EU: 0/7(0%)

5/15 (33.3%) EU: 3/8 (37.5%)

Non-EU: 2/7 (28.6%)

9/15 (60%) EU: 4/8 (50%)

Non-EU: 5/7(71.4%)

47. Trainers performance 0/15 (0%)

0/15 (0%)

5/15 (33.3%) EU: 4/8 (50%)

Non-EU: 1/7 (14.3%)

10/15 (66.7%) EU: 4/8 (50%)

Non-EU: 6/7 (85.7%)

48. Teaching methods 0/15 (0%)

0/15 (0%)

5/15 (33.3%) EU: 2/8 (25%)

Non-EU: 3/7 (42.9%)

10/15 (66.7%) EU: 6/8 (75%)

Non-EU: 4/7 (57.1%)

49. Length of course 0/15 (0%)

1/15 (6.7%) EU: 1/8 (12.5%)

Non-EU: 0/7 (0%)

4/15 (26.7%) EU: 2/8 (25%)

Non-EU: 2/7 (28.6%)

10/15 (66.7%) EU: 5/8 (62.5%)

Non-EU: 5/7 (71.4%)

50. Time keeping 0/15 (0%)

0/15 (0%)

6/15 (40%) EU: 3/8 (37.5%)

Non-EU: 3/7 (42.9%)

9/15 (60%) EU: 5/8 (62.5%)

Non-EU: 4/7 (57.1%)

51. Opportunity for

networking

0/15 (0%)

0/15 (0%)

6/15 (40%) EU: 4/8 (50%)

Non-EU: 2/7 (28.6%)

9/15 (60%) EU: 4/8 (50%)

Non-EU: 5/7 (71.4%)

52. Relevance to job 0/15 (0%)

0/15 (0%)

5/15 (33.3%) EU: 3/8 (37.5%)

Non-EU: 2/7 (28.6%)

10/15 (66.7%) EU: 5/8 (62.5%)

Non-EU: 5/7 (71.4%)


53. Enjoyment 0/15 (0%)

0/15 (0%)

4/15 (26.7%) EU: 2/8 (25%)

Non-EU: 2/7 (28.6%)

11/15 (73.3%) EU: 6/8 (75%)

Non-EU: 5/7 (71.4%)

54. Work environment 0/15 (0%)

0/15 (0%)

4/15 (26.7%) EU: 3/8 (37.5%)

Non-EU: 1/7 (14.3%)

11/15 (73.3%) EU: 5/8 (62.5%)

Non-EU: 6/7 (85.7%)

Another training course was organised under WP5 activities on cluster analysis. In Table 12

the results of the evaluation of this course are presented. All of the responders rated as

excellent or good the coordination and administration of the course. The course material and

administrative support were rated as excellent by more than 90% of the responders. The

course content, the trainees’ selection requirements and instructions for the course were

rated as excellent by more than 80% of the responders. More than 70% of the responders

rated as excellent the following aspects: details of the venue, comfort of meeting venue,

publicity of the course to target group, description of the objectives, participants, trainers’

performance, teaching methods, enjoyment. Five out of 11 (45.5%) rated the relevance to

job as good and another 45.5% as excellent.

Table 12: Evaluation Questionnaire 1: WP5Training Activities

Poor Average Good Excellent

Coordination and administration

55. Contact with EPISOUTH prior to

the training courses 0/11 (0%)

1/11 (9.1%) EU: 0/6 (0%)

Non-EU:

1/5 (20%)

4/11 (36.4%) EU: 4/6 (66.7%)

Non-EU: 0/5 (0%)

6/11 (54.5%) EU: 2/6 (33.3%)

Non-EU: 4/5 (80%)

56. Information regarding your

participation 0/11 (0%) 0/11 (0%) 5/11 (45.5%) EU: 3/6 (66.7%)

Non-EU: 2/5 (40%)

6/11 (54.5%) EU: 3/6 (33.3%)

Non-EU: 3/5 (60%)

57. Details on the venue 0/11 (0%) 0/11 (0%)

3/11 (27.3%) EU: 1/6 (16.7%)

Non-EU: 2/5 (40%)

8/11 (72.7%) EU: 5/6 (83.3%)

Non-EU: 3/5 (60%)

58. Instructions for the course 0/11 (0%)

1/11 (9.1%) EU: 0/6 (0%)

Non-EU: 1/5 (20%)

1/11 (9.1%) EU: 1/6 (16.7%)

Non-EU: 0/5 (0%)

9/11 (81.8%) EU: 5/6 (83.3%)

Non-EU: 4/5 (80%)

59. Administrative support 0/11 (0%) 0/11 (0%)

1/11 (9.1%) EU: 0/6 (0%)

Non-EU: 1/5 (20%)

10/11 (90.9%) EU: 6/6 (100%)

Non-EU: 4/5 (80%)

60. Comfort of meeting venue 0/10 (0%) 0/10 (0%)

3/10 (30%) EU: 2/5 (40%)

Non-EU: 1/5 (20%)

7/10 (70%) EU: 3/5 (60%)

Non-EU: 4/5 (80%)

61. Trainee’s selection requirements 0/11 (0%) 0/11 (0%)

2/11 (18.2%) EU: 1/6 (16.7%)

Non-EU: 1/5 (20%)

9/11 (81.8%) EU: 5/6 (83.3%)

Non-EU: 4/5 (80%)

62. Publicity of the course to target

audience 0/10 (0%) 1/10 (10%) EU: 0/5 (0%)

Non-EU: 1/5 (20%)

2/10 (20%) EU: 1/5 (20%)

Non-EU: 1/5 (20%)

7/10 (70%) EU: 4/5 (80%)

Non-EU: 3/5 (60%)

WORK PACKAGE 5: TRAINING COURSE ON CLUSTER ANALYSIS

63. Description of the objectives 0/10 (0%) 0/10 (0%)

3/10 (30%) EU: 1/5 (20%)

Non-EU: 2/5 (40%)

7/10 (70%) EU: 4/5 (80%)

Non-EU: 3/5 (60%)

64. Course content 0/10 (0%)

1/10 (10%) EU: 0/5 (0%)

Non-EU: 1/5 (20%)

1/10 (10%) EU: 0/5 (0%)

Non-EU: 1/5 (20%)

8/10 (80%) EU: 5/5 (100%)

Non-EU: 3/5 (60%)


65. Course materials 0/10 (0%) 0/10 (0%)

1/10 (10%) EU: 0/5 (0%)

Non-EU: 1/5 (20%)

9/10 (90%) EU: 5/5 (100%)

Non-EU: 4/5 (80%)

66. Participants (number and

adequacy) 0/10 (0%) 1/10 (10%) EU: 0/5 (0%)

Non-EU:

1/5 (20%)

2/10 (20%) EU: 1/5 (20%)

Non-EU:

1/5 (20%)

7/10 (70%) EU: 4/5 (80%)

Non-EU:

3/5 (60%)

67. Trainers performance

0/10 (0%) 1/10 (10%) EU: 0/5 (0%)

Non-EU:

1/5 (20%)

2/10 (20%) EU: 1/5 (20%)

Non-EU: 1/5 (20%)

7/10 (70%) EU: 4/5 (80%)

Non-EU: 3/5 (60%)

68. Teaching methods 0/10 (0%)

1/10 (10%) EU: 0/5 (0%)

Non-EU: 1/5 (20%)

2/10 (20%) EU: 0/5 (0%)

Non-EU: 2/5 (40%)

7/10 (70%) EU: 5/5 (100%)

Non-EU: 2/5 (40%)

69. Length of course 0/10 (0%)

2/10 (20%) EU: 1/5 (20%)

Non-EU: 1/5 (20%)

2/10 (20%) EU: 1/5 (20%)

Non-EU: 1/5 (20%)

6/10 (60%) EU: 3/5 (60%)

Non-EU: 3/5 (60%)

70. Time keeping 0/11 (0%)

1/11 (9.1%) EU: 0/6 (0%)

Non-EU: 1/5 (20%)

4/11 (36.4%) EU: 2/6 (33.3%)

Non-EU: 2/5 (40%)

6/11 (54.5%) EU: 4/6 (66.7%)

Non-EU: 2/5 (40%)

71. Opportunity for networking 0/11 (0%)

2/11 (18.2%) EU: 0/6 (0%)

Non-EU: 2/5 (40%)

2/11 (18.2%) EU: 1/6 (16.7%)

Non-EU: 1/5 (20%)

7/11 (63.6%) EU: 5/6 (83.3%)

Non-EU: 2/5 (40%)

72. Relevance to job

0/11 (0%) 1/11 (9.1%) EU: 0/6 (0%)

Non-EU:

1/5 (20%)

5/11 (45.5%) EU: 3/6 (50%)

Non-EU: 2/5 (40%)

5/11 (45.5%) EU: 3/6 (50%)

Non-EU: 2/5 (40%)

73. Enjoyment 0/11 (0%)

1/11 (9.1%) EU: 0/5 (0%)

Non-EU: 1/5 (20%)

2/11 (18.2%) EU: 0/6 (0%)

Non-EU: 2/5 (40%)

9/11 (72.7%) EU: 6/6 (100%)

Non-EU: 2/5 (40%)

74. Work environment 0/11 (0%) 0/11 (0%)

2/11 (18.2%) EU: 0/6 (0%)

Non-EU: 2/5 (40%)

9/11 (81.8%) EU: 6/6 (100%)

Non-EU: 3/5 (60%)

3.6.3 Comments by the responders for WP4 training

� Besides only Dengue and West Nile virus are a priority for the WP4 activities, we

suggest including other emerging infectious diseases like the novel corona virus. The

network could take advantage of this situation because the participation of countries

from the Middle East where the new virus was emerged and go ahead , helping this

countries to improve the EPI and lab surveillance, acting as a bridge between Europe

and this countries to share knowledge and methodology.

3.6.4 Comments by the responders for WP5 training

� The stay would be more supported by the Project organisation even after the period

after the course/training. It would be better to give a bit more information during the

transfers (arrival/departure) at the meeting.


3.7 Documents and deliverables produced Evaluation


Table 13 present the evaluation of the deliverables and documents produced by the partnership. The majority of the responders rated the documents as excellent and satisfactory.

Table 13: Evaluation Questionnaire 1: Documents and deliverables produced

Deliverables /

Documents

Methodology

Used

How do you rate the

deliverable

/document?

Performance criteria

Clarity* Consistency** Usefulness/ Effectiveness Conformity to Objectives Completeness***

E* S* RS* RI* E S RS RI E S RS RI E S RS RI E S RS RI E S RS RI E S R RI

75. WP1:

1st

Steering

Committee and

2nd

SC/AB

Meeting reports

2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-

EU:

1/1

(100%)

0 0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-EU:

1/1

(100%)

0 0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-

EU:

1/1

(100%)

0 0 3/6

(50%)

EU:

3/5

(60%)

3/6

(50%)

EU:

2/5

(40%)

Non-EU:

1/1

(100%)

0 0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-

EU:

1/1

(100%)

0 0 3/6

(50%)

EU:

3/5

(60%)

3/6

(50%)

EU:

2/5

(40%)

Non-

EU:

1/1

(100%)

0 0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-EU:

1/1

(100%)

0 0

76. WP1:

Project Work

Plan 1st and 2nd

11/21

(52.4%)

EU:

10/15

(66.7%)

Non-

EU:

1/6

(16.7%)

10/21

(47.6%)

EU:

5/15

(33.3%)

Non-

EU:

5/6

(83.3%)

0 0 11/20

(55%)

EU:

9/14

(64.3%)

Non-

EU:

2/6

(33.3%)

9/20

(45%)

EU:

5/14

(35.7%)

Non-EU:

4/6

(66.7%)

0 0 12/19

(63.2%)

EU:

11/13

(84.6%)

Non-

EU:

1/6

(16.7%)

7/19

(36.8%)

EU:

2/13

(15.4%)

Non-

EU:

5/6

(83.3%)

0 0 9/18

(50%)

EU:

8/12

(66.7%)

Non-

EU:

1/6

(16.7%)

9/18

(50%)

EU:

4/12

(33.3%)

Non-EU:

5/6

(83.3%)

0 0 13/19

(68.4%)

EU:

11/13

(84.6%)

Non-

EU:

2/6

(33.3%)

6/19

(31.6%)

EU:

2/13

(15.4%)

Non-

EU:

4/6

(66.7%)

0 0 10/17

(58.8%)

EU:

9/12

(75%)

Non-EU:

1/5

(20%)

7/17

(41.2%)

EU:

3/12

(25%)

Non-

EU:

4/5

(80%)

0 0 10/16

(62.5%)

EU:

9/11

(81.8%)

Non-EU: 1/5 (20%)

5/16

(31.3%)

EU:

2/11

(18.2)

Non-EU: 3/5

(60%)

1/16

(6.3%)

Non-

EU: 1/5

(20%)

0

77. WP1:

Sustainability

Plan

8/17

(47.1%)

EU:

7/11

(63.6%)

Non-

EU:

1/6

(16.7%)

8/17

(47.1%)

EU:

3/11

(27.3%)

Non-

EU:

5/6

(83.3%)

1/17

(5.9%)

EU:

1/11

(9.1%)

0 8/15

(53.3%)

EU:

6/9

(66.7%)

Non-

EU:

2/6

(33.3%)

5/15

(33.3%)

EU:

2/9

(22.2%)

Non-EU:

3/6

(50%)

2/15

(13.3%)

EU:

1/9

(11.1%)

Non-

EU:

1/6

(16.7%)

0 7/15

(46.7%)

EU:

6/9

(66.7%)

Non-

EU:

1/6

(16.7%)

7/15

(46.7%)

EU:

2/9

(22.2%)

Non-

EU:

5/6

(83.3%)

1/15

(6.7%)

EU:

1/9

(11.1%)

0 6/15

(40%)

EU:

5/9

(55.6%)

Non-

EU:

1/6

(16.7%)

7/15

(46.7%)

EU:

3/9

(33.3%)

Non-EU:

4/6

(66.7%)

2/15

(13.3%)

EU:

1/9

(11.1%)

Non-

EU:

1/6

(16.7%)

0 7/15

(46.7%)

EU:

5/9

(55.6%)

Non-

EU:

2/6

(33.3%)

7/15

(46.7%)

EU:

3/9

(33.3%)

Non-

EU:

4/6

(66.7%)

1/15

(6.7%)

EU:

1/9

(11.1%)

0 6/14

(42.9%)

EU:

5/9

(55.6%)

Non-EU: 1/5 (20%)

7/14

(50%)

EU:

3/9

(33.3%)

Non-

EU:

4/5

(80%)

1/14

(7.1%)

EU:

1/9

(11.1%)

0 6/12

(50%)

EU:

5/8

(62.5%)

Non-EU:

1/4(25%)

6/12

(50%)

EU:

3/8

(37.5%)

Non-EU:

3/4

(75%)

0 0

78. WP1:

D1: Interim

technical

Implementation

report

9/18

(50%)

EU:

8/13

(61.5%)

Non-

EU:

1/5

(20%)

9/18

(50%)

EU:

5/13

(38.5%)

Non-

EU:

4/5

(80%)

0 0 9/16

(56.3%)

EU:

7/11

(63.6%)

Non-

EU:

2/5

(40%)

7/16

(43.8%)

EU:

4/11

(36.4)

Non-EU:

3/5

(60%)

0 0 8/16

(50%)

EU:

6/11

(54.5%)

Non-

EU:

2/5

(40%)

8/16

(50%)

EU:

5/11

(45.5%)

Non-

EU:

3/5

(60%)

0 0 6/16

(37.5%)

EU:

5/11

(45.5%)

Non-

EU:

1/5

(20%)

10/16

(62.5%)

EU:

6/11

(54.5%)

Non-EU: 4/5 (80%)

0 0 9/16

(56.3%)

EU:

7/11

(63.6%)

Non-

EU:

2/5

(40%)

7/16

(43.8%)

EU:

4/11

(36.4%)

Non-

EU:

3/5

(60%)

0 0 6/14

(42.9%)

EU:

5/10

(50%)

Non-EU: 1/4 (25%)

8/14

(57.1%)

EU:

5/10

(50%)

Non-

EU:

3/4

(75%)

0 0 8/14

(57.1%)

EU:

7/10

(70%)

Non-EU: 1/4 (25%)

6/14

(42.9%)

EU:

3/10

(30%)

Non-EU:

3/4

(75%)

0 0

79. WP1:

DEVCO 1st 2nd

and 3rd

Technical

Biannual Reports

11/17

(64.7%)

EU:

9/13

(69.2%)

Non-

EU:

2/4

(50%)

6/17

(35.3%)

EU:

4/13

(30.8%)

Non-

EU:

2/4

(50%)

0 0 9/16

(56.3%)

EU:

7/12

(58.3%)

Non-

EU:

2/4

(50%)

7/16

(43.8 %)

EU:

5/12

(41.7%)

Non-EU:

2/4

(50%)

0 0 8/15

(53.3%)

EU:

6/11

(54.5%)

Non-

EU:

2/4

(50%)

7/15

(46.7%)

EU:

5/11

(45.5%)

Non-

EU:

2/4

(50%)

0 0 6/15

(40%)

EU:

5/11

(45.5%)

Non-

EU: 1/4

(25%)

9/15

(60%)

EU:

6/11

(54.5%)

Non-EU:

3/4

(75%)

0 0 9/15

(60%)

EU:

7/11

(63.6%)

Non-

EU:

2/4

(50%)

6/15

(40%)

EU:

4/11

(36.4%)

Non-

EU:

2/4

(50%)

0 0 7/14

(50%)

EU:

6/11

(54.5%)

Non-EU:

1/3

(33.3%)

7/14

(50%)

EU:

5/11

(45.5%)

Non-

EU:

2/3

(66.7%)

0 0 8/13

(61.5 %)

EU:

7/10

(70%)

Non-EU:

1/3

(33.3%)

5/13

(38.5 %)

EU:

3/10

(30%)

Non-EU:

2/3

(66.7%)

0 0

80. WP2:

Dissemination

plan

10/19

(52.6%)

EU:

9/13

(69.2%)

Non-

EU:

1/6

9/19

(47.4%)

EU:

4/13

(30.8%)

Non-

EU:

5/6

0 0 9/17

(52.9%)

EU:

8/11

(72.7%)

Non-

EU:

1/6

8/17

(47.1 %)

EU:

3/11

(27.3%)

Non-EU:

5/6

(83.3%)

0 0 7/16

(43.8%)

EU:

6/10

(60%)

Non-

EU:

1/6

9/16

(56.3%)

EU:

4/10

(40%)

Non-

EU:

5/6

0 0 9/17

(52.9%)

EU:

8/11

(72.7%)

Non-

EU:

1/6

8/17

(47.1 %)

EU:

3/11

(27.3%)

Non-EU:

5/6

(83.3%)

0 0 10/17

(58.8%)

EU:

8/11

(72.7%)

Non-

EU:

2/6

7/17

(41.2%)

EU:

3/11

(27.3%)

Non-

EU:

4/6

0 0 7/15

(46.7%)

EU:

6/10

(60%)

Non-EU:

1/5

(20%)

8/15

(53.3%)

EU:

4/10

(40%)

Non-

EU:

4/5

0 0 8/13

(61.5%)

EU:

7/8

(87.5%)

Non-EU:

1/5

(20%)

5/13

(38.5%)

EU:

1/8

(12.5%)

Non-EU:

4/5

(80%)

0 0


Deliverables /

Documents

Methodology

Used

How do you rate the

deliverable

/document?




(16.7%) (83.3%) (16.7%) (16.7%)

(83.3%) (16.7%) (33.3%) (66.7%) (80%)

81. WP2:

Evaluation

survey of

website

9/19

(47.4%)

EU:

7/13

(53.8%)

Non-

EU:

2/6

(33.3%)

10/19

(52.6%)

EU:

6/13

(46.2%)

Non-

EU:

4/6

(66.7%)

0 0 9/18

(50%)

EU:

7/12

(58.3%)

Non-

EU:

2/6

(33.3%)

8/18

(44.4%)

EU:

4/12

(33.3%)

Non-EU:

4/6

(66.7%)

1/18

(5.6%)

EU:

1/12

(8.3%)

0 7/17

(41.2%)

EU:

6/11

(54.5%)

Non-

EU:

1/5

(20%)

10/17

(58.8%)

EU:

5/11

(45.5%)

Non-

EU:

5/6

(83.3%)

0 0 8/17

(47.1%)

EU:

6/11

(54.5%)

Non-

EU:

2/6

(33.3%)

8/17

(47.1%)

EU:

4/11

(36.4%)

Non-EU:

4/6

(66.7%)

1/17

(5.9%)

EU:

1/11

(9.1%)

0 7/17

(41.2%)

EU:

5/11

(45.5%)

Non-

EU:

2/6

(33.3%)

10/17

(58.8%)

EU:

6/11

(54.5%)

Non-

EU:

4/6

(66.7%)

0 0 6/15

(40 %)

EU:

5/10

(50%)

Non-EU: 1/5 (20%)

9/15

(60%)

EU:

5/10

(50%)

Non-

EU:

4/5

(80%)

0 0 6/14

(42.9 %)

EU:

5/9

(55.6%)

Non-EU: 1/5 (20%)

8/14

(57.1%)

EU:

4/9

(44.4%)

Non-EU: 4/5

(80%)

0 0

82. WP2:

Project Outline

10/19

(52.6%)

EU:

9/13

(69.2%)

Non-

EU:

1/6

(16.7%)

9/19

(47.4%)

EU:

4/13

(30.8%)

Non-

EU:

5/6

(83.3%)

0 0 9/18

(50%)

EU:

8/12

(66.7%)

Non-

EU:

1/6

(16.7%)

9/18

(50%)

EU:

4/12

(33.3%)

Non-EU:

5/6

(83.3%)

0 0 8/17

(47.1 %)

EU:

7/11

(63.6%)

Non-

EU:

1/6

(16.7%)

9/17

(52.9%)

EU:

4/11

(36.4%)

Non-

EU:

5/6

(83.3%)

0 0 8/17

(47.1

%) EU:

7/11

(63.6%)

Non-

EU:

1/6

(16.7%)

9/17

(52.9%)

EU:

4/11

(36.4%)

Non-EU:

5/6

(83.3%)

0 0 9/17

(52.9%)

EU:

7/11

(63.6%)

Non-

EU:

2/6

(33.3%)

8/17

(47.1 %)

EU:

4/11

(36.4%)

Non-

EU:

4/6

(66.7%)

0 0 8/15

(53.3%)

EU:

7/10

(63.6%)

Non-EU:

1/5 (20%)

7/15

(46.7%)

EU:

3/10

(30%)

Non-

EU: 4/5

(80%)

0 0 7/14

(50%)

EU:

6/9

(66.7%)

Non-EU:

1/5 (20%)

7/14

(50%)

EU:

3/9

(33.3%)

Non-EU:

4/5

(80%)

0 0

83. WP3:

Evaluation Plan

10/16

(62.5%)

EU:

8/11

(72.7%)

Non-

EU:

2/5

(40%)

4/16

(25%)

EU:

1/11

(9.1%)

Non-

EU:

3/6

(50%)

2/16

(12.5%)

EU:

2/11

(18.2%)

0 8/16

(50%)

EU:

6/10

(60%)

Non-

EU:

2/6

(33.3%)

6/16

(37.5%)

EU:

2/10

(20%)

Non-EU:

4/6

(66.7%)

2/16

(12.5%)

EU:

2/10

(20%)

0 6/15

(40%)

EU:

5/9

(55.6%)

Non-

EU:

1/6

(16.7%)

8/15

(53.3%)

EU:

3/9

(33.3%)

Non-

EU:

5/6

(83.3%)

1/15

(6.7%)

EU:

1/9

(11.1%)

0 7/15

(46.7%)

EU:

5/9

(55.6%)

Non-

EU:

2/6

(33.3%)

7/15

(46.7%)

EU:

3/9

(33.3%)

Non-EU:

4/6

(66.7%)

1/15

(6.7%)

EU:

1/9

(11.1%)

0 6/15

(40%)

EU:

5/9

(55.6%)

Non-

EU:

1/6

(16.7%)

7/15

(46.6%)

EU:

2/9

(22.2%)

Non-

EU:

5/6

(83.3%)

2/15

(13.3%)

EU:

2/9

(22.2%)

0 5/13

(38.5%)

EU:

4/8

(50%)

Non-EU:

1/5

(20%)

6/13

(46.2%)

EU:

2/8

(25%)

Non-

EU:

4/5

(80%)

2/13

(15.4%)

EU:

2/8

(25%)

0 6/11

(54.5%)

EU:

5/6

(83.3%)

Non-EU:

1/5

(20%)

5/11

(45.5 %)

EU:

1/6

(16.7%)

Non-EU:

4/5

(80%)

0 0

84. WP4:

Report of Lab

Experts Meeting

16/28

(57.1 %)

EU:

11/19

(57.9%)

Non-

EU:

5/9

(55.6%)

12/28

(42.9 %)

EU:

8/19

(42.1%)

Non-

EU:

4/9

(44.4%)

0 0 16/27

(59.3 %)

EU:

12/19

(63.2%)

Non-

EU:

4/8

(50%)

11/27

(40.7 %)

EU:

7/19

(36.8%)

Non-EU:

4/8

(50%)

0 15/27

(55.6 %)

EU:

10/18

(55.6%)

Non-

EU:

5/9

(55.6%)

12/27

(44.4%)

EU:

8/18

(44.4%)

Non-

EU:

4/9

(44.4%)

0 0 14/27

(51.9%)

EU:

9/18

(50%)

Non-

EU:

5/9

(55.6%)

12/27

(44.4%)

EU:

8/18

(44.4%)

Non-EU:

4/9

(44.4%)

1/27

(3.7%)

EU:

1/18

(5.6%)

0 12/27

(44.4 %)

EU:

7/18

(38.9%)

Non-EU

EU:

5/9

(55.6%)

14/27

(51.9%)

EU:

10/18

(55.6%)

Non-

EU:

4/9

(44.4%)

1/27

(3.7 %)

EU:

1/18

(5.6%)

0 12/26

(46.2%)

EU:

8/17

(47.1%)

Non-EU:

4/9

(44.4%)

13/26

(50%)

EU:

8/17

(47.1%)

Non-

EU:

5/9

(55.6%)

1/26

(3.8%)

EU:

1/17

(5.9%)

0 14/25

(56%)

EU:

9/16

(56.3%)

Non-EU:

5/9

(55.6%)

11/25

(44%)

EU:

7/16

(43.8%)

Non-EU:

4/9

(44.4%)

0 0

85. WP4:

Report of Heads

of Lab Meeting

18/31

(58.1%)

EU:

8/18

(44.4%)

Non-

EU:

10/13

(76.9%)

13/31

(41.9%)

EU:

10/18

(55.6%)

Non-

EU:

3/13

(23.1%)

0 0 19/30

(63.3%)

EU:

9/18

(50%)

Non-

EU:

10/12

(83.3%)

11/30

(36.7%)

EU:

9/18

(50%)

Non-EU:

2/12

(16.7%)

0 0 17/30

(56.7%)

EU:

9/17

(52.9%)

Non-

EU:

8/13

(61.5%)

12/30

(40 %)

EU:

7/17

(41.2%)

Non-

EU:

5/13

(38.5%)

1/30

(3.3%)

EU:

1/17

(5.9%)

0 13/29

(44.8%)

EU:

6/16

(37.5%)

Non-

EU:

7/13

(53.8%)

14/29

(48.3%)

EU:

8/16

(50%)

Non-EU:

6/13

(46.2%)

2/29

(6.9%)

EU:

2/16

(12.5%)

0 15/29

(51.7%)

EU:

5/16

(31.3%)

Non-

EU:

10/13

(76.9%)

14/29

(48.3%)

EU:

11/16

(68.8%)

Non-EU

3/13

(23.1%)

0 0 15/28

(53.6%)

EU:

7/16

(43.8%)

Non-EU:

8/12

(66.7%)

12/28

(42.9%)

EU:

8/16

(50%)

Non-

EU:

4/12

(33.3%)

1/28

(3.6%)

EU:

1/16

(6.3%)

0 15/27

(55.6%)

EU:

7/15

(46.7%)

Non-EU:

8/12

(66.7%)

11/27

(40.7%)

EU:

7/15

(46.7%)

Non-EU:

4/12

(33.3%)

1/27

(3.7%)

EU:

1/15

(6.7%)

0

86. WP5:

Questionnaire –

in depth core

capacities

assessment

(Note: to be

officially shared

in the 1st Project

Meeting, but

annexed to the

SANCO interim

Report and in

12/20

(60%)

EU:

10/16

(62.5%)

Non-

EU:

2/4

(50%)

6/20

(30%)

EU:

4/16

(25%)

Non-

EU:

2/4

(50%)

2/20

(10%)

EU:

2/16

(12.5%)

0 11/18

(61.1%)

EU:

9/14

(64.3%)

Non-

EU:

2/4

(50%)

6/18

(33.3%)

EU:

4/14

(28.6%)

Non-EU:

2/4

(50%)

1/18

(5.6%)

EU:

1/14

(7.1%)

0 9/18

(50%

EU:

8/14

(57.1%)

Non-

EU:

1/4

(25%)

8/18

(44.4%)

EU:

5/14

(35.7%)

Non-

EU:

3/4

(75%)

1/18

(5.6%)

EU:

1/14

(7.1%)

0 9/18

(50%)

EU:

7/14

(50%)

Non-

EU:

2/4

(50%)

7/18

(38.9%)

EU:

5/14

(35.7%)

Non-EU:

2/4

(50%)

2/18

(11.1%)

EU:

2/14

(14.3%)

0 9/17

(52.9%)

EU:

8/14

(57.1%)

Non-

EU:

1/3

(33.3%)

5/17

(29.4%)

EU:

3/14

(21.4%)

Non-

EU:

2/3

(66.7%)

3/17

(17.6%)

EU:

3/14

(21.4%)

0 8/17

(47.1%)

EU:

7/14

(50%)

Non-EU:

1/3

(33.3%)

8/17

(47.1%)

EU:

6/14

(42.9%)

Non-

EU:

2/3

(66.7%)

1/17

(5.9%)

EU:

1/14

(7.1%)

0 11/16

(68.8%)

EU:

10/13

(76.9%)

Non-EU:

1/3

(33.3%)

4/16

(25%)

EU:

2/13

(15.4%)

Non-EU:

2/3

(66.7%)

1/16

(6.3%)

EU:

1/13

(7.7%)

0


Deliverables /

Documents

Methodology

Used

How do you rate the

deliverable

/document?




WP5ENWA)

87. WP5:

Workshop and

capacity building

session 1

Training Material

13/22

(59.1%)

EU:

11/18

(61.1%)

Non-

EU:

2/4

(50%)

7/22

(31.8%)

EU:

6/18

(33.3%)

Non-

EU:

1/4

(25%)

2/22

(9.1%)

EU:

1/18

(5.6%)

Non-

EU:

1/4

(25%)

0 12/20

(60%)

EU:

10/16

(62.5%)

Non-

EU:

2/4

(50%)

5/20

(25%)

EU:

4/16

(25%)

Non-EU: 1/4 (25%)

3/20

(15%)

EU:

2/16

(12.5%)

Non-

EU:

1/4

(25%)

0 10/20

(50%)

EU:

8/16

(50%)

Non-

EU:

2/4

(50%)

8/20

(40%)

EU:

7/16

(43.8%)

Non-

EU:

1/4

(25%)

2/20

(10%)

EU:

1/16

(6.3%)

Non-

EU:

1/4

(25%)

0 9/19

(47.4%)

EU:

8/15

(53.3%)

Non-

EU:

1/4

(25%)

8/19

(42.1%)

EU:

6/15

(40%)

Non-EU:

2/4

(50%)

2/19

(10.5%)

EU:

1/15

(6.7%)

Non-

EU:

1/4

(25%)

0 10/20

(50%)

EU:

8/16

(50%)

Non-

EU:

2/4

(50%)

7/20

(35%)

EU:

6/16

(37.5%)

Non-

EU:

1/4

(25%)

2/20

(10%)

EU:

1/16

(6.3%)

Non-

EU:

1/4

(25%)

1/20

(5%)

EU:

1/6

(6.3%)

10/18

(55.6 %)

EU:

9/15

(60%)

Non-EU:

1/3

(33.3%)

6/18

(33.3%)

EU:

5/15

(33.3%)

Non-

EU:

1/3

(33.3%)

2/18

(11.1 %)

EU:

1/15

(6.7%)

Non-

EU:

1/3

(33.3%)

0 9/17

(52.9%)

EU:

5/14

(57.1%)

Non-EU:

1/3

(33.3%)

6/17

(35.3%)

EU:

5/14

(35.7%)

Non-EU:

1/3

(33.3%)

1/17

(5.9%)

Non-

EU:

1/3

(33.3%)

1/17

(5.9%)

EU:

1/14

(7.1%)

88. WP6:

Plan of Actions

Note: Internal:

Road Map with

ECDC – annexed

to the interim

Report

4/10

(40%)

EU:

4/9

(44.4%)

6/10

(60%)

EU:

5/9

(55.6%)

Non-

EU:

1/1

(100%)

0 0 3/9

(33.3%)

EU:

3/8

(37.5%)

4/9

(44.4%)

EU:

4/8

(50%)

Non-EU:

1/1

(100%)

2/9

(11.1%)

EU:

1/8

(12.5%)

0 3/9

(33.3%)

EU:

3/8

(37.5%)

4/9

(44.4%)

EU:

3/8

(37.5%)

Non-

EU:

1/1

(100%)

2/9

(22.2%)

EU:

2/8

(25%)

0 3/9

(33.3%)

EU:

3/8

(37.5%)

5/9

(55.6%)

EU:

4/8

(50%)

Non-EU:

1/1

(100%)

1/9

(11.1%)

EU:

1/8

(12.5%)

0 3/9

(33.3%)

EU:

3/8

(37.5%)

5/9

(55.6%)

EU:

4/8

(50%)

Non-

EU:

1/1

(100%)

1/9

(11.1%)

EU:

1/8

(12.5%)

0 4/9

(44.4%)

EU:

4/8

(50%)

5/9

(55.6%)

EU:

4/8

(50%)

Non-

EU:

1/1

(100%)

0 0 3/8

(37.5%)

EU:

3/7

(42.9%)

5/8

(62.5%)

EU:

4/7

(57.1%)

Non-EU:

1/1

(100%)

0 0

89. WP6:

SOP’s (Note: to

be officially

shared in the 1st

Project Meeting,

but annexed to

the SANCO

interim Report

and in

WP6ENWA)

9/18

(50%)

EU:

8/13

(61.5%)

Non-

EU:

1/5

(20%)

8/18

(44.4%)

EU:

4/13

(30.8%)

Non-

EU:

4/5

(80%)

1/18

(5.6%)

EU:

1/13

(7.7%)

0 8/17

(47.1%)

EU:

7/12

(58.3%)

Non-

EU:

1/5

(20%)

8/17

(47.1%)

EU:

4/12

(33.3%)

Non-EU:

4/5

(80%)

1/17

(5.9%)

EU:

1/12

(8.3%)

0 8/16

(50%)

EU:

7/11

(63.6%)

Non-

EU:

1/5

(20%)

8/16

(50%)

EU:

4/11

(36.4%)

Non-

EU:

4/5

(80%)

0 0 7/16

(43.8%)

EU:

6/11

(54.5%)

Non-

EU:

1/5

(20%)

9/16

(56.3%)

EU:

5/11

(45.5%)

Non-EU:

4/5

(80%)

0 0 8/16

(50%)

EU:

7/11

(63.6%)

Non-

EU:

1/5

(20%)

6/16

(37.5%)

EU:

2/11

(18.2%)

Non-

EU:

4/5

(80%)

1/16

(6.3%)

EU:

1/11

(9.1%)

1/16

(6.3%)

EU:

1/11

(9.1%)

6/15

(40%)

EU:

5/10

(50%)

Non-EU:

1/5

(20%)

7/15

(46.7%)

EU:

3/10

(30%)

Non-

EU:

4/5

(80%)

2/15

(13.3%)

EU:

2/10

(20%)

0 7/14

(50%)

EU:

6/9

(66.7%)

Non-EU:

1/5

(20%)

6/14

(42.9%)

EU:

2/9

(22.2%)

Non-EU:

4/5

(80%)

1/14

(7.1%)

EU:

1/9

(11.1%)

0

90. WP6:

Evaluation of

secure platform

13/18

(72.2%)

EU:

11/14

(78.6%)

Non-

EU:

2/5

(50%)

4/18

(22.2%)

EU:

2/14

(14.3%)

Non-

EU:

2/5

(50%)

1/18

(5.6%)

EU:

1/14

(7.1%)

0 10/16

(62.5%)

EU:

9/12

(75%)

Non-

EU: 1/4

(25%)

5/16

(31.3%)

EU:

2/12

(16.7%)

Non-EU:

3/4

(75%)

1/16

(6.3%)

EU:

1/12

(8.3%)

0 12/16

(75%)

EU:

10/12

(83.3%)

Non-

EU:

2/4

(50%)

3/16

(18.8%)

EU:

1/12

(8.3%)

Non-

EU:

2/4

(50%)

1/16

(6.3%)

EU:

1/12

(8.3%)

0 10/16

(62.5%)

EU:

9/12

(75%)

Non-

EU: 1/4

(25%)

5/16

(31.3%)

EU:

2/12

(16.7%)

Non-EU:

3/4

(75%)

1/16

(6.3%)

EU:

1/12

(8.3%)

0 12/16

(75%)

EU:

10/12

(83.3%)

Non-

EU:

2/4

(50%)

3/16

(18.8%)

EU:

1/12

(8.3%)

Non-

EU:

2/4

(50%)

1/16

(6.3%)

EU:

1/12

(8.3%)

0 10/15

(66.7%)

EU:

8/11

(72.7%)

Non-EU:

2/4

(50%)

4/15

(26.7%)

EU:

2/11

(18.2%)

Non-

EU:

2/4

(50%)

1/15

(6.7%)

EU:

1/11

(9.1%)

0 10/14

(71.4%)

EU:

8/10

(80%)

Non-EU:

2/4

(50%)

4/14

(28.6%)

EU:

2/10

(20%)

Non-EU:

2/4

(50%)

0 0

91. WP7:

Final report of

1st investigation

Note: Shared

only with the

WP7ST members

- annexed to the

interim Report

4/12

(33.3%)

EU:

4/11

(36.4%)

7/12

(58.3%)

EU:

6/11

(54.5%)

Non-

EU:

1/1

(100%)

1/12

(8.3%)

EU:

1/11

(9.1%)

0 4/11

(36.4%)

EU:

4/10

(40%)

6/11

(54.5%)

EU:

5/10

(50%)

Non-EU:

1/1

(100%)

1/11

(9.1%)

EU:

1/10

(10%)

0 3/11

(27.3%)

EU:

2/10

(20%)

Non-

EU:

1/1

(100%)

7/11

(63.6%)

EU:

7/10

(70%)

1/11

(9.1%)

EU:

1/10

(10%)

0 3/11

(27.3%)

EU:

3/10

(30%)

7/11

(63.6%)

EU:

6/10

(60%)

Non-EU:

1/1

(100%)

1/11

(9.1%)

EU:

1/10

(10%)

0 4/11

(36.4%)

EU:

4/10

(40%)

6/11

(54.5%)

EU:

5/10

(50%)

Non-

EU:

1/1

(100%)

1/11

(9.1%)

EU:

1/10

(10%)

0 4/11

(36.4%)

EU:

4/10

(40%)

5/11

(45.5%)

EU:

4/10

(40%)

Non-

EU:

1/1

(100%)

2/11

(18.2%)

EU:

2/10

(20%)

0 5/10

(50%)

EU:

4/9

(44.4%)

Non-EU:

1/1

(100%)

5/10

(50%)

EU:

5/9

(55.6%)

0 0

92. WP7:

Report from

meeting in

Rome, July 2011

2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-

EU:

1/1

(100%)

0 0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-EU:

1/1

(100%)

0 0 3/6

(50%)

EU:

2/5

(40%)

Non-

EU:

1/1

(100%)

2/6

(33.3%)

EU:

2/5

(40%)

1/6

(16.7%)

EU:

1/5

(20%)

0 2/6

(33.3%)

EU:

2/5

(40%)

2/6

(33.3%)

EU:

2/5

(40%)

2/6

(33.3%)

EU:

1/5

(20%)

Non-

EU:

1/1

(100%)

0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-

EU:

1/1

(100%)

0 0 2/6

(33.3%)

EU:

2/5

(40%)

3/6

(50%)

EU:

3/5

(60%)

1/6

(16.7%)

Non-

EU:

1/1

(100%)

0 2/6

(33.3%)

EU:

2/5

(40%)

4/6

(66.7%)

EU:

3/5

(60%)

Non-EU:

1/1

(100%)

0 0

93. WP7:

Final report of in

10/20

(50%)

9/20

(45%)

1/20

(5%)

0 9/19

(47.4%)

9/19

(47.4%)

1/19

(5.3%)

0 7/17

(41.2%)

10/17

(58.8%)

0 0 7/17

(41.2%)

10/17

(58.8%)

0 0 9/18

(50%)

8/18

(44.4%)

1/18

(5.6%)

0 6/17

(35.3%)

10/17

(58.8%)

1/17

(5.9%)

0 9/16

(56.3%)

7/16

(43.8%)

0 0


Deliverables /

Documents

Methodology

Used

How do you rate the

deliverable

/document?




depth analysis of

priority area

(literature

review and

analysis of

existing

monitoring

frameworks)

(NOTE: Shared

only with the

WP7ST members

- annexed to the

interim Report &

website)

EU:

9/16

(56.3%)

Non-

EU: 1/4

(25%)

EU:

6/16

(37.5%)

Non-

EU:

3/4

(75%)

EU:

1/16

(6.3%)

EU:

8/15

(53.3%)

Non-

EU: 1/4

(25%)

EU:

6/15

(40%)

Non-EU:

3/4

(75%)

EU:

1/15

(6.7%)

EU:

6/13

(46.2%)

Non-

EU: 1/4

(25%)

EU:

7/13

(53.8%)

Non-

EU:

3/4

(75%)

EU:

6/13

(46.2%)

Non-

EU: 1/4

(25%)

EU:

7/13

(53.8%)

Non-EU:

3/4

(75%)

EU:

8/14

(57.1%)

Non-

EU: 1/4

(25%)

EU:

5/14

(35.7%)

Non-

EU:

3/4

(75%)

EU:

1/14

(7.1%)

EU:

5/13

(38.5%)

Non-EU:

1/4 (25%)

EU:

7/13

(53.8%)

Non-

EU:

3/4

(75%)

EU:

1/13

(7.7%)

EU:

8/13

(61.5%)

Non-EU:

1/3

(33.3%)

EU:

5/13

(38.5%)

Non-EU:

2/3

(66.7%)

* Clarity: free from obscurity and easy to understand **Consistency: logical coherence and accordance with the facts ***Completeness: complete and entire; having everything that is needed

*E: Excellent S: Satisfactory RS: Relatively Satisfactory RI: Requires Improvement


3.8 Meetings/Working Groups Evaluation

3.8.1 Previous Evaluation of the 1st WP7 Steering Team meeting

An evaluation of the 1st

Work Package 7 Steering Team Meeting was conducted. 14 out of the 17 participants completed the questionnaire. The meeting was rated as successful by 85% of the participants. 83% thought enough

information was available to identify the capacities to address as priority. A total of 77% agreed that the method adopted favoured the development of a wide sense of ownership of the results obtained; no participant felt this sense of

ownership was not reached altogether.


In Table 14 the results of the evaluation of the EpiSouth Plus Meetings conducted from the period of October 2010 to September 2012 are presented.

Table 14: Evaluation Questionnaire 1: Meetings and Workshops

The meeting’s usefulness and effectiveness

for the development of the project’s

processes

Material for discussion disseminated

contributed to the achievement of the

project objectives

Quality of the material

disseminated for discussion

How do you rate the coordination of

discussion?

How do you rate the quality of the

meetings report (did it include all

issues raised during the meeting)?

E* S* RS* RI* E S RS RI E S RS RI E S RS RI E S RS RI

94. 1st

Steering Committee

(WP1)

4/7 (57.1%) EU: 3/6 (50%)

Non-EU: 1/1 (100%)

2/7 (28.6%) EU: 2/6 (33.3%)

1/7 (14.3%) EU: 1/6 (16.7%)

0

3/7 (42.9%) EU: 2/6 (33.3%)

Non-EU:

1/1 (100%)

4/7 (57.1%) EU: 4/6(66.7%)

0 0

6/7 (85.7%) EU: 5/6 (83.3%)

Non-EU:

1/1 (100%)

1/7 (14.3%) EU: 1/6 (16.7%)

0 0

4/7 (57.1%) EU: 3/6 (50%)

Non-EU:

1/1 (100%)

3/7 (42.9%) EU: 3/6 (50%)

0 0

6/7 (85.7%) EU: 5/6 (83.3%)

Non-EU:

1/1 (100%)

1/7 (14.3%) EU: 1/6 (16.7%)

0 0

95. 2nd

Steering Committee

/Advisory Board

Meeting (WP1) **

3/5 (60%)

2/5 (40%)

0 0

2/5 (40%)

3/5 (60%)

0 0

4/5 (80%)

1/5 (20%)

0 0

2/5 (40%)

3/5 (60%)

0 0

3/5 (60%)

2/5 (40%)

0 0

96. WP4

Meeting of expert

committee - January

2012

4/7 (57.1%) EU: 3/5 (60%)

Non-EU:

1/2 (50%)

3/7 (42.9%) EU: 2/5 (40%)

Non-EU:

1/2 (50%)

0 0

4/7 (57.1%) EU: 3/5 (60%)

Non-EU:

1/2 (50%)

3/7 (42.9%) EU: 2/5 (40%)

Non-EU:

1/2 (50%)

0 0

3/6 (50%) EU: 2/4 (50%)

Non-EU:

1/2 (50%)

3/6(50%) EU: 2/4 (50%)

Non-EU:

1/2 (50%)

0 0

4/7 (57.1%) EU: 3/5 (60%)

Non-EU:

1/2 (50%)

2/7 (28.6%) EU: 1/5 (20%)

Non-EU:

1/2 (50%)

1/7 (14.3%) EU: 1/5 (20%)

0

5/7 (71.4%) EU: 4/5 (80%)

Non-EU:

1/2 (50%)

2/7 (28.6%) EU: 1/5 (20%)

Non-EU:

1/2 (50%)

0 0

97. WP4

Meeting with heads

of laboratories -

March 2012

11/15 (73.3%) EU: 3/7 (42.9%)

Non-EU: 8/8

(100%)

3/15 (20%) EU: 3/7 (42.9%)

1/15 (6.7%) EU: 1/7 (14.3%)

0

9/15 (60%) EU: 2/7 (28.6%)

Non-EU:

7/8 (87.5%)

5/15 (33.3%) EU: 4/7 (57.1%)

Non-EU:

1/8 (12.5%)

1/15 (6.7%) EU: 1/7 (14.3%)

0

9/14(64.3%) EU: 2/6 (33.3%)

Non-EU:

7/8 (87.5%)

5/14(35.7%) EU: 4/6 (66.7%)

Non-EU:

1/8 (12.5%)

0 0

9/14

(64.3%) EU: 3/7

(42.9%)

Non-EU: 6/7

(85.7%)

3/14 (21.4%) EU: 2/7

(28.6%)

Non-EU:

1/7 (14.3%)

2/14

(14.3%) EU: 2/7

(28.6%)

0

11/15 (73.3%) EU: 3/7 (42.9%)

Non-EU:

8/8 (100%)

4/15(26.7%) EU: 4/7(57.1%)

0 0

98. WP5:

Workshop and

capacity building

session 1 **

1/1 (100%)

0 0 0

1/1 (100%)

0/1 (0%)

0 0

1/1 (100%)

0/1(0%)

0 0

1/1 (100%)

0 0 0

1/1 (100%)

0 0 0

99. WP6 ST

Meeting in

Montenegro (2011)

**

2/3 (66.7%)

1/3 (33.3%)

0 0

3/3 (100%)

0/3 (0%) 0 0

1/2 (50%)

1/2(50%)

0 0

1/2 (50%)

1/2 (50%)

0 0

1/2 (50%)

0 0

1/2 (50%)

100. WP6 ST

Meeting in Turkey

(2011)

3/4 (75%) EU: 2/3 (66.7%)

Non-EU:

1/1 (100%)

0

1/4 (25%) EU: 1/3 (33.3%

0

3/4 (75%) EU:2/3 (66.7%)

Non –EU:

1/1 (100%)

1/4 (25%) EU: 1/3 (33.3%)

0 0

2/3 (66.7%) EU: 1 /2 (50%)

Non-EU:

1/1 (100%)

1/3(33.3%) EU: 1/2 (50%)

0 0

3/3

(100%) EU: 2/2

(100%)

Non-EU:

1/1 (100%)

0 0 0

2/3 (66.7%) EU: 1/2(50%)

Non-EU:

1/1 (100%)

0 0

1/3

(33.3%) EU: 1/2

(50%)

101. WP7:

Meeting in Rome,

July 2011

2/5 (40%) EU: 2/4 (50%)

3/5 (60%) EU: 2/4 (50%)

Non-EU:

1/1 (100%)

0 0

2/5 (40%) EU: 2/24 (50%)

3/5 (60%)

EU: 2/4 (50%) Non-EU: 1/1

(100%)

0 0

2/5 (40%)

EU: 2/4 (50%)

3/5(60%)

EU: 2/4 (50%) Non-EU: 1/1

(100%)

0 0

3/5

(60%)

EU: 2/4

(50%) Non-EU: 1/1

(100%)

2/5 (40%)

EU: 2/4

(50%)

0 0

3/5 (60%) EU:3 /4 (75%)

2/5 (40%) EU: 1/4 (25%)

Non-EU: 1/1

(100%)

0 0


102. WP7:

Meeting in Lyon, July

2012

4/5 (80%) EU: 3/ 4 (75%)

Non-EU: 1/1

(100%)

1/5 (20%) EU: 1/4 (25%)

0 0

2/5 (40%) EU: 1/4 (25%)

Non-EU: 1/1

(100%)

3/5 (60%) EU: 3/4 (75%)

0 0

3/5(60%) EU: 2/4 (50%)

Non-EU:

1/1 (100%)

2/5 (40%) EU: 2/4 (50%)

0 0

4/5

(60%) EU: 3/4 (75%)

Non-EU:

1/1 (100%)

1/5 (40%) EU: 1/4 (25%)

0 0

2/5 (80%) EU: 2/4 (50%)

3/5 (20%) EU: 2/4 (50%)

Non-EU:

1/1 (100%)

0 0

*E: Excellent S: Satisfactory RS: Relatively Satisfactory RI: Requires Improvement

** Only EU responders

3.8.3 Comments by the responders:

�� The meeting was very useful provided very important feedback from countries and provides NFP a real opportunity to express their needs expectation and concerns

�� Too much time spent on discussing issues that were not directly related o the implementation of the activities and the EpiSouth itself .The 1st SC meeting was especially difficult do to the time spent on re-discussing issues that were

supposed to be agreed when grant was signed, lot of discussion between the different representative of the EU institutions to the detriment of the project

3.9 Network Consolidation Evaluation

3.9.1 Previous evaluation report

An evaluation of the WP6 – Cross border epidemic intelligence was conducted. Twenty two

persons out of 54 potential have answered the questionnaire (40% response rate). Seven of

the 22 responders (32%) were from Non –EU countries.

Main findings:

� 95% of the respondents consider that the access to the secure platform is easy

(password)

� 65% of the respondents think it is easy enough to sort, search an alert, and 27% think

it is very easy

� 73% think there are enough alerts/posting on the platform

� 54.5% think that the information of the posted alerts is relevant

� 50% think that the content/data provided in the alert are of good quality, 45.5% think

it is of excellent quality

� 68% think that the quality of the analysis of the information is good, 23% think it is

excellent.

� 54% think that the information is usually provided at the same time as other similar

sources of information

� 41% think the timeliness is very good, the information is usually provided in advance

compared to other similar sources of information

� 5% think timeliness is not good, the information is provided with too much delay

� 64% of the respondents usually transfer the information to key stakeholders

� 82% think that, on average, the posted alerts were useful for their institution or

country

� 41% always consult the alert, 23% do it frequently, and 32% consult sometimes the

alerts

� For 77% of the respondents, they usually use only the email to consult the alert (due

to problems of connection to the secure area, or because it is easier and/or faster),


and for 23%, they use both the email and the secured area (one mentioned that the

summary in the email was not readable enough)

� 45% have never posted an alert

� 36% think it is easy enough to post an alert, and 12% think it is very easy

� 46% of the respondents have a direct access to their national alerts and are able to

post them while 27% have not. The remaining respondents (27%) had no opinion on

the question. Most of the respondents who answered no to the question specified

that they do have access but are not (always) allowed posting alerts.

� 45% said they would need an EpiSouth criteria procedure or a formal EpiSouth

manual to help in posting alerts, against 32% who would not need

� 82% of the respondents think that the layout of the platform is very good and that

there is no need to change anything and 68% think the same thing regarding the

colour of the platform.

3.9.2 Evaluation of the EpiSouth Network

In Table 15 the evaluation of the Episouth Network by the partnership is presented. Around

80% (26/32) of the responders believes that the EPISOUTH Network is fully facilitating the

exchange of alerts and health information. Eighteen out of 30 (60%) responders believe that

the EPISOUTH Network has fully enhanced the coordinated response to public health events

in the Mediterranean area whereas 40% (12/30) believe that this is partially achieved. About

56% (18/32) believe that the EPISOUTH Network is fully contributing in the strengthening of

Mediterranean countries capacities building whereas about 40% (13/32) responded that this

is only partially achieved. Almost 80% (26/32) of the responders believe that the EPISOUTH

Network fully works towards building reliable and collaborative relationships among public

health professionals. Eighteen out of 30 (60%) of the responders believe that the EPISOUTH

Network has only partially contributed to the development of interoperability with other

early warning systems and about 62% (18/29) that only partially has enhanced their capacity

to better respond to alerts. It should be noted that a higher percentage of EU responders

rated that the EpiSouth Network has partially enhanced their capacity to better respond to

alerts (14/19, 73.7%) in comparison to Non-EU responders (4/10, 40%).


The enhancement of their capacity for communicable disease surveillance and response has

only been achieved fully according to 48.4% (15/31) of the responders whereas the same

percentage of 48.4% (15/31) believes it was achieved partially. Responders feel the same for

the capacity of the Network to strengthen their laboratory capacity in terms of diagnoses

procedure (50% fully and 45.8% partially). It should be noted that a higher percentage of Non

EU countries (7/10, 70%) believe that the network will partially strengthen their capacity of

the Mediterranean region in terms of diagnoses procedure in comparison to the EU countries

(8/21, 38.1%) (p-value: 0.10). About 53% (17/32) of the responders believed that the

network has fully used their organisations strengths and expertise in implementing its

activities and almost 44% (14/32) that it was used only partially.

Table 15: Evaluation Questionnaire 2: EpiSouth Network

Fully Partially Not at all

103. Do you think that the EPISOUTH Network is

facilitating the exchange of alerts and health

information?

26/32 (81.3%) EU: 18/22 (81.8%)

Non-EU: 8/10 (80%)

6/32 (18.8%) EU: 4/22 (18.2%)

Non-EU: 2/10 (20%) 0/32 (0%)

104. Do you think that the EPISOUTH Network

has enhanced the coordinated response to public

health events in the Mediterranean Area?

18/30 (60%) EU: 11/20 (55%)

Non-EU: 7/10 (70%)

12/30 (40%) EU: 9/20(45%)

Non-EU: 3/10 (30%) 0/30 (0%)

105. Do you think that that the EPISOUTH

Network is contributing in the strengthening of

Mediterranean countries capacities building?

18/32 (56.3%) EU: 13/22 (59.1%)

Non-EU: 5/10 (50%)

13/32 (40.6%) EU: 8/22 (36.4%)

Non-EU: 5/10 (50%)

1/32 (3.1%) EU: 1/22 (4.5%)


work towards building reliable and collaborative

relationships among public health professionals?

26/32 (81.3%) EU: 19/22 (86.4%)

Non-EU: 7/10 (70%)

6/32 (18.8%) EU: 3/22 (13.6%)

Non-EU: 3/10 (30%)

0/32 (0%)


has contributed to the development of

interoperability with other early warning systems?

11/30 (36.7%) EU: 8/20 (40%)

Non-EU: 3/10 (30%)

18/30 (60%) EU: 11/20 (55%)

Non-EU: 7/10 (70%)

1/30 (3.3%) EU: 1/20 (5%)

Non-EU: 0/10 (0%)


has enhanced your capacity to better respond to

alerts?

11/29 (37.9%) EU: 5/19 (26.3%)

Non-EU: 6/10 (60%)

18/29 (62.1%) EU: 14/19 (73.7%)

Non-EU: 4/10 (40%)

0/29 (0%)


has enhanced your capacity for communicable

disease surveillance and response?

12/29 (41.4%) EU: 7/19 (36.8%)

Non-EU: 5/10 (50%)

16/29 (55.2%) EU: 12/19 (63.2%)

Non-EU: 4/10(40%)

1/29 (3.4%) EU: 0/19 (0%)

Non-EU: 1/10

(10%)


will strengthen the laboratory capacities of the

Mediterranean region in terms of diagnosis and

procedures?

15/31 (48.4%) EU: 12/21 (57.1%)

Non-EU:3/10 (30%)

15/31 (48.4%) EU: 8/21 (38.1%)*

Non-EU: 7/10 (70%)*

1/31 (3.2%) EU: 1/21(4.8%)

Non-EU: 0/10 (0%)


has used your organisation’s strengths and expertise

in implementing its activities?

17/32 (53.1%) EU: 13/22 (59.1%)

Non-EU:4/10 (40%)

14/32 (43.8%) EU: 9/22 (40.9%)

Non-EU:5/10 (50%)

1/32 (3.1%) EU: 0/22 (0%)

Non-EU: 1/10

(10%)

*p-value: 0.1

3.9.3 Evaluation of the EpiSouth Platform

In Table 16 the evaluation of the Episouth platform by the partnership is presented. The

functionality of the platform was rated as excellent by almost 44% (13/30) of the responders

whereas 50% (15/30) rated it as satisfactory. The usefulness and effectiveness of the

platform was rated as excellent by almost 54% (16/30) and as satisfactory by 40% (33.3). The

accessibility was rated as excellent by almost 50% (15/30) and as satisfactory by 10% (10/30)


whereas 13.3% (4/30) rated it as relatively satisfactorily. One out of 30 (3.3%) commended

that it requires improvement. Regarding time response, 43.3% (13/30) rated it as excellent

and almost 47% (14/30) as satisfactory. The user friendliness of the platform was rated as

excellent by 40% (12/30) and as satisfactory by 50% (15/30). More than 43% (13/30)

believed that the navigation mechanism was excellent whereas 30% (9/30) thought that it

was satisfactory and another 20% (6/30) that it was relatively satisfactory. The quality of the

content of the alerts was rated as excellent by >53% (16/30) from the responders and as

satisfactory by 40% (12/30). Finally regarding the quality of the analysis of the information

50% (15/30) of the responders believe that is excellent and almost 37% (11/30) rated it as

satisfactory. From the results in Table 16 is shown that EU responders rated with a higher

percentage the functionality, usefulness, accessibility, time response, user friendliness and

quality of the analysis of the information in contrast with the Non EU that rated these as

satisfactory.

Table 16: Evaluation Questionnaire 2: EpiSouth Platform

System/ Platform Excellent Satisfactory Relatively

Satisfactory

Requires

Improvement

112. Functionality 13/30 (43.3%)

EU: 11/21 (52.4%)

Non-EU:

2/9 (22.2%)

15/30 (50%)

EU: 8/21 (38.1%)

Non-EU:

7/9 (77.8%)

2/30 (6.7%)

EU: 2/21 (9.5%)

Non-EU: 0/9 (%)

0/30 (0%)

113. Usefulness/Effectiveness

16/30 (53.3%)

EU: 12/21(57.1%)

Non-EU:

4/9 (44.4%)

12/30 (40%)

EU: 7/21 (33.3%)

Non-EU:

5/9 (55.6%)

1/30 (3.3%)

EU: 1/21 (4.8%)

Non-EU: 0/9 (%)

1/30 (3.3%)

EU: 1/21 (4.8%)

Non-EU: 0/9

(%)

114. Accessibility 15/30 (50%)

EU: 11/21 (52.4%)

Non-EU:

4/9 (44.4%)

10/30 (33.3%)

EU: 6/21 (28.6%)

Non-EU:

4/9 (44.4%)

4/30 (13.3%)

EU: 3/21 (14.3%)

Non-EU:

1/9 (11.1%)

1/30 (3.3%)

EU: 1/21 (4.8%)

Non-EU: 0/9

(%)

115. Time response 13/30 (43.3%)

EU: 10/21 (47.6%)

Non-EU:3/9 (33.3%)

14/30 (46.7%)

EU: 8/21 (38.1%)

Non-EU:

6/9 (66.7%)

3/30 (10%)

EU: 3/21 (14.3%)

Non-EU: 0/9 (%)

0/30 (0%)

116. User friendly

12/30 (40%)

EU: 10/21 (47.6%)

Non-EU: 2/9 (22.2%)

15/30 (50%)

EU: 8/21 (38.1%)

Non-EU:

7/9 (77.8%)

2/30 (6.7%)

EU: 2/21 (9.5%

Non-EU: 0/9 (%))

1/30 (3.3%)

EU: 1/21 (4.8%)

Non-EU: 0/9

(%)

117. Easy navigation

mechanism 13/30 (43.3%)

EU: 9/21 (42.9%)

Non-EU:

4/9 (44.4%)

9/30 (30%)

EU: 6/21 (28.6%)

Non-EU:

3/9 (33.3%)

6/30 (20%)

EU: 4/21 (19%)

Non-EU:

2/9 (22.2%)

2/30 (6.7%)

EU: 2/21 (9.5%)

Non-EU: 0/9

(%)

118. Quality of the content of

the Alerts 16/30 (53.3%)

EU: 10/21 (47.6%)

Non-EU: 6/9 (66.7%)

12/30 (40%)

EU: 10/21 (47.6%)

Non-EU:

2/9 (22.2%)

2/30 (6.7%)

EU: 1/21 (4.8%)

Non-EU:

1/9 (11.1%)

0/30 (0%)

119. Quality of the analysis of

the information 15/30 (50%)

EU: 11/21 (52.4%)

Non-EU: 4/9 (44.4%)

11/30 (36.7%)

EU: 7/21 (33.3%)

Non-EU:

4/9 (44.4%)

4/30 (13.3%)

EU: 3/21 (14.3%)

Non-EU: 1/9

(11.1%)

0/30(0%)


3.9.4 Comments by the responders regarding different questions presented in Tables 15

and 16

� Question 103: This is a unique network that needs to be maintained. It has became a

very important source of information for potential health threat in the area

� Question 104: This WAS NOT the objective of EpiSouth.

� Question 105: It did contribute to capacity building through meetings training and

exchanges. Nevertheless, resources (financial and human resources) were far too

limited to allow considering a major and sustainable impact.

� Question 107: It must be continued with Non EU networks and WHO

� Question 110: It is really too early to ask the question whether the network has

strengthen the laboratory capacities of the Mediterranean region in terms of

diagnosis and procedures although lab activities implemented were highly valued

� EpiSouth is a complex and ambitious network that has proven its usefulness, its

pertinence and that actually contribute to reinforce the health security in the area. Its

composition (1/3 EU countries 2/3 non-EU countries and the sources of funding i.e.

mainly EU commission) is its main strength. The lack of long term perspective (i.e.

sustainability) is an obstacle to adequate problem solving elaboration of adequate

governance and further development. Nevertheless, despite all these difficulties

EpiSouth is a very successful network that have been able to cope with very complex

issues including (political instability (eg Arab Spring) that has been able to build trust

among participating countries (and the EpiSouth Teams). Throughout the year a

genuine EpiSouth identity has emerged. Countries are more and more participating

(at least in the health information exchanges) and these achievements must be

secured for the long term.

� Information provided by countries has so far always been very useful and appropriate

� As Focal point for EpiSouth, EWRS, and IHR I do find the platform simple easy to use.

Also as Focal point for EpiSouth, EWRS, and IHR I am fully entitled o “judge” the

pertinence and usefulness of the alerts shared by the other countries and indeed this

information is very useful for my department in our activity (not EpiSouth related) but


more widely for my institute and the MoH of my country and other public health

agency in my countries (just to mention one as an example: blood transfusion agency)

� It would need more time to put into place capacity building adapted to the different

level of countries system of response and it would be more important to have time to

work together for building up relevant common procedures

� We have not used it, so we cannot answer

� Constant difficulties for accessing to the NWA and constant ask for change the

password must be solved

� Find documents require much time than expected. In countries where the connection

is worst this could be a real threat

� Putting a short summary would improve notably

� It is not yet clear to me how to enter data on e/g/ number of cases of WNV, how

often (weekly, periodically), how do I guarantee that no other will use this data

without …my permission?, who is the person to contact for such entries?…Can you

resend the user name for this?


44 EEppiiSSoouutthh PPlluuss SSWWOOTT aannaallyyssiiss In the following section the results of the SWOT analysis are presented as these were

identified by all responders of the questionnaires and interviews.

Strengths

� The partnership:

o Arabic and Mediterranean

Countries that would not have any

other opportunity to meet

o Bringing the cross border

dimension between EU and Non

EU countries

o Countries sharing the same

environment but do not belong in

the same WHO region and do not

have the opportunity to work

together.

� Valuable Network -

o All work packages have their

expertise

� Good opportunities to share

information/problems between

countries - Countries have the

opportunity to learn from each other.

� Trustworthiness within the network

o Good spirit.

o A network of friends and not only

of professionals.

o Scientists and Epidemiologists

sharing the same vision/ goals.

� The commitments.

� The vision of the project and the

unique objectives.

o New subjects and issues addressed

that are very important for the

countries (IHR, coordination of

information etc).

� The absence of political differences.

� Good coordination.

� User-friendly website- member’s area

� The WP4 activities-development of

laboratory network

� The WP5 activities

� The WP6 activities – Epidemic

Intelligence - The timely reliable alerts.

� Non EU countries are very committed

� Voluntary participation

Weaknesses/Limitations

� Sharing of information between North

and South. Southern countries send

information but they do not receive

information from Northern countries

due to EU legislation constrains.

� It is financially dependant.

� Many members which raises many and

different obligations.

� Difficult to agree between so many

countries.

� Duplication - The information shared is

data that already circulate through other

networks.

� The reporting must meet the project

objectives together with the objectives of

the funding organisations which is not

always the case.

� A weak link is the EU MS – the directors

– the fact that collaboration only is at

central/ high level at the laboratory

network.

� No real specific and measurable

objectives. Confusion between methods

and objectives.

� Added value of the network has not been

clearly identified and especially for EU

countries.

� The participation is difficult sometimes

due to problems like the Arab Spring.

� Unequal resources

� Need to have more meetings and eye to

eye discussion

� Voluntary participation

� Topic is very broad

� Language issue

� Weak on the surveillance – the platform

is not used – low information on it

� Not enough information sharing

� NFP not authorised to share information

� EpiSouth does not have the expertise to

work on IHR issues and especially on

Points of Entry.


Opportunities

� Enhance Information Sharing:

o Facilitate information between North and

South

o Facilitate the experience to share

information between the laboratories.

o Improve direct collaboration between

laboratories.

o Enhance the exchange of information

through the platform (WP6) which is very

well developed. Not yet fully used.

� Find advocates for the network and make the

network known.

� More active participation from the side of the

members as the project output is for their

protection. Active participation through

immediate notification of any threat

encountered and not to be influenced by

political critic or internal blames.

� Bring the project closer to the countries – have

meeting in their locations and facilitate the

attendance of a larger number of that and

neighbouring countries.

� Self funding is an important step for progress.

� Identify the added value of the network.

� Look at the gaps identified by the countries

and see how the network can be mobilised.

� Expand to other southern countries (Sudan

etc).

� EpiSouth currently has a bottom up approach

on technical issues. However the opposite, a

top to bottom approach should also exist.

� Link with existing international organisations.

� Use what it was achieved in the past and

prepare new projects - A network of more

specific networks - Create smaller project

teams, not all countries need to be involved in

all WPs - Should be a FLEXIBLE – INFORMAL

network.

� Partner more with bigger institutions -

enhance further WHO collaboration.

� Find the network’s part in the world of

preparedness and response.

� Improve coordination.

� Sticking to the timetable.

� Deliver what was agreed.

� Other issues to address are immigration and

zoonoses.

Threats

� Major threat is the

sustainability of funding

� Dispute with other bodies

like IHR and ECDC and WHO

should be resolved.

� The burden of the countries

in North Africa and Middle

East with volatile situations

which is beyond control of

the project.

� Exchange data and provide

further authorisation for

data sharing.

� Countries are not willing to

share – political instabilities

delaying the exchange of

experience.

� The lack of motivation.

� Avoid duplication.

� Financial constrains.

� Time consuming.

� Low participation.

� The partnership should note

that they might find

competitors that might have

economic goals.

� Completeness.

� Not clear that the

deliverables will be used by

all countries


55 SSuussttaaiinnaabbiilliittyy In the question regarding which activities implemented so far did they believe should be

sustainable, the interviewees answered the following:

� WP6- The epidemic intelligence system (8 interviewees) followed by printable guide

to manage such alerts (1 interviewee)

� The regional laboratory networking (4 interviewees)

� The training activities (3 interviewees)

� The WP7 Points of Entry (3 interviewees): The promotion of the implementation of

the PoE guidelines: A possibility to be used by MS to improve the ability of PoE to

respond is something that all countries are interested in. Regarding the PoE countries

will collaborate easier because they are all very interested in this issue. These

activities will contribute to future activities of WHO

� The Network of Epidemiologists (2 interviewee)

� The coordination and active communications with similar projects (1 interviewee)

� EpiSouth should build on bridges between health and agriculture (1 interviewee)

� Sharing of information

� Translate key documents in key languages

In the question regarding what are the means that the partnership should use in order to

become sustainable, the interviewees responded the following:

• Sustainability should come through the countries (4 interviewees).

• Financial support from international organisations or the EC or from Arab

countries league or EUROMED collaboration (2 interviewees).

• Avoid duplications (2 interviewees).

• WHO and ECDC should be on board. Southern countries will not act if not in

agreement with WHO (2 interviewees).

• G7 and Mexico Network was also approached and they showed an interest in

EPISOUTH.

• Advocacy.


• Define the objectives and the vision.

• Define surveillance at PoE, identify action and countries that have the expertise to

assist.

• The network is a tool but need to define its activities.

• Different structure of the project in order to become sustainable, create more

concrete projects The network should be a political network - Enhance the

segmentation that currently exists and become a Network of Networks (3

interviewees)

• The project should have a political role and work to create a network structure

like EUROMED has.

• Motivate countries to use the platform

• Create a small project and network

• Key is to know how to prepare a successful proposal and to build on existing

achievements.

• WP5 training activities should become sustainable ECDC and DENCO aim to use

the work of EpiSouth.

• Funding should not be an issue because there are technical available tools to be

used

• To be voluntary and to engage more the Member States

• Find the best way to enhance the border surveillance

• Establish technical assistance in relation to IHR

• EpiSouth should partner with other networks

• There is no need for WP7 since this is work covered by WHO

• WP6 and the Epidemic Intelligence system should go under ECDC

• The only viable is WP4 : they should define the pathogens (since ECDC already

covers some)

• Could become eventually a laboratory network coordinated by ECDC


66 AAddddeedd VVaalluuee ooff tthhee EEppiiSSoouutthh PPrroojjeecctt In the following section the added value of the WP7 activities are presented as these were

identified by the WP 7 Leader. It is highly recommended to all Work Package Leaders and Co-

leaders to prepare a summary like the one presented below of the added value of the

activities of their WPs. The main objective is to promote the dissemination of the added

value documentation to the stakeholders including EU and international organisations in

order to increase the visibility and recognition of the project.

6.1 Added value of the collaboration between the EpiSouth Network and

WHO in the framework of the EpiSouth Plus WP7

Action undertaken Output Added value Level of

implementation

Analysis of WHO

data for the

EpiSouth Region

(disaggregation

not available

before)

Report on the

level of

implementati

on of IHR in

the EpiSouth

Region

Knowledge sharing

This data was not available

before, need for it was

expressed by countries and

the collaboration between

WHO and the EpiSouth

network made the

knowledge sharing possible

Completed

Prioritization of

capacity

development

needs in the

Mediterranean

Two ST

meetings

Mutual learning

Countries presented level of

implementation of IHR in

their national settings and

contributed in the definition

of the need for and contents

of the WHO guidance on the

priority area selected through

an enlarged consensus

process.

Completed

Situational analysis

on coordination of

surveillance

between Points of

Entry (PoE) and

National

Surveillance

Systems in key

EpiSouth countries

Study report Fostering best practice

exchange

The methodology chosen is

aimed at providing best

practices developed in

Countries that are

representative of the

geopolitical diversity of the

EpiSouth Region

Ongoing


Feasibility study of

the applicability of

WHO guidance on

coordination of

surveillance

between PoE and

National

Surveillance

Systems

Study Report Unlocking the potential of

innovation in health.

The contribution of the

EpiSouth network is in

analysing the possible impact

of the WHO guidance and the

opportunity of suggesting

nationally identified

strengths/constraints/propos

als. This is an opportunity for

the Countries to participate

in the formulation of the

guidance and in its national

application

Planned

WP7 Strategic

Document

Strategic

document

Identify, disseminate and

promote the up-take of

validated best practices

enriched by the revision of

experts in the network’s

advisory board, the strategic

document will build on the

activities performed to

provide a regional policy

paper on IHR implementation

in the region, the priority

need for coordination of

surveillance and solutions

adopted in key EpiSouth

Countries.

Planned

Additional aspects to consider:

- No Country could have achieved this alone

- No network could have achieved this in absence of a close and productive

collaboration with WHO

- Participants are part of a strategically relevant region comprising 27 countries of

which only 9 EU Member States, and stretching across three WHO Regions.

- Duplication was avoided (no additional data was collected for the initial assessment

and for the mapping phase of the in depth study).

- Notwithstanding the complexity of the topic identified by participants as priority,

coordination with WHO subject matter experts and with the network Focal Points

that work in this specific domain, were valued and their expert opinion was requested

in the definition of methods, tools and in the validation of results.


77 LLiimmiittaattiioonnss A total of seven questionnaires with common questions/variables were disseminated to eight

different target groups. The fact that the questionnaires were disseminated according to the

target group meant that partners participating in more than one target group were obliged

to answer the same questions more than once. The common variables answered more than

once from the same responder were excluded from the analysis. As explained above, since

certain Focal Points responded collectively, ten questionnaires were identified as Not

Applicable and were excluded from the original list in order to calculate the response rate.

Moreover, four questionnaires were further excluded because the partner had not

participated in activities or was not the focal point anymore or was a work package leader

and feared would be biased if had responded to the questionnaires.

A total of 90 out of 178 (51%) questionnaires were disseminated to partners from the EU

countries and a total of 88 out of 178 (49%) were disseminated to Non EU Countries.

Lower response rate was recorded in Non EU partners (36 out of 88, 41%) in comparison

with EU countries (73 out of 90, 81%) (p-value: <0.001). As a result the Non EU countries’

views and opinions are not equally represented. This finding should be investigated further to

identify reasons and consequences. One possible explanation could be the political instability

in the Non-EU countries.

Taking into consideration that statistical significant differences were identified between the

EU and Non EU responses with the EU rating higher the activities in comparison to the Non-

EU, the overall evaluation results could potentially be different if a higher response rate was

achieved for Non-EU countries. This finding should be further investigated and addressed by

the coordination of the project.

Another limitation was the limited time available to conduct the evaluation. The period of

two months was given to revise the indicators, review all documents and activities and

prepare the evaluation tools. The collection period of the questionnaire was only two weeks

and limited time was also provided to assess the evaluation results and prepare the report.


88 CCoonncclluussiioonnss The midterm evaluation covering 24 months of the implementation of the EpiSouth Plus

project has led to the following concluding remarks reflecting the views of the partnership as

expressed by the target groups (Focal Points, Advisory Board, Steering Committee, Steering

Teams) through the questionnaires and interviews.

The evaluation of milestones and indicators has shown that minor and major delays occurred

in all work packages. The reason for the delays as identified by the responders includes,

amongst others, the political instability in certain countries and the long-term negotiations

between EpiSouth Plus and the European Commission (EAHC, DEVCO) regarding the Epidemic

Intelligent platform (WP6) which were also characterised by a lack of flexibility. During the 1st

Project Meeting in Rome, the Work Package Co-leaders presented the revised methodologies

and timeframe to successfully achieve the objectives.

It was noted by all target groups that EpiSouth Plus is a large and complex project to manage

with partners that have geopolitical and cultural differences. The overall belief of the

majority of the responders is that the coordination structure of the project and the problem–

solving process were effective in resolving the delays, however, responders from the EU and

International organisations commented that the project would have benefited from a more

proactive approach in coordination activities. More than 80% of the responders identified

that the project has great potential and that it has achieved reliable and collaborative

relationships among public health professionals. It is characterised as a valuable network that

has fully facilitated the exchange of alerts and health information, enhancing the coordinated

response to public health events in the Mediterranean Area. The website, the Quarterly

EpiSouth Electronic Bulletin and especially the Weekly Epi Bulletin are valued by the partners

and they identify that the information shared is useful and needed. The training activities of

both the WP4 and WP5 were highly rated by the responders. Responders also identified that

the network is not used to its full potential and opportunities for further improvement exist,

especially in the capacity of the Network to strengthen participants’ laboratory capacity in

terms of diagnoses response and in using the participant organisation’s strengths and

expertise. It should be noted that some partners from the Mediterranean countries

mentioned as a shortcoming the limited access to European data, however, this is due to EU

legislation constraints. A fear of duplication of information shared through the Epidemic

Intelligence was expressed as a threat as well as uncertainty over the quality of data shared.

The lack of long-term concrete vision for the sustainability of the project was identified by


the key stakeholders (EU and International organisations) as a limitation and a threat to the

project, and the identification and dissemination of the added value of the project an

immediate need.

It is highly recommended that all Work Package Leaders and Co-leaders should prepare a

summary of the added value of the activities of their WPs similar to what was prepared for

WP7 (see Section 6).

Through this midterm evaluation, recommendations and opportunities for improvement

have been identified. One of the strengths of EpiSouth is the double co-financing received by

DG SANCO and DG DEVCO which has allowed the concomitant involvement of EU and non-EU

partners. However, this has created additional managerial efforts since the coordinators had

to respect double regulations, requirements and expectations. This is still considered by the

partnership as an opportunity to be exploited for EpiSouth as well as other similar

projects/programmes. The coordinating structure of the project should consider enhancing

their ability to overlook the activities of the project and the extra managerial effort required

due to double co-financing by employing a full time scientist. This will allow the coordinating

structures to dedicate their time to the sustainability of the project and the quality of the

work completed.

We would highly recommend that the coordination should take into consideration the

differences of EU and Non-EU countries in the evaluation results and explore the reasons for

the low participation of the Non-EU countries in the evaluation process.

It was evident from the results of the EU and International organisations that different

opinions were expressed by the partners. The evaluators highly recommend enhancing the

communication channels with the EU and international organisations to promote all the

positive results of the project. Moreover, the added value of the network and the platform

should be better defined and disseminated to the stakeholders including EU and

International organisations.

The political instability has certainly not made easy the communication and participation of

Non EU countries in the project leading to low participation in the project’s activities, and

might possibly be the explanation for the low response rate observed to the evaluation

questionnaires by the Non EU countries. More active participation from the partners is

required in order for the network to achieve sustainability.


One important recommendation identified by the responders is that facilitation of

information sharing between the laboratories should be enhanced and direct collaboration

between the laboratories should be improved. However, it should be noted that this

facilitation will be better demonstrated following implementation of the activities in the

following year.

The EpiSouth Plus partnership should review the gaps identified by the countries and identify

the methodology by which the network can be mobilised and used to its full potential. The

partnership could explore within the sustainability plan the future of EpiSouth Plus to

consider possible re-structuring where they will alter their role to more of a political public

health network/association overlooking other smaller networks that will possibly be

operated under International and European organisations.

Taking into consideration that the work completed in the 24-month period evaluated is

smaller than that scheduled in the technical annex, the partnership will have to work hard in

the next months and during the no cost extension in order to complete all scheduled tasks as

these are described in the amended technical annex and to identify the means for

sustainability.


Annexes

Annex 1- Questionnaire 1 - Completed by the Advisory Board of the EpiSouth

Network


Annex 2- Questionnaire 1 - Completed by all Focal Points of

Institution partners and Countries involved in the EpiSouth Network


Annex 3- Questionnaire 1 – Completed by the Steering Committee of

the EpiSouth Network


Annex 4- Questionnaire 1 - Completed by the WP4 Steering Team


Annex 8- Questionnaire 2 - Completed by all Focal Points of

Institution partners and Countries involved in the EpiSouth Network

Annex 9- Questionnaire 3 – Interview Checklist