mid-term evaluation of the third health programme (2014 ... · the third health programme (3hp) is...

Written by Coffey International Development SQW and Economisti Associati

Mid-term Evaluation of the third Health Programme

(2014 – 2020)

Final report

EUROPEAN COMMISSION

Directorate-General for Health and Food Safety Directorate C — Public Health Unit C1 — Programme Management and Chronic Diseases E-mail: [email protected]

European Commission B-1049 Brussels

mailto:[email protected]

EUROPEAN COMMISSION

Mid-term Evaluation of the third Health Programme

Final report

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

More information on the European Union is available on the Internet (http://www.europa.eu).

Luxembourg: Publications Office of the European Union, 2017

ISBN 978-92-79-68447-0 doi: 10.2875/359384

© European Union, 2017 Reproduction is authorised provided the source is acknowledged.

Europe Direct is a service to help you find answers to your questions about the European Union.

Freephone number (*):

00 800 6 7 8 9 10 11

(*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

http://europa.eu.int/citizensrights/signpost/about/index_en.htm%23note1%23note1

Mid-term evaluation of the third Health Programme (2014-2020)

Contents LIST OF ACRONYMS ...................................................................................................... 2 0. EXECUTIVE SUMMARY ............................................................................................ 3 1. INTRODUCTION ................................................................................................... 13 2. BACKGROUND TO THE EVALUATION ....................................................................... 18

2.1. Report structure .......................................................................................... 18 2.2. Purpose and rationale .................................................................................. 18 2.3. Scope ........................................................................................................ 18 2.4. Evaluation approach and methodology ........................................................... 19 2.4.1. Programme assessment ............................................................................... 19 2.4.2. Case studies ............................................................................................... 20 2.4.3. Open public consultation .............................................................................. 21 2.5. Validity and limitations ................................................................................. 22

3. RELEVANCE ......................................................................................................... 24 3.1. Validity of objectives .................................................................................... 24 3.2. Appropriateness of thematic priorities ............................................................ 29 3.3. Relevance of actions .................................................................................... 35 3.4. Objectives in Article 168 .............................................................................. 37

4. EFFECTIVENESS ................................................................................................... 38 4.1. Process for defining Annual Work Programmes ................................................ 38 4.2. Multi-Annual Planning .................................................................................. 42 4.3. “Exceptional utility” criteria........................................................................... 46 4.4. Contribution of the HP to objectives and priorities ........................................... 53

5. EFFICIENCY ......................................................................................................... 58 5.1. Allocation of resource among objectives / thematic priorities ............................ 58 5.2. Efficiency of programme management ........................................................... 63 5.3. Improvements ............................................................................................ 68 5.4. Monitoring .................................................................................................. 69

6. EU ADDED VALUE ................................................................................................. 73 6.1. Consideration of EU added value in action proposals ........................................ 73

7. COHERENCE ........................................................................................................ 77 7.1. Internal coherence ...................................................................................... 77 7.2. External coherence ...................................................................................... 81

8. OVERALL CONCLUSIONS AND RECOMMENDATIONS .................................................. 86 8.1. Overall conclusions ...................................................................................... 86 8.2. Recommendations ....................................................................................... 91 8.2.1. New recommendations ................................................................................. 91 8.2.2. Recommendations where progress is still required ........................................... 94

1


List of acronyms

1HP First EU Health Programme (2003 – 2008) 2HP Second EU Health Programme (2008 – 2013) 3HP Third EU Health Programme (2014 – 2020) AWP Annual Work Programme Chafea Consumers, Health, Agriculture and Food Executive Agency CoR Council of Europe DG Directorate-General DGA Direct Grant Agreement DG SANTE Directorate General for Health and Food Safety EC European Commission EIP European Innovation Partnership EP European Parliament ERDF European Regional Development Fund ERN European Reference Networks ESF European Social Fund ESIF European Structural and Investment Funds EUR Euro, currency of the Eurozone FP Framework Programme GDP Gross Domestic Product GNI Gross National Income HP Health Programme HTA Health Technology Assessment JA Joint Actions JRC Joint Research Centre MAP Multi-Annual Planning MFF Multiannual Financial Framework MS Member States NFP National Focal Point NGO Non-governmental organisation OECD Organisation for Economic Co-operation and Development OPC Open Public Consultation PC Programme Committee SC Service Contract SF Structural Funds TFEU Treaty on the Functioning of the European Union WHO World Health Organisation

2


0. EXECUTIVE SUMMARY

Background and scope

The third Health Programme (3HP) is the Commission’s main vehicle for supporting policy coordination in health. It is the subject of the current evaluation and has a budget of €449.4m for the 2014-2020 funding period. It aims “To complement, support and add value to the policies of Member States, in terms of improving the health of EU citizens and reducing health inequalities”. To do this, 3HP action is organised around four specific and operational objectives that are broken down into a 23 thematic priorities. These aim to focus the 3HP on types of issues and types of action where the potential to generate EU added value is greatest.

This mid-term evaluation of the 3HP has as its purpose to report on the achievement of the objectives of the programme, the state-of-play regarding the implementation of the thematic priorities and the efficiency of the use of resources and the EU added value of the programme. It covers the years 2014-2016, though given the early state of implementation of many of the actions funded so far, the assessment of effectiveness is limited to the management of the HP and likely impact.

Approach and validity

The evaluation used a methodology comprised of distinct pillars that, taken together, allowed us to examine the HP from three different angles. These consisted of: (1) a programme assessment, which looked at the 3HP as a whole using documentary sources, consultation with key programme stakeholders and online focus groups; (2) case studies of eight of the 23 thematic priorities, which provided an in-depth examination of how the 3HP is working towards its objectives and explored details of how actions are implemented and delivered; and (3) an open public consultation which allowed stakeholders to provide views on different aspects of the 3HP.

The evaluation had to grapple with several challenges. Most importantly, the diverse and loosely related nature of funded actions meant it was not possible to assess impact in quantitative terms. This was exacerbated by the timing of the evaluation, since the research was conducted at a time when less than half of the programming period had elapsed and before many of the actions under review had been completed. Moreover, the sheer number and heterogeneity of thematic priorities and individual actions precluded in-depth study of all them. Our approach took these issues into account and sought to provide as much early-stage insight as possible. In doing so, we prioritised key aspects of the programme such as the thematic priority structure, multi-annual planning process, efforts to increase participation among organisations from poorer Member States (MS) and the consideration of EU added value in funding applications.

Overall Findings

The HP has taken a long journey since its inception in 2003. The case for EU action has always been clear from Article 168 of the Treaty on the Functioning of the EU, and previous evaluations have consistently praised the achievements of funded actions and the HP as a whole. At the same time, earlier versions of the HP were criticised both for a lack of focus and management difficulties which to some extent undermined their potential added value.

The mid-term evaluation examined implementation of the 3HP in terms of specific aspects that are new in this Programme (e.g. multi-annual planning) or have been previously under-examined (e.g. process for defining the Annual Work Programmes). Particular attention was given to the state of implementation of the 23 thematic priorities agreed with the European Parliament and the Council (Health Programme Regulation N°(EU) 282/2014) and their continued relevance vis à vis the Programme

3


objectives and their contribution to the Commission priorities for years 2014-2020. This was an explicit request enshrined in the Programme legal basis (Article 14) in view of eventual modifications through a delegated act if any of the thematic priorities become obsolete or new needs appear.

The 3HP was designed with past criticisms in mind, and represents a concerted effort to tackle them while also making the most of the momentum gained so far. Most importantly, the programme structure agreed with European Parliament and Council has been designed so that actions are organised around four specific and operational objectives that are broken down into a 23 thematic priorities. These aim to focus the 3HP on types of issues and types of action where the potential to generate EU added value is greatest.

On the management side, multi-annual planning has been introduced to increase the coherence of the programme and various systems and processes have been simplified and digitised. Indicators have been put in place to monitor progress at action and programme levels, while funding applications are now assessed against specific EU added value criteria. An incentive structure to boost participation in the programme among organisations from poorer MS has been ramped up (called the “exceptional” criteria). Communicating about and publicising the HP, persistently a challenge, is being addressed through increased resources and a new Dissemination Strategy.

A few years after the 3HP began, it would be too early to assess these changes in terms of the HP’s impact on the public health of European citizens. Instead, the evaluation sought to give an early indication as to their effects on planning and implementation, and thereby ascertain the extent to which the pre-conditions for success are in place.

Taken as a whole, we found that the 3HP represents a major improvement compared to what came before. The new structure has increased the HP’s ability to target important health needs where it can add value (such as anti-microbial resistance and “e-Health” in the context of the digital single market to name just a few). It is also channelling efforts to identify common, structural challenges for member states. For instance, mechanisms for pooling expertise at EU level and supporting MS in their health reforms have been set up (namely the Expert Panel on Health and the Expert Group on Health Systems Performance Assessment) and the 3HP is providing direct financial support to the OECD and WHO to produce country profiles which give a clear understanding of country specific needs. This focus is recognised by Member States: the consultation of Programme committee members and national focal points representing Member States’ interests confirmed the 3HP structure matches the main health challenges in their country. A perception of the appropriateness of the 3HP design and structure was also confirmed in replies to the open public consultation.

At the same time as becoming increasingly focused on identified important issues such as those mentioned above, the 3HP structure provides flexibility. This has allowed it to be responsive to shifting circumstances and trends over its seven-year funding period, for instance in relation to a need for crisis management. The migrant crisis of 2015 presented an early and unpreceded test of the programme’s adaptability, given its pan-European nature and the strain it put on existing public health infrastructure. The HP’s intervention, which included the quick deployment of nearly €15m to support healthcare professionals and NGOs dealing with migrants on the frontline, was a major success that highlighted its potential to react decisively in uncertain times.

Where there is more predictability, the 3HP is being used as a tool to support the implementation of EU health legislation in areas of clear EU added value. These include cross border health care, health technology assessment, substances of human origin1,

1 Substances of human origin include: blood, tissues and cells, and organs. 4


and medicinal products and medical devices . It is also being used to address important policies beyond those limited to “traditional” public health. For example, the new legal basis on serious cross-border health threats has meant DG SANTE can play a role in managing the EU response to health-security crises (i.e. the Ebola and Zika outbreaks). The on-going migrant crisis provides a topical example of the opportunity for synergies and coordination and deployment of significant funds in quickly evolving conditions. There is still work to be done to increase visibility and coordination with other actors, however the evaluation found the 3HP is coherent with other EU action in areas such tobacco, and the international development / global health arena.

Promisingly, the available evidence suggests that the funded actions themselves are producing more concrete results and linking better to wider initiatives than under the 2HP, including the EU Budget Focused on Results initiative2. In part this is because support provided over the long term through several actions (spanning successive iterations of the HP) is finally gaining traction and bearing fruit. But in part it is because actions in the 3HP are more focused and purposeful, especially where there is a clearly defined legal basis.

For instance, actions funded through the 2HP laid much of the groundwork for establishing European Reference Networks (ERNs) under the EU Directive on Patients' Rights in Healthcare (2011/24/EU), which also makes it easier for patients to access information on healthcare and thus increase their treatment options. During the first years of the 3HP preparatory steps have been taken (see Annex B - case study ”Thematic priority 4.1”). Since March 2017 24 thematic ERNs, gathering over 900 highly specialised healthcare units from 26 countries, have begun working together on a wide range of diseases3. Similarly, the HP’s work on Health Technology Assessment began with a project funded during the 1HP. This led to two JAs during the 2HP to test and pilot methods and generate the buy-in necessary for common approaches in this area. It is only with the third JA, funded through the 3HP, that these approaches are finally being operationalised and anchored in MS practices and at the same time DG SANTE is preparing future legislation on Health Technology Assessment.

In other areas where there is a legislative basis for EU action and clear EU added value, there has been a clear path of progress. For instance in relation to substances of human origin which is underpinned by Union legislation for safe standards4, actions funded in the 3HP take further previous work by developing methodologies and guidelines in new areas (i.e. novel therapies and products), creating new models for sustainable updating of technical standards, defining procedures in areas where this is lacking (e.g. clinical follow-up)5. This is also true in the case of medicinal products and medical devices, where efforts in the 3HP are expected to increase transparency of prices paid for medicinal products, to create more favourable conditions for the introduction of advanced therapies across the EU, among other benefits. These are important steps and demonstrate the importance of EU support in areas where there is a clear EU role to lead.

That being said, improvements to the programme’s design did not resolve all its problems. The importance of continuity between funding periods meant that the changes were incremental rather than drastic, and that some previously-noted challenges remain. For example, while the four specific objectives provide a workable overarching framework for the 3HP, some of them are more clearly relevant than others. More concretely, objectives 2 (Protect Union citizens from serious cross-border health

2 See here for more information on this initiative: http://ec.europa.eu/budget/budget4results/initiative/index_en.cfm 3 See here for more information http://europa.eu/rapid/press-release_IP-17-323_en.htm 4 Summaries of EU legislation in the field of blood, tissues and cells and organs, can be found here: http://eur-

lex.europa.eu/summary/chapter/public_health/2902.html?root=2902 5 See Annex B, case study on “Thematic priority 4.5”.

5

http://ec.europa.eu/budget/budget4results/initiative/index_en.cfm

http://europa.eu/rapid/press-release_IP-17-323_en.htm


threats) and 4 (Facilitate access to better and safer healthcare) relate to relatively narrow challenges that either flow between countries (e.g. health threats and anti-microbial resistance under objective 2) or create opportunities for collaboration to generate economies of scale (e.g. ERN for rare diseases under objective 4).

By comparison, Objectives 1 (Health promotion) and 3 (Health systems) deal with very broad challenges that are already high on Member State policy agendas. For these objectives, EU added value is typically generated through the sharing of best practices. Actions under these objectives can be highly appropriate, but more care is needed to ensure they fit well with existing initiatives and contexts, and have feasible plans for the eventual implementation of any best practices to be identified and shared. This is possible if care is taken to ensure this is designed into actions. For example an area where the identification, dissemination, and take up of best practices has already been strong is in relation to action on mental health (specifically, dementia6) and cancer. During the remaining period of implementation of the 3HP, DG SANTE intends to strategically enhance the transfer of best practices and established a Steering group on Promotion and Prevention in November 2016 for this purpose.

In addition, some thematic priorities are more precisely and narrowly defined than others in the legal basis, and there are a few minor potential overlaps and differences in scope. For example, while thematic priority 4.1 only deals with action related to the establishment of ERNs for rare diseases, thematic priority 1.4 on chronic diseases has been designed to address a huge array of public health issues (as agreed with the European parliament and Council). Each specific objective also includes two cross-cutting thematic priorities, on the implementation of EU legislation and health information. These can help focus attention on important areas, but they also create ambiguity in a programme structure that otherwise revolves around specific public health topics (i.e. themes) rather than types of action.7

These structural decisions had knock-on effects for individual actions, some of which were overly broad, insufficiently aligned with MS action and / or experiencing difficulties to disseminate results among key stakeholders. For example, the evaluation found that several actions funded under thematic priority 1.4 on chronic diseases had an overly broad scope, leading to a risk that the HP’s resources would be spread too thinly and making it difficult to generate momentum in an area which is central to the health of EU citizens. There is a need to further ensure the thematic priorities are as well defined as possible to focus on actions which adequately address pressing issues.

Our examination of improvements to the programme’s management processes reveals a similar pattern of generally positive change punctuated with some remaining room for further development. The introduction of multi-annual planning has enabled programme managers to take a more strategic approach to funding decisions and smooth the formal process of drafting AWP. The continued trends towards joint actions and away from projects reflects the growing maturity of the programme and serves to increase its cost-effectiveness (given the different relative administrative costs associated with the two financial mechanisms). Simplified and digitised application and grant management procedures have lessened the administrative burden both on DG SANTE / Chafea and applicants / beneficiaries.

The areas where there are still challenges in terms of project management indicate not a lack of progress, but rather highlight the difficulty in addressing longstanding problems. For example, while the process for setting Annual Work Programmes was generally considered objective, some stakeholders felt it gave more weight to the needs

6 For example the joint action: “Act on Dementia”. 7 We note that the programme designers considered grouping all such thematic priorities under an additional specific objective, but decided that, on balance, it would be more confusing than the current set-up.

6


of the Commission and some Member States over others ones (as demonstrated by the survey of NFP and PCs where one in five participants reported involvement in drawing up the AWP was “not very adequate”). However, it is hard to envisage how DG SANTE could take the highly diverse needs of all Member States and interested stakeholders into account while at the same time funding purposeful and focused action in a field as broad as public health.

The increased scope of the “exceptional utility” criterion used to encourage participation among organisations from poorer MS (by providing higher levels of co-funding) has also not yet achieved great success. This is due in part to the complex and interrelated set of barriers other than difficulties securing co-funding that impede such organisations to apply to the HP. These include insufficient administrative capacity, the perceived complexity of the application process and concerns about the administrative burden. But tinkering with the parameters of the criterion and publicising it better might increase its uptake and make the distribution of HP funding somewhat more equitable. In a similar vein, despite improvements to the application and grant management procedures, smaller organisations in particular expressed concerns about their complexity and the reporting requirements.

To better integrate the seven EU added value criteria that were defined during the previous programme (and enshrined in paragraph 19 of the Regulation establishing the 3HP), many applications for funding are now screened and scored for EU added value. This has helped to mainstream the criteria among stakeholders and focus minds on the ways the programme can add value. However, we also found that the assessment panels responsible for awarding funding lack the guidance to apply the criteria in a systematic and objective way. Consequently, it was clear neither how scores were allocated nor how these weighed on funding decisions.

With regard to dissemination, the previous evaluation recommended DG SANTE and Chafea to “develop a formal communication strategy to define key communication objectives, actors, messages, audiences and channels”. Such a strategy now exists, and Chafea has in place a full-time Dissemination Officer to oversee its implementation. However, at the time of writing the strategy’s implementation was still in its initial stages. This represents a missed opportunity given achievements in such politically salient areas as support in dealing with the migrant crisis and the increased ability of the programme to demonstrate its EU added value. It also makes it harder for new potential beneficiaries to learn about the HP, and for results to be taken up. While it understandably takes time to generate the buy-in and marshal the resources needed to design and carry out new activities, the current funding period is nearly halfway complete. In the meantime, previous criticism about inadequate stakeholder engagement and the poor quality of accessible information about the HP remain valid.

Similarly, while the systems for monitoring implementation of the 2HP were harshly criticised, evidence so far does not suggest that major changes have been made to improve the efficiency of these processes in relation to monitoring of implementation. Concerning monitoring of actions’ outputs and outcomes, programmatic and action-specific indicators have now been introduced. Although the evaluation has found that these programmatic indicators are not as comprehensive as they could be, it is a significant step forward to have them in place and revisiting them to ensure better coverage can be an action going forwards. Regarding the action-specific monitoring, the evaluation was unable to find evidence showing how information is being gathered, collated and used. This leaves doubt about whether they will provide meaningful data for the ex-post evaluation and highlights a need for more attention to this area when actions finish and start to produce results.

Looking forward, we re-iterate that the changes brought in for the 3HP have been substantial and positive, and they augur well for the likely effectiveness of the

7


programme and its ability to provide value for money. The next section builds on these conclusions by making recommendations for the immediate and longer-term.

Main findings and conclusions

The research concentrated on specific issues spread across the evaluation criteria of relevance, effectiveness, efficiency, EU added value and coherence. The paragraphs below summarise the main findings for each of them.

Relevance

• Validity of programme objectives: the present set of specific and operational objectives are broadly valid and appropriate. They have helped increase the 3HP’s focus and concentrate limited resources on issues generating the most EU added value, while accommodating existing needs and emerging challenges.

• Appropriateness of thematic priorities: In setting 23 thematic priorities for action, the designers of the 3HP succeeded in better defining the purpose of the 3HP and increasing its coherence. There are still some inconsistencies in terms of how the thematic priorities are formulated (e.g. some are more specific or general than others). There are also some overlaps and potential duplications. These are not damaging per se, but efforts to refine or streamline the thematic priorities could make the HP structure more coherent in the future.

• Relevance of actions: the 3HP’s structure of relevant objectives and thematic priorities to operationalise them has served to ensure the relevance of individual actions. Despite the diversity of issues addressed, the actions funded have corresponded to public health needs and demonstrated clear and suitable objectives. However, the actions under more broadly defined thematic priorities and open-ended funding mechanisms (i.e. operating grants) sometimes lacked focus, highlighting the need for particular attention to the planning stages of such actions, and monitoring and evaluation processes to gauge performance.

Effectiveness

• Process for defining Annual Work Programmes (AWPs): the evaluation found that the process for setting AWPs works well, with widespread agreement that the priorities defined on a yearly basis correspond to the public health needs of the MS and that the consultation process is well-defined and impartial. However, there were also some concerns about lacking transparency in informal consultation, perceived disparities among the MS in feeding into the AWP process and the pre-eminent role of DG SANTE in setting the HP’s agenda. This demonstrates a need to explain the process to stakeholders and ensure their buy-in.

• Multi-annual planning (MAP): the newly adopted MAP exercise has proven to be a valuable tool which has facilitated a quicker, less controversial, more efficient adoption of the AWP according to those involved. Additionally, there is evidence that the MAP has enabled a more focused and strategic approach to planning in the medium-run (i.e. up to 3-4 years).

• “Exceptional utility” criterion aimed at increasing participation among organisations from poorer MS: by increasing the proportion of co-funding available to actions that include partners from low-GNI MS, the criterion addresses a real problem. However, the evidence suggests that take-up so far is low. In part this seems due to lacking awareness among potential applicants. It is also possible that the criteria do not create a sufficient incentive to overcome capacity and skills shortages that also act as barriers to participation in the 3HP.

8


• Contribution of the HP to objectives and priorities: based on case studies of two thematic priorities per each of the 3HP’s four specific objectives, the evaluation found evidence of many potential benefits from funded actions. While it is impossible to ascertain whether the case study findings are indicative of the programme as a whole, it is also worth noting that the actions examined under objectives 2 (cross-border health threats) and 4 (access to healthcare) typically appeared likelier to generate tangible benefits in the near future. For the other objectives, the potential to generate EU added value depends to a greater extent on identifying gaps that can be filled with the sharing and eventual uptake of best practices. The path to making an impact then varies depending on the specific context.

Efficiency

• Allocation of resources among objectives and thematic priorities: the actions with the strongest EU added value are typically under objective 2 and 4 due to the clarity of purpose gained from EU legislation in these areas and cross-border nature of the issues at stake. Action under other areas can also add significant value and be cost-effective, provided that actions are sufficiently well-designed and outcome-focused. This has been reflected in the design and implementation of the 3HP, which allows for clarity in the level of funding directed towards objectives 2 and 4.

• Efficiency of programme management: irrespective of the size of an action, the biggest driver of efficiency is how effective the action is in achieving its goals and therefore the value added by EU action. While it was not possible to measure effectiveness in quantitative terms, early indications are positive, with evidence of many potential benefits from funded actions and generally good planning. Moreover, a high-level comparison of administrative costs with selected other Commission programmes demonstrated that the 3HP is relatively cheap to administer, while recently-introduced simplification measures have led to cost savings and been favourably received (though some concerns about administrative burdens remained).

• Monitoring: the 3HP has responded to previous criticism by introducing programmatic indicators and action level e-monitoring, as well as investing in strategic dissemination activities. While these are welcome steps, more work is needed to develop and operationalise the indicators. In terms of dissemination activities, the beginning stages of new plans are promising (i.e. the definition of a Dissemination Strategy for 2017 – 2020) but there is a need to progress faster.

EU added value

• Consideration of EU added value in action proposals: building the seven EU added value criteria into the application process represents a major improvement and has ensured that the majority of relevant potential beneficiaries consider EU added value when preparing their proposals, and that assessment panels in turn take it into account as part of the decision to award funding. However, applicants and evaluators sometimes lack a common understanding of the seven criteria, meaning they are not always considered in a consistent fashion. This points to a need for clearer guidance, as well as efforts to better define (and potentially refine) the criteria.

Coherence

• Internal coherence: the revised structure of the programme has substantially improved the coherence of the programme, providing a framework to fund actions in well-defined areas of intervention and fostering synergies between

9


actions. However, due to the ambitious and broad nature of the programme, some funded actions in given thematic priorities are not closely related, while in other cases potential links have not been identified or fully exploited.

• External coherence: the 3HP strengthens and emphasises the links between economic growth and a healthy population to a greater extent than the previous programmes, bringing it into line with the Commission’s policy priorities. It has also demonstrated many practical complementarities with other programmes, particularly Horizon 2020, though such links could be publicised more and be further exploited (particularly for structural funds). The 3HP is also being used as a tool to address policies related to such issues as the Ebola and Zika outbreaks and migrant crisis, where HP funding has been quickly deployed to make a significant difference. In the international development / global health arena, the 3HP is coherent with other EU action but, suffers from low visibility and insufficient coordination between the HP / action beneficiaries and other actors.

Recommendations

Following on the findings and conclusions, we make the following recommendations for the immediate and longer term. These distinguish between issues that are newly identified and those relating to recommendations that have been previous made but where work remains to be done.

Recommendations on newly identified issues

Thematic focus of the programme

1. Maintain a focus on thematic areas of strong EU added value: DG SANTE should maintain a clear thematic focus in areas where the EU has a clearly defined role (such as ERN, and HTA) will strengthen the delivery of results in these areas by the end of the Programming period

2. DG SANTE should strengthen and build links between the HP and the wider policy agenda to maximise impact: the example of the support provided through the 3HP on health needs arising from the 2015 migrant crisis illustrates the value of policy coordination in ensuring an adequate response to emerging health needs.

Programme design

3. Spell out how action targeting health promotion and health systems should generate EU added value: in the immediate term, DG SANTE should define in as much detail as possible the mechanisms by which best practices should be taken up in practical terms and reasonable timescales for doing so (either in general or with regard to specific funding calls). For the longer term, the operational objectives should include more detail about how the HP should generate EU added value and complement the actions of other actors. In addition, we suggest that a greater proportion of funding is directed towards the actions with the highest EU added value.

4. Refine the thematic priorities as part of the continuing effort to focus programme spending: the structure of thematic priorities should be maintained in the immediate term, but efforts could be made to increase the clarity of certain priorities. In the longer term, DG SANTE should consider streamlining thematic priorities to avoid any overlaps, ambiguities or redundancies. In particular, thematic priority 1.2 on drugs related health damage

10


could potentially be included under 1.1 risk factors. Thematic priorities 2.4 and 4.6 on “fostering a health information and knowledge system” were included for the sake of consistency but in practice could be removed, while thematic priority 2.1 on risk assessment is also considered unnecessary (and could be subsumed under 2.2 on capacity dealing with health threats). It is also worth considering whether the scope of the thematic priorities is sufficiently distinct or whether some re-wording could improve clarity.

Programme management

5. Refine the EU added value and fully integrate criteria into the application process: in the immediate term, in order to help potential applicants and assessment panels to build a common understanding of the EU added value criteria as they currently stand, DG SANTE and / or Chafea could develop detailed but accessible guidance for applicants about what each of the criteria means, with practical examples. As a longer-term goal, the criteria could be streamlined to make them clearer. We suggest the following criteria: addressing cross-border health threats; improving economies of scale; and fostering the exchange and implementation of best practices.

6. Integrate multi-annual planning with existing programme processes: in order to engage stakeholders more in the multi-annual planning, we recommend DG SANTE consider its formal integration it into the consultation and priority-setting process.

Recommendations on previously identified issues where progress is still needed


7. Develop a broader strategy to increase participation from poorer Member States and underrepresented organisations: DG SANTE should consider steps to address the barriers to participation in a more holistic way. For instance more consideration could be given for the equitability and capacity building element of the HP, while ensuring the technical quality of the actions. Further efforts should also be made to reduce the administrative burden of applying for and receiving funding, led by Chafea.

8. Invest in the resources necessary to improve the systems for monitoring programme implementation: DG SANTE and Chafea should invest the resources needed to put in place and manage a simple and effective system for monitoring implementation of the programme.

9. Implement and use programmatic and action specific monitoring indicators: it is important that together, DG SANTE and Chafea put in place a system for reporting on, collecting and presenting data on the action specific indicators as soon as possible and revisiting the programmatic indicators to ensure that the key programme goals are covered. This could be built into the electronic reporting system for beneficiaries managed by Chafea.

10. As with previous HP, there is a need to continue to step up efforts to communicate about the HP with core stakeholders and wider audiences: given that only half of the programming period remains, it is crucial that DG SANTE and Chafea assemble the political will and resources to roll out the newly developed communication strategy in the near term.

11


KEY MESSAGES

Relevance • The 3HP has valid and appropriate objectives in place which has led to actions

which are focused and generate EU added value while accommodating existing needs and challenges.

• The thematic priorities are a positive development and facilitate synergies and coherent action. However, these could still be streamlined.

• The structure in place has supported relevant actions, especially in fields where there is legislative clarity and / or a clear cross-border dimension to activity. In areas where action is more open-ended or broadly defined, there is a danger of the actions being less focused.

Effectiveness

• The AWP and MAP processes work well. The MAP in particular has enabled a more strategic approach to medium-term planning. The AWP process is clear and well-defined and impartial but to avoid confusion and ensure greater buy-in, the process could be better explained to stakeholders.

• The 3HP has improved how it attracts the participation of poorer MS through the “exceptional utility” criterion. Still, securing co-funding is only one part of the explanation for lower participation and a more holistic approach is needed.

• To date, the 3HP has contributed significant progress in several areas of public health. In particular, in the establishment of European Reference Networks, in the adoption of Health Technology Assessment, in supporting capacity building of Member States to respond to outbreaks and continuous updating of skills to take into account emergent issues such as the migrant crisis.

Efficiency

• The allocation of resources in the 3HP has been found to be efficient overall. • Programme management has been mostly effective, and has improved since the

previous HP. For instance new indicators are in place for monitoring the HP and specific actions. Nevertheless, there are persistent inefficiencies and inadequacies with the monitoring of implementation data, which holds back the ability of managers to keep an up-to-date overview of the HP’s achievements.

• While significant strides have been made to ramp up dissemination, going forwards progress in this area must be prioritised.

EU added value

• The HP has increased its ability to target important health needs where it can add value (e.g. anti-microbial resistance, e-health, accreditation schemes for breast cancer screening, etc.)

• The fact that the seven EU added value criteria are written into the regulation and are built into the proposal evaluation process are positive achievements allowing the majority of relevant potential beneficiaries to consider EU added value when preparing their proposals and in turn, for assessment panels to take it into account as part of the decision to award funding.

• However, we consider there to be scope to streamline the added-value criteria to focus on three key areas: addressing cross-border health threats; improving economies of scale; and fostering the exchange and implementation of best practices. This will make it easier to provide clear guidance of what the criteria mean and make it easier for them to be addressed more effectively.

Coherence

• The 3HP has been found to be internally coherent, in part due to the revised structure of the programme. However, we found that where the definition of action remains broad and ambitious, coherence is harder to achieve.

• The 3HP is also coherent with the Commission’s policy priorities and has been shown to be an effective tool to respond to evolving needs.

12


1. INTRODUCTION

Overview

The third Health Programme (3HP) is the Commission’s main vehicle for supporting policy coordination in health. It is the subject of the current evaluation and has a budget of €449.4m for the 2014-2020 funding period.

Based on the EU’s supporting competence in public health as defined in Article 168 of the Treaty on the Functioning of the European Union, the Regulation establishing the 3HP8 defined its general objective as “To complement, support and add value to the policies of Member States, in terms of improving the health of EU citizens and reducing health inequalities’.

To support this general objective, 3HP action is organised around four specific and operational objectives that are broken down into a 23 thematic priorities. These aim to focus the 3HP on types of issues and types of action where the potential to generate EU added value is greatest. The specific objectives and thematic priorities are briefly listed in the table below.

Table 1: Specific objectives and thematic priorities of the 3HP

Objective 1: Promote health, prevent disease and foster supportive environments for healthy lifestyles

1.1 Risk factors such as use of tobacco and passive smoking, harmful use of alcohol, unhealthy dietary habits and physical inactivity

1.2 Drugs-related health damage, including information and prevention 1.3 HIV / AIDS, tuberculosis and hepatitis 1.4 Chronic diseases including cancer, age-related diseases and neurodegenerative diseases 1.5 Tobacco legislation 1.6 Health information and knowledge system to contribute to evidence-based decision making

Objective 2: Protect Union citizens from serious cross border health threats

2.1 Risk assessment additional capacities for scientific expertise 2.2 Capacity building against health threats in Member States, including, where appropriate, cooperation

with neighbouring countries 2.3 Implementation of Union legislation on communicable diseases and other health threats, including

those caused by biological, and chemical incidents, environment and climate change 2.4 Health information and knowledge system to contribute to evidence-based decision-making

Objective 3: Contribute to innovative, efficient and sustainable health systems

3.1 Support voluntary cooperation among MS on HTA 3.2 Innovation and e-health 3.3 Health workforce forecasting and planning 3.4 Setting up a mechanism for pooling expertise at Union level 3.5 European Innovation Partnership on Active and Healthy Ageing 3.6 Implementation of Union legislation in the field of medical devices, medicinal products and cross-border

healthcare 3.7 Health information and knowledge system including support to the Scientific Committees set up in

accordance with Commission Decision 2008/721/EC

Objective 4: Facilitate access to better and safer healthcare for Union citizens

4.1 Establishment of a system of European reference networks 4.2 Effectively help patients affected by rare diseases 4.3 Strengthen collaboration on patient safety and quality of healthcare 4.4 Measures to prevent Antimicrobial resistance and control healthcare-associated infections 4.5 Implementation of Union legislation in field of tissues and cells, blood, organs, 4.6 Health information and knowledge system to contribute to evidence-based decision making

8 Regulation (EU) N° 282/2014 of 11 March 2014 on the establishment of a third Programme for the Union's action in the field of health (2014-2020) and repealing Decision No 1350/2007/EC (OJ L 86/1).

13


The 3HP is implemented through a series of financial mechanisms that reflect the different kinds of actions and objectives pursued. Their main features can be summarised as follows.

Table 2: Financial mechanisms of the 3HP

Financial mechanism Description

Project grants

They are used to fund a collaborative effort between different organisations in various EU MS, which join forces to perform various tasks on a common set of objectives for a defined period of time9

E.g.: Projects for the promotion of early diagnosis and screening of preventable chronic diseases

Operating grants

They provide financial support towards the functioning of a non-governmental body or network, over a period that is equivalent to its accounting year, in order to carry out a set of core activities10

E.g.: Operating grant to Alzheimer Europe

Direct grants to international organisations

They are awarded to international organisations with the capacities needed to tackle relevant health priorities

E.g.: Direct grants to the WHO on the monitoring of the national policies related to alcohol consumption and harm reduction

Joints actions

They have a clear EU added value and are co-financed either by competent authorities that are responsible for health in the Member States or in the participating third countries, or by public sector bodies and non-governmental bodies mandated by those competent authorities11

E.g.: Joint actions on Dementia

Procurement contracts

These contracts cover specific needs related to the support of EU health policies (e.g. studies, development of IT tools, etc.)12

E.g.: Study on the added value of the strategic and life-course approach to vaccination

Presidency Conferences

Thematic conferences on health topics such as personalised medicine to mark Presidency of the EU.

Others E.g. “Payment of membership fee and reimbursement of expert mission costs", "Reimbursement of auditor mission costs", "Cross sub-delegation to EUROSTAT", etc.

Source: http://ec.europa.eu/chafea/health/index.html

The Regulation establishing the 3HP provides the legal basis for actions, which are realised through the adoption of Annual Work Programmes (AWPs). Each year, the AWP sets out the priorities for actions to be funded. They are developed by the Commission and adopted through a collaborative process involving consultation with a representative from each Member State (which together form the Programme Committee). A Multi-Annual Planning (MAP) supports the AWP process by incorporating a more holistic, longer-term mind-set into the programming process. Once the AWPs are adopted, the actions are procured using the appropriate procedure for the funding mechanism in question.

In terms of managing the 3HP, overall responsibility lies with DG SANTE of the European Commission. The Commission is supported by the Consumers, Health, Agriculture and Food Executive Agency (Chafea) which takes charge of its implementation. In this role, Chafea is supported by “National Focal Points” (appointed by national health authorities), which support the promotion and implementation of the HP at national level (through, for example, holding information days), helping to

9 http://ec.europa.eu/chafea/documents/health/hp-factsheets/project-grants/factsheets-hp-pg_en.pdf 10 http://ec.europa.eu/chafea/documents/health/hp-factsheets/operating-grants/factsheets-hp-og_en.pdf 11 http://ec.europa.eu/chafea/documents/health/hp-factsheets/joint-actions/factsheets-hp-ja_en.pdf 12 http://ec.europa.eu/chafea/health/tenders.html

14

http://ec.europa.eu/chafea/health/index.html

http://ec.europa.eu/chafea/documents/health/hp-factsheets/project-grants/factsheets-hp-pg_en.pdf

http://ec.europa.eu/chafea/documents/health/hp-factsheets/operating-grants/factsheets-hp-og_en.pdf

http://ec.europa.eu/chafea/documents/health/hp-factsheets/joint-actions/factsheets-hp-ja_en.pdf

http://ec.europa.eu/chafea/health/tenders.html


disseminate results, discussing national-level experiences and concerns and providing information on the HP’s impact.

The following paragraphs provide a snapshot of how the funding for the programme has been allocated in practice during the years 2014-2016. This section is structured in terms of allocation across funding mechanisms, objectives and thematic priorities and beneficiary involvement (note that a full analysis of these issues is provided in Annex A2).

Overall budget allocation and split across funding mechanisms

So far, €165.6m of the programme budget has been allocated during its first three years. More precisely, this was €53.8m for 2014 and €55.4m for 2015. The budget for 2016 is €56.4m.

The figure below illustrates how the budget has been split across the different funding mechanisms so far and compares this to the 2HP. In the first three years of the 3HP, joint actions (30%) and procurement contracts (27%) have received the highest proportion of funding, followed by projects (24%). This continues a trend started during the 2HP, whereby a progressively larger share of the programme budget went to joint actions while a decreasing amount was allocated to projects. Operating grants and direct grant agreements (DGA) took a consistently smaller proportion of the budget each year but still as much 9% and 7% in total, respectively. Allocations to “other” mechanisms (such as communication and dissemination activities, IT services for databases and pan-European online platforms and administrative and technical assistance) and conferences on pressing health issues (under the auspices of the rotating EU presidency) took a very small share of the budget.

Figure 1: Proportion of total funding by mechanism for 2HP and 3HP (2014 – 2016)

Sources: 3HP DG SANTE and Chafea; 2HP Annual implementation reports

Note: Percentages are rounded to the nearest whole percentage point

Spending by objective and thematic priority

Over the first three years of the 3HP, the largest share of funding has supported objectives 1 (health promotion, 33%) and 3 (health systems, 31%). The remaining funding has been

15


split between objective 4 (access to healthcare, 19%), horizontal or cross-cutting activities (10%)13 and objective 2 (cross border health threats, 7%).

A more refined breakdown of spending by thematic priority shows significant variation of funding across the 23 priorities; the range is from over €17 M awarded to cross-cutting elements (“horizontal” activities), and tackling chronic diseases (priority 1.4), to none for “health information and knowledge system” (priority 2.4).

Table 3: Allocation of budget by thematic priority14, 2014 - 2016

Source: DG SANTE and Chafea Note: figures are rounded; 2016 allocated budget is based on available data. *While the funding to thematic priority 2.1 drugs-related health damage appears low, five actions which address among other secondary prevention for drugs users have been funded under the 3HP and are accounted under thematic priority 1.3 HIV/AIDS, Tuberculosis and hepatitis as the main subject was on these communicable diseases.15 **while very little funding appears to be allocated to thematic priority 4.4 on antimicrobial resistance, this topic is in fact addressed through some of the action funded through thematic priority 4.3 on patient safety and healthcare quality.

Beneficiary involvement

The involvement of different organisations and participating countries in 3HP actions depends heavily on the financial mechanism in question. The table below gives an overview of the different types (and numbers) of organisations and also shows patterns in terms of participating countries.

13 Horizontal activities include actions which are not specifically attributable to a thematic priority (for instance support and administrative activities, such as communication, dissemination, IT services for databases and platforms)which cut across the objectives of the HP). 14 Full text of the Regulation and further information at: http://ec.europa.eu/health/programme/policy/index_en.htm 15 For example, the 2014 Joint Action on HIV and Co-infection Prevention and Harm Reduction (HA REACT), (http://www.hareact.eu/en) and 2015 HEPCARE EUROPE: Early diagnosis and treatment of viral hepatitis, (http://www.ucd.ie/medicine/hepcare/)

2014 2015 2016 Total Total

Horizontal €3.7 M €10.0 M €3.6 M €17.3 M1.4 Chronic diseases €6.6 M €0.8 M €9.9 M €17.2 M1.1 Risk factors €5.2 M €4.8 M €4.6 M €14.6 M1.3 HIV / AIDS, TB & hepatitis €3.3 M €5.3 M €4.6 M €13.2 M3.1 Health Technology Assessment €0.3 M €12.0 M €0.4 M €12.7 M4.1 European Reference Networks €5.5 M €0.4 M €6.7 M €12.6 M3.5 EIP on Active & Healthy Ageing €5.4 M €6.8 M €0.0 M €12.2 M3.6 Union legislation on medicinal €4.0 M €3.8 M €4.2 M €12.0 M3.7 Health information €5.0 M €1.3 M €2.9 M €9.2 M2.2 Capacity building €1.8 M €1.4 M €4.3 M €7.5 M4.5 Union legislation tissues and cells etc. €3.3 M €1.9 M €2.1 M €7.3 M4.3 Patient safety & healthcare quality €0.9 M €1.0 M €4.2 M €6.0 M4.2 Rare Diseases €0.8 M €2.3 M €1.6 M €4.7 M1.5 Tobacco legislation €0.2 M €1.4 M €3.1 M €4.7 M1.6 Health information €1.5 M €0.4 M €1.8 M €3.8 M2.3 Union legislation health threats €3.5 M €0.0 M €0.0 M €3.5 M3.2 Innovation and e-health €2.4 M €0.1 M €0.3 M €2.8 M3.4 Mechanism to pool expertise €0.3 M €1.0 M €0.5 M €1.8 M3.3 Health workforce €0.2 M €0.2 M €1.0 M €1.3 M1.2 Drugs-related health damage* €0.0 M €0.0 M €0.6 M €0.6 M4.4 Antimicrobial resistance** €0.0 M €0.4 M €0.0 M €0.5 M4.6 Health information €0.0 M €0.2 M €0.0 M €0.2 M2.1 Risk assessment €0.0 M €0.0 M €0.1 M €0.1 M2.4 Health information €0.0 M €0.0 M €0.0 M €0.0 M

16

http://ec.europa.eu/health/programme/policy/index_en.htm

http://www.hareact.eu/en

http://www.ucd.ie/medicine/hepcare/


Table 4: Comparative information for financial mechanism and beneficiaries

Financing mechanism

Typical organisation(s) involved (# organisations per action)

Implications for country involvement

Joint actions

Collaborations between public bodies, research organisations, academic organisations (the number of participating organisations ranged from 10 to 61 but on average 28 organisations were involved in a given joint action).

The involvement of organisations is based on nominations from national health authorities; all MS authorities are asked to put forward suggestions for participants when a joint action is negotiated. Most MS had organisations involved in at least half of all the joint actions.

Projects

Collaborations between research institutes, academic organisations, public bodies16.

Most participating countries are involved in projects but to varying degrees.


DG SANTE17

Private entities and EU institutions but occasionally individuals (single recipients).

Participation concentrated in organisations based in BE18, and five other MS: NL, LU, FR, the UK and DE19. A further 14 MS, managed 1 or 2 actions each. Many actions are not tied to a particular country20.


Chafea

Typically international organisations (single recipients or consortia).

These organisations typically have their headquarters in LU (e.g. SOGETI), BE (e.g. PWC), the UK (e.g. Public Health England) and NL21.

Operating grants

NGOs and umbrella organisations (single recipient, which has many members22).

These organisations are typically EU-wide with membership spanning the MS (more detailed information below).

Direct grant agreements /conferences

International organisations (e.g. WHO and OECD) / MS authorities23 (single recipients).

Typically international / MS authorities determined by the presidency of the EU.

Others Sub-delegations to, and Administrative agreements with other EC services (e.g. Eurostat or JRC), including reimbursement of costs to individuals for their participation in experts groups, international conferences, audits and joint assessments.

Involvement ranges from individuals/ experts to EU institutions (e.g., Eurostat issuing grants to national statistical services, the JRC recruiting additional external staff and delivering various services and outputs).

Source: DG SANTE and Chafea24

16 Number of organisations ranged from 3 to 31 but on average 9 organisations were involved in a given project 17 As previously mentioned, DG SANTE manages actions which provide the HP with strategic and institutional support, for example the payment of experts, IT services and solutions and membership fees, among others. Some of these are horizontal, cross-cutting support and others are more specific to the implementation of thematic priorities, such as the establishment of a European Reference Network. 18 40% of all actions 19 Combined to 46% of actions 20 (such as payments to experts, promotional tools and materials) 21 There are some exceptions, such as the consultancy “NOOKOM” with headquarters in SK. 22 the total number of members can vary from around 30 (for instance “The European network for smoking prevention” http://ensp.org/ and “The Smoke Free Partnership” http://www.smokefreepartnership.eu/ ) to over 130 (for instance “Aids Hilfe” https://en.aidshilfe.de/aids-service-organizations) 23 Presidency conferences are also funded through this mechanism and have to date been held in Latvia, Italy, Luxembourg and the Netherlands. 24 Note that data on beneficiary involvement is only available for 2014 and 2015.

17

http://ensp.org/

http://www.smokefreepartnership.eu/

https://en.aidshilfe.de/aids-service-organizations


2. BACKGROUND TO THE EVALUATION

2.1. Report structure

The evaluation is structured in terms of answers to 16 evaluation questions spread across six criteria. These are relevance, effectiveness, efficiency, EU added value, consistency / coherence and utility. Each of the criteria has a chapter devoted to it in this report, with the last of these, on utility, presented as overall conclusions concerning the 3HP’s implementation so far. Based on the findings and conclusions of the evaluation, the last sub-section of the report provides recommendations to improve the HP during the remainder of the current funding period and further into the future.

Additional insight is provided in two annexes. Annex A presents the Terms of Reference for the assignment, as well as detailed findings for most of the data collected and analysed. Annex B provides the full reports of the case studies conducted on eight of the 3HP’s thematic priorities. The evaluation approach and methodology, included a description of the various research tools employed, can be found later in this chapter.

2.2. Purpose and rationale

The legal basis for the mid-evaluation is contained in Article 13 (3) of Regulation (EU) 282/2014. The Regulation asks the Commission to present a mid-term evaluation of the 3HP no later than 30 June 2017.

The purpose of the evaluation is to report on the achievement of the objectives of the programme, the state-of-play regarding the implementation of the thematic priorities and the efficiency of the use of resources and the EU added value of the programme. Based on this, a decision should be taken on the renewal, modification or suspension of (some of) these thematic priorities. The evaluation should also examine possibilities for simplification, the internal and external coherence of the programme, the continued relevance of objectives and the contribution to the objectives of Article 168 of the Treaty on the Functioning of the European Union (which defines the EU’s supporting competence in the field of public health). Particular consideration should also be given to improvements made since the last evaluation of the programme and any problems that might still persist.

Leading from this, the evaluation results will be used by the Commission for the purpose of evidence-based policy-making. As noted above, the evaluation results will be an important input for the Commission’s upcoming review of the HP’s thematic priorities, in addition to work to be done to develop the next programming period. Taken together, these will play an important role in determining the future of the HP and its fit within the broader policy context and the EU’s supporting competence for public health defined in Article 168 of the TFEU.

2.3. Scope

Regulation (EU) No 282/2014 entered into force at the end of March 2014. The first Annual Work Plan (AWP) was agreed May 2014, the second in June 2015 and the third in March 2016. The evaluation was launched in April 2016, less than two years since the first actions under the 3HP had been launched, and a month after actions under the 2016 AWP were determined. As such, the evaluation fully covers the first two years of the implementation (2014-2015), while 2016 is assessed based on data available. However, the research for the evaluation took place before much of the action already funded under the 3HP was concluded; rather, many funded actions are still underway. Impacts can in any case take years to emerge and be difficult to measure. In keeping with these limitations (which are described in detail in section 2.5 below), the assessment of effectiveness is limited to the management of the programme and potential impact.

18


More concretely, the evaluation examined implementation of the 3HP in terms of specific aspects that they are either new or previously under-examined. These include the process for defining Annual Work Programmes (AWPs), the multi-annual planning (MAP) used to provide a framework for several years at a time and the “exceptional utility” criterion aimed at encouraging participation in the 3HP among organisations from low-GNI countries.

With regard to impact, the evaluation looked in depth at a subset of eight of the 23 thematic priorities used to organise the 3HP in order to assess the suitability of this structure and the programme’s likely contribution to objectives and priorities. See annex B for the cases studies where 29 actions under the different financial mechanisms have been examined.

2.4. Evaluation approach and methodology

In order to respond to the requirements outlined above and provide a useful contribution to evidence-based policy-making, the evaluation used a methodology comprised of distinct pillars that, taken together, allowed us to examine the HP from three different angles. The diagram below depicts these pillars and how they fit together, and is followed by a brief summary of the research methods they entailed.

Figure 2: Evaluation approach

2.4.1. Programme assessment

A programme assessment collected and analysed data on the HP as a whole. This focused on what the programme is doing in terms of both its implementation and its objectives and themes (in addition to identifying issues that helped refine the methodology for the case studies, which are described below). This was comprised of three main methods, the findings for which can be found in annex A.

1. An analysis of existing sources provided hard evidence on a range or pertinent issues relating to public health and wider policies and programme

3. Public consultation

2. Case studies on eight thematic priorities (two

from each specific objective)

1. Programme assessment covering entire HP

Analysis of existing sources

Consultation with key HP stakeholders

Online focus groups with public health

experts

Overall evaluation results

1. Health promotion

2. Cross-border health threats

3. Health systems

Questionnaire design

Stakeholder analysis

Test conclusions / recommendations

4. Access to healthcare

19


implementation (see annex B, section 11). With regard to the former, as part of the assessment of the 3HP’s relevance we reviewed studies, reports and other documentation on public health problems and Member State health strategies (see annex A, section 4). In addition, EU policy and programming documentation helped us examine the 3HP’s internal and external coherence. To examine programme implementation and monitoring provisions, we collated and analysed financial and other data provided by DG SANTE and Chafea. This covered such issues as the budget structure and management, allocation of funding according to financial mechanisms, objectives and thematic priorities, and beneficiary organisations and countries of origin (see annex A, section 2).

2. Consultation with key programme stakeholders (annex A, sections 5 - 7) allowed us to elicit the views and perceptions of those with direct experience of the HP regarding its relevance and implementation and performance so far. The consultation took three forms.

a. An online survey gathered quantitative data from members of the Programme Committee (PC) and National Focal Points (NFP). The survey focused on the suitability of the HP’s objectives, MS health needs and various aspects of implementation, and received 45 responses, covering all participating countries except Estonia.

b. In order to obtain additional feedback, which MS health needs should be prioritised for EU action and barriers to participation in the 3HP, we focus particularly on the most important MS health priorities, we carried out a follow-up survey with PCs and NFPs. This was distributed in paper form at the PC and NFP meeting of December 2016, and led to 25 responses.

c. A series of semi-structured interviews with officials from DG SANTE and Chafea helped us shed light on all aspects of the design and implementation of the 3HP, particularly regarding new features such as the thematic priorities and multi-annual planning. These were complemented by interviews with officials from other DGs in the Commission and other stakeholders (i.e. international organisation and external HP evaluation experts) who shared their views on the coherence of the 3HP with other initiatives.

3. Online focus groups with public health experts on three of the 3HP’s specific objectives (namely health promotion; health systems; and access to healthcare) were conducted in order to engage relevant public health experts in group discussions on a variety of topics. These centred on issues concerning the relevance, consistency and coherence of the programme, particularly regarding the 23 thematic priorities and engaged an average of five participants. In practical terms, the focus groups took place online, using specialised chat software, and were moderated by members of the evaluation team’s expert panel (see annex A, section 8). Given the political sensitivity of the specific objective on cross-border health threats (objective 2), this focus group was replaced with a series of five interviews with relevant representatives of national ministries and research institutes (see annex A, section 9).

2.4.2. Case studies

Case studies of eight thematic priorities (annex B) looked in depth at how the HP is working towards its objectives and aims in theory and practice. These enabled us to explore the details of how actions in the thematic areas are implemented and delivered, their specific results, and the main factors and processes that facilitated or hindered their success, particularly regarding such aspects as EU added value, the relationship between various funding mechanisms and the ability of the HP to engender in specific

20


actions the key success factors identified during the ex post evaluation of the second HP.

The methodology for the case studies consisted of eight mini theory-based evaluations, each consisting of two main steps. First, we reconstructed the thematic priority’s theory in the form of an intervention logic. The intervention logic depicts a full causal chain from the desired underlying strategy through delivery and to desired benefits, with a focus on key assumptions that need to hold for the desired changes to occur. Then, to test the theory, we collected and analysed evidence about implementation so far, above all through an examination of five funded actions. The evidence came from several sources, namely contextual literature and programme documentation and about eight interviews per case study with Chafea project officers and representatives of organisations responsible for implementing the sampled actions wherever possible. In addition, the data collection and analysis was supported through on-going consultation with members of the expert panel.

2.4.3. Open public consultation

An open public consultation took place from 23 November 2016 to 23 February 2017. The purpose of the consultation was to allow stakeholders to provide views on different aspects of the evaluation questions. It also fulfils the consultation requirement stipulated in the Better Regulation Guidelines published in May 2015.25 By definition, all citizens and organisations across the European Union (EU) were welcome to contribute to this consultation, however the OPC targeted those with an interest in health policy, public health, and/or healthcare in Europe. It was disseminated via the Chafea and SANTE websites, shared on the Health Policy Platform and distributed via stakeholder mailing lists and National Focal Points.

The OPC gave the possibility to interested parties to express their views and opinions on the 3HP and focused on the following topics:

• The objectives and priorities of the 3HP, and the extent to which these are appropriate and in line with health needs in the EU;

• The way the 3HP is implemented, and the extent to which this is effective and efficient; and

• The overall added value and usefulness of the 3HP.

Participants were invited to complete the online questionnaire provided and available in English, though responses were accepted in all official EU languages. In total, the consultation received 133 responses. These covered all Member States except for Estonia and Malta (though nearly four in five responses came from EU-15 countries) and a wide range of profiles, as illustrated in the chart below. Importantly, while the consultation did not engage a representative sample of potentially interested individuals and organisations, it did allow the evaluation to reach beyond core HP stakeholders to gather the perceptions of a wider range of stakeholders than would otherwise have been possible. A full report of the consultation findings can be found in annex A.

25 For details, see the Commission’s Better Regulation “Toolbox”, url: http://ec.europa.eu/smart-regulation/guidelines/toc_tool_en.htm.

21

http://ec.europa.eu/smart-regulation/guidelines/toc_tool_en.htm

http://ec.europa.eu/smart-regulation/guidelines/toc_tool_en.htm


Figure 3 : Profile of respondents to OPC26

2.5. Validity and limitations

The evaluation had a grapple with a number of challenges that led us to take certain decisions regarding the approach and pose some limitations on the results. The main challenge was already present in previous evaluations. As described in the evaluation of the 2HP, the programme “co-funds a large number of loosely related actions that attempt to foster collaboration and develop solutions to a wide range of health-related issues and problems in a variety of different ways. This means that assessing the overall impact of the programme (or even measuring it in quantitative terms) is very challenging.”27 This challenge remains valid, meaning we were not required (nor did we seek) to attribute impact in any quantifiable way.

The timing of this mid-term evaluation created additional challenges. The Regulation establishing the 3HP notes “the fact that the achievement of the Programme’s objectives could require a longer time period than its duration”. Nonetheless, DG SANTE must report on the mid-term results of the programme (based on the present evaluation) by 30 June 2017. This means that the research was undertaken at a time when less than half of the programming period had elapsed. Indeed, the actions financed under the Work Programme 2014 started from January 2015 onwards, and the actions under the Work Programme 2015 only at the beginning of 2016. So even if a large share of funding was already allocated to supporting projects and joint actions (as much as 54% of the budget in the first three years), the research was conducted when most actions were near their beginning, while for others (especially under Work Programme 2016) contracts had not yet been signed. In most cases these are actions that last between 12 months (procurement contracts, operating grants, some projects for immediate action on refugees) and36 months (the length of the majority of the other actions, including projects, joint actions and direct grants to international organisations). In practice, this means that over half of the funding under

26 As respondents were able to select more than one answer to describe the two profile questions, the total number of responses add up to more than 133. 27 See in particular section 2.3 of the ex post evaluation of the 2HP.

Provision of healthcare services, 18

Other, 29

Patients and

health service

users, 22

Health professional(s),

35

Health / public health policy making and planning, 61

Health research / education, 42

3

5

11

11

12

25

39

43

Private company

Other

Professionalassociation or trade

union

Internationalorganisation

Academic / researchorganisation

Public authority(national, regional or

local)

Non-governmentalorganisation

Individual / privateperson

22


review is still being implemented. Moreover, the sheer number and heterogeneity of thematic priorities and individual actions precluded in-depth study of all them.

In order to take account of these limitations and provide as much early-stage insight as possible, we proposed an approach which used a relatively high-level assessment of the 3HP’s general features (described above as the ‘programme assessment’) combined with eight case studies of a sample of the programme’s thematic priorities (involving a review of up to five funded actions per case study). To make the most of limited time and resources, we focused throughout the evaluation on features of the 3HP that are either new (such as the re-designed objectives and thematic priorities, Multi-Annual Planning process and response to the migrant crisis) or previously under-examined (such as the “exceptional utility” criterion aimed at incentivising participation among organisations from low-income Member States).

The case studies let us examine the practical implementation of the new thematic priority structure and performance of a number of individual actions. By using purposive sampling, we were able to cover a large part of the 3HP despite focusing on only eight of the programme’s 23 thematic priorities and 29 of the 536 actions funded during the period under review. More specifically, we selected two thematic priorities under each of the 3HP’s four specific objectives, ensuring consideration of all of the 3HP’s broader subject areas, while also covering areas which comprise nearly half of the funding allocated to date (i.e. €76.4m of €165.6m). Similarly, we chose actions under those thematic priorities that as a whole included the range of financial mechanisms. We also included examples of actions that benefit from the “recurrent spending”28 that makes up a large and growing proportion of HP funding.

Overall, the approach chosen provides for a detailed assessment of key new aspects of the 3HP (for instance the new management of multi-annual planning, exceptional utility and the assessment of EU added value criteria in applications). But it does not let us pronounce decisively on the wider applicability of certain findings. Moreover, due to the timing of the evaluation, we have focused the study of effectiveness on implementation and potential impacts that are likely to be generated later. Nonetheless, the evaluation draws evidence-based (and, where necessary, qualified) conclusions and provides insight that can be used to improve the 3HP and feed into the design of the next programme.

28 Recurrent spending is funding which is allocated on a recurring multiannual basis. It includes a range of different types of actions addressing on-going needs. While it accounted for 27% and 28% of funding during the first two years of the 3HP, it is projected to increase to 37% of funding and 2016 and reach 53% in 2017.

23


3. RELEVANCE

Given the EU’s supporting competence in the field of public health, EU action in this field needs to fit strategically into the complex and ever-changing context of health needs of the EU and its Member States. This is also important given the relatively small amount of EU spending in public health (the 3HP has a budget of EUR 449.4 million for the 2014-2020 funding period) compared with such spending by Member State governments and other actors29. Indeed, previous evaluations emphasised the need for more focus and a better concentration of the HP’s limited resources on issues where they can add the most value.

The present chapter deals with the relevance of (in other words ‘need for’) the 3HP and will consider in turn: the validity of its objectives; sufficiency of thematic priorities; and the relevance of actions with the above in mind.

3.1. Validity of objectives

EQ 1: To what extent are the 3HP objectives still valid and in accordance with health needs in Europe?

In line with the EU’s supporting competence in public health as defined in Article 168 of the Treaty on the Functioning of the European Union, the Regulation establishing the 3HP30 defined its general objective as ‘To complement, support and add value to the policies of Member States, in terms of improving the health of EU citizens and reducing health inequalities’. The 3HP’s four specific and operational objectives are:

Table 5: Specific and general objectives of the 3HP

Specific objective Operational objective 1. Promote health, prevent diseases and foster

supportive environments for healthy lifestyles taking into account the ‘health in all policies’ principle;

Identify, disseminate and promote the up-take of evidence-based and good practices for cost-effective disease prevention and health promotion measures by addressing in particular the key lifestyle related risk factors with a focus on the Union added value

2. Protect citizens from serious cross-border health threats;

Identify and develop coherent approaches and promote their implementation for better preparedness and coordination in health emergencies.

3. Contribute to innovative, efficient and sustainable health systems; and

Identify and develop tools and mechanisms at Union level to address shortages of resources, both human and financial, and facilitate the voluntary up-take of innovation in public health intervention and prevention strategies.

4. Facilitate access to better and safer healthcare for Union citizens.

Increase access to medical expertise and information for specific conditions also beyond national borders, facilitate the application of the results of research and develop tools for the improvement of healthcare quality and patient safety.

29 While public health expenditure is difficult to tally and compare, figures from 2014 indicate that Germany alone spends EUR 321bn annually, while France spends EUR 237bn. For more information, see http://ec.europa.eu/eurostat/statistics-explained/index.php/Healthcare_expenditure_statistics#Healthcare_expenditure

30 Regulation (EU) 282/2014. 24

http://ec.europa.eu/eurostat/statistics-explained/index.php/Healthcare_expenditure_statistics%23Healthcare_expenditure

http://ec.europa.eu/eurostat/statistics-explained/index.php/Healthcare_expenditure_statistics%23Healthcare_expenditure


The subject of this evaluation question is to examine the extent to which the specific objectives are:

• in accordance with evolving health needs in Europe;

• valid, in that the correct issues were identified during the design of the 3HP.

Accordance with EU health needs

For the most part the objectives represent continuity with the previous HP31. However, as noted in the literature for the 3HP32 and health experts consulted through focus group discussions, the shifting health landscape has brought a number of new and existing issues to the fore, namely:

• An ageing population, threatening the financial sustainability of health systems and causing health workforce shortages;

• A fragile economic recovery, limiting the availability of resources to invest in healthcare;

• An increase in health inequalities between and within Member States;

• An increase in the prevalence of chronic disease;

• Pandemics and emerging cross-border health threats;

• The rapid development of health technologies;

• Increase in mental health problems (particularly among the young);

• Other specific emergency situations which expose EU health professionals to unprecedented challenges (for example, dealing with the repercussions of the influx of refugees); and

• Threats to environmental health such as air quality and pollution monitoring.

The objectives of the 3HP were designed with many of these needs and challenges in mind, but also aimed to build in the flexibility to respond to new challenges. More specifically:

The operational objective for Objective 1 now explicitly mentions the need to promote cost-effective measures, reflecting the growing strain on national health systems in the wake of the economic crisis. Under Objective 1 there is also a specific focus on chronic diseases which have become increasingly prevalent and therefore increasingly costly for MS to tackle (under thematic priority 1.4)33. Objective 1 includes the only reference to mental health and it is in relation to tackling chronic diseases including “neurodegenerative diseases” (thematic priority 1.4).

The operational objective for Objective 2 specifically addresses preparedness and coordination in recognition of the need for greater EU capacity to deal with health emergencies or crisis situations (including pandemics). Although the 3HP has not explicitly introduced migrant health as a topic, however the following reference was incorporated under thematic priority 2.2: "address the increasing

31 The objectives of the 2HP: 1. Improve citizens’ health security; 2. Promote health; 3. Generate and disseminate health information and knowledge (Annex to the 2HP) 32 For example, the Impact Assessment for the 3HP {SEC(2011) 1322 final} describes many of these challenges in section 2.2.2 on ‘Major health challenges for health in the EU’. 33 Five conditions (diabetes, cardiovascular diseases, cancer, chronic respiratory diseases and mental disorders) account for an estimated 86% of the deaths and 77% of the disease burden in the European Region (which includes the EU)

25


health threats resulting from global population movements". Similarly, at present the 3HP does have a reference to climate change and the environment as part of Objective 2 (thematic priority 2.334).

Objective 3 on innovative, efficient and sustainable health systems, includes a number of new elements. It addresses innovation more directly than in previous funding periods. By mentioning the role innovation can play in dealing with resource shortages, this objective places further emphasis on the EU added value criterion of fostering economies of scale and thereby ensuring a more efficient use of limited resources. The objective also deals with the rapid development of health technologies and how to harness them for more efficient health systems. The new reference to sustainability is important in light of an increasingly challenging demographic context, which is threatening the sustainability of health systems.

Objective 4 places emphasis on access to medical expertise and thereby brings together previously disparate aspects regarding patient safety and treatment of rare diseases in a way that aims to create efficiencies and tackles issues which are difficult for MS to deal with alone. The objective also deals with the increasingly pressing global challenge of anti-microbial resistance.

Given the above, it is unsurprising that health experts consulted through focus group discussions found all of the objectives highly relevant and responsive to important health needs (in the ways already expressed above). Some individual experts would have put greater emphasis on certain issues (such as migrant health and environmental health), but for the most part these concerns more reflected their specialisms more than shortcomings in the programme objectives.

The only criticism where a consensus emerged was in the field of mental health. Here, a large proportion of focus group participants felt that the 3HP does not adequately reflect the current needs facing EU population (such as those relating to depression, stress and burnout). It could therefore make a more explicit commitment to raising awareness and driving policies to tackle mental health. Indeed, the framework is already there, since the Regulation establishing the 3HP cites the WHO definition of health (“health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity”). At present this holistic approach is not encapsulated in the objectives (or thematic priorities), except in relation to neurodegenerative diseases under chronic diseases.

Relevance of health programme objectives

At a high level the evidence shows that the specific objectives of the 3HP remain valid. According to the wide variety of stakeholders consulted, including public health experts, NFPs and PCs, as well as EU officials and the wider public (through the open public consultation), all four objectives address key health needs.

However, stakeholders also felt the rationale for EU action and potential EU added value were stronger for some objectives than for others. For instance, NFPs and PCs responding to the online survey distinguished between the different objectives, finding the first two (on health promotion and cross-border health threats) even more important as areas for EU action than the second two (on innovative, efficient and sustainable health systems and facilitating access to safer healthcare). This was also true of the replies to the open public consultation35. The follow-up mini-survey which targeted NFP

34 Which refers to health threats such as “those caused by … environment and climate change”. 35 The results of the OPC (annex A, section 10) show that respondents consider all objectives of the health programme relevant but are more strongly in agreement regarding the relevance of Objectives 1 and 3.

26


and PC members asked for elaboration on the responses provided, namely by describing the single most important reason for EU action (as opposed to action at the national, regional or local level). The responses for each objective are summarised below.

Table 6: Most important reasons for EU action

Objective Most important reason(s) given for EU action by mini-survey respondents

1. Promote health, prevent diseases & foster supportive environments

Exchange of best practice and expertise, potential for mutual learning between MS. At the EU level it is possible to promote activities and engagements that would not otherwise be a priority for national government. Also pooling resources was mentioned.

2. Protect citizens from serious cross-border health threats

The fact that communicable diseases are trans-boundary in nature and require a trans-boundary response (the EU can support development of early warning systems which prevent the rapid spread of communicable diseases).

3. Innovative, efficient & sustainable health systems

Exchange of best practices and innovation. EU action supports MS especially vis-à-vis eHealth and the EIP on active and healthy ageing.

4. Facilitate access to better and safer healthcare

Responses here were wide ranging, somewhat reflecting the objective itself. Reponses ranged from citing specific areas like AMR and ERN where economies of scale can be achieved through EU level action to the establishment of common standards to prevent disease and ensure equal treatment of all EU citizens thus creating greater cohesion among MS.

The exercise illustrates that while respondents consider EU action necessary under all of the objectives, their reasoning and relative prioritisation differs. What emerges is a dichotomy between the rationale for EU action under Objectives 1 and 3 on the one hand and Objectives 2 and 4 on the other.

• Objectives 1 and 3 concern issues that are very high on MS’ health policy agendas (such as prevention of chronic diseases, or healthcare system reform).36 At the same time, they are also issues which require much greater resources than those at the HP’s disposal. In these areas the HP is seen less as a big source of funding than as a facilitator in the exchange of best practices and mutual learning. This can in turn lead to a bigger contribution than the 3HP’s relatively small budget would suggest.

• Objectives 2 and 4, on the other hand, are better rooted in EU legislation and relate to cross-border challenges (such as epidemics) where coordinated action is necessary and / or issues where collaboration between countries could generate economies of scale (such as rare diseases, where the incidence in individuals in Member States is often too low to justify substantial action). These areas attract less attention in the Member States, but the EU role is clearer in the sense that MS would struggle to address them on their own.

36 As evidence in the review of Member State health strategies in annex A (section 4) 27


Conclusion

Previous evaluations emphasised the need for more focused objectives that concentrate the EU’s limited resources on the issues where they can add the most value37. The present set of specific and operational objectives have largely achieved this while providing a framework to accommodate both existing needs, such as chronic diseases, and emerging challenges, such as an ageing population and cross-border health threats. The only obvious area concerns mental health problems, such as depression. Experts consulted for the evaluation felt that despite the ability of the HP to accommodate action on mental health, that this area deserved greater emphasis given the growing scale of the problem and its myriad effects on young people and the health workforce.

The objectives also allow for action geared towards providing EU added value in different ways. While objectives 1 (health promotion) and 3 (health systems) cover areas where national-level action pre-dominates, the 3HP can add value by facilitating and promoting the spread of best practices and mutual learning. Objectives 2 (cross-border health threats) and 4 (access to healthcare) address issues that are smaller in scale, but where coordinated action is needed to ensure security and / or generate economies of scale.

Thus the evaluation finds that the objectives of the 3HP are broadly valid and appropriate. While it could be argued that they should be even more specific, this would make it difficult to target action at the wide range of policy areas that the programme needs to support. It could also undermine the flexibility that has seen the 3HP respond quickly to emerging health needs (such as the migrant crisis).

37 As stated in the Regulation establishing the 3HP: “The Programme should … concentrate support on a smaller number of activities in priority areas. The emphasis should be placed, in accordance with the principle of subsidiarity, on areas where there are clear cross-border or internal market issues at stake, or where there are significant advantages and efficiency gains from collaboration at Union level” (EU Regulation 282/2014, Para 5)

28


3.2. Appropriateness of thematic priorities

EQ 2: To what extent are the 3HP thematic priorities sufficient and sufficiently covered to achieve the 3HP objectives and Commission wider priorities?

The objectives outlined above are further broken down into 23 “thematic priorities” to operationalise the HP. Abridged versions of the thematic priorities are presented below. This structure is new to the 3HP (under the 2HP the three specific objectives were divided into six priorities which were further broken down into 19 sub-priorities).

As stipulated in the Regulation establishing the 3HP, the mid-term evaluation is expected to provide findings on the adequacy of the thematic priorities for achieving the 3HP’s objectives. This in turn should help the Commission to determine whether: “renewal, modification or suspension of the thematic priorities” is necessary. Here, we examine the relevance of the thematic priorities, which forms an important part of any judgement about potential changes to them (see also Chapter 4 and Chapter 5 on effectiveness and efficiency respectively, for more discussion surrounding the implementation and design of the thematic priorities).

Table 7: Thematic priorities of the 3HP

Thematic priorities (abridged text - provided in full in Annex I of 3HP Regulation)

Objective 1: Promote health, prevent disease and foster supportive environments for healthy lifestyles

1.7 Risk factors such as use of tobacco and passive smoking, harmful use of alcohol, unhealthy dietary habits and physical inactivity

1.8 Drugs-related health damage, including information and prevention 1.9 HIV / AIDS, tuberculosis and hepatitis 1.10 Chronic diseases including cancer, age-related diseases and neurodegenerative diseases 1.11 Tobacco legislation 1.12 Health information and knowledge system to contribute to evidence-based decision making

Objective 2: Protect Union citizens from serious cross border health threats

2.5 Risk assessment additional capacities for scientific expertise 2.6 Capacity building against health threats in Member States, including, where appropriate, cooperation

with neighbouring countries 2.7 Implementation of Union legislation on communicable diseases and other health threats, including

those caused by biological, and chemical incidents, environment and climate change 2.8 Health information and knowledge system to contribute to evidence-based decision-making

Objective 3: Contribute to innovative, efficient and sustainable health systems

3.8 Support voluntary cooperation among MS on HTA 3.9 Innovation and e-health 3.10 Health workforce forecasting and planning 3.11 Setting up a mechanism for pooling expertise at Union level 3.12 European Innovation Partnership on Active and Healthy Ageing 3.13 Implementation of Union legislation in the field of medical devices, medicinal products and cross-border

healthcare 3.14 Health information and knowledge system including support to the Scientific Committees set up in

accordance with Commission Decision 2008/721/EC

Objective 4: Facilitate access to better and safer healthcare for Union citizens

4.7 Establishment of a system of European reference networks 4.8 Effectively help patients affected by rare diseases 4.9 Strengthen collaboration on patient safety and quality of healthcare 4.10 Measures to prevent Antimicrobial resistance and control healthcare-associated infections 4.11 Implementation of Union legislation in field of tissues and cells, blood, organs, 4.12 Health information and knowledge system to contribute to evidence-based decision making

29


This evaluation question considers the thematic priorities in terms of the extent to which they are:

• well defined (i.e. clear and focused);

• consistent with the specific objectives and MS needs; and

• consistent with the Commission’s wider priorities.

Clarity and focus of the thematic priorities

The definition of 23 thematic priorities represents an important step for the HP as it seeks to become more coherent and focused. As explained in the ex post evaluation of the 2HP, the previous programme was organised in such a way that nearly any topic could have been considered relevant, while there was significant overlap in terms of how actions were classified. In contrast, the thematic priorities are the result of efforts to define discrete areas for EU action. In theory, this should lead to more focused action and make it easier to monitor the deployment of programme resources. Numerous examples from case studies illustrate that in practice the thematic priorities have helped increase the focus of HP action. For instance, one thematic priority under objective 2 (i.e. cross-border health threats) relates to capacity building against health threats in the MS38. This consolidates into a single budget line a set of issues which were previously spread across at least six of them39. More broadly, the OPC asked whether the objectives and priorities were clear and easy to understand to which the response was positive (over 60% agreed or strongly agreed).

Despite these generally positive findings, the evaluation uncovered some criticism of the way some of the thematic priorities are worded. For example, some participants in focus groups with public health experts felt that the thematic priorities were both too general and too numerous to help increase the focus of the HP. Reponses to the open public consultation included: “The programme is too fragmented.”40 Similarly, some EC officials thought the thematic priorities remain too vague in some cases to facilitate the drafting of precise calls for proposals which is a necessary precursor for well-defined actions. There was also a recognition that among EC officials interviewed that there are significant synergies between certain thematic priorities. While this is positive, there is a fine balance between synergies and overlap.

This was evident in the case studies on specific thematic priorities. For instance, while chronic diseases are addressed through a distinct thematic priority (1.4), several other thematic priorities touch on related aspects. In particular:

• Some major chronic diseases, such as specific types of cancer, are linked to communicable agents, which therefore connect thematic priority 1.4 with 1.3 on “HIV/AIDS, tuberculosis and hepatitis”.

• In many cases, cancer is related to life-style health risk factors such as smoking, alcohol consumption and obesity. This links thematic priority 1.4 with 1.1 on “risk

38 see Point 2.2 of Annex I of the 2014 Programme Regulation 39 The 2HP had three objectives and was then divided into priorities and sub-priorities. Some examples of the sub-priorities from the 2HP which in the 3HP were consolidated into a single thematic priority are: developing strategies and mechanisms for preventing, exchanging information on and responding to health threats (1.1.1), improving partnerships, networks, tools and reporting systems for immunisation status and adverse events monitoring (1.1.2), developing risk management capacity and procedures (1.1.3), and others 40 Although as explained under Coherence (Chapter 7), overall the perception was that the 3HP was more focused than its predecessor.

30


factors such as use of tobacco and passive-smoking, harmful use of alcohol, unhealthy dietary habits and physical inactivity”.

• Thematic priority 1.4 builds on the actions funded under the European Innovation Partnership on Active and Healthy Ageing (EIP-AHA), which is also the focus of thematic priority 3.541.

• Thematic priority 1.4 also covers innovative prevention and management of chronic diseases, connecting it to objective 3 of the 3HP which supports actions that “contribute to innovative, efficient and sustainable health systems".

In addition, as detailed in the case study on thematic priority 2.3 because the unique purpose of this thematic priority is to support the implementation of Decision 1082/2013/EU42, there is potentially significant overlap with the other three thematic priorities under this objective, which also support this Decision in different way and more implicitly.

In part, these criticisms illustrate the complexity, inextricable linkages and mutually supportive nature of public health needs and challenges. Nonetheless, there are legitimate concerns regarding the disparities between the thematic priorities. Some are narrower and more focused (i.e. 4.1. “Support the establishment of a system of European reference networks for patients requiring highly specialised care and a particular concentration of resources or expertise”) than others (i.e. 1.4. “Support cooperation and networking in the Union in relation to preventing and improving the response to chronic diseases…”). This can lead to problems for monitoring and evaluation. For example one criticism from the case study on thematic priority 3.4 (which seeks to support the pooling of expertise at Union level) was that “the underpinning logic for the priority does not sufficiently consider how to measure impact and account for the expenditure of funding.” As discussed in section 5.4 on the monitoring provisions of the 3HP, without reference to a measurable goal, it is difficult to focus activity on effecting behavioural or policy change.

In many cases, the formulation of thematic priorities may be linked to the differing rationale for EU action. Where action is justified on the basis of reducing disparities and / or spreading best practices (rather than the result of a legal or practical necessity), the scope for EU action is generally broad and difficult to define in advance. This makes it harder to define narrow and precise thematic priorities. For example, thematic priority 4.1 on European Reference Networks is rooted in EU legislation43. By contrast, thematic priority 1.4 on chronic diseases does not relate to specific EU legislation, but rather provides a framework for action where it makes sense, on a case-by-case basis. As with our findings in the previous section on the relevance of the 3HP’s objectives, this is not to say that there is less room to generate EU added value under these thematic priorities. Rather, it means simply that the scope and parameters for such action is to a lesser extent defined in advance, and must instead be considered carefully during the planning process (for more information see sections 4.1 and 4.2 on Annual Work Programmes and Multi-annual planning, respectively).

41http://ec.europa.eu/health//sites/health/files/programme/docs/factsheet_healthprogramme2014_2020_en.pdf 42 which aims to improve health security in the European Union and the protection of the Union's citizens from communicable diseases, and other biological, chemical and environmental events. 43 Specifically, Directive 2011/24/EU on the application of patients’ rights in cross-border healthcare has a chapter on European Reference Networks, stating among other things that “The Commission shall support the Member States in the development of European Reference Networks […] in particular in the area of rare diseases”.

31

http://ec.europa.eu/health/sites/health/files/programme/docs/factsheet_healthprogramme2014_2020_en.pdf

http://ec.europa.eu/health/sites/health/files/programme/docs/factsheet_healthprogramme2014_2020_en.pdf


Other thematic priorities have been formulated to ensure coverage of certain cross-cutting issues across the four specific objectives. For example, while to “Generate and disseminate health information and knowledge” was an Objective under the 2HP, for the 3HP each objective includes a thematic priority which allows for actions to “Foster a health information and knowledge system to contribute to evidence-based decision-making…” A similar approach is taken to actions which support the implementation of EU legislation44.

This way of organising thematic priorities contributes some uniformity and also makes it easier to track the allocation of spending in different thematic areas. However it contrasts with the more targeted, subject-specific focus of the rest of the thematic priorities. This creates some ambiguities and makes it harder to identify or justify the organising principles behind the design of the HP. As discussed in Chapter 5 on efficiency, this has played a role in the sometimes uneven allocation of funding across thematic priorities.

Consistency of the thematic priorities with MS priorities and needs

Defining thematic priorities presented DG SANTE with a significant challenge. On the one hand, it is widely recognised that the scope of the 3HP had to be defined more narrowly than was previously the case. At the same time, the 3HP still needs to allow for enough flexibility to fund relevant actions across a seven-year funding period, within which needs and priorities may change and evolve.

In general, the desk research and consultation with key stakeholders show that the thematic priorities are relevant and reflect health needs. More specifically, they mostly relate to explicit health problems that MS face (e.g. diseases such as non-communicable chronic diseases, infectious chronic diseases such as HIV/AIDs and pandemics as well as AMR and issues relating to healthcare capacity such as the status of the health workforce). This was evident from our review of Member State health strategies (see annex A, section 4) and corroborated by consultation of stakeholders (through our online survey of NFP and PC members, as well as the open public consultation). Concerning the online survey of NFPs and PCs, where respondents largely agreed that the thematic priorities reflected important issues for their countries. Likewise, replies to the OPC also confirmed that the priorities (and objectives) are consistent with the health policy objectives of a respondents’ country.

While Member State health policy documents were not consistently available enough for us to rank their priorities, a follow-up survey with NFPs and PCs asked them to identify the top three thematic priorities per objective45. The OPC was also designed to solicit views on which are the most important thematic priorities46. The responses to both these consultations provide insight into which thematic priorities are considered most relevant in aggregate by competent MS authorities and wider stakeholders.

The key findings from the survey of the NFP and PC and the OPC were as follows:

• Objective 1 (health promotion): Risk factors (1.1) and chronic diseases (1.4) were overwhelmingly cited as most important in both surveys (by 92% and 96% of

44 In all objectives, there is a pattern of one thematic priority which is dedicated to supporting the implementation of Union legislation in the relevant field. 45 Indeed, in light of the issues faced by this absence of complete information, the 3HP is funding direct grant agreements for the WHO and OECD to prepare country specific reports to develop a clear understanding of specific needs of each country (this is also financed by the Health Programme – direct grants in 2016 and 2017). 46 Although it is important to note that the questions were structured differently for the targeted survey (for NFP and PC members) and OPC. More specifically, the targeted survey asked respondents to select the top three thematic priorities under each objective, whereas the OPC asked for the most important thematic priorities overall.

32


follow-up survey respondents, respectively and 48% and 60% of OPC respondents respectively).

• Objective 2 (cross-border health threats): The most frequently selected thematic priority under objective 2 was capacity-building against health threats in Member States (2.2) (76% of the follow-up survey respondents and 28% of OPC respondents). Health information and knowledge systems to evidence-based decision-making (2.4, where action to date has been undertaken by the European Centre for Disease Control rather than the HP) and the Implementation of EU legislation on communicable diseases (2.3) were also selected as most relevant / important by respondents (more than 60% of follow-up survey respondents and just over 20% of OPC respondents).

• Objective 3 (health systems): Almost every NFP and PC member who responded to the follow-up survey considered that Innovation and eHealth (3.2) are the most important thematic priority in this context (92% of respondents). This was also the frequently selected of the thematic priorities under objective 3 for the OPC respondents (with 23% of respondents selecting this thematic priority).

• Objective 4 (access to healthcare): Respondents to the follow-up survey were nearly unanimous that the leading thematic priorities are the development of Measures to prevent AMR and control healthcare-associated infections (4.4), as well as Patient safety and quality of healthcare (4.3) (92% and 80% respectively). The same pattern emerged from the OPC replies (with these thematic priorities deemed the most important by 40% and 32% of respondents respectively).

While survey respondents did not point out areas that were missing from the 3HP, focus groups with public health experts and responses to the OPC provided some suggestions for areas which could be better covered. Some focus group participants pointed to mental health, which was cited as an area where the pooling of resources and sharing of best practices could help reduce the considerable disparities in capacity between Member States47. Other stakeholders provided additional feedback through the OPC, but the replies were diverse and sometimes cited areas (such as diabetes, hepatitis C and cancer prevention) which are covered by the HP already, or very specific areas (such as endocrine disruptors in air, soil and water pollution) which are arguably better covered by other initiatives48.

Consistency of the thematic priorities with EC policy objectives

The last part of the question relates to whether the thematic priorities are consistent with EC policy priorities. This was explored as part of the case studies but also through interviews with EC officials. While the Chapter on coherence (Chapter 7) deals with the internal and external coherence of the 3HP more fully, it is worth emphasising here that evidence from the thematic case studies illustrates that EC policy priorities are well reflected in the thematic priorities.

To give a concrete example, there are strong links between actions under thematic priority 1.4 and EU initiatives such as the Europe 2020 Strategy and Horizon 2020, which are funding a significant amount of research to improve the health and quality of life of older people. There are also links with the New Skills Agenda for Europe, which forms part of Commission President Junker’s broader EU strategy where employment is a key pillar, as well as with international initiatives supported by the EU such as the

47 Although mental health is considered as a major chronic disease (and therefore included under thematic priority 1.4), the point made was that this does not receive sufficient emphasis. 48 For instance, DG Environment is doing a lot of work in this area, see

http://ec.europa.eu/environment/chemicals/endocrine/index_en.htm 33

http://ec.europa.eu/environment/chemicals/endocrine/index_en.htm


Sustainable Development Goals (SDGs), WHO Global Action Plan for the Prevention and Control of Chronic Diseases and the Observatory on Dementia.

Conclusion

In setting 23 thematic priorities for action, the designers of the 3HP sought to define more precisely what the programme is for and increase the coherence of its structure. The evaluation found not only that this effort has been successful, but that in general the thematic priorities are relevant and valid. They have served to consolidate previously diffuse actions under coherent thematic priorities, while focusing minds on specific areas that are high priorities for the Member States and where there is a potential to generate EU added value. They also align well with wider EC policy objectives.

Nonetheless, a number of inconsistences remain. For example, thematic priorities rooted in EU legislation (such as European Reference Networks) are typically more precise than those related to larger policy areas (such as chronic diseases). We also noted some overlaps and potential duplications. While these are not damaging to the programme per se, a future effort to refine or streamline the thematic priorities could lead to a more coherent programme structure. For example, arguably, objective 2 could be streamlined further to focus on capacity building and implementation of legislation (i.e. the present 2.2. and 2.3). In addition, there are cases where thematic priorities have not been utilised (for instance 2.4 and 4.6 which cover “fostering a health information and knowledge system”) and were included for the sake of consistency but actually in the case of objective 2, this priority is already mainly served by the ECDC's mission.

34


3.3. Relevance of actions

EQ 3: To what extent are the actions prioritised in the Annual Work Plans (AWPs) relevant vis-à-vis the 3HP thematic priorities?

Each year an AWP sets out a plan for the practical implementation of the 3HP. Among other things, the AWP identifies which types of actions (in which subject areas) will be pursued to support given thematic priorities and defines the funding mechanisms provisional budgets that will be available. The 3HP makes use of several financial mechanisms to operationalise the thematic priorities, which are described in chapter 1, Table 1 of this report. This question seeks to understand whether actions prioritised in the AWP support progress under the each thematic priority by exploring whether actions:

• are clear and well defined; and

• consistent with the thematic priorities.

To answer this question, we rely primarily on the evidence collected through the case studies of eight thematic priorities as these have provided the most detailed insights into individual actions. Where relevant we also draw on other sources, such as interviews with EC officials.

Generally well-defined action with clarity of purpose

The case studies showed that the vast majority of actions have indeed been well-defined and have a clear purpose. Indeed, the case study on thematic priority 3.649 (on the implementation of EU legislation on medical devices) was typical. Based on interviews with Chafea, DG SANTE and DG GROW, it found that with clear objectives and a good description of technical aspects, few unexpected problems (with actions) are arising.

Other examples were numerous. The clear, precise definition of thematic priority 4.1 on European Reference Networks has led to a well-designed and purposeful joint action to put recommendations on ERNs into practice, as well as service contracts that are being used for support. The case study on thematic priority 2.3, which deals with the implementation of Decision 1082/2013 on cross border health threats, provides another positive example. This cast light on a joint action designed with the express aim of filling gaps in technical expertise among Member State authorities and thereby increasing their ability to fulfil the requirements of EU legislation. In both cases, the highly-focused nature of the thematic priorities helped ensure that funded actions corresponded to identified needs.

In the few cases where room for improvement was identified, the precise reasons vary but interviews with EC officials suggest that the most important factor relates to the thematic priorities themselves. More specifically, the actions funded under more vaguely worded or wide-ranging thematic priorities could end up with unrealistic or overly ambitious goals.

An example of this can be found in the case study on thematic priority 1.4, which relates to the very broad topic of chronic diseases. Here, we found that the broad scope of thematic priority is leading to over-ambitious actions with limited focus on their objectives. More specifically, the initial proposal for a Joint Action on Dementia 2015-

49 (which concerns the implementation of legislation in the field of medical devices, medicinal products and cross-border healthcare)

35


2018 listed only vague objectives50. While the parameters of this action were narrowed down during the negotiated procedure, it shows that broadly defined calls could lead to highly varied applications, making it hard for assessment panels to compare proposals and identify the strongest ones.

Another example comes from the case study of thematic priority 3.4 which deals with pooling of expertise at Union level. Here the case study found that the thematic priority had an unclear purpose, making it difficult to measure impact and account for the expenditure of funding.

These examples illustrate that, for the broader thematic priorities, substantial thought is needed in the planning and design phases to ensure actions fit well with public health needs. This is especially the case for projects and operating grants, which deliberately allow beneficiaries considerable leeway in defining how and what HP funding is used to do. While issues can and should be overcome through the evaluation process, there are nonetheless real concerns as expressed by interviewed stakeholders that operating grants in particular are not sufficiently focused and thus risk a situation whereby the HP is financing the functioning of non-governmental bodies whose actions and objectives are very general.51

Alignment between actions and thematic priorities

In terms of the link between actions with the thematic priorities and objectives they support, the findings from the case studies which specifically examined this aspect were overwhelmingly positive. However, as explained previously, this does not necessarily translate into clarity of purpose. Contrarily, the more ambiguous or wide-ranging the thematic priorities the more likely it is that actions are consistent with them. This is explored more in Chapter 4 on effectiveness.

Conclusion

In general, the 3HP’s structure of relevant objectives and appropriate thematic priorities to operationalise these objectives has served to ensure the relevance of individual actions. Despite the diversity of issues addressed, the eight case studies have shown that, by and large, the actions funded have corresponded to public health needs and demonstrated clear and suitable objectives.

However, there were some disparities depending on the thematic priority in question. The actions we examined under the more precisely worded and narrowly defined ones (e.g. 4.1 on European Reference Networks) were all highly relevant, whereas some of those funded under the broader thematic priorities (e.g. 1.1 on risk factors) raised concerns. This highlighted the need for particular attention in the planning stages of actions in such areas. Moreover, the nature of certain funding mechanisms (i.e. operating grants) is that they support a mission rather than specific activities. This speaks to the importance of assessing the relevance of that mission and monitoring and evaluation processes that could keep track of how organisations benefiting from this funding mechanism perform in relation to the HP policy priorities.

50 Once this was flagged by the evaluation of the proposal - the beneficiary was able to refine the objectives, increasing their focus and making a more clear reference to the action’s potential EU added value - i.e. to support the successful uptake of evidence-based practices on improving the quality of life for people living with dementia and their carers across the EU. 51 As an example, look to the case study on thematic priority 1.1 on risk factors. The operating grant proposal for OBSTAINS (implemented by the World Obesity Federation - WOF) had a list of activities but no strategy behind them.

36


3.4. Objectives in Article 168

EQ 4: To what extent are the actions co-funded through the AWPs relevant to achieving the objectives set out in Article 168 TFEU?

This evaluation question is addressed under evaluation question 15 on the coherence and consistency of the 3HP (see chapter 7).

37


4. EFFECTIVENESS

In this section we examine the Programme’s effectiveness. Given the relatively early stage of implementation of the 3HP, the focus is on:

• management processes, including those introduced as part of the 3HP;

• new criteria for co-financing, which aim to increase participation in the HP of organisations from low GNI countries; and

• early indications of the 3HP’s contribution to programme and EU objectives.

The ensuing subsections elaborate on each of these aspects in turn before forming a conclusion regarding effectiveness as a whole.

4.1. Process for defining Annual Work Programmes

EQ 5: To what extent is the process for defining and prioritising actions through Annual Work Programmes (AWPs) transparent, equitable and impartial?

The Regulation establishing the 3HP provides the legal basis for Union action in the field of health. This is then operationalised through the adoption of AWPs52. Draft AWPs are prepared by DG SANTE in consultation with the Programme Committee, which is comprised of representatives of the HP’s Participating Countries. The AWP is adopted after a formal vote in the Committee53.

This section examines the extent to which the process for defining and prioritising actions through the AWP is:

• transparent in terms of being open to scrutiny to relevant stakeholders;

• equitable, in that it includes fair participation of and consideration of concerned actors; and

• free from political bias.

Transparency: a (mostly) well-documented process

To figure out whether the process for drafting and adopting AWPs is transparent, we looked into how it has been documented. The more comprehensive and accessible the ‘paper trail’ behind the AWPs, the more transparent we would determine it to be. A review of the available documentation and interviews with interested actors on both sides of the process show a high level of transparency but also some room/perceptions that it could be improved.

The formal consultation process includes two rounds of discussions and a final agreement with the Programme Committee, which is comprised of Member State representatives.54 Draft versions of the AWP are presented, commented and voted on, while a summary of the procedure, views expressed and decisions taken is recorded, with minutes submitted to the European Parliament. Given this is made publicly

52 The process for drafting these is facilitated by a Multi-Annual Planning (MAP) process which is an internal planning and consultation process involving relevant units (this process is new to the 3HP and is examined in detail in the next sub-section). 53 The provisions for the Committee Procedure are set out in Article 17 of the Regulation establishing the 3HP, and regulated by the EU Regulation no 182 /2011 http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32011R0182&from=EN 54 As outlined in the Rules of Procedure for the Committee for the 3HP (Ref. Ares (2015)705325)

38

http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32011R0182&from=EN

http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32011R0182&from=EN


available, it can thus be considered to be both well-documented and open to scrutiny.

Other actors are also formally consulted at various stages. An Inter-service consultation mechanism55 allows other European Commission DGs with relevant expertise to share their views. It is through this mechanism that DG SANTE gathers the views of other DGs on the AWP. One of the purposes of this consultation is to ensure compliance with the legal basis and complementarity of actions, for example to cross-reference complementary actions pursued by different DGs. Given its more practical expertise, Chafea is also formally consulted both before and during the inter-service consultation.

There is also a certain amount of informal consultation. This is not explicitly required, but it allows for other interested actors to share their views and feed into the AWP development process. For example, in addition to the formal consultation, successive drafts of the AWP may be shared with Chafea for feedback. External stakeholders, such as the WHO, may also be consulted informally about the actions of the draft AWP.

While this informal consultation is valuable both to DG SANTE and to the actors whose views are elicited, it is by definition less transparent than the formal consultation process. Moreover, stakeholders expressed some confusion as the purpose and fit of this informal consultation within the decision-making structures of the 3HP. In particular, the WHO reported that it is difficult to follow and understand the decisions being made.

Moreover, some OPC respondents also asked for greater transparency. This implies that both the formal and informal consultation processes are poorly understood outside the 3HP’s core stakeholder groups.56 This shows that the AWP process and role of different types of consultation could be better explained and potentially refined in order to buy in actors such as the WHO.

The establishment of the Multi-annual planning (MAP) process (which is assessed in detail in section 4.2) has also influenced the transparency of the AWP process. Since the start of the 3HP, this systematically feeds into the drafting of the AWP and is well-documented: the background and results are filed using the Commission e-domec system57. However, because the MAP is essentially an internal planning tool it is only accessible to those involved in the strategic and day-to-day management of the HP, until it is finalised. At this point it is made available to the Programme Committee.

The order of procedure is important because the early scoping and preparatory work of the MAP defines the strategic direction of the HP and thereby provides a conceptual framework for the more operational aspects that are discussed on a yearly basis as part of the AWP process. In other words, the AWP consultation process takes place after key discussions have already taken place and decisions made.

The establishment of MAP is a welcome development (and indeed reflects recommendations in previous evaluations of the HP). However, since it does not incorporate wider stakeholder consultation, particularly with the MS, in some ways it reduces the transparency of the programming process in favour of coherence and

55 C(2014) 9004 http://ec.europa.eu/transparency/regdoc/rep/3/2014/EN/3-2014-9004-EN-F1-1.Pdf 56 For instance, one respondent said: “The content of the annual work programmes and priorities setting is not transparent and does not sufficiently involve health related stakeholders.” While another said: ““The current arrangement does not support EU stakeholder participation, this should be made a requirement and the whole process made more transparent.” 57 See, for example “Multi annual planning for the Third Health Programme”, Ref. Ares (2014)1616439 – 19/05/2014 and “Background to the multi-annual planning exercise 2017 - 2020”, Ref Ares (2016) 1542265 – 31/03/2016

39

http://ec.europa.eu/transparency/regdoc/rep/3/2014/EN/3-2014-9004-EN-F1-1.Pdf


expediency. While this was a conscious choice, the evidence from interviews suggests that it has not been well understood by some external stakeholders (e.g. international organisations) and could thus be more clearly explained, or adapted to bring them into the process.

Scope for more equitable consultation

The equity of the process is judged by the existence of participation mechanisms for relevant stakeholders and their perceptions about the fairness and usefulness of these mechanisms.

Regarding the existence of participation mechanisms, as outlined above the evidence suggests these are systematically available to some but not all relevant stakeholders. The Rules of Procedure for the Committee for the 3HP58 ensures the representation of MS (and participating countries) as well as the possibility for nominated experts. This is especially important in the context of joint actions which, under the 3HP, are agreed by negotiated procedure and all MS have the opportunity to state their preferences as well as to nominate a participant. The Inter-Service Consultation mechanism also involves other DGs and Chafea.

However, other stakeholders are consulted in a less systematic and / or formal way, if at all. For instance, when it comes to external stakeholders (e.g. the WHO or the OECD), since there is no legal requirement, there is no formal process to gather views on the appropriateness of activities proposed59. Such stakeholders felt that, given the complementarity of HP actions and activities carried out by external actors, this could result in a lost opportunity to create or strengthen synergies over the longer term. As pointed out above, at the least this shows that the decision-making processes are not fully understood by external stakeholders.

Using an online survey, we also asked Programme Committee members about their views on the equity of the AWP process. The results showed that PCs were generally positive, in particular with regard to the participation of other concerned actors (such as other DGs, Chafea, etc.). The participants also appeared satisfied with the weight given to stakeholders’ views in the adopted AWP. Interviews with representatives from other DGs showed that they were also positive about the consultation process.

Where the PC members were slightly less complimentary was regarding the participation of national governments (not necessarily their own) through their involvement in the 3HP Programme Committee (one in five respondents thought it was not very adequate). This illustrates that even with a process in place, certain MS may not be as motivated or may even struggle for practical reasons (for instance, insufficient capacity) to be proactive in ensuring their views are reflected in the activities supported by the 3HP. At the other end of the spectrum, external stakeholders such as the WHO reiterated that the absence of a formal consultation process is sub-optimal60.

Responsibility for setting the HP agenda

The examination on the impartiality of the AWP process is based on the perceptions of key stakeholders and experts. Overall, the evaluation finds that the HP’s priorities and actions are defined based on evidence and public health needs (as explained in the previous chapter on relevance). However, some Member State representatives have expressed concerns with DG SANTE’s ultimate responsibility for setting the HP’s

58 The Rules of Procedure for the Committee for the 3HP (Ref. Areas (2015)705325. 59 with the notable exception of the open public consultation which is required for evaluations. 60 As noted previously, this is not a legal requirement. According to the Comitology rules, only MS must be formally consulted. Organisations like the WHO are able to submit views to DG SANTE in writing on an ad hoc basis.

40


agenda and defining priorities in the AWP. While such stakeholders are consulted and must approve of the AWP, there was an impression that DG SANTE, in its role as final arbiter, did not always take Member State preferences fully into account, but rather to an increasing degree set t the HP's agenda according to the Commission's priorities.

More specifically, the committee procedure, as described above, does allow for consultation of MS representatives and experts and Committee members must agree on the activities to be supported. However, the extent of involvement of MS is dependent on how proactive each MS and participating country is. As such, the responsibility to ensure that MS have the health needs of their citizens addressed is not wholly in their control.

The trend for increased use of joint actions and procurement contracts (whose parameters are set in a top-down fashion) over projects (which are more bottom up) serves to increase the feeling that the Commission sets the HP agenda. None of this is wrong per se (indeed, the last chapter showed that the objectives, priorities and actions of the 3HP are relevant) and reflects the difficulty of identifying consensus positions among stakeholders with disparate interests. Nonetheless, the findings show that there is a balance to be struck between equitability and expediency, and that there is some room for increasing buy-in and ownership among core HP stakeholders, particularly the Member States.

Evidence from the case studies illustrates that this balance is a delicate one, with responsibility lying both with the Commission and MS to ensure that MS derive the maximum potential from HP action. For example, our case study on the setting up of a mechanism to pool Union expertise (thematic priority 3.4) touched on the need to ensure MS ownership in the process as part of ensuring that ultimately the results of the mechanism are used in MS national policy setting. The need for MS ownership also came out of the case study on European Reference Networks (thematic priority 4.1). In both cases the MS must take responsibility for ensuring ownership / involvement, while the Commission must ensure actions are designed to encourage and accommodate this.

Conclusion

Overall, the evaluation finds that the process for setting AWPs works well. There was widespread agreement that the priorities defined on a yearly basis correspond to the public health needs of the Member States, allowing the 3HP to fund actions in the most important areas. The consultation process is well-defined and impartial, especially regarding formal consultation with the Member States, Chafea and other Commission DGs. These stakeholders expressed largely positive views and were satisfied with the weight given to their views in the AWP. The involvement of other DGs was singled out for special praise, since it increased the likelihood that the HP would complement other EU action.

Criticism was in turn mostly minor and focused on identifying areas where transparency could be improved. For example, some interviewees and survey respondents felt that informal consultation did not allow them to feed sufficiently into the programming process, showing that their role was not fully understood. There were also disparities among Member States, some of whose representatives seemed better able than others to have their preferences taken into account.

Regarding the pre-eminent role of DG SANTE in setting the HP agenda, it was acknowledged that there is a balance to be struck. On the one hand, leadership is necessary to reconcile the many disparate interests involved in setting priorities for the HP. On the other hand, the increased expediency from the MAP process, as well as trends towards more top-down financial mechanisms (especially joint actions and procurement contracts) create a need to explain the process to stakeholders and ensure their buy-in and the coherence of HP spending.

41


4.2. Multi-Annual Planning

EQ 6: How effective was the Multi-Annual Planning (MAP) for the preparation of the AWPs?

For the 3HP the Commission introduced Multi-Annual Planning (MAP) in order to incorporate a more holistic, longer-term mind-set into the programming process. More precisely, the MAP is an internal planning exercise managed by DG SANTE that aims to put in place the “more structured approach to planning spending”61 recommended in audits and previous evaluations.

As the name suggests, the MAP provides for planned spending across several years (2014 – 2017; 2017 – 2020). An internal note summarises the key objective and purpose of the MAP as follows:

“The key objective of the MAP exercise is to provide a funding plan based on the priorities and clusters of actions related to the thematic priority areas of the programme”62 (emphasis added).

As such the MAP has sought not only to help focus HP action and improve the coherence of the programme but also to improve the efficiency of the adoption of AWP by providing an initial plan for how to implement the HP. The first MAP covered the years 2014-2016 and is being updated to cover the rest of the MFF (2017-2020).

Based on documentary evidence and interviews with key stakeholders, we examined the effectiveness of the MAP in terms of the extent to which it has contributed to:

• a more strategic approach to setting AWP priorities;

• more clarity regarding the availability of resources and budget allocation over time; and

• proven coherence between priorities set in the MAP and AWP.

Building a culture of strategic priority setting

The available evidence (mainly feedback from EC officials and other stakeholders, but also documentary evidence of the process itself) suggests that the MAP exercise has allowed for an increased focus and a clustering of spending on a limited number of priorities in a given year. This has contributed to building a culture of more strategic priority setting and helped address previous criticism that programme funding was spread too thinly. However, it has not completely transformed the planning process, which in formal terms still relies on setting and amending priorities annually.

The main advantage of the MAP process is that by enabling more multi-year planning, it encourages more consideration of the medium- to long-term and strategic thinking behind the purpose of supporting different actions. The fact that the plans cover multiple years means that even if a certain issue is not addressed in a given year, the intention to address it in another is credible and can be written into the MAP, thereby assuaging the potential concerns of different units and allowing for greater focus and clustered spending (the coherence between planned and actual spending is presented below).

61 See “Multi-annual planning for the implementation of the second half of the 3rd Health Programme from 2017 t0 2020 included” Ref. Ares (2016)1542265 – 31/05/2016 62 Internal note “Subject: Third Health Programme 2014-2020 - Multiannual planning (MAP)”.

42


However, as explained by EC officials consulted, the varied nature of the actions means that there are certain activities which are more predictable than others meaning that it was reportedly easier to anticipate and estimate the required level of funding for some areas / types of action. For example, there was reported to be “low predictability” regarding the take up of low value procurement contracts (especially studies) which meant there was unspent budget (particularly in 2015) which in the event could be re-directed to issues relating to the migrant crisis63. With routine activity such as annual reviews / reports such as the “Monitoring of the EU Platform for Action on Diet, Physical Activity and Health”64, it is easier to plan and including these in the MAP makes clear what funding is available for annual priorities.

One caveat, presented in an internal note, refers to the challenges of planning ahead in the more distant future. The note explains that the effectiveness of the MAP is highest for the earlier years because the further away from the year of planning, the higher the likelihood that the needs / policy is unpredictable (e.g. new policies and legislation require different responses / actions)65. Indeed, this provides the rationale for an MAP process which covers three or four years, rather than the full seven years.

Streamlined processes and AWP planning

Again, the evidence from stakeholders involved suggests that the exercise of drawing up a MAP and corresponding actions / budget allocations is directly leading to a more efficient adoption of AWPs. In the view of stakeholders consulted due to this investment in multi-year planning and the establishment of prior agreement, the process for agreeing the AWPs has been found to be significantly smoother and faster. This is evidence that the exercise is perceived to be a valuable internal planning tool.

Having conducted the MAP, much of the difficult thinking and decision-making behind what the key priorities for a given year should be and how best to fund them has already been agreed within DG SANTE; as a result, the AWP process does not have to (in the words of one interviewee) “start from scratch”. Indeed, the MAP provides an indicative plan for what the AWP should contain, not just in terms of the priority itself but also in terms of which funding mechanism(s) to use.

A comparison of the adoption of AWP in the 2HP and 3HP shows although the process does not appear to have been faster under the first years of the 3HP, there were extenuating circumstances outside of the control of the HP managers which explain the timeline66. The 2016 AWP was adopted in March 2016 and the 2017 AWP was adopted in January of that year, which is a similar situation as for the 2HP67.

It is also worth pointing out that even after adopting the AWP, there is sometimes a need to be flexible to evolving circumstances. The AWPs of the 2HP saw a considerable number of amendments one year after their adoption (this is the case for three out of the six AWP68). Amendments to address changed circumstances in this way is not

63 As expressed in an internal note “Background to the multi-annual planning exercise 2017 - 2020”, Ref Ares (2016) 1542265 – 31/03/2016 64 http://ec.europa.eu/health/archive/ph_determinants/life_style/nutrition/platform/docs/eu_platform_mon-framework_en.pdf 65 Ibid. 66 The 2014 and 2015 AWP were adopted in May and June of the respective years (with an amendment to the 2015 AWP in November to tackle the health needs arising due to the high influx of refugees in October of that year). In 2014, the AWP was held up by the need to await the adoption of the Programme Regulation (EU)N° 282/2014 (which occurred on 11 March 2014) because only after that was it possible to call the Programme Committee for formal consultation (as required by the Regulation). In 2015, the delay relating to complications in receiving approval from representatives of the new College of Commissioners which was inaugurated in January 2015 and to launch the Inter-Service Consultation also after this date. 67 The 2008, 2009 and 2011 AWP were adopted in February of the respective years, with the exception of AWP 2010 which was adopted on 22 December 2009. 68 For years 2009, 2010 and 2011.

43

http://ec.europa.eu/health/archive/ph_determinants/life_style/nutrition/platform/docs/eu_platform_mon-framework_en.pdf

http://ec.europa.eu/health/archive/ph_determinants/life_style/nutrition/platform/docs/eu_platform_mon-framework_en.pdf


dissimilar to what has occurred under the 3HP in 2015, when the AWP was amended in November that year to direct action towards the refugees' crisis in summer 2015.

Follow through: coherence between MAP and AWP

Coherence between the MAP and AWP is a measure of its effectiveness as it illustrates the MAP process not only facilitated the drafting of the AWP but that it was realistic and accurate. As mentioned above, the MAP does not seek to set plans in stone but rather sets out a plan which will be subject to changes to adapt to new / unexpected developments (for instance the migrant crisis and corresponding emerging health needs) or to adapt changing timetables (for instance when policies or legislation is delayed).

The Commission has described how “a comparison of MAP with the actual AWP shows generally high coherence especially in the first two years of the Programme”69. This coherence implies that the decision-making process has been effective (i.e. useful). This saves time and therefore creates efficiencies in resource / planning. Our analysis of spending by thematic priority for the first two years of the 3HP and additionally for 2016 confirms the envisaged focus was achieved.

As per an internal note, the MAP for 2014 – 2016 identified the following priorities:

• 2014: prevention of chronic diseases70;

• 2015: health systems cooperation, focusing on technology and innovation, and;

• 2016: establish European Reference Networks (ERN) including those for rare diseases71.

Our assessment of the actual allocation shows that these priorities were indeed reflected in the emphasis of funding (as presented below). While they did not match the MAP figures precisely, this is not the purpose of the MAP. Rather the purpose is to define in broad terms the scope / priorities and, indeed, these are evidenced in actual spend.

• In 2014: prevention of chronic diseases: non-communicable chronic diseases including cancer and neurodegenerative (i.e. thematic priority 1.4 which tackles chronic disease) received the highest proportion of funding awarded to a thematic priority in that year (€6.6 m or 12% of all 3HP funding in 2014) but also communicable chronic diseases such as HIV/AIDS (through thematic priority 1.3; €3.3 m or 6% of total spending in 2014).

• In 2015: health systems cooperation, focusing on technology and innovation thematic priority 3.1 focused on developing Health Technology Assessment (HTA) was awarded €12 m or 22% of total spending in 2015 (this will be deployed through a joint action from 2016-202072) making it the single highest financed priority and action in any year.

• In 2016: ERN received nearly €7 m (or 12%). But it is also worth noting that thematic priority 1.4 on chronic diseases was awarded a total of nearly €10 m (or 17%)

69 Ibid. 70 Non-communicable chronic diseases including cancer and neurodegenerative, but also communicable chronic diseases such as HIV/AIDS. 71 these are taken from an internal note (Note for the attention of Ms Testori-Coggi, Director-General of DG SANCO” 72 More information is available here: http://www.eunethta.eu/activities

44

http://www.eunethta.eu/activities


Conclusion

In sum, the newly adopted MAP exercise has proven to be a valuable tool which has facilitated a quicker, less controversial, more efficient adoption of the AWP according to those involved. Additionally, there is evidence that the MAP has enabled a more focused and strategic approach to planning in the medium-run (i.e. up to 3-4 years) which is confirmed in the analysis of allocations to date. Breaking the 7-year programming period down into two MAP processes seems to be appropriate; any longer time horizon would likely present too many unknowns.

45


4.3. “Exceptional utility” criteria

EQ 7: How effective is the introduction of "exceptional utility" criteria in the Regulation establishing the 3HP (paragraph 19) in order to incentivise the participation of low GNI MS?

Although HP funding is available to organisations from all participating countries, the take up of funding varies markedly across them (Annex A provides a full data analysis). While some differences should be expected, the patterns in participation and receipt of funding have been shown to correlate with patterns in Gross National Incomes (GNI) in MS. For example the ex-post evaluation of the 2HP found that lead beneficiaries of actions were far more likely to come from the EU-15 MS (which tend to have higher GNI)73. This continues to be the case under the 3HP with far fewer organisations from low GNI countries leading projects or joint actions compared to high GNI countries74.

This trend is worrying, among other things because one of the general objectives of the 3HP is to reduce health inequalities. Low GNI countries face numerous barriers to participation in the HP, such as difficulties in dealing with administrative burdens and application processes and relatively low public health capacity. In addition, organisations from low GNI countries could have trouble to secure the required amount of co-funding, due to associated issues such as the scarcity of other funding sources in such countries and less developed public health networks.

To address the co-funding issue, during the 2HP DG SANTE introduced an “exceptional utility” selection criterion for projects, joint actions and operating grants.75 This allows for a higher proportion of co-funding (for all partners irrespective of where they are from) for a given action provided certain conditions are met relating to the amount of participation of organisations from lower GNI countries. The idea is to incentivise the inclusion of such organisations on funding applications and thereby increase their participation in the programme.

This section focuses on the “exceptional utility” criterion, with a view to gauging its suitability and success so far in increasing participation in the 3HP among organisations from low GNI counties. To lay a foundation for the assessment, we first provide more detail on how the criterion works and has evolved over time. This is followed by an assessment of participation of low-GNI countries in the 3HP and take-up of the possibilities afforded by the criterion. Finally, we place this in context by holding up the criterion against other barriers to participation faced by organisations from low GNI countries.

73 Ex-post Evaluation of Health Programme (2008 – 2013) reported at concerning lead beneficiaries “95% were based on the EU-15, with only 4% based in the EU12” (p.31). 74 Of the 12 joint actions funded, just one (“ADVANTAGE” which was launched in 2015) was led by a low GNI country (Spain) - meaning that 8% of joint actions were led by a low GNI country. Of the 25 projects funded, six were led by organisations qualifying as low GNI. These were: TOB-G and EUR HUMAN led by a Greek organisation, EURIPID – led by Hungarian Research organisation, Euro-GTP and SH-CAPAC led by Spanish organisations, and ALLCOOL led by a Portuguese organisation. The proportion for projects led by organisations from low GNI was nearly a quarter, at 24%. Half of these projects were awarded higher co-funding. 75 Under the 2HP the EU funding co-financing rate could be raised to 70 or 80 % (depending on whether the action was co-financed by the competent authorities of participating countries i.e. joint actions or by NGOs mandated by competent authorities). The specific criteria for different mechanisms are found in Annex VII to the AWP from 2011 onwards and require a certain level of involvement of countries with a GDP per capita in the lower quartile of all MS (“as published by Eurostat in the latest statistical report”) see discussion below.

46


The exceptional utility criteria

As explained above, the exceptional utility criterion applies to three funding mechanisms, namely joint actions, projects and operating grants.76 It allows for a higher rate of co-funding for all organisations in an action that includes a certain proportion of members, with a certain level of involvement, from low GNI participating countries. However, the precise parameters of the criterion differ depending on the funding mechanism in question. The criterion has also been refined and adapted over time and includes provisions relating to several types of circumstances.77 The table gives a brief overview of the criterion’s key features, while Annex A (Section 3) presents a comprehensive description of the criterion and contrasts with the early version in effect during the 2HP.

Table 8: Key features of the “exceptional utility” criterion

Funding mechanism

Criteria for exceptional utility under 3HP (2014 – 2016)

Joint Actions

1. At least 30% of the budget of the proposed action is allocated to MS whose gross national income (GNI) per inhabitant is less than 90% of the Union average. This criterion intends to promote the participation from MS with low GNI.

2. Bodies from at least 14 participating countries participate in the action, out of which at least four are countries whose gross national income (GNI) is less than 90% of the Union average. The criterion promotes wide geographical coverage and the participation of MS authorities from countries with a low GNI.

Projects

1. At least 60% of total budget must be used to fund staff. This criterion intends to promote capacity building for development and implementation of effective health policies

2. At least 30% of the budget of the proposed action is allocated to MS whose gross national income (GNI) per inhabitant is less than 90% of the Union average. This criterion intends to promote the participation of health actors from MS with low GNI.

3. The proposal most demonstrate excellence in furthering public health in Europe and a very high EU added value78.

Operating grants

1. At least 25% of the members or candidate members of the non-governmental bodies come from MS whose gross national income (GNI) per inhabitant is less than 90% of the Union average … to promote the participation of non-governmental bodies from MS with a low GNI.

2. The reduction of health inequalities at EU, national or regional level is manifested in the mission as well as the AWP of the applicant… to ensure that co-funded non-governmental bodies directly contribute to 1 of the main objectives of the 3HP, i.e. to reduce health inequalities

Importantly, the scope of the “exceptional utility” criterion has been greatly expanded for the 3HP in relation to its predecessor. This reflects a big push for a more equitable allocation of HP funding across MS. The most obvious change relates to the list of eligible countries. While this was formerly limited to seven MS (“MS with a GDP per capita in the lower quartile of all EU MS”), for the 3HP a country needs a GNI of less than 90% of the EU average. In practice, this has led to a more than doubling of the number of eligible countries, as shown in the table below.

76 Additionally, the 2016 AWP specifies exceptional utility criteria for “Mono beneficiary ERN grants”. Which include: a quarter of the members of the ERN must come from low GNI MS and the mission / annual work programme of the ERN must include activities which foster capacity building such as training, pooling knowledge and close collaboration with centres of expertise. (Annex VII of the 2016 AWP) 77 Indeed, as is discussed in the next section on uptake of the criterion, difficulties understanding and taking advantage of the criterion may be in part responsible for its relatively low utilisation so far 78 This last part “and a very high EU added value” was removed in the 2016 AWP.

47


Table 9: MS triggering higher co-funding in 2HP and 3HP BG CY CZ EE ES EL HR HU LT LV MT PL PT RO SI SK TOTAL

2HP X X X X X X X 7

3HP X X X X X79 X X X X X X X X X X X 16

The ability to benefit from the criterion has also been widened in other important ways under the 3HP as follows:

• Most importantly, when an action is awarded “exceptional utility” all participating organisations can receive the higher rate of funding (of up to 80% compared to the “regular” co-funding rate of 60%) irrespective of the type of mechanism80.

• Conditions have been simplified and made less restrictive, especially for joint actions where there were previously five criteria, including the proportion of funding which needed to be allocated to staff, which needed to be fulfilled and now there are just two.

• Conditions aiming to “promote the involvement of new actors for health” no longer need to be satisfied.

• There is no longer an explicit upper limit on the proportion of projects which can be awarded exceptional utility, whereas under the second half of the 2HP the conditions stipulated that: “No more than 10% of funded projects should receive EU co-funding of over 60%”.

• For both projects and joint actions, the threshold of funding awarded to poorer GNI countries has risen to 30% (compared to 25% for the lower quartile under the 2HP).

Taken together, these changes mean that a larger number of actions and organisations could qualify for the higher co-funding rate under the 3HP than was previously the case.

Take up of the “exceptional utility” but continued disparities in participation

Despite the efforts to make the “exceptional utility” criterion more accessible, in practice take-up has been limited so far81. More specifically:

• Two out of the eight joint actions supported under the 2014 AWP were awarded the higher co-financing rate (there were no joint actions receiving the higher co-funding in 2015). Together these two projects accounted for 15% of joint action funding in the first years of the HP82.

• Four projects (which together accounted for 10% of project funding) met the conditions to be recognised as a situation of “exceptional utility”. Two were funded under the 2014 AWP and the other two were under the 2015 AWP83.

79 2015 only 80 This compared to a rate of up to 70% for joint actions (compared to the “regular” co-funding of up to 50%) and up to 80% for projects and operating grants (compared to the “regular” 60%) under the second half of the 2HP. 81 Detailed data on beneficiary involvement is available for the first two years of the 3HP, i.e. 2014 and 2015. 82 VISTART” (dealing with Blood transfusion and tissue and cell transplantation) and “HA REACT” (tackling HIV and co-infection prevention) 83 Euro-GTP II” (set up the good practices applied to tissues and cells preparation processes and patient follow-up procedures: http://goodtissuepractices.eu/); “SIMPATHY” (Stimulating Innovative Management of

48

http://goodtissuepractices.eu/


• A higher proportion of operating grant recipients were successful in their application for higher co-funding: five out of a possible 14 different organisations receiving operating grants were awarded higher co-funding (such that 35% of total funding awarded to organisations receiving operating grants in the first two years of the 3HP)84.

For projects this seems in part due to unusually low success rates among applicants seeking to benefit from the criterion85. For example, the rate of success for projects seeking higher co-funding was less than half the success rate compared to the regular co-funding rate (13% compared to 30%)86. If success rates were as high as for regular funding, you would expect nine projects to have been awarded exceptional utility (i.e. 30% of the project applications for exceptional utility of which there were 31). For operating grants the difference was less pronounced, it was still slightly lower for those organisations applying for exceptional utility but overall 14 out of 40 applications applied for higher funding rate (or 35%) and this ratio was preserved in the allocation of funding.

The evaluation also found evidence which implies that the conditions and practical details of the “exceptional utility” criterion are either poorly understood or not sufficiently attractive. More specifically, we uncovered numerous examples of both projects and joint actions which fulfilled the criteria but did not apply for the higher co-funding rate87. In one case, “JANPA”, the evaluation team have access to the evaluation summary report because this action was examined through our case study on thematic priority 1.1, here it is noted that “in principle the JA proposal ought to be eligible for Exceptional Utility payment, except that not enough budget is allocated to new MSs”.

We also found a substantial number of projects (five) and one joint action88 where the proportion of organisations from low GNI countries was only slightly lower than the 30% threshold to qualify for the higher co-funding rate89. This suggests that in the majority of cases, the incentive for extra funding was not sufficiently attractive to stimulate an increased role / share for organisations from low GNI countries, even in cases where the required increase would have been small. Overall, we found no evidence to show that the expanded scope of the “exceptional utility” criterion is having a measurable impact on participation from low GNI countries overall. While the sample size is limited to the second half of the 2HP and the first two years of the 3HP for which data was available, the chart below illustrates that differences in participation are small. Indeed, Spain (which was eligible for higher funding in 2015) and Slovenia (which was eligible in both 2014 and 2015) have so far participated less in the 3HP than in its predecessor.

Polypharmacy and Adherence in the Elderly: http://www.simpathy.eu/); “ALCOOL” (Raising awareness and action-research on Heavy Episodic Drinking among low income youth and young adults in Southern Europe) and “TOB-G” (To develop and implement an innovative and cost effective approach to prevent chronic diseases related to tobacco dependence: http://tob-g.eu/ 84 So Europe Eurasia Foundation; Forum Europeen des Patients; European network for smoking prevention; Deutsche Aids-Hilfe and Smoke Free Partnership (in 2015 only). 85 Data shared with the evaluation team indicates that: 18 out of 50 project proposals received in 2014 and 13 out of 50 applications in 2015, applied for higher co-funding. Given that 4 were successful, the success rate is calculated at 4/31 (or 13%). 86 The success rate for regular funding is calculated at 21 successful projects, out of a total of 69 “regular funding” applications. 87 i.e. joint actions: ADVANTAGE, JANPA, JARC; projects: SH-CAPAC, EPIC, EUR HUMAN, EURIPID and CARE. 88 Note that operating grants are awarded to a single organisation, making it impossible to ascertain how funding is allocated across countries. 89 i.e. joint action: JAseHN and projects: SIE; 8 NGOs; SCIROCCO; PATHWAYS and ACT at scale.

49

http://www.simpathy.eu/

http://tob-g.eu/


Figure 4: Share of funding received in 2HP (2008-2013) & 3HP (2014 – 2015)90 for “new” 3HP low GNI countries

Source: Chafea

Furthermore, a comparison of the overall participation of organisations of (i.e. funding allocated to) low GNI countries compared to high GNI countries indicates that under the 3HP their involvement has fallen slightly from 29% to 25% in the case of joint actions and from 28% to 26% in the case of projects. In part this is due to the continued importance of big envelope joint actions (such as “EUnetHTA”) which have low involvement of low GNI countries and pull the figure down.

The evidence is too limited to attribute continued disparities in HP participation to the lack of effectiveness of the “exceptional utility” criterion. Rather, it shows that further encouragement and potentially awareness raising are needed to increase uptake among interested organisations. The findings also imply that low GNI countries face many barriers to participation, of which co-funding is only one.

Barriers to participation in the HP

The above analysis illustrates that increasing the participation of low GNI MS is not straightforward. Consultations with stakeholders have shed light on the other barriers (financial and non-financial) which continue to make it difficult for certain MS to increase their participation.

Feedback from Chafea in particular provided insight into structural barriers typically facing low GNI countries. These are summarised below:

• Issues related to the absence of sufficient skills (capacity and knowledge) required to organise and apply for EU funding through the 3HP: typically MS joining the EU after 2004 do not have as much experience in applying for funding from international organisations and need to build up this expertise and experience. Also issues relating to staff capacity, especially for smaller countries with smaller public health departments.

• Issues relating to co-funding arrangements: One of the fallouts of the economic crisis has been the tightening of public spending, including the ability to hire staff for specific projects / periods. This has created difficulties in some MS where staff costs would typically be used to meet co-funding requirements.

Feedback from NFPs and PCs responding to the online survey confirmed that irrespective of the relative income of a given MS, lacking capacity to lead or even participate in funding applications presents a serious barrier. Moreover, when asked how the barriers

90 Note that Spain was not classified as a low GNI country in 2014; however in 2015 (and 2016) it was and is included in the chart for this reason.

50


they face compared to the barriers facing other countries, low GNI countries were more likely to report that organisations in their country face more difficulties in applying for funding than other countries. Responses also indicated that, in addition to securing co-financing for actions and the administrative burden of participating in an action were thought to have the greatest impact on organisations’ interest.

In order to further explore the thinking behind these responses, the evaluation conducted a follow-up mini-survey. This asked respondents to name the single most important barrier to participation and to explain why. While both high and low GNI countries considered the administrative burden and securing co-financing to be the most important barriers, their explanations differed substantially. More specifically:

• administrative burden is considered most prohibitive by respondents from high GNI countries (11 respondents or 44% of total selected this as the single most significant barrier - nine of the 11 were from low GNI countries);

• securing co-financing is more likely to be considered most difficult for low GNI countries (11 respondents or 44% of total thought this is the most important barrier for potential participants from their countries - seven of the 11 were from low GNI countries).

At first glance this would seem to imply that the “exceptional utility” criterion would go a long way towards boosting participation from low GNI countries. However, it is important to bear in mind that the vast majority of lead applicants are from high GNI countries. The criterion does not explicitly aim to change this. Rather, by offering all partners a higher rate of co-funding, it seeks to incentivise applicants to include organisations from low GNI countries. The fact that fewer high GNI countries consider co-funding to be a major barrier means that the prospect of higher rates does not necessarily increase their motivation to work with (potentially new or reduced capacity) partners from low GNI countries.

Findings from the case studies, which included interviews with Chafea project officers but also beneficiaries themselves, add further weight to the explanations above. For example, interviews conducted for the case study on thematic priority 1.4 (which covers chronic diseases) confirmed that organisations from low GNI MS are keen to participate but struggle to secure co-funding. In addition there is the simple problem that smaller countries (of which relatively more fall into the low GNI category) do not have sufficient administrative capacity in their ministries to manage multiple actions.

Conclusion

The “exceptional utility” criterion seeks to address an important and persistent problem: low participation in the HP by organisations from low GNI countries. Its expanded scope and simplified eligibility criteria are a welcome development. Yet, the figures suggest that the use of the mechanism has been low for projects and joint actions, and slightly better, but still low for operating grants.

There is unlikely to be a single reason for the low take-up, but one possible factor is that the criteria appear to be poorly understood among applicants. There is also the possibility that the incentive is not sufficiently attractive to overcome the other barriers to participation for these countries, or that it is difficult to obtain a consensus among partners to allocate the necessary proportion of responsibility (and resources) to low GNI MS. Indeed, higher co-funding does not tackle some of the other crucial underlying structural barriers to participation such as skills (capacity and knowledge) and institutional resourcing challenges. These other barriers may explain why the participation rates of low GNI countries remain persistently low overall, despite notable improvements / exceptions in specific cases (including but not limited to those where co-funding has been received).

51


All this suggests better monitoring and feedback systems are needed to troubleshoot problems ahead of time and to be in a better position to identify with more certainty the reasons for low take up.

52


4.4. Contribution of the HP to objectives and priorities

EQ 8: In practice, to what extent are the actions in the AWP contributing to the:

• 3HP objectives and thematic priorities

• Commission policy priorities?

This question looks at effectiveness at a higher level, in terms of the HP’s contribution so far to its objectives and Commission policy more broadly.

It is important to point out that the ex-post evaluation of the 2HP focused on identifying the key features of successful actions. However, this focused on the relative advantages and disadvantages of the different financial mechanisms. For instance, joint actions are strong vehicles for achievements in areas where political buy-in is necessary, and projects are better for early action, when a solution to a given problem is not known and more exploratory work is required.

Given the early timing of this evaluation and its focus on the re-structured thematic priorities and objectives of 3HP, here we concentrate more on the likely contributions in given themes and on identifying the factors that render success more or less likely. As explained in section 2.4 on validity and limitations, our findings in terms of the effectiveness of actions are based primarily on the case studies of eight thematic priorities (two per specific objective of the 3HP). Through these we draw out cross-cutting themes and bring in additional forms of evidence in order to explore the effectiveness so far of the 3HP as a whole in terms of the contribution to 3HP objectives and thematic priorities and Commission policy priorities.

Notable achievements so far

The evaluation found that individual actions can contribute to the 3HP’s objectives and thematic priorities in a wide variety of ways, and that notable achievements are already being made and / or are evidently likely. These achievements are provided in detail in the introduction to the case studies presented in annex B. Here, we point out that evidence from the case studies shows the funded actions are producing concrete results in line with the 3HP objectives and thematic priorities, as well as linking better to wider Commission priorities than in previous iterations of the HP.

Some key examples illustrate areas of meaningful achievement, in many cases building on foundations laid in previous funding periods and underpinned by EU legislation.

• In the area of serious cross border health threats, competent authorities from the MS have been working together to ensure laboratories are technically equipped to effectively identify and respond to threats. For example the improving and making more accurate diagnostics of emerging pathogens will ultimately support the rapid containment of pathogens and protect citizens from health threats (in line with Decision 1082/2013/EU).91

• Significant advances have been made to create more innovative, efficient and sustainable health systems under the 3HP, notably to operationalise approaches to Health Technology Assessment (which have in development since the 1HP). Two joint actions were funded during the 2HP to test and pilot methods

91 Refer to Annex B, case study report on thematic priority 2.3 “Implementing EU legislation (health threats)” for more detailed explanation and information on activity in this area.

53


and generate the buy-in necessary for common approaches in this area. Now, under the 3HP, the largest joint action to date under the HP will anchor these approaches in MS practices. While this is voluntary, a network set up to implement Directive 2011/24/EU provides the legal framework and consensus around the need for action in this area.92 Mechanisms for pooling expertise at EU level and support MS in their health reforms were set up such as the Expert Panel on Health93 and the Expert Group on Health Systems Performance Assessment94; intensified cooperation with international organisations such as OECD and WHO is expected for the second half of the 3HP while the first results becoming publicly accessible concern tools and methodologies for accessing integrated care in Europe95.

• The HP has also supported the development of better and safer healthcare. For instance regarding measures aiming to increase both the availability of substances of human origin and access to transplant and transfusion therapies for EU citizens. For instance under the 3HP, action has begun to support the systematic evaluation of the efficacy of tissues and cells and promoting good practices for the implementation of novel therapies with human tissues and cells, building on previous efforts under the 2HP in the context of a clear legal framework for EU action in this field96. In addition, under the same objective of better and safe healthcare, the 3HP has achieved meaningful progress in establishment of European Reference Networks (ERNs). Projects to be funded during the second half of the 3HP are expected to build on previous action and support further the establishment and functioning of new and existing ERNs for numerous diseases97. Anti-microbial resistance is another topic of growing interest and importance, and increased funding is foreseen for a Joint action in 2016 while other initiatives for the years to come are being agreed (an open public consultation for possible activities under a Commission Communication on a One Health Action Plan to support Member States in the fights against Antimicrobial Resistance (foreseen for mid-2017) was still open when this evaluation report was being finalised98).

• The HP has supported the identification, dissemination and take up of best practices. For instance while some actions have been criticised for petering out, our case study on thematic priority 1.4 on chronic diseases found work supported under the 3HP has built on previous action in the identification of best practices and is leading to the practical uptake of results in the field of dementia (see Joint Action “Act on Dementia”99). For instance through the examination of the delivery of tested principles of good practice in localities and by improving the understanding of the challenges of implementation in different settings. Other actions such as the operating grant awarded to Alzheimer Europe (a non-governmental organisation), are helping the WHO Global Observatory on Dementia. Other actions serve the ambitious goal set by the Commission to

92 See more information of the EU’s work on Health Technology Assessment here: http://ec.europa.eu/health/technology_assessment/policy_en 93 http://ec.europa.eu/health/expert_panel/ 94 http://ec.europa.eu/health/systems_performance_assessment/policy_en 95http://ec.europa.eu/health/sites/health/files/systems_performance_assessment/docs/2017_blocks_en_0.p

df 96 Refer to Annex B, case study report on thematic priority 4.5 “Implementing EU legislation (Substances of human origin)” for more detailed explanation and information on activity in this area. 97 Refer to Annex B case study report on thematic priority 4.1 “European Reference Networks” for more detailed explanation and information on activity in this area. 98 http://ec.europa.eu/dgs/health_food-safety/amr/index_en.htm 99 http://www.actondementia.eu/

54

http://ec.europa.eu/health/technology_assessment/policy_en

http://ec.europa.eu/dgs/health_food-safety/amr/index_en.htm

http://www.actondementia.eu/


reduce cancer incidence by 15% by 2020100. For example, an operating grant for “Cancer Leagues Collaborating in Cancer Prevention and Control at the EU and National Level” (ECL) is increasing public awareness surrounding cancer-related risk factors such as obesity, promoting cancer prevention and communicating the fourth revision of the European Code Against Cancer101. Also the European Initiative on Breast Cancer (EIBC)102 is implementing a revision of the European Guidelines on Breast Cancer Screening taking into account all the new scientific knowledge in the field. At the same time, a voluntary scheme for accreditation of breast cancer screening units is being developed in participating countries.

These examples illustrate that where 3HP activity has been directed towards activities with clearly defined goals, and a strong well-prescribed EU added value (i.e. legislative framework and agreement) real achievements have been made. By contrast, we observed that in areas which have less enforceable, long term continuous goals (for example, “support the exchange of evidence-based and good practice…” as per thematic priority 1.1 or “support cooperation and networking in the Union in relation to preventing and improving the response to chronic diseases”) which makes it harder (not impossible) to ensure results are taken up. For this reason, during the further implementation of the 3HP, DG SANTE intends to strategically enhance the transfer of best practices and has established the Steering Group on Health Promotion and Prevention in November 2016. The Group, which is composed of representatives from all EU and EEA countries, was set up by the Commission to support Member States in meeting the WHO/UN 2025 global voluntary targets on non-communicable diseases103. The Group's objective is to provide the Commission with advice on investments and priorities for the 3HP AWP, to ensure the coordination of actions by groups working in numerous sectors (e.g. cancer, mental health and rare diseases), and provide a platform for decision makers from national health authorities to decide on implementing actions with the EU financial support related to health promotion and disease prevention. This highlights the importance of design and planning for actions and provides a solution for working under the broader thematic priorities (particularly those under objectives 1 on health promotion and 3 on health systems).

Indeed, it is also symptomatic of the fact that the nature of the contribution is different depending on the specific objective in question. For priorities under specific objective 1 (health promotion and prevention) the EU’s soft competence means that actions primarily involve supportive / coordinating actions, developing synergies in approaches of different Member States. This is also typically the case for action supporting specific objective 3, where actions typically involve coordination and “pooling of expertise.” By contrast, under specific objective 2, the EU has a clear legal mandate to act because the nature of the challenge requires a multi-national response if it is to be effective. As such, the actions have been focused on technical capabilities, capacity building and systems to ensure compliance. The added value was also found to be more linked to efficiencies and necessity of EU level action for the activity under specific objective 4. As already mentioned, this objective included the establishment of ERN which only makes sense at the EU level, while the safe and optimum transfer of

100 Commission Communication on Action Against Cancer: European Partnership COM(2014) 584 final of 23.9.2014, http://ec.europa.eu/health/archive/ph_information/dissemination/diseases/docs/com_2009_291.en.pdf 101 The European Code Against Cancer, developed through a direct grant with IARC under the previous health Programme, is a set of 12 recommendations on how people can take action to reduce their cancer risk, see at http://cancer-code-europe.iarc.fr/index.php/en/ 102 This is an Administrative Agreement with the Commission Joint Research Centre financed under the Annual Work Programme 2016. 103 http://www.who.int/nmh/ncd-tools/definition-targets/en/

55

http://ec.europa.eu/health/archive/ph_information/dissemination/diseases/docs/com_2009_291.en.pdf

http://cancer-code-europe.iarc.fr/index.php/en/

http://www.who.int/nmh/ncd-tools/definition-targets/en/


substances of human origin across borders and the global economic and societal challenge of anti-microbial resistance are clearly cross border issues.

In terms of supporting the Commission’s wider priorities such as the Digital Single Market, the 3HP assists Member States in including eHealth into health policy while better aligning eHealth investments to health needs (see Joint Action “JAseHN 2014”104, for example). A great number of actions reviewed were explicitly or implicitly supportive of Commission priorities. For example, actions in the promotion of health and prevention of diseases are often inextricably connected to the Commission’s broader strategy and agenda. For instance action to tackle chronic diseases supported by the HP is linked with the New Skills Agenda for Europe, which forms part of Juncker’s broader EU strategy where employment is a key pillar105. Additionally, under the same objective of health promotion, the HP has notably addressed needs in areas relating to migrant health which fits in with the Commission’s prioritisation of issues relating to migration106. This is covered in more detail under external coherence of the 3HP (section 7.2).

Cross-cutting challenges and areas for improvement

In terms of the remaining challenges and areas for improvement, the following three themes recurred.

Firstly, a recurring theme across the objectives relates to the need for generating sustainable impacts, for example by ensuring that activities are anchored in the MS and local contexts, that there is ownership of and input in the results (as demonstrated by the findings in thematic priority 4.1 on European Reference Networks and 3.4 on the pooling of Union expertise and 3.6 on the implementation of Union legislation covering medical devices, medicinal products and cross-border healthcare). But also, in regards to areas where there is a reliance on EU funding. This is illustrated by the example of action under thematic priority 2.3 which develops capacities in identifying and responding to highly threat pathogens (as per Article 4 of Decision 1082/2013/EU). The challenge is the need to continually maintain a high level of expertise in diagnostics of rare but extremely dangerous pathogens (i.e. a discontinuation of investment means capabilities may be lost and new threats not sufficiently well addressed). Importantly, this issue is already high on the agenda and the results of a study on the costs / benefits of a reference laboratory for human pathogens already delivered107.

Another recurring widespread challenge is the accessibility of the programme to new players. While the involvement of organisations which have previously been involved in the HP has clear benefits (in terms of the understanding of the processes and procedures, co-ordination efforts, etc. and a confidence in the capability to deliver on time and to a high standard), the flip side is that the 3HP could be accused of repeatedly addressing and supporting the same beneficiaries. Indeed, feedback from interviewees highlighted that this is a perception among stakeholders. Further, our case studies provide more weight to this view108. As discussed in greater depth under Coherence

104 http://jasehn.eu/ 105 Refer to Annex B, case study reports on thematic priority 1.1 “Risk factors” and 1.4 “Chronic diseases” for more detailed explanation and information on activity in this area. 106 See more information on Commission policy here: https://ec.europa.eu/commission/priorities/migration_en 107 Study on cost-benefit analysis of reference laboratories for human pathogens, June 2016, Civic Consulting. Available online here: http://ec.europa.eu/health//sites/health/files/preparedness_response/docs/2016_laboratorieshumanpathogens_frep_en.pdf 108 We note that the way that the beneficiaries are recorded does not facilitate a direct comparison / analysis of beneficiaries over time (for instance names have been found to be recorded differently meaning a manual process of comparing names would be required but more importantly, sometime only the lead organisation is recorded and this is misleading). If it were of sufficient interest, this could be investigated going forwards.

56

http://jasehn.eu/

https://ec.europa.eu/commission/priorities/migration_en

http://ec.europa.eu/health/sites/health/files/preparedness_response/docs/2016_laboratorieshumanpathogens_frep_en.pdf

http://ec.europa.eu/health/sites/health/files/preparedness_response/docs/2016_laboratorieshumanpathogens_frep_en.pdf


(Chapter 7) the repeated involvement of some beneficiaries can enhance the synergies between actions and across iterations of the programme. As such, the question is whether this is carefully managed and whether there is sufficient attention to inclusive access (i.e. from across the EU). The evidence from section 4.3 on “exceptional utility” would suggest that more could be done to enable wider access to the HP.

Finally, we also noted the need for results of actions to be communicated for the linkages to be as effective as possible and that in many cases dissemination is sub-optimal. For instance, we note that under thematic priority 3.4109, currently the results (which are called “Opinions”) are disseminated via relevant sector journals but that there is not a systematic the dissemination process to ensure Opinions are received by MS yet. Since the expert group is relatively new, putting this in place may encourage both the suggestion of Mandates by MS and the uptake of Opinions. A similar approach is already in place for scientific opinions for instance, or studies ordered by the Commission and could easily be emulated.

Conclusion

Given the early stage of the 3HP’s implementation, we have focused on key areas where meaningful achievements have been uncovered and examined the pre-conditions necessary for future success of the programme in terms of the its objectives and thematic priorities. Based on case studies of two thematic priorities per each of the 3HP’s four specific objectives, the evaluation found evidence of many potential benefits from funded actions. Some key examples have been highlighted, especially concerning ERNs, Health Technology Assessment and tackling cross border health threats.

While it is impossible to ascertain whether the case study findings are indicative of the programme as a whole, it is also worth noting that the actions examined under specific objectives 2 (cross-border health threats) and 4 (access to healthcare) typically appeared likelier to generate tangible benefits in the near future. In part, this was because some of the actions in question (such as those relating to European Reference Networks for rare diseases) are part of long-term efforts to reach specific goals. It also related to the nature of the specific objectives. While objectives 2 and 4 focus on relatively narrow issues where the case for EU action is clear and established under EU law, objectives 1 (health promotion) and 3 (health systems) address much bigger health needs and are less likely to have a defined legal basis.

For the latter two objectives, the potential to generate EU added value depends to a greater extent on identifying gaps that can be filled with the sharing and eventual uptake of best practices. The path to making an impact then varies depending on the specific context. Most of the actions under review presented a credible case. However, some of them appeared overly broad (a frequent issue given the scope of the specific objectives). They also lacked clear communication plans, making it hard to see how the best practices, once identified, would be adapted to diverse circumstances and implemented. This demonstrates the importance, especially for actions funded under specific objectives 1 and 3, of requiring a plausible intervention logic and credible plans for follow-up work.

109 “Provide expertise and share good practices to assist Member States undertaking health systems reforms by setting up a mechanism for pooling expertise at Union level…”

57


5. EFFICIENCY

This chapter holds up various aspects of the HP’s effectiveness against the costs involved. Given the difficulty in quantifying the programme’s contribution to higher-level objectives (especially given the early state of implementation), the focus is on more operational elements of the 3HP. More specifically, the next sections consider evaluation questions relating in turn the allocation of resources across objectives and thematic priorities, practical implementation and monitoring processes and resources110. The answers to these questions are based on all of the sources of data collected and analysed for the evaluation but draw particularly on the quantitative analysis of implementation data that is presented in full in annex A).

5.1. Allocation of resource among objectives / thematic priorities

EQ 9: To what extent does the design of the 3HP lead to an efficient allocation of resources among objectives / thematic priorities?

As discussed in chapter 3 on relevance, the 3HP has a more focused and purposeful structure than its predecessors. Here we explore whether the new structure (of four specific objectives divided into 23 thematic priorities) has provided programme managers with the framework needed to direct funding towards the (1) most relevant issues and (2) potentially impactful actions. The following sub-sections explore each of these issues in turn by looking at the analysis presented in chapters 3 and 4 (on relevance and effectiveness, respectively) against implementation data on the allocation of financial resources.

Alignment of resources with identified relevance of objectives

Section 3.1 on the relevance of the programme objectives found that all four objectives corresponded to important health needs and evolving challenges. However, it also found that their potential EU added value differed in nature. Objectives 2 and 4 (on cross-border health threats and facilitating access to healthcare, respectively) relate to a relatively narrow set of issues that are cross-border in nature or entail clear economies of scale from pooling resources or coordinated action largely due to the fact that they are closely linked with legislation. Objectives 1 and 3 (on health promotion and health systems, respectively) concern much larger issues that are high on MS agendas but where EU legislation is soft (i.e. Council recommendations) rather than hard (e.g. Decision 1082/2013/EU on serious cross border health threats). While HP funding in these areas pales in comparison to spending at other levels, it can add value and reduce disparities by facilitating the exchange and uptake of best practices.

It is also worth referring to the Regulation establishing the 3HP, which states that:

“The emphasis should be placed, in accordance with the principle of subsidiarity, on areas where there are clear cross-border or internal market issues at stake, or where there are significant advantages and efficiency gains from collaboration at Union level”111.

These findings in themselves do not spell out how the limited available funding should be divided between the objectives, not least because the relative scale of the problems related to each of them could vary, as can the budgets required for different activities;

110 Improvements are dealt with as part of recommendations in section 8.2. 111 Regulation 242/2014 (paragraph 5)

58


the HP does not exist in a vacuum and significant budgets may be directed to these areas through other avenues). But they do suggest that, other things being equal, at the level of EU intervention (of which the HP is an important part), greater emphasis should be given to objectives 2 and 4 than to 1 and 3.112

As shown in the chart below, such an emphasis is not immediately identifiable in spending allocation during the first half of the 3HP: actions under Objective 1 received the most funding (€54m), followed closely by objective 3 (€52m). Objective 4 (€31m) received substantially less, while objective 2 (€11m) received by far the least amount of funding through the HP113.

However, it is important to stress that the HP is not the only means for funding activity in these areas. An important example of work to tackle cross-border health issues is the EU’s annual subsidy of nearly €60m to the European Centre for Disease Control (ECDC), which was set up in 2005 dedicated to strengthening Europe's defences against infectious diseases114. This means from 2014 – 2016, roughly €180m was dedicated to fighting infectious diseases. This funding encompasses emerging and vector-borne diseases (which is part of objective 2) and anti-microbial resistance (which is part of objective 4) but also chronic communicable diseases (such as HIV which is covered under objective 1) among others. As such, while read in isolation, the data would seem to indicate that the allocation of resources across the objectives has not been in line with the emphasis proposed in the Regulation (and therefore is not as efficient as it could be), the HP operates in a broader context which provides some of the explanation for how its limited budget is prioritised (i.e. to complement other activity).

Figure 5: Budget allocation by objective 2HP and 3HP (2014 – 2016)

Source: Chafea and SANTE Note: the coverage of HP objectives has not been stable over time and thus any comparisons can only be made with sufficient context

Furthermore, although comparisons over time are complicated by the fact that different iterations of the HP have seen different ways of organising spending115 the share of funding which is allocated to health promotion and health information is not as

112 It should also be noted that the dichotomy between specific objectives is not clear cut. Rather, objectives 1 and 3 include possibilities for action that goes beyond the sharing of best practices benchmarking and networking, while objectives 2 and 4 do contain such elements. However, these are exceptions to the general rule. 113 Note that a further €17.3m was allocated to “horizontal activities”. 114 Information on the ECDC budget is provided at the webpage on Financial Information on the ECDC website. http://ecdc.europa.eu/en/aboutus/access_documents/Pages/Financial_information.aspx 115 Some examples include: spending on AMR and HIV (which for the purposes of the 3HP are categorised under objective 4 – “access to health” and objective 1 “health promotion” respectively) were categorised as objective 1 “health security”. Another example is spending on rare diseases, which was previously incorporated under objective 3 “health information”, is addressed as part of objective 4 “access to health” in the 3HP.

59


significant as previously. For example, over the course of the 2HP, the health promotion objective (which falls primarily under the current objective 1) received 50% of all programme funding, while health promotion was allocated about one third of funding in the 3HP so far. Under the 2HP, 19% of spending went to “health information”, but under the 3HP this is streamlined into each objective (as the final thematic priority) it has received a total of 9%116.

While spending on health security and access to health is not directly comparable to spending in the 2HP, what is important is that the revised structure of the programme objectives is a way of directing resources to issues with the most potential EU added value. For the HP to make the best of limited resources, it is important that actions complement other EU support, and that the positive trends described above continue during the second half of the current funding period and beyond.

Alignment with identified relevance of thematic priorities

In section 3.2 we discussed the relevance of the HP’s 23 thematic priorities, concluding among other things that they represent a valuable effort to define the most important areas for programme support. While making some suggestions about how the thematic priorities could be refined or streamlined for the next HP, in general terms the evaluation found them to be appropriately organised with relevant content. We also found that the process for setting AWPs broadly reflects the needs of MS and other stakeholders.

This means that in theory the thematic priorities provide a suitable framework for allocating programme resources efficiently. To examine whether they are doing so in practice, we drew on the detailed analysis of implementation data and interviews with DG SANTE and Chafea officials.

In particular, we considered the data in light of the aim, expressed in the Regulation establishing the 3HP, to concentrate funding on “a smaller number of activities in priority areas”. The evidence suggests that the thematic priorities have provided an enabling environment to focus actions in a relatively small number of key areas.

While there are 23 thematic priorities, in fact funding has been concentrated among a relatively small number of them. Over €70m (42%) of funding was allocated to the five highest funded thematic priorities, while nearly €50m went to the next five best funded thematic priorities (29%). The remaining €47m went to the remaining thirteen thematic priorities and horizontal actions combined.

To provide more detail, the largest amount of funding was allocated to the following thematic priorities:

• Tackling chronic diseases (priority 1.4); risk factors (priority 1.1)117 and HIV /AIDS, TB and hepatitis (1.3) which fall under health promotion (objective 1).

• European Reference Networks (priority 4.1) which contribute to facilitating access to better and safer healthcare (under objective 4) have also been built up through the 3HP and,

• Health Technology Assessment (priority 3.1).

116 Mostly through spending under objective 1 and 3 (which accounted for 3% and 6% of the total respectively). 117 Both of these thematic priorities were deemed highly relevant by stakeholders consulted through the OPC and a targeted survey of NFPs and PCs.

60


As already eluded to, there is significant variation of funding across the 23 priorities; the range is from over €17m awarded to tackling chronic diseases (priority 1.4) and cross-cutting elements (“horizontal” activities), and to one of the four thematic priorities covering “health information and knowledge system”, specifically dealing with health information in the context of health threats118 (priority 2.4). In part this is a reflection of how different the thematic priorities are, in terms of their scope and, in part, the different rationale for their inclusion. In terms of scope, we can compare thematic priority 1.1 on tackling a host of risk factors (from tobacco and alcohol to nutrition and physical exercise), with thematic priority 1.2 which deals with the narrow subject of drugs related health damage or priority 2.1 which deals specifically with risk assessment and additional capacities for scientific expertise. In the case of 2.1, the priority serves to address a gap in the competence of the ECDC. This lack of comparability in scope was criticised by health experts because it lacks logic and consistency. From the point of view of the Commission officials interviewed, the main criticism of the thematic priority structure is that they have led to fragmentation in the way programme funding is allocated.

As discussed in more detail below in section 5.4 on monitoring, the large number of thematic priorities makes it harder to keep focused and to ensure that a critical mass of funding is allocated to enough actions to generate results. That being said, we did not find evidence to suggest that the thematic priority structure has undermined the programme’s efficiency in any meaningful way. But a somewhat smaller number of more similarly structured priorities could make it easier to allocate and account for resources efficiently and, more importantly to achieve results in those areas.

Finally, as already discussed under effectiveness, the introduction of the MAP has improved the ability to purposely focus spending on key areas, as reflected in the data and discussed in more detail under effectiveness (Chapter 4).

Expected results

In terms of how these findings relate to expected results, while it is too early to assess results in a comprehensive way given that many actions are still underway, the findings from the case studies offer insight into broad trends, namely that action funded through the thematic priorities falling under specific objectives 2 (cross-border health threats) and 4 (access to better and safer healthcare) tended to be more focused and likelier to lead to tangible results in the near-term. Importantly, this is in large part because there is an established legal basis for action (e.g. Decision 1082/2013/EU which outlines a clear path and remit for EU action in the field of serious cross-border threats to health119 and Directive 2011/24/EU on the application of patients’ rights in cross-border healthcare120). Actions under objectives 1 (health promotion) and 3 (health systems) addressed highly relevant issues but could be criticised for their overly broad scope, in addition, typically these areas are less likely to have legal basis with few exceptions (e.g. legislation related to tobacco products under objective 1 and the voluntary cooperation between Member States on Health Technology Assessment under objective 3). Although this does not capture the nuance of the effectiveness of individual actions, under thematic priorities, it does imply that in broad terms it might be better to direct a higher proportion of funding to areas where the EU added value of action has been established by legislation and is thus clearer and more focused (i.e. objectives 2 and 4).

118 Note, as explained elsewhere, this thematic priority is repeated under each objective and under other objectives, there is activity to support health information and knowledge systems to contribute to tackling chronic diseases. 119http://ec.europa.eu/health/sites/health/files/preparedness_response/docs/decision_serious_crossborder_threats_22102013_en.pdf 120 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2011:088:0045:0065:en:PDF

61

http://ec.europa.eu/health/sites/health/files/preparedness_response/docs/decision_serious_crossborder_threats_22102013_en.pdf

http://ec.europa.eu/health/sites/health/files/preparedness_response/docs/decision_serious_crossborder_threats_22102013_en.pdf

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2011:088:0045:0065:en:PDF


Conclusion

The structure of the 3HP, with its four specific objectives split into 23 thematic priorities, allows for an efficient spread of funding across a wide variety of potentially relevant actions. However, given the limited funding available there is a need to prioritise. The Regulation establishing the 3HP provides a basis on which to do so, referring to the need to emphasise action in areas where the EU added value relates to clear cross-border or internal market issues (or where there are significant advantages or efficiency gains from collaboration at Union level). As such, the actions with the strongest EU added value are typically under objective 2 and 4 due to the clarity of purpose gained from EU legislation in these areas and cross-border nature of the issues at stake. Action under other areas can also add significant value and be cost-effective, provided that actions are sufficiently well-designed and outcome-focused.

In terms of actual implementation, the balance of funding has been gradually flowing more towards the areas addressed under specific objectives 2 (cross-border health threats) and 4 (access to health care), where the case for EU action is clearer and more close linkages with EU legislation and cross border issues. Action related to cross-border health threats (i.e. communicable diseases) is also delegated to the dedicated agency (the ECDC). We note that more funding (33%) is directed to objective 1 (health promotion). While this is not inefficient per se, actions under this and objective 3 (health systems) need particular attention to EU added value, as mentioned above. This highlights the importance of sound design and management in these areas to ensure impact.

It would also be expected that the trend towards objectives 2 and 4 would continue as the programme continues to mature and hone in on the issues where it can generate tangible results.

62


5.2. Efficiency of programme management

EQ 10: To what extent does the allocation of resources allow for an efficient implementation of the 3HP in terms of: funding mechanisms, simplification measures and operational costs?

While the main driver of a programme’s ability to provide value for money is the effectiveness of its funded actions, it is also important to manage resources in a supportive but cost-minimising way. This section examines the efficiency of the 3HP’s management in terms of the following elements:

• Financial mechanisms: these allow the HP to address a wide variety of health needs for a diverse group of stakeholders. As described in the final evaluation of the 2HP, the funding mechanisms each have specific strengths and weaknesses and are best deployed in certain circumstances. They also entail different administrative burdens and costs. In the first sub-section we examine these burdens and costs in relation to the split of resources across the funding mechanisms and their evolution over time.

• Programme administration: operational costs relate to the resources, mainly borne by DG SANTE and Chafea, needed to administer the programme. In order to gauge their appropriateness, we have held up the 3HP against its predecessors as well as benchmarking against another Commission-funded programme (presented in Table 11 in the second sub-section below). Management improvements have been introduced during the current funding period, namely electronic tools for the submission of funding proposals, grant management and reporting. In addition, all electronic tools have been centralised on the Participant Portal and the procedures for awarding joint actions and grants have been simplified. Our examination focuses on views of these new features and implications for the efficient running of the programme.

Financial mechanisms

The different financial mechanisms available under the 3HP each carry with them specific attributes regarding administrative costs. While individual actions may deviate from general trends, larger actions typically allow funding to be administered more cheaply than smaller actions. This means that joint actions typically cost less to administer per EUR of funding than other actions (although we note that from the perspective of DG SANTE, the design and negotiation involved in joint actions means they are also resource intensive). Similarly, projects, operating grants and (most) procurement contracts involve competitive tendering processes that reportedly take relatively more time and money for Chafea to administer given their size (although we note that this was not the case for DG SANTE). This does not imply that larger actions should be prioritised, especially given that cost-effectiveness is mainly driven by the ability of individual actions across the different mechanisms to generate EU added value. It rather means simply that, other things being equal, larger actions entail proportionately smaller administrative costs.

In this context, it is worth considering the split across the financial mechanisms and how it has been evolving. In the first three years of the 3HP, there has been a continuation of trends observed during the 2HP, namely a decrease in spending allocated to projects, a continued increase of spending on joint actions and an increase in spending on DG SANTE- and Chafea-managed procurement contracts (see Figure 1 in the introduction).

63


Taken together, the substantial increase in funding for joint actions and commensurate drop in projects imply an improvement in administrative efficiency once an action is up and running121. While this is to some extent offset by the continued increase in procurement contracts, it is important to distinguish between those contracts which are managed by DG SANTE and those managed by Chafea. Chafea procurement contracts are typically higher value than DG SANTE-managed ones (on average €430,000 compared to €70,000, as shown in the table below). This is due to the large number of low value (often IT-related) contracts that are managed by DG SANTE.

In fact, a development under the 3HP is for higher-value procurement contracts: of the 50 service contracts managed by Chafea, six are valued at €1m or more122. This means that, even considering the proliferation of funding for procurement contracts, the funding mechanism trends have been positive for the efficiency of the 3HP. Moreover, the pattern of increasingly large actions holds also true for other financial mechanisms (with the exception of direct grant agreements) as shown below.

Table 10: Average budget of financial mechanisms, 2HP and 3HP

Financial mechanism Average budget per action % Change 2HP – 3HP 2HP 3HP

Joint action €2.1 m €3 m +39% Project €0.7 m €1.3 m +74% Operating grant €0.2 m €0.3 m +45% Direct grant agreement €0.4 m €0.3 m -23% Service contracts – Chafea €0.2 m €0.4 m +101% Service contracts – SANTE €0.2 m €0.07 m -58%123 Conferences €0.07 m €0.07 m +0.1%

The most notable change is in relation to projects: not only is there a lower proportion of funding being awarded to projects but the average budget per project has risen by 74% from an average of €700,000 to €1.3 m.

From the averages above, it is clear that conferences are the smallest and thereby most “inefficient” actions. This was confirmed by internal analysis done by Chafea which confirmed that conferences with low budgets (less than €200,000) are not cost-efficient given the human and financial resources which must be deployed for a specific call for conferences124. Therefore, for the 3HP, only those conferences with a substantial budget and policy relevance were retained125, while the smaller ones have been integrated as

121 It has been stressed by all those concerned in the set-up of joint actions (which require a long negotiation process typically lasting a year) that these are immensely resource intensive from the point of view of SANTE and Chafea and MS authorities. 122 In 2016: Training programme for first-line health professionals, border officers and trainers working at local level with migrants and refugees (€3.5 M); European Reference Networks -Assessment of applications of Network and membership proposals - March-June 2016 (€1.1 M); Pilot specific training modules for health professionals, border guards and trainers in migrants and refugees (€1 M) and Support to health workforce planning and forecasting expert network (€1 M). In 2015: Surveys and target prevention projects for training of health professionals in the area of HIV/AIDS (€2 M) and Implementation analysis regarding the technical specification and other key elements for a future EU system for traceability and security features in the field of tobacco products (€1.4 M) 123 A word of caution in the interpretation of this figure: the size of SANTE procurement contracts appears to fallen dramatically, and yet this may be simply the result of accounting techniques i.e. where individual actions were previously reported in aggregate, like “IT services”, a detailed list of individual service contracts has been shared – increasingly the number of actions and meaning the average is lower. 124 As reported by a Chafea representative 125 These are the high level health conferences held by MS authorities assuring the EU Presidency.

64


deliverables into other actions (mainly in projects/operating grants and joint actions). An exception is made for conferences in which Commission either is the organiser or plays an important role and in this case this falls under the Communication costs.126

Programme administration

All spending programmes entail operational costs. Given the HP’s diverse funding mechanisms and the wide range of issues addressed, it would be expected for these costs to be substantial. At the same time, it is difficult to define in more precise terms how large these costs should be. To provide some benchmark of where the HP fits compared to other programmes financed by the EC, we held up the HP against one programme implemented by the same agency, and one programme which shares some common goals with the HP. This provides a high-level point of reference for assessing the operational efficiency of the HP. The comparison is not intended to compare like for like, not least because of the different size and scope of the benchmark programmes. However, our choice of comparators is purposeful and shows how the HP which has a modest budget, compares to a programme with a significantly larger budget, and one with an even more modest budget.

As the table below shows, the costs of the HP compare favourably with the overall programme budget than for the Consumer programme, also managed by Chafea and are slightly higher than for Horizon 2020. This result is the pattern that would be expected given the size of the programmes.

Table 11: Budgetary data for 3HP, Consumer Programme and Horizon 2020 (2014 – 2020)

Programme Administrative

support budget

Executive Agency

budget

Total budget Proportion of total budget

to administrative

support and executive

agency

Health Programme €10.5 m €29.2m €449.4 m 9%

Consumer Programme

€7.5 m €10.9 m €188.8 m 10%

Horizon 2020 €3,594.6 m €946.2 m €74,320.4 m 6%

Source: The Draft general budget for the European Union for the financial year 2016, DG BUDGET COM(2015) 300 – June 2015

On the one hand, it might be concluded that, based on the above, the HP is run relatively efficiently. However, given that a common challenge voiced by interviewees relates to the lack of resources (both within DG SANTE and Chafea) there is a possibility that more needs to be done to identify potential efficiency gains. For instance, by reviewing how resource-intensive different administrative procedures are and seeking to identify those which could be simplified.

At the operational level previous evaluations have sought to identify areas for improvement to make use of efficiency savings by removing costly complexity and building in the use of more efficient (time-saving) systems wherever possible. This evaluation finds that, in line with findings from previous evaluations, over time there has been a clear trend towards improvements and simplifications to the 3HP in terms of programme management and operationalisation. These simplifications have

126 E.g. Annual Conference on European Reference Networks, EUPHA, Gastein. 65


been welcomed by Chafea and applicants alike and illustrate how ongoing lesson learning from each iteration of the HP is allowing the management to be refined to gradually remove complexity.

To give some examples of simplifications to the design of the programme: the rules of the programme have been changed to make them less complex, i.e. through the harmonisation of the level of co-financing rates to 60% (or up to 80% in cases of exceptional utility, as discussed in section 4.3); allowing operating grants to be funded through framework contracts (which run for up to three years) and introducing a negotiation process for joint actions. But also the introduction of new processes to streamline the implementation of the HP, i.e. the introduction of MAP process (as discussed under section 4.2). Right down to the level of the funded actions there have been simplifications to requirements for amendment procedures, most important the ability for beneficiaries to transfer resources between different cost categories without the need for an amendment (which made up more than 40% of amendments under 2HP)127.

Simplifications to the management of applications and actions have also been facilitated by the introduction of electronic tools for the submission of proposals, management of grants and e-reporting and monitoring. In addition, all electronic tools have been centralised on the Participant Portal. Given that the HP management is spread between Luxembourg City and Brussels, the advantages of electronic management are especially important. Indeed, interviewees confirmed the time-saved through the increased automation of these procedures and the reduction in paper handling.

The main message is that as a whole interviewees felt these improvements have substantially enhanced the administrative efficiency of the 3HP compared with its predecessor. Although, as evidenced in the feedback from beneficiaries through case studies, there was an adjustment cost, while new and repeat beneficiaries get to grips with new systems. And, despite the improvements described, NFPs and PC who responded to the targeted follow-up survey still felt that the administrative burden128 was a key barrier to participation among potential beneficiaries (especially respondents representing high GNI countries). Complexity of the application process was also deemed important by survey respondents, but when (through the follow-up survey) respondents were asked to only select the most important barrier, administrative burden was a lot more prominent (with 44% of respondents selecting this option). Further, there was also anecdotal evidence that the administrative burden was prohibitive for some smaller organisations which lack the experience of applying for EU funding, which might have otherwise shared the goals of the HP. In the words of a Chafea representative: “… the requirements of the EU financial regulation compliance cannot be fulfilled by small organisations that do not have the operational and financial capacities. Most of these organisations have a national scope, and are not active at EU level.” Finally, the results from the open public consultation confirm that administrative burden in the broader sense (i.e. complying with rules and legal requirements as well as the manner in which this is done) is widely perceived to be the most important barrier (with 42% of respondents selecting this as an area of concern). We discuss possibilities to address these barriers in the recommendations contained in section 8.2.

127 Note that certain elements, such as changes to the technical or financial annex as well as transfer of funds between partners, still require amendments. 128 “Administrative burden” encompasses a range of administrative elements relating to the management of actions from the legal, economic, financial and technical requirements which need to be complied with before an action can be procured / launched, to the reporting requirements for actions once they are up and running (reporting of implementation activities and deliverables, as well as indicators where applicable).

66


Conclusion

Our assessment of the efficiency of programme management focused on aspects of the 3HP that are new or have been evolving. We have found the trends in split between the funding mechanisms has been broadly positive, above all the increase in funding for joint actions compared to projects (and the simultaneous increase in average size of projects), as well as an increase in the size of procurement contracts managed by Chafea. Nonetheless, it is important to recognise that joint actions in particular require significant investment to prepare from all stakeholders. Perhaps even more importantly, irrespective of the size of an action, the biggest driver of efficiency is how effective the action is in achieving its goals and therefore the value added by EU action.

A high-level comparison of administrative costs with other Commission programmes for which data was readily available demonstrated that the 3HP is relatively cheap to administer. Moreover, the evaluation found that recently-introduced simplification measures have led to cost savings and been favourably received among programme managers and other stakeholders, though some concerns about administrative burdens remained. These related particularly to perceived complexities of the application process and reporting requirements during the implementation of funded actions.

67


5.3. Improvements

EQ 11: How may the efficiency of the 3HP be improved regarding: number of priorities; funding mechanisms; application and implementation procedures and available resources?

Given that this question relates to the future, it has been addressed in section 8.2 on recommendations to improve the HP during the current funding period and further ahead. More specifically:

• The number of thematic priorities are examined in Recommendation 4.

• We did conclude that the efficiency of the HP could be improved by making changes to the funding mechanisms, so this is not taken up in a specific recommendation.

• Application and implementation procedures are discussed in recommendations 5-9.

• Making the most of available resources is addressed in all ten recommendations, through suggestions to enhance the focus of the HP, maximise impact, increase awareness among various stakeholders and improve systems and processes.

68


5.4. Monitoring

EQ 12: To what extent are the monitoring processes and resources (at the Commission and MS level) sufficient and adequate to plan and promote the results of the Health Programme?

The question seeks to assess the monitoring processes and resources available at the Commission and MS level and the extent to which they were sufficient and adequate, which we define as fit for purpose. The purpose of monitoring is twofold: on the one hand for internal planning / assessment of results and on the other for promotion of results to stakeholders.

To answer this question we first assess the adequacy of the different monitoring processes and resources and their drawbacks and then go on to consider how drawbacks could best be overcome. Our analysis is based on desk review of the action documentation (i.e. from the Terms of Reference, proposals, to the final reports and deliverables including various monitoring reports), as well as interviews.

Some improvements to monitoring for internal purposes

Monitoring of the 3HP has two important aspects: on the one hand it concerns monitoring of outputs and outcomes of actions (and the programme as a whole) and on the other hand relating to the monitoring of implementation.

One of the recommendations stemming from previous Programme evaluations and underlined again in the ex-post evaluation of the 2HP is to define indicators for better monitoring the actions co-funded under the HP. Leading from these recommendations, substantial efforts have been made to improve monitoring processes for the 3HP. A working group was set up to establish indicators which has led to the introduction of programme-level indicators, which are presented in the HP Regulation itself (Article 3).

The annual budget provides for monitoring of these high level indicators for each specific objective as well as for the general objective and also includes an assessment of expenditure related outputs (foreseen and actual). The indicators for each specific objective are included below (in some cases these have been further operationalised based on the HP Regulation).

• Specific objective 1: Number of MS involved in health promotion and disease prevention, using evidence-based and good practices through measures and actions taken at appropriate level in MS

o MS with national initiative on reduction of saturated fat;

o MS in which the European accreditation scheme for breast cancer services is implemented – establishment of the scheme;

• Specific objective 2: Number of MS integrating coherent approaches in the design of their preparedness plans;

• Specific objective 3: Advice produced and the number of MS using the tools and mechanisms identified in order to contribute to effective results in their health systems;

o Advice produced, in particular the number of HTA produced per year

69


o Number of MS using the tools and mechanisms identified in order to contribute effective results in their health systems: patient summaries data/ePrescription in line with EU guidelines

• Specific objective 4: Number of ERN established in accordance with Directive 2011/24/EU (Draft General EU Budget, 2016); number of healthcare providers and centres of expertise joining ERN; and number of MS using the tools developed.

While having defined these indicators is a major improvement and the fact that they have been defined in a way that is measurable is also commendable, a remaining drawback is that they are far from comprehensive. The indicators are notably narrower and more focused than the priorities outlined in the annex to the 3HP. For example, the indicator defined for specific objective 4 is exclusively focused on thematic priority 4.1 (the establishment of European Reference Networks) while leave the remaining five thematic priorities, including aspects of patient safety related to healthcare associated infections and antimicrobial resistance, the implementation of legislation related to substance of human origin unaddressed.

In addition to the programme level indicators, e-management tools that allow for electronic monitoring at action level (on a continuous and periodic basis) have been developed to include specific indicators. For instance, operating grants recipients are required to fill in an e-questionnaire which includes the five areas, ranging from the high level indicators to more specific indicators on support given on the development of health policies to the EC, and support for patients’ empowerment and increase health literacy. In 2015 this requirement was extended to joint actions. This idea is to gather information on type of beneficiary, deliverables and the concerned and target population as well as dissemination strategy and activities implemented as part of the actions. Among other things, data on these indicators should facilitate later assessment of EU added value. Stakeholders interviewed remarked on the positive changes. In particular, they felt that the online migration of action-level monitoring has reduced the administrative burden for both Chafea and beneficiaries while making it easier to monitor progress in real time.

However, interviewees noted some persistent shortcomings. Most importantly, it was reported that the diversity of the actions funded makes it impossible to design and implement standardised action-level indicators beyond the delivery of activities. The indicators which beneficiaries have been providing have according to interviewees been of highly variable quality and usefulness, and have not allowed for objective performance measurement or comparison across actions.

As such, while in principle these are positive developments, and there is evidence that the programme specific indicators are being tracked and reported on in annual budget reports129, there is little evidence of how the specific indicators are used in practice, indeed for the information to be useful it needs to be analysed and reported on. While part of the problem may be that they have only very recently been launched, in the case of operating grants the system has been sufficiently long that it should have been available for the evaluation since it has been in place for a while130.

Regarding the monitoring of implementation at the aggregate level, this was harshly criticised in the ex-post evaluation of the 2HP. The Action Plan following the Final ex post Evaluation of the 2HP requires that “information relating to budgetary expenditure,

129 Each year DG SANTE reports to DG BUDGET and the report produced includes a record of progress against the programme level indicators. 130 As per the Action Plan following the Final ex-post evaluation of the 2HP, these were included in the 2016 calls in 2016 for the applicants.

70


specific objective, thematic priority, type of beneficiary and geographical and organisational representativeness will be available for the conduct of the mid-term evaluation. This information will be updated on an annual basis and is part of the annual report on the 3HP implementation.”

However, the analysis carried out for this evaluation has shown that significant shortcomings continue to be unaddressed. Information is not stored or organised in an efficient manner, so that data is ready to be analysed. Instead, considerable efforts to manually link up data with thematic priorities from the AWP, collate and cross-check the information were necessary. Again, as before, the evaluation found little evidence that monitoring data were used beyond the purposes of financial accountability for individual actions. It seems that this is not prioritised at all and the system is simply not fit for the purpose of internal monitoring of programme implementation. This undermines the efficient monitoring of the programme, leading to resource-intensive collation exercises which are error-prone due to a lack of automation. These data must be maintained and regularly updated to facilitate the monitoring of the HP (from the thematic coverage to the involvement of partners from different Member States and organisations).

Adequacy and drawbacks of monitoring processes and resources - promotion

As well as serving the purpose of supporting the internal monitoring of the Programme, ideally data on the actions and implementation of the HP would also be used to publicise the activity of the HP. Further, making results of actions and other information publicly available and trying to ensure that the results reach the right kinds of stakeholders to facilitate their take-up beyond those who are directly involved in the actions is an integral part of the HP’s eventual impact. As per the evaluation of the previous HP, significant efforts were called for to ramp up dissemination of the programme and better promote its results to stakeholders. While important strides have been made to address the weaknesses identified and build internal capacity for dissemination, significant challenges remain and much of the work has yet to bear fruit.

In terms of positive developments to improve dissemination, info sheets covering 12 topics are now available on the Chafea website in 23 EU languages131. For outreach, Chafea, together with the National Focal Points have run a variety of well-attended events under the 3HP: a total of 16 events were organised in 2016 and 11 have been planned or implemented already in 2017.132 A report on dissemination activities produced by Chafea documents the high level of satisfaction and success of these events, as well as the positive reception of the info sheets133.

In terms of efforts made, we note the increasing prioritisation and professionalism awarded to dissemination. Evidence of this is the launch of the Health Policy Platform in April 2016134 (and this has been used to disseminate information among relevant stakeholders, for instance training material for health professionals and frontline staff working with refugees, for instance).

More importantly than these specific achievements, is the adoption of an overarching dissemination strategy, which is an important step to steer successful promotion of the results of the 3HP. Although the Dissemination Strategy outlines a long-term vision, there is some evidence of how the strategy is already being operationalised. For instance, the appointment of a designated Dissemination Officer within Chafea whose role it is to ensure effective communication. The Strategy itself has been developed by

131 http://ec.europa.eu/chafea/health/hp-infosheets_en.html 132 list of events at: http://ec.europa.eu/chafea/health/events_en.html 133 Evaluation of the dissemination activities 2014-2015 report; REPORT ON THE DISSEMINATION ACTIONS FUNDED UNDER the EU HEALTH PROGRAMME WP2013 AND WP2014 (Cinthia Menel Lemos Chafea Health unit, 09/10/2015). 134 https://webgate.ec.europa.eu/hpf/

71

http://ec.europa.eu/chafea/health/hp-infosheets_en.html

http://ec.europa.eu/chafea/health/events_en.html

https://webgate.ec.europa.eu/hpf/


the Dissemination officer, it has an overall aim (to “enhance impact and benefit of the actions funded under the 3rd Health Programme, to the benefit of EU citizen Health”) and outlines principles to achieve this, target audiences and tools which will be used. In terms of a concrete annual work plan and key performance indicators, the first year for which this was developed was 2017.

At this early stage (the role began in August 2016) and given the timescales for some of the Dissemination Plan which lasts until 2021 and would be enacted in part through a contract for a communication service provider who has not yet been contracted, it is too early to assess results of the entire strategy. What we were able to establish in discussions with the Commission and with the newly appointed Dissemination Officer is the scale of ambition as well as the high level commitment to substantially improve dissemination building on successes of the previous years.

Notwithstanding the commitment to improve aspects of dissemination, we note that progress in some areas has been disappointingly slow. Most notably, despite earmarking around €300,000 for the creation / publication of a new database135, it has been repeatedly postponed and, in the meantime, the existing project database is incomplete (i.e. it is missing information on procurement contracts and direct grant agreements).

While this seems to be a frustration to those involved, more importantly its delay has a negative impact on the visibility of the Programme and its results. Indeed, the low visibility of the HP was also a finding from the OPC which found that awareness and information on the Programme remain relatively limited, even amongst stakeholders: more emphasis should be put on the communication and dissemination on the 3HP, the opportunities created and its results, while the support to the networking of stakeholders could also be strengthened.

Conclusion

In conclusion, the 3HP has set out to make meaningful progress to address the criticisms of previous evaluations in relation to internal monitoring and promotion of results. In terms of monitoring, this has meant establishing programmatic indicators and action level e-monitoring. In terms of promoting results, the 3HP has seen investments in more professional and strategic dissemination activities to consolidate and develop existing strengths.

However we found that in relation to the indicators, the programme level indicators are not at all comprehensive and that the action specific indicators do not seem to be used in practice. For instance, although data for action level indicators have been collected for operating grants since 2015, they do not seem to be actively reviewed or utilised. We interpret this to be a demonstration of the lack of confidence in their usefulness but also might be related to their relative newness.

Regarding the dissemination of results, while important efforts have been made and achievements noted, again, in some respects it is too early to determine whether at the level of the overarching strategy, dissemination activities will contribute substantially to the HP’s aims. 2017 will be the first year that a work plan with key performance indicators is in place. What we can say at this early stage is that the commitments made are moving in the right direction with a concrete plan, and that progress in certain areas is not fast enough (as shown by the repeatedly delayed online database).

135 Internal Excel with consultation plan and estimated budget for dissemination, Chafea and SANTE 72


6. EU ADDED VALUE

6.1. Consideration of EU added value in action proposals

The EU’s supporting competence in public health means that action can only be justified if it adds value above and beyond what the Member States and other actors could achieve on their own. In order to ensure sufficient attention to EU added value, during the 2HP Chafea introduced a set of criteria that described in more concrete terms how the programme’s action could potentially provide EU added value. These were later revised and streamlined into the seven criteria presented in the table below.

Table 12: Seven areas where EU added value of action should be demonstrated

Criteria as per 3HP Regulation Description

Fostering best practice exchange between MS

To apply best practice in all participating Member States, e.g. by identifying procedures, approaches, methods or tools that could be applied by healthcare professionals or others

Supporting networking for knowledge sharing or mutual learning

To make sure that networking activities among stakeholders, which contribute to knowledge sharing and building health capacity in the EU, are supported and sustained

Addressing cross-border threats to reduce risks and mitigate their consequences

To reduce risks and to mitigate the consequences of cross border health threats by establishing relevant structures for coordination

Addressing certain issues related to the internal market where the EU has substantial legitimacy to ensure high-quality solutions across MS

To facilitate the movement of patients, healthcare personnel and goods and services between EU Member States, thereby contributing to a better match between supply and demand, and to ensure the consistent implementation of EU legislation in such areas as tobacco, medicinal products and medical devices

Unlocking the potential of innovation in health

To support the deployment of innovative solutions for healthcare provision, in terms of both products and services

Actions that could lead to a system for benchmarking for decision-making at Union level

To facilitate evidence-based decision making, e.g. by providing scientific information, real time data for comparison, and/or indicators that can impact on decision making at a higher political / policy level

Improving efficiency by avoiding waste of resources due to duplication and optimising use of financial resources

To save money and provide a better service to citizens by avoiding a duplication of efforts and by cooperating across national health systems

While the EU added value criteria were meant to be taken into consideration as part of the application process during the 2HP, the process was informal and inconsistent. In order to increase the consistency and rigorousness with which the criteria are applied, the system was formalised for the 3HP. Indeed, EU added value is a guiding principle for establishing AWP in the 3HP136.

After a brief overview of the process for assessing EU added value in funding proposals, this section examines the extent to which it has been successful in the 3HP so far. The analysis draws mainly on the case studies, which looked in depth at up to five actions for eight of the 3HP’s thematic priorities.

136 As per Annex II of the Regulation 282/2014 establishing the 3HP: “The added-value of the proposed actions in line with the thematic priorities in Annex I”

EQ 13: To what extent are the seven EU added value criteria addressed in proposals?

73


EU added value criteria as addressed in proposals

Firstly, it is important to note that it is not the case that a proposal must meet as many of the criteria as they can, as some are not relevant for particular thematic priorities (e.g. reducing cross border threats is not relevant for combatting chronic diseases). Instead, potential beneficiaries must highlight in applications the area(s) where they can add the most EU value, and they are then graded on how well their proposal meets this / these particular criteria.

Although EU added value is not treated separately as a specific criterion, most applications (aside from procurement contracts, whose EU added value is determined before requests for services are issued) are screened and awarded a score for an aspect which encompasses EU added value. For example their “contribution to public health in Europe” (joint action) or “Policy and contextual relevance” (operating grants and projects)137. Indeed the justifications for these scores made reference to EU added value specifically.

Scores for EU added value are allocated on a 1-10 scale, where 10 indicated maximum contribution / relevance. The table below summarises the scores for the 12 actions reviewed as part of the case studies for which they were available138.

Table 13: EU added value of sampled actions (scores from application process)

Action name Funding mechanism

Score (from 0-10)

Section of application evaluation report where EU added value is assessed

Thematic priority 1.1 Risk factors Joint Action on Nutrition and Physical Activity (JANPA)

Joint action 7 Contributions to public health in Europe

Obesity Training And Information Services for Europe (OBTAINS)

Operating grant

10 Policy and contextual relevance of the operation of the non-governmental body

Thematic priority 1.4 Chronic diseases Joint action on Dementia 2015-2018 (DEM 2)

Joint action 6 Contribution to public health in Europe

Participation to Healthy Workplaces And Inclusive Strategies in the Work Sector (PATHWAYS)

Project 9 Policy and contextual relevance

Alzheimer Europe (2015-2017) Operating grant

10 Coherence with the Multi-annual work plan

Cancer Leagues Collaborating in Cancer Prevention and Control at the EU and National Level (ECL)

Operating grant

9 Policy and contextual relevance of the operation of the non-governmental body

Thematic priority 2.3 Implementation of Union legislation on health threats

Efficient response to highly dangerous and emerging pathogens at EU level 2014 (EMERGE)


Thematic priority 3.6 Implementation of Union legislation in the field of medical devices, medicinal products and cross-border healthcare Statistical data and Guidance Document for medicinal product pricing and for the use of ERP (EURIPID)


Market surveillance of medical devices (JAMS)

Joint action 8 Policy and contextual relevance

137 service contracts and direct grant agreements do not include these criteria 138 out of 31 examined in total, many of which were procurement contracts

74


Thematic priority 4.5 Implementation of Union legislation in the field of tissues and cells, blood, organs

Vigilance and inspection for the safety of transfusion, assisted reproduction and transplantation (VISTART)


Good practices for demonstrating safety and quality through recipient follow-up (EURO-GTHP II)


European Cornea and Cell Transplantation Registry (ECCTR)


Improved but continued efforts needed

As already mentioned in the introduction, the first point to make is that the process for assessing EU added value is improving and it is becoming more systematic, as evidenced by the approach described above. Feedback from Chafea project officers confirmed that added value descriptions have improved as applicants learn and develop approaches to describing EU added value and this ability to score well in the area which comprises EU added value gives credibility to this claim.

Nevertheless, there are still gaps and drawbacks to the assessment of EU added value in proposals at present. Evidence of gaps is illustrated by the fact that there are examples where the criteria were not consistently assessed (for example, the descriptions of EU added value in action proposals presented in table 15 above, often go beyond the seven criteria defined for the 3HP) or where (despite the requirement to assess EU added value) no explicit reference to EU added value was made in the evaluation summary report (e.g. EURIPID, a web-based database for medicine prices139). Further, even where EU added value was presented and evaluated, in the view of external evaluation experts consulted for this evaluation they were not necessarily convinced of the quality of these descriptions, which they described as sometimes generic.

In view of these issues, there is clearly scope to become still more systematic in the assessment of EU added value and to make it clearer to applicants how the assessment of EU added value feeds into the award process. For example, making EU added value a separate criterion in proposals to ensure its importance is highlighted and allow for a systematic assessment for all actions (across funding mechanisms) would address the present gaps.

In addition, there is also scope to improve the quality of the descriptions of EU added value in proposals. For example, the current guidance documents and the Chafea FAQ refer to the slightly different EU added value criteria which were used under the 2HP140, while a factsheet also refers to the “old” added-value criteria141. Moreover, there are no examples provided to help applicants understand how to improve their own descriptions of EU added value. This could help them avoid confusion. Further, while an explanation of what the EU added value criteria are is welcome, further elaborating these with examples to help would likely help bring these descriptions to life in the proposals and make it easier to evaluators to identify the proposals which offer the best EU added value.

Finally, it is worth noting that the evaluation of the 2HP noted that some of the EU added value criteria (namely those relating to best practice exchange, networking and

139 Perhaps the EU added-value was deemed self-evident but in this case it should nonetheless be outlined. 140 I.e. previously “free movement of persons” was included and has been removed but the phrasing and emphasis of others has also changed. See http://ec.europa.eu/chafea/health/faq.html 141 See http://ec.europa.eu/chafea/documents/health/hp-factsheets/added-value/factsheets-hp-av_en.pdf

75

http://ec.europa.eu/chafea/health/faq.html

http://ec.europa.eu/chafea/documents/health/hp-factsheets/added-value/factsheets-hp-av_en.pdf


benchmarking) were less outcome-focused than the others. Leading from this, the evaluation recommended refining these so that demonstrating EU added value would require e.g. plausible plans to see best practices actually implemented (rather than merely shared). This recommendation has not been taken up to date, most likely due to the short period which has elapsed since the roadmap to act on the recommendations was agreed in March 2016. Once this is enacted, it should mean the EU added value criteria can be better used as a way to distinguish the worthiest actions.

Conclusion

Building the seven EU added value criteria into the application process for funding has been a major improvement on the informal approach to considering EU added value used during the 2HP. It has ensured that the majority of relevant potential beneficiaries consider EU added value when preparing their proposals, and that assessment panels in turn take it into account as part of the decision to award funding.

However, the evidence also suggests that applicants and evaluators sometimes lack a common understanding of the seven criteria, and they are not always considered in a consistent fashion. This implies that the guidance for stakeholders is not sufficiently clear, which we have indeed found to be the case. In turn, this makes it difficult to know how the scores are allocated and how this weighs on funding decisions. Furthermore, some of the criteria (namely those related to sharing best practices, networking and benchmarking) are less outcome-focused than others which also makes it harder to use EU added value as a way of identifying potentially worthy actions.

76


7. COHERENCE

This chapter examines the extent to which the 3HP is both internally coherent and well aligned with other policies and programmes. Given the scale of the health challenges affecting the EU, both aspects are crucial. The former relates to the HP’s structure and focus of its own actions and resources, while the latter refers to exploiting synergies and avoiding duplication with other (typically larger) initiatives at national, European and international levels. The analysis provided in the ensuing sections begin with an examination of internal coherence, followed by external coherence.

7.1. Internal coherence

The assessment of internal coherence of the 3HP involves looking at a how well different funded actions are working together to deliver the programme’s objectives. The previous HP was criticised for a “certain lack of structure and prioritisation”, leading to structural improvements that aimed to “introduce a better focus” into the 3HP.142

This section examines the extent to which the revised structure, which organises planning processes and funding in terms of 23 thematic priorities, has improved the internal coherence of the 3HP so far. Based on the findings of focus groups with experts and interviews and surveys of key stakeholders, we first explore this at the higher level, in terms of the new structure as a whole. Then, to further substantiate and examine how well this structure is working in practice, we draw on examples from the case studies, which allowed for in-depth research into a subset of eight thematic priorities and up to five specific actions for each of them.

An improved structure to support internal coherence

As explained under Relevance (Chapter 3), the design of the HP has changed significantly under the 3HP. The consensus from interviews, case studies and the focus groups shows that the new structure provides a substantially improved framework for a more coherent set of focused actions in more concretely defined areas of intervention. We note that this was also the prevailing view of stakeholders consulted through the open public consultation. Participants were asked whether the way that objectives and thematic priorities have been defined allowed for more focused action than under predecessors: 45% of respondents agreed or strongly agreed that it had, while 33% were neutral, just 4% didn’t agree or strongly didn’t agree (and 17% didn’t know). Below is an elaboration of how the new structure provides a framework for more coherent, focused action:

• Specific objectives at have been refocused and made more precise. For example, instead of “Promote health” (as per 2HP), the 3HP seeks to “Promote health, prevent diseases and foster supportive environments for healthy lifestyles taking into account the “health in all policies” principle”. And, while the 2HP had just three objectives the 3HP has four, all of which tackle distinct, relevant areas: health promotion, health security / threats, health systems and access to health.

• An effort has been made to further operationalise the 3HP’s specific objectives with the definition of operational objectives (which did not exist

142 Page 82, Ex post evaluation of the Health Programme (2008-2013), Coffey and SQW, 2015.

EQ 14: To what extent have the thematic priorities of the 3HP led to more synergy, focus and coherence between the funded actions in delivering on the objectives?

77


under the 2HP). For instance: for the objective on health promotion, “identify, disseminate and promote the uptake of evidence based and good practices for cost-effective health promotion and disease prevention measures by addressing in particular the key lifestyle related risk factors with a focus on Union added value”

• Compared to the 2HP, there are now no longer priorities and sub-priorities but instead consolidated, concrete thematic priorities. Just as the name suggests these are now a majority of specific, relating to themes / topic areas which fall under the more specific objectives already discussed above. For example, under each specific objective there are two cross-cutting thematic priorities. There are some exceptions, one of these is on implementing Union legislation, while the other relates to fostering “a health information and knowledge systems to contribute to evidence-based decision making”. There were also calls to further increase the focus of thematic priorities and reduce the number of thematic priorities, for example by consolidating some of the narrower thematic priorities (such as 1.2 which tackles drug related health damage) into broader thematic priorities (such as 1.1 which tackles risk factors and / or 1.3 on communicable chronic diseases HIV / AIDs). This would not only be in line with the approach taken by most MS health promotion agencies and programmes but also reduce existing overlap between actions which tackle drug-related health damage already but are part of a broader approach.

• In almost all cases, actions now fall under one thematic priority, even if synergies with other priorities are possible, the increased purposiveness and topical focus of the thematic priorities enables greater clarity and coherence in the scope of actions, as noted in the case studies (and as compared to the situation in the 2HP). It is not that the situation is perfect and exceptions have already been highlighted under Relevance (Chapter 3), but these are notable for their rarity143. It is also worth noting that under AWP 2016, there was a call for proposal for actions to provide “Best practices in care provision for vulnerable migrants and refugees” and these relate to both thematic priorities 1.3 (relating to communicable diseases) and 1.4 (relating to chronic diseases)144. This is not necessarily a drawback as it illustrates the HP can be flexible in targeting two thematic priorities simultaneously with sizable actions.

Synergies developed but some reliance on relationships

While the new structure of the 3HP provides the backbone for a more coherent programme, the evaluation question specifically asks about the extent to which this has led to funded actions which actually display more coherence and synergies. Since our case studies looked in detail at a sample of funded actions under the eight thematic priorities, the major insights are based on this evidence, supported where relevant with findings from interviews.

The case studies illustrate that the most significant risk to coherence is the risk of duplication, given how closely some actions (and thematic priorities) relate to one another. However the evidence suggests to this risk is currently mostly being managed, in particular through a proactive approach to the design and implementation of individual actions that encourages them to build on each other and / or communicate closely.

143 Most notably thematic priority 1.1 on risk factors. 144 An example of a contract which was funded is the: “Pilot specific training modules for health professionals, border guards and trainers in migrants and refugees”.

78


For example, thematic priority 1.4, which tackles chronic diseases, at first glance would have a high risk of incoherence due to the wide breadth of issues it covers. However, the case study shows that the majority of actions under review have developed links and cooperated with existing actions, mainly as a result of numerous well exploited relationships between beneficiaries. For instance, the Cancer Leagues Collaborating in Cancer Prevention and Control at the EU and National Level (ECL)145 which was awarded an operating grant, is an associated partner in the ongoing Cancer Control Joint Action (or “CANCON”)146 and provided research inputs on survivorship and rehabilitation throughout 2016. It also used the CANCON newsletter to promote the European Code Against Cancer, one of the key activities funded by the operating grant. Other examples from this case study are Alzheimer Europe (AE)147, which has been asked to contribute to work packages of the joint action on Dementia (DEM2)148. The lead beneficiary of the project dealing with work-related issues for people suffering from chronic diseases - PATHWAYS149 - will participate in another joint action tackling chronic diseases (CHRODIS)150. “DEM2” is also a continuation of the joint action on Alzheimer Cooperative Valuation in Europe (ALCOVE)151, which built a wealth of evidence and knowledge to support MS in developing their dementia policies and operational capacity. The new joint action will now support the uptake of evidence-based practices.

There are also examples of synergies which illustrate that the 3HP is building on the work undertaken in the previous HP and additionally becoming more focused. For example, the joint action “EMERGE152” launched in 2014 and coordinated by RKI DE and INMI IT, is the result of the merging of two networks of highly pathogenic agents, funded in 2010 as the “QUANDHIP”153 joint action.

Actions examined under thematic priority 4.1 on European Reference Networks also demonstrate how actions build on one another. The case study found that “There is coherence between their aims and their design”. The sequencing of the three actions attempts to contribute to the creation, implementation and operation of ERNs starting with:

• The joint action “RD-Action” (which built on the work of the previous joint action in the 2HP) and supports the adoption of a codification and knowledge management system for rare diseases, which will be necessary for ERNs to diagnose patients and share knowledge and expertise across the EU.

• The Study on the Manual and Toolbox for Assessing ERNs will develop an assessment manual that will serve as an evaluation framework for ERNs. It is designed to coincide chronologically with applications received to the call for ERN status.154

145 http://ec.europa.eu/chafea/projects/database.html?prjno=709864 146 CANCON was funded under the 2HP and will run until 2017. 147 http://ec.europa.eu/chafea/projects/database.html?prjno=707934 148 http://ec.europa.eu/chafea/projects/database.html?prjno=678481 149 http://ec.europa.eu/chafea/projects/database.html?prjno=663474 150 http://ec.europa.eu/chafea/projects/database.html?prjno=20132201 151 http://ec.europa.eu/chafea/projects/database.html?prjno=20102201 152 http://ec.europa.eu/chafea/projects/database.html?prjno=677066 153 http://ec.europa.eu/chafea/projects/database.html?prjno=20102102 This joint action itself brought together two networks of highly pathogenic agents led by the same organisations which coordinated QUANDHIP and now EMERGE (“Establishment of Quality Assurances for Detection of Highly Pathogenic Bacteria of Potential Bioterrorism Risk – EQADeBa coordinated by the Robert Koch-Institut (RKI) from 2008 -2011, and European Network of Level 4 Laboratories – EuronetP4, coordinated by L. Spallanzani National Institute for Infectious Diseases (INMI), Italy from 2005-2008). In turn, these networks were set up on the basis on results from previous research”. 154 The call for interest to establish ERNs took place in March, 2016. The call for grants for ERN closed on July 22 2016. http://ec.europa.eu/health/ern/implementation/call/index_en.htm

79

http://ec.europa.eu/chafea/projects/database.html?prjno=709864








http://ec.europa.eu/health/ern/implementation/call/index_en.htm


• and finally, the Study on Services to be provided by ERNs (which will support the implementation and functioning of the networks by identifying the possible sets of services that Network Members will provide alone as well as together in a network).155

Furthermore, synergies between thematic priories are being exploited. For example, the European Cancer League (ECL), which is funded through an operating grant under thematic priority 1.4 on chronic diseases, has a memorandum of understanding with the Smoke Free Partnership156 (which receives an operating grant from the 3HP under thematic priority 1.1), creating a framework for joint work and the sharing of perspectives.

These examples show that the scope for synergies can be well managed and avoid risks of duplication. However, the ability to make use of synergies and avoid duplication has not necessarily been the result of the design of the programme per se, but rather on good programme management, communication and reliance on established networks between beneficiary organisations.

A counterexample serves to illustrate this point. While the actions examined under thematic priority 3.6 showed a strategic fit with the priority (which seeks to contribute to the implementation of Union legislation in the field of medical devices, medicinal products and cross-border healthcare), a conclusion of the case study was that some actions were not necessarily linked to others to ensure their mutual support and complementarities.

Conclusion

The revised structure of the 3HP shows considerable attention has been paid to improve on the coherence of previous iterations of the HP. The consensus from the range of stakeholder views collected for the evaluation leads us to conclude that this new structure has been a success. The funding framework has enabled more concretely defined areas of intervention.

Evidence from the cases studies suggests that extensive synergies are being fostered between ongoing and previous actions. While this supports the internal coherence of the HP, we found that this is not entirely attributable to the structure of the HP. Rather, it is facilitated by this structure but in practice requires a successful exploitation of relations between different actors / beneficiaries, as well as good programme management and communication with core stakeholders.

Despite the strengths of the revised structure, the 3HP has ambitious goals and there are still areas where its scope is considered too broad. This can create knock-on effects on individual actions, which either do not relate closely to other ones or fail to make the most of potential links.

155 The Study will seek to conceptualize in concrete terms the services that ERNs will be expected to provide, aiming to catalogue and develop a typology and establish what are the characteristics and costs of services that can be provided by the ERNs and their Members. 156 http://ec.europa.eu/chafea/projects/database.html?prjno=709003

80



7.2. External coherence

This section explores the 3HP and its objectives (which relate to Article 168, the legal base for EU action in public health) and other initiatives at EU level. Given its relatively limited financial envelope and the supporting role played by the EU, real links with other policies and programmes help leverage the 3HP’s benefits while reducing duplication of efforts or unintended results.

The ex post evaluation of the previous HP found that it was highly coherent with the EU’s overarching policy objectives as embodied in the Europe 2020 strategy, as well as the Seventh Framework Programme for Research and Innovation (FP7, which has since been superseded by the Horizon 2020 programme). More specifically, the previous evaluation concluded that “The Health Programme is highly coherent with the EU’s overarching policy objectives embodied in the Europe 2020 strategy, in that it funds actions that have the potential to contribute to a healthier population and workforce (a key prerequisite for smart growth), and/or to reducing inequalities (a key component of inclusive growth).” Important examples of synergies and cross-fertilisation with FP7 were identified, but synergy effects with the Structural Funds were less obvious and limited to discrete examples.

The current evaluation notes that the underlying dynamics for complementarities were present at start of the 3HP and takes them as a baseline. Rather than going back over old ground, we focus on three issues. First, we look at how changes to the programme structure and governance have influenced its coherence with other policies and programmes. This is followed by an examination of how the 3HP has responded to new EU priorities, most notably the refugee crisis and recent epidemics. Finally, we explore the relationship between the HP and international-level initiatives, which was not discussed in depth in the evaluation of the 2HP. The assessment is based on desk research, interviews with a range of stakeholders and case studies.

Continued strong coherence with EU policy and priorities

The links which have been built relate to high level priorities such as DG SANTE’s strategic objectives, the Europe 2020 Strategy, political goals such as the “Juncker priorities” defined in 2015 (including migration) but also other EU programmes such as Horizon 2020 and the European Structural and Investment Funds.

The 3HP strengthens and emphasises the links between economic growth and a healthy population to a greater extent than the previous programmes. For instance, there is now an explicit reference to (and significant support for) the European Innovation Partnership (EIP) on Active and Healthy Ageing, one of the flagship policies of the Europe 2020 Strategy. In addition, as per the 2014 AWP, work under operating grants must contribute to the priorities of the EU 2020 Strategy.

In concrete terms, this prioritisation can be seen firstly in how funding is channelled. For instance, the thematic priority which explicitly supports the EIP on Active and Healthy Ageing (3.5) received over €12m in the first years of the 3HP, making it the sixth highest-funded thematic priority. Three of the thematic priorities under objective 1, which most obviously support the EU2020 Strategy through Health

EQ 15: To what extent are the objectives and thematic priorities of the 3HP externally consistent/coherent i.e. is there correspondence between:

• the 3HP and health objectives of Article 168 TFEU

• those of other public interventions (e.g. national health policies, EU policies and Programmes, other international actions)?

81


Promotion (by focusing support on “healthy lifestyles taking into account the “health in all policies” principle”) comprise the highest-funded thematic priorities (namely: 1.4 which tackles chronic diseases which relate to reduced ability to work; 1.1 which tackles risk factors and preventable diseases; and 1.3 which tackles communicable chronic diseases, as shown in table 3 on the allocation of budget by thematic priority in Chapter 1 of this report). This helps to illustrate that the aims of the 3HP match the Commission’s policy priorities.

In terms of coherence with other programmes, the links between the 3HP and Horizon 2020 programme (which are built into the 3HP’s design through an explicit reference in the annex to the Regulation establishing it) deserve special mention. As pointed out in the ex post evaluation of the 2HP, the Commission’s research programme (then FP7, now Horizon 2020) occupies a different position in the ‘health intervention process’ than the 3HP. More specifically, relevant aspects of the research programmes (the Health research component of FP7 / Health, demographic change and wellbeing societal challenge of Horizon 2020) focus on fundamental research to improve understanding of various aspects of health and to test new methods. Actions funded under the 3HP then focus on further developing the products of this research so that they can be applied in policy terms.

The evaluation revealed several examples of how this is happening in practice. For instance, thematic priority 3.1 on Health Technology Assessment makes reference to facilitating the uptake of results stemming from research projects funded through the Seventh Framework Programme, and is expected to be taken over by Horizon 2020 in the long term. Thematic priority 3.4 also makes reference to support for the uptake of results stemming from research projects supported under research programmes old and new. These concrete references help focus on the need to coordinate activities in specific areas but the scope for synergies goes beyond the research programmes to work in other DGs in relation to e-Health, occupational health, even animal health (for example in combatting anti-microbial resistance). Despite this, some stakeholders engaged during the evaluation felt that the potential synergies between the 3HP and research programmes were not being exploited, and that more efforts were needed to publicise their complementary structures.

EU structural funds, most importantly the European Social Fund (ESF) and European Regional Development Fund (ERDF), sit at the other end of the ‘health intervention process’, related to large-scale implementation. The objectives and thematic priorities are coherent with these instruments, and it would be expected that numerous mechanisms developed through HP actions could be scaled up in this way. However, we did not find specific examples of this happening in practice due to the early stage of the 3HP’s implementation. Moreover, interviewees reported that awareness of the possible complementarities is not as widespread as for those between the 3HP and Horizon 2020. Indeed, according to many interviewees the coherence between the 3HP and other EU funding programmes or activities is best exploited when their distinguishing features are properly understood and when adequate coordination strengthen the links in practice. As discussed in section 4.1, the inter-services group provides the formal structure for coordinating activities, via review of the Regulation and AWP to include references to on-going work or policy. However interviewees explained how much of the collaboration is organised by theme, as the different individuals / units concerned are most intimately involved with the topics concerned they would be part of working groups on issues of shared relevance (for example, there is a working group on reducing alcohol related harm and working group on European Workforce for health) or ongoing discussions.

In terms of exploiting the relative strengths of the HP, interviewees stressed how lean and responsive the HP is perceived to be in comparison to other funding programmes. This, especially given the limited budget allocated to the HP, means that the HP is better suited to implement “incubator” type actions, as expressed by one interviewee, which

82


can be scaled up through other programmes (such as the ESF and ERDF, as mentioned above). Another comparative advantage of the HP is that funds can be used to fund Member States directly, which is not the case for all EU funding programmes and makes the HP the best instrument to take forward actions / research which requires policy coordination.

Strong response to wider EU agenda

Our analysis shows that the 3HP places more emphasis than its predecessor on coherence with wider EU polices relating not only to economic growth but to initiatives at all levels relating to such topic issues such as climate change and the refugee crisis.

The 3HP is also being used as a tool to address policies beyond those strictly limited to public health. While the evaluation did not find much evidence to show how addressing climate change had been meaningfully mainstreamed into HP spending, the new legal basis on serious cross-border health threats (Decision 1082/2013/EC) has given DG SANTE a role to coordinate reactions to health-security crises (the relevance of which has been recently illustrated by the Ebola and Zika outbreaks).157

The migrant crisis provides another important example of how synergies and coordination have been exploited to tackle an all-encompassing emerging challenge which goes beyond the traditional public health remit, in quickly evolving conditions. The spectrum of health issues and needs of the refugees as well as the challenges faced by the MS health systems (and the health workforce) in providing the services to the migrant population illustrates how the coordination of actions in different areas is required to produce desired impacts. Comprehensive policy approaches are favoured and the adoption of the action plan on the integration of third countries’ nationals by the Commission (which integrates education, employment and access to services in general with a specific reference to health services) is one example of concrete implementation. With regard to the 3HP specifically, actions it supports contribute to health capacity-building in the EU MS where hotspots are located (see below). In this way, the expertise and funding available under 3HP complements the other dimensions of the EU’s response to the migrant crisis supported by DG HOME (and ECHO in Greece only) and focuses on the longer-term impact of the action.

Nine actions totalling nearly €15m launched in the 2015 (through an amendment to the AWP) and 2016 AWP have been identified as providing the financing for emerging capacity gaps or needs relating to the health of migrants and refugees.

157 Decision No 1082/2013 83


Table 14: Actions directed towards capacity gaps of migrant / refugee health

Year Description Budget

2015

Project supporting health coordination, assessments, planning, access to health care and capacity building in Member States under particular migratory pressure (SH-CAPAC)

€537,044

Project supporting 8 NGOs for migrants/refugees' health needs in 11 countries

€2,756,269

Project on “Common Approach for REfugees and other migrants' health” €1,689,045

Project “EUropean Refugees - HUman Movement and Advisory Network” €1,251,841

Direct grant to the IOM to support Member States under particular migratory pressure in their response to health related challenges.

€1,000,000

2016

Project on Migrants health: Best practices in care provision for vulnerable migrants and refugees (RE-HEALTH)

€2,484,885

Service contract: Pilot specific training modules for health professionals, border guards and trainers in migrants and refugees

€1,000,000

Service contract: Training programme for first-line health professionals, border officers and trainers working at local level with migrants and refugees

€3,500,000

Direct grant to WHO - Migrant's health €500,000 Total €14,719,084

International level

Lastly, at the international level, the EU is an important player in global health and pursues aims that are largely consistent with the 3HP, as defined in the 2010 Communication and Council Conclusions on ‘the EU role in global health’.158 This outlines the Commission’s rights-based approach to health and support for countries to design policies that equitably maximise health benefits.

While the 3HP focuses mainly on the EU Member States and other participating countries, many health challenges cross borders not just within the EU and there are complementarities with action on global health. For example, the EU speaks with one voice in the WHO’s governing bodies, and through the 3HP the EU makes an annual contribution to the WHO’s European Observatory on Health Systems and Policies. Complementarity between the 3HP (as well as contributing directly to monitoring and specific areas of WHO action). Under the 3HP, complementarity with the ECDC’s actions on infectious disease in a specific areas of close cooperation is ensured by a memorandum of understanding with the joint action EMERGE (funded under thematic priority 2.3 on implementing Union legislation on communicable diseases).

Through EuropeAid, the EU supports such actions as the Global Fund to fight AIDS, Tuberculosis and Malaria, which complements thematic priority 1.3 on HIV/AIDS, tuberculosis and hepatitis and several of the thematic priorities under objective 2 on cross-border health threats. In this vein, the EU has also been working with the WHO and G7 within the Global Health Security Initiative to create an effective and well-organised global strategy for preparedness and responses to potential health threats.159

158 See COM(2010) 128 final, url: https://ec.europa.eu/europeaid/sites/devco/files/communication-eu-role-in-global-health-com2010128-20100331_en.pdf. 159 See for more information at http://ec.europa.eu/health/preparedness_response/cbrn_threats/ghsi/index_en.htm

84

https://ec.europa.eu/europeaid/sites/devco/files/communication-eu-role-in-global-health-com2010128-20100331_en.pdf

https://ec.europa.eu/europeaid/sites/devco/files/communication-eu-role-in-global-health-com2010128-20100331_en.pdf

http://ec.europa.eu/health/preparedness_response/cbrn_threats/ghsi/index_en.htm


In addition, the EU is a party to the Framework Convention on Tobacco Control. This aligns with thematic priorities 1.1 on risk factors and 1.5 on tobacco legislation.

More broadly, many of the EU’s health-related development initiatives relate to areas also addressed in the 3HP. Seventeen countries with development partnerships with the EU have health programmes for the 2014-2020 funding period. Multilaterally, the EU supports the 2030 Agenda for Sustainable Development, for which Goal 3 relates to ensuring health lives and promoting well-being for all at all ages.160 This focuses not only on reducing mortality from communicable and non-communicable diseases, but also addresses mental health and substance and tobacco abuse and facilitating access to medicines. The EU also supports the UN General Assembly initiative on non-communicable diseases.

While the role of the HP is reportedly well known in the tobacco and access to medicines areas, interviewees stressed that the HP is not as visible as an international policy instrument as it could be. This is logical, given the focus of the HP on capacity building within the EU and other participating countries. And yet, there are areas with the potential to take a more global approach; the impacts of climate change and environmental health, the spread of vector borne and communicable diseases and the increasingly pressing need to tackle anti-microbial resistance are just a few examples. As pointed out above, a lot of HP-funded action is relevant for neighbouring and developing countries. From 2016 onwards, Serbia and Moldova participate in the 3HP, while other Western Balkan countries are in the process of signing bilateral agreements for their participation in the Programme (i.e. Bosnia and Herzegovina).

To better exploit synergies, consultation with the WHO revealed that there is significant scope for further collaboration on a range of topics and health challenges as outlined in the Vilnius Declaration161. This could be done by increasing systematic collaboration, making key actors aware of others’ activities and thereby avoiding duplication.

Conclusion

Our analysis shows that to date the 3HP strengthens and emphasises the links between economic growth and a healthy population to a greater extent than the previous programmes, which brings it in line with Commission policy priorities. The HP makes explicit reference to Research funding (i.e. Horizon 2020), and there are many examples of practical complementarities. However, there is scope to make the identification and exploitation of such synergies more systematic, while for structural funding awareness of potential complementarities is low.

Importantly, the 3HP is also being used as a tool to address policies beyond those strictly limited to public health. For example, the new legal basis on serious cross-border health threats has meant DG SANTE can play a role in managing the EU response to health-security crises (i.e. the Ebola and Zika outbreaks).162 The on-going migrant crisis provides a particularly topical example of the opportunity for synergies and coordination and deployment of significant funds in quickly evolving conditions. In the international development / global health arena, the HP is coherent with other EU action and, particularly active in areas relating to cross-border health threats as well as widespread problems such as those relating to tobacco. However, low visibility and sometimes insufficient coordination between the HP / action beneficiaries and other actors holds in back in playing a larger role.

160 See for more information at http://www.un.org/sustainabledevelopment/health/. 161 http://www.euro.who.int/__data/assets/pdf_file/0003/235443/Vilnius-Declaration-Nov2013.pdf 162 Decision No 1082/2013

85

http://www.un.org/sustainabledevelopment/health/

http://www.euro.who.int/__data/assets/pdf_file/0003/235443/Vilnius-Declaration-Nov2013.pdf


8. OVERALL CONCLUSIONS AND RECOMMENDATIONS

The following pages bring together findings from across the evaluation to draw an overall conclusions on the utility of the third Health Programme and the way it was implemented. This answers the last evaluation question, which is:

8.1. Overall conclusions

The Health Programme has taken a long journey since its inception in 2003. The case for EU action has always been clear from Article 168 of the Treaty on the Functioning of the EU, and previous evaluations have consistently praised the achievements of funded actions and the programme as a whole. At the same time, earlier versions of the HP were criticised both for a lack of focus and management difficulties which to some extent undermined their potential added value.

The mid-term evaluation examined implementation of the 3HP in terms of specific aspects that are new in this Programme (e.g. multi-annual planning) or have been previously under-examined (e.g. process for defining the Annual Work Programmes). Particular attention was given to the state of implementation of the 23 thematic priorities agreed with the European Parliament and the Council (Health Programme Regulation N°(EU) 282/2014) and their continued relevance vis à vis the Programme objectives and their contribution to the Commission priorities for years 2014-2020. This was an explicit request enshrined in the Programme legal basis (Article 14) in view of eventual modifications through a delegated act if any of the thematic priorities become obsolete or new needs appear.

The 3HP was designed with past criticisms in mind, and represents a concerted effort to tackle them while also making the most of the momentum gained so far. Most importantly, the programme structure agreed with European Parliament and Council has been designed so that actions are organised around four specific and operational objectives that are broken down into a 23 thematic priorities. These aim to focus the 3HP on types of issues and types of action where the potential to generate EU added value is greatest.

On the management side, multi-annual planning has been introduced to increase the coherence of the programme and various systems and processes have been simplified and digitised. Indicators have been put in place to monitor progress at action and programme levels, while funding applications are now assessed against specific EU added value criteria. An incentive structure to boost participation in the programme among organisations from poorer Member States has been ramped up (called the “exceptional utility” criteria). Communicating about and publicising the HP, persistently a challenge, is being addressed through increased resources and a new Dissemination Strategy.

A few years after the programme began, it was too early for the present evaluation to assess these changes in terms of the HP’s impact on the public health of European citizens. Instead, we sought to give an early indication as to their effects on planning and implementation, and thereby ascertain the extent to which the pre-conditions for success are in place.

EQ 16: To which extent is the 3HP overall useful and, if necessary, how could its overall utility be increased? What are the specific needs of Member States to which the Programme could provide a concrete solution but has not done so yet?

86


Taken as a whole, we found that the 3HP represents a major improvement compared to what came before. The new structure has increased the HP’s ability to target important health needs where it can add value (such as anti-microbial resistance and “e-Health” in the context of the digital single market to name just a few). It is also channelling efforts to identify common, structural challenges for Member States. For instance, mechanisms for pooling expertise at EU level and supporting MS in their health reforms have been set up (namely the Expert Panel on Health and the Expert Group on Health Systems Performance Assessment) and the 3HP is providing direct financial support to the OECD and WHO to produce country profiles which give a clear understanding of country specific needs. This focus is recognised by Member States: the consultation of Programme committee members and national focal points representing Member States’ interests confirmed the 3HP structure matches the main health challenges in their country. A perception of the appropriateness of the 3HP design and structure was also confirmed in replies to the open public consultation.

At the same time as becoming increasingly focused on identified important issues such as those mentioned above, the 3HP structure provides flexibility. This has allowed it to be responsive to shifting circumstances and trends over its seven-year funding period, for instance in relation to a need for crisis management. The migrant crisis of 2015 presented an early and unpreceded test of the programme’s adaptability, given its pan-European nature and the strain it put on existing public health infrastructure. The HP’s intervention, which included the quick deployment of nearly €15m to support healthcare professionals and NGOs dealing with migrants on the frontline, was a major success that highlighted its potential to react decisively in uncertain times.

Where there is more predictability, the 3HP is being used as a tool to support the implementation of EU health legislation in areas of identified EU added value. These include cross border health care, health technology assessment, substances of human origin163, and medicinal products and medical devices . It is also being used to address important policies beyond those limited to “traditional” public health. For example, the new legal basis on serious cross-border health threats has meant DG SANTE can play a role in managing the EU response to health-security crises (i.e. the Ebola and Zika outbreaks). The on-going migrant crisis provides a topical example of the opportunity for synergies and coordination and deployment of significant funds in quickly evolving conditions. There is still work to be done to increase visibility and coordination with other actors, however the evaluation found the 3HP is coherent with other EU action in areas such tobacco, and the international development / global health arena.

Promisingly, the available evidence suggests that the funded actions themselves are producing more concrete results and linking better to wider initiatives than under the 2HP, including the EU Budget Focused on Results initiative164. In part this is because support provided over the long term through several actions (spanning successive iterations of the HP) is finally gaining traction and bearing fruit. But in part it is because actions in the 3HP are more focused and purposeful, especially where there is a clearly defined legal basis.

For instance, actions funded through the 2HP laid much of the groundwork for establishing European Reference Networks (ERNs) under the EU Directive on Patients' Rights in Healthcare (2011/24/EU), which also makes it easier for patients to access information on healthcare and thus increase their treatment options. During the first years of the 3HP preparatory steps have been taken (see Annex B - case study “Thematic priority 4.1”). Since March 2017 24 thematic ERNs, gathering over 900 highly

163 Substances of human origin include: blood, tissues and cells, and organs. 164 See here for more information on this initiative: http://ec.europa.eu/budget/budget4results/initiative/index_en.cfm

87

http://ec.europa.eu/budget/budget4results/initiative/index_en.cfm


specialised healthcare units from 26 countries, have begun working together on a wide range of diseases165. Similarly, the HP’s work on Health Technology Assessment began with a project funded during the 1HP. This led to two JAs during the 2HP to test and pilot methods and generate the buy-in necessary for common approaches in this area. It is only with the third JA, funded through the 3HP, that these approaches are finally being operationalised and anchored in MS practices and at the same time DG SANTE is preparing future legislation on Health Technology Assessment.

In other areas where there is a legislative basis for EU action and clear EU added value, there has been a clear path of progress. For instance in relation to substances of human origin which is underpinned by Union legislation for safe standards166, actions funded in the 3HP take further previous work by developing methodologies and guidelines in new areas (i.e. novel therapies and products), creating new models for sustainable updating of technical standards, defining procedures in areas where this is lacking (e.g. clinical follow-up)167. This is also true in the case of medicinal products and medical devices, where efforts in the 3HP are expected to increase transparency of prices paid for medicinal products, to create more favourable conditions for the introduction of advanced therapies across the EU, among other benefits. These are important steps and demonstrate the importance of EU support in areas where there is a clear EU role to lead action.

That being said, improvements to the programme’s design and strategic focus did not resolve all its problems. The importance of continuity between funding periods has meant that the changes were incremental rather than drastic, and that some previously-noted challenges remain. For example, while the four specific objectives provide a workable overarching framework for the 3HP, some of them are more clearly relevant than others. More concretely, objectives 2 (Protect Union citizens from serious cross-border health threats) and 4 (Facilitate access to better and safer healthcare) relate to relatively narrow challenges that either flow between countries (e.g. health threats and anti-microbial resistance under objective 2) or create opportunities for collaboration to generate economies of scale (e.g. European Reference Networks for rare diseases under objective 4).

By comparison, Objectives 1 (Health promotion) and 3 (Health systems) deal with very broad challenges that are already high on Member State policy agendas. For these objectives, EU added value is typically generated through the sharing of best practices. Actions under these objectives can be highly appropriate, but more care is needed to ensure they fit well with existing initiatives and contexts, and have feasible plans for the eventual implementation of any best practices to be identified and shared. This is possible if care is taken to ensure this is designed into actions. For example an area where the identification, dissemination, and take up of best practices has already been strong is in relation to action on mental health (specifically, dementia168) and cancer (dissemination of the "European Code against cancer", the revision of the European Guidelines on Breast Cancer Screening and the development of a voluntary scheme for accreditation of breast cancer screening units in Member states and other participating countries in the Health Programme". During the remaining period of implementation of the 3HP, DG SANTE intends to strategically enhance the transfer of best practices also in other key health areas and established a Steering group on Promotion and Prevention in November 2016 for this purpose.

165 See here for more information http://europa.eu/rapid/press-release_IP-17-323_en.htm 166 Summaries of EU legislation in the field of blood, tissues and cells and organs, can be found here: http://eur-lex.europa.eu/summary/chapter/public_health/2902.html?root=2902 167 See Annex B, case study on “Thematic priority 4.5”. 168 For example the joint action: “Act on Dementia”.

88

http://europa.eu/rapid/press-release_IP-17-323_en.htm


In addition, some thematic priorities are more precisely and narrowly defined than others in the legal basis, and there are a few minor potential overlaps and differences in scope. For example, while thematic priority 4.1 only deals with action related to the establishment of ERNs for rare diseases, thematic priority 1.4 on chronic diseases has been designed to address a huge array of public health issues (as agreed with the European parliament and Council). Each specific objective also includes two cross-cutting thematic priorities, on the implementation of EU legislation and health information. These can help focus attention on important areas169, but they also create ambiguity in a programme structure that otherwise revolves around specific public health topics (i.e. themes) rather than types of action.170

These structural decisions had knock-on effects for individual actions, some of which were overly broad, insufficiently aligned with MS action and / or experiencing difficulties to disseminate results among key stakeholders. For example, the evaluation found that several actions funded under thematic priority 1.4 on chronic diseases had an overly broad scope, leading to a risk that the HP’s resources would be spread too thinly and making it difficult to generate momentum in an area which is central to the health of EU citizens. There is a need to further ensure the thematic priorities are as well defined as possible to focus on actions which adequately address pressing issues.

Our examination of improvements to the programme’s management processes reveals a similar pattern of generally positive change punctuated with some remaining room for further development. In concrete terms, the introduction of multi-annual planning has enabled programme managers to take a more strategic approach to funding decisions and smooth the formal process of drafting Annual Work Programmes. The continued trends towards joint actions and away from projects reflects the growing maturity of the programme and serves to increase its cost-effectiveness (given the different relative administrative costs associated with the two funding mechanisms). Simplified and digitised application and grant management procedures have lessened the administrative burden both on DG SANTE / Chafea and applicants / beneficiaries.

The areas where there are still challenges in terms of project management indicate not a lack of progress, but rather highlight the difficulty in addressing longstanding problems. For example, while the process for setting Annual Work Programmes was generally considered objective, some stakeholders felt it gave more weight to the needs of the Commission and some Member States over others ones (as demonstrated by the survey of NFP and PCs where one in five participants reported involvement in drawing up the AWP was “not very adequate”). However, it is hard to envisage how DG SANTE could take the highly diverse needs of all Member States and interested stakeholders into account while at the same time funding purposeful and focused action in a field as broad as public health.

The 3HP has improved how it attracts the participation of poorer MS through the “exceptional utility” criterion. However, the increased scope of the “exceptional utility” criterion used to encourage171 Member States and other participating countries to involve in their actions organisations from poorer Member States and spend a part of the action budget for that participation has not yet achieved the expected success. This is due in part to the complex and interrelated set of barriers other than difficulties securing co-funding that impede such organisations to apply to the HP. These include insufficient administrative capacity, the perceived complexity of the application process and concerns about the administrative burden. But tinkering with the parameters of the

169 For example, significant progress has been made in developing country specific profiles as part of the 3HP’s focus on health information. 170 We note that the programme designers considered grouping all such thematic priorities under an additional specific objective, but decided that, on balance, it would be more confusing than the current set-up. 171 This is done by providing higher levels of co-funding up to 80% to all Member states and participating countries involved in actions that fulfil the exceptional utility criterion.

89


criterion and publicising it better might increase its uptake and make the distribution of HP funding somewhat more equitable. In a similar vein, despite improvements to the application and grant management procedures, smaller organisations in particular expressed concerns about their complexity and the reporting requirements.

To better integrate the seven EU added value criteria that were defined during the previous programme (and enshrined in the Regulation establishing the 3HP), many applications for funding are now screened and scored for EU added value. This has helped to mainstream the criteria among stakeholders and focus minds on the ways the programme can add value. However, we also found that the assessment panels responsible for awarding funding lack the guidance to apply the criteria in a systematic and objective way. Consequently, it was clear neither how scores were allocated nor how these weighed on funding decisions.

With regard to dissemination, the previous evaluation recommended DG SANTE and Chafea to “develop a formal communication strategy to define key communication objectives, actors, messages, audiences and channels”. Such a strategy now exists, and Chafea has in place a full-time Dissemination Officer to oversee its implementation. Both of which signal the increased professionalism, and scope for a more structured approach to dissemination which is welcome. At the time of writing the strategy’s implementation was still in its initial stages but the signs were positive. For instance, achievements in such politically salient areas as support in dealing with the migrant crisis provide an increased ability for the programme to demonstrate its EU added value and this is incorporated into the strategy through a dedicated cluster meeting on Migrants’ health “Paths towards integration” planned for 20 June in Brussels. A key part of the strategy is the design and publication of a new projects database. The new database will enable new potential beneficiaries and the wider public to learn about the HP. It will also make it easier for results to be communicated and taken up through its new emphasis on outputs and results of the HP. While it understandably takes time to generate the buy-in and marshal the resources needed to design and carry out activities such as this, the current funding period is nearly halfway complete. In the meantime, previous criticism about inadequate stakeholder engagement and the poor quality of accessible information about the HP cannot be entirely dismissed.

Similarly, the systems for monitoring implementation of the 2HP were harshly criticised. However unlike efforts to improve dissemination, the evidence so far does not suggest that major structural changes have been made or resources made available to improve the efficiency of monitoring processes in relation to monitoring of implementation. This could stem from such issues as the division of management responsibilities between DG SANTE and Chafea (which means that information is stored separately) and teething difficulties associated from the new organisational structure of the 3HP. Perhaps more importantly, programme managers face the challenge of reporting on a Programme which is complex (with many different financing mechanisms operating on different timelines). For instance, joint actions take a long time to be negotiated and signed off, and in the meantime budgetary implementation information can only be preliminary. But the fact remains that vital information about the implementation of the programme is not maintained in a user-friendly format and remains fragmented.

Concerning monitoring of actions’ outputs and outcomes, programmatic and action-specific indicators have now been introduced. Although the evaluation has found that these programmatic indicators are not as comprehensive as they could be, it is a significant step forward to have them in place and revisiting them to ensure better coverage can be an action going forwards. Regarding the action-specific monitoring, the evaluation was unable to find evidence showing how information is being gathered, collated and used. This leaves doubt about whether they will provide meaningful data for the ex-post evaluation and highlights a need for more attention to this area when actions finish and start to produce results.

90


Looking forward, we re-iterate that the changes brought in for the 3HP have been substantial and positive, and they augur well for the likely effectiveness of the programme and its ability to provide value for money. The next section builds on these conclusions by making recommendations that can be implemented either during the second half of the current funding period or in the years following 2020.

8.2.Recommendations

Based on the findings of the evaluation and the conclusions presented above, here we provide a series of recommendations to improve the HP. Where relevant we highlight recommendations which could and should be taken in the immediate term, and those which require more lead in and are thus more appropriate for longer term planning.

The recommendations are presented to distinguish between those which are new (because they relate to recently introduced or substantially changed elements of the HP, i.e. thematic focus, design and structure, programme management) and those which are similar to those made in previous evaluations of the HP but for which more action is needed for the future success of the programme.

8.2.1. Recommendations on newly identified issues

Thematic focus of Programme

1. DG SANTE should maintain a focus on thematic areas of strong EU added value: as explained in our conclusions above, the 3HP has built on the momentum established under previous HP in a number of areas with strong EU added value, for instance the establishment of European Reference Networks, Health Technology Assessment, capacity building in Member States for responding to cross-border health threats, health information and country knowledge among others. This has led to important and tangible progress. Maintaining a clear thematic focus in areas where the EU has a clearly defined role (such as those mentioned) will strengthen the delivery of results in these areas by the end of the Programming period. Linkages to sustainable development goals and targets in WHO agreed processes may be a good way to achieve prioritisation. In addition the cycle of health information and knowledge reports launched by the Commission in 2016 will also allow identification of needs at Member States level to which the health programme could contribute.

2. DG SANTE should strengthen and build links between the HP and the wider Commission and EU policy agenda to maximise impact: the example of the support provided through the 3HP on health needs arising from the 2015 migrant crisis as well as from the work under the EU agenda for effective accessible and resilient health systems172 illustrates the value of policy coordination in ensuring an adequate response to emerging needs and challenges faced by MS health systems.

Programme design

3. DG SANTE should spell out how action targeting health promotion and health systems should generate EU added value: the evaluation found that the 3HP’s structure of four specific objectives provides a more purposeful framework for support than the three strands of the 2HP. However, our assessment also showed that at present objectives 1 (which targets health promotion) and 3 (which seeks to strengthen health systems) encompass a wide range of (relevant) issues that are mainly addressed at the level of individual

172 Commission Communication COM (2014) 215 91


Member States, and that this risked trickling down into funded actions which were overly broad. In part, this could be addressed by making stakeholders (particularly applicants and assessment panels) more aware of how action under these specific objectives is expected to generate EU added value.

a. More concretely, we suggest in the immediate term, DG SANTE define in as much detail the mechanisms by which best practices should be taken up in practical terms and reasonable timescales for doing so (either in general or with regard to specific funding calls). This information should then be shared with key stakeholders such as potential applicants and NFPs.

b. For the longer term beyond 2020, operational objectives should be revised to include more detail about how the HP should generate EU added value and complement the actions of others. In addition, we suggest that a greater proportion of funding is directed towards the actions with the highest EU added value.

4. DG SANTE should continue its effort to focus programme spending on identified thematic priorities: the definition of 23 thematic priorities represents a major step in the HP’s gradual evolution from a relatively general programme to one focused on specific issues and types of action where the potential to generate EU added value is greatest. The relatively large number of thematic priorities was useful in that it provided continuity with the 2HP while allowing a degree of flexibility for the seven-year funding period. However, the first years of implementation have served to highlight some ambiguities and inconsistencies in the thematic priorities that should be corrected over time.

a. We do not suggest changing the structure of thematic priorities in the immediate term given their importance for monitoring spend over time. However, this does not preclude other efforts to enhance their clarity. For instance, as with the recommendation for specific objectives 1 and 3, it would be worth considering whether thematic priorities are concrete enough in terms of their expected results and EU added value and refining their descriptions accordingly.

b. In the longer term beyond 2020, consider further streamlining any thematic priorities to avoid any potential overlap or ambiguities (for instance, thematic priority 1.2 on drugs related health damage could potentially be included under 1.1 risk factors) but also to remove apparent redundancies in cases where thematic priorities have not been utilised (for instance 2.4 and 4.6 which cover “fostering a health information and knowledge system” and were included for the sake of consistency in, but have been shown to be less relevant under objectives 2 and 4 up to now; thematic priority 2.1 on risk assessment has not yet been used even though every year an amount of 70 000€ is earmarked in case risk assessment is requirement for health threats caused by non-biological agents l173). As part of the next funding period, it is also worth considering whether the scope of thematic priorities is sufficiently distinct or could be enhanced for greater clarity. For instance, in the current phrasing much of the work funded under thematic priority 2.1 on risk assessments and 2.2 on capacity building in health threats could be considered to be geared towards implementing Decision 1082/2013/EU on serious cross-border health threats which is the specific subject of thematic priority 2.3.

173 Risk assessment of communicable diseases is the mission of the European Centre for Diseases Control. 92



5. DG SANTE and Chafea should refine the EU added value and fully integrate criteria into the application process: the 3HP made two important steps forward with regard to EU added value. It enshrined the seven EU added value criteria in the Regulation establishing the programme and built them into the process for assessing funding applications. Both of these steps have been recognised in the evaluation as helping to focus the 3HP in the areas where the case for EU action is strongest. However, we also found that the criteria are not necessarily well understood by applicants and assessment panels, and therefore applied somewhat arbitrarily as part of decisions to award funding. We recommend the following solutions over the short- and longer-term.

a. In the immediate term, in order to help potential applicants and assessment panels to build a common understanding of the EU added value criteria as they currently stand, DG SANTE and / or Chafea could develop detailed but accessible “how to” guidance for applicants about what each of the criteria means, including practical examples. NFPs should be briefed about the criteria given their role in supporting potential applicants. Additional guidance could also be provided to assessment panels and published, so that the system for scoring applications for EU added value is made more transparent.

b. As a longer term goal the criteria should be revisited alongside any re-structuring of a new programme. As part of this, we recommend reducing their number to eliminate overlap and make the criteria easier for stakeholders to understand. Given that some of the criteria can be considered sub-sets of others, this would not entail a real change in the conceptualisation of the HP’s added value but would rather alleviate confusion and help the programme continue to focus more. In concrete terms, we suggest defining the criteria as follows:

• Addressing cross-border health threats

• Improving economies of scale

• Fostering the exchange and implementation of best practices between Member States

The process for describing and considering EU added value in funding applications could also be formalised for the next programming period beyond 2020, with guidance and literature updated accordingly. Given the importance of generating EU added value, it could form a central part of a revised funding application, replacing and incorporating rather than adding to the existing administrative burden.

6. DG SANTE should integrate multi-annual planning with existing programme processes: in a short space of time since its introduction, multi-annual planning has led to a more focused and strategic approach to programming and increased the efficiency of the formal Annual Work Programme process. However, as an informal, internal exercise, it has been somewhat disconnected from core HP stakeholders, especially the Member States. This served to exacerbate the feeling expressed by some interviewees and survey respondents that priorities for the HP are set in a top-down fashion. In order to engage stakeholders more in the multi-annual planning, we recommend integrating it into the formal priority-setting process for the future. Making the draft plan publicly available and building it into structured consultations would help the programme managers increase the transparency of the next HP and increase its ability to meet Member States’ needs.

93


8.2.2. Recommendations on previously identified issues where progress is still needed


7. DG SANTE and Chafea should develop a broader strategy to increase participation from poorer Member States and underrepresented organisations: despite increasing the scope of the “exceptional utility” criterion (which provides for a higher level of co-funding for actions that include a certain proportion of members from low-GNI countries), participation in the HP among organisations from the Member States joining the EU after 2004 remains consistently low. Moreover, the HP continues to have trouble attracting strong applications from organisations who have not already benefited from HP funding. The evaluation has showed that these problems stem in part from the fact that potential applicants face numerous barriers to participation aside from the need to secure sufficient co-funding. These include concerns about the administrative burden, low capacity and lacking awareness about the HP. In order to address these challenges, DG SANTE (with the support of Chafea) should consider steps to address the barriers in a more holistic way. This could include form of positive discrimination that would allow more consideration for the equitability of the HP over the technical quality of the actions. As mentioned in the previous evaluation, the tendency for low GNI countries to be less involved in certain funding mechanisms (i.e. service contracts, and to some extent joint actions) should also be considered as the Programme is shifting toward fewer projects (which might be more likely to involve more new players, including those from poorer Member States). Further efforts should also be made to reduce the administrative burden of applying for and receiving funding. Given the difficulties organisations from low-GNI countries face, the benefits of such efforts would be widespread but help them in particular.

8. DG SANTE and Chafea should invest in the resources necessary to improve the systems for monitoring programme implementation: in order to take well-informed decisions about future spending, ascertain whether the programme is on track and publicise its activities and achievements, it is imperative that real-time, accurate information about the HP’s implementation is readily available to programme managers. The previous evaluation noted difficulties with this and recommended the adoption of an electronic monitoring system as a potential solution. However, no such system has yet been put in place. Rather, key data on the allocation of spending across different objectives, thematic priorities, funding mechanisms and participating countries is still fragmented and extremely time-consuming to compile and therefore difficult to feed into day-to-day decision making or the Annual Work Programme process. DG SANTE and Chafea should invest the resources needed to put in place and manage a simple and effective system for monitoring implementation of the programme.

9. DG SANTE and Chafea must implement and use programmatic and action specific monitoring indicators: while programmatic indicators have been developed for the 3HP they are not comprehensive enough, and concerning action-specific indicators it is still unclear whether they are being implemented and how they will be used. Given that actions funded during the first years of the years of the programme will soon produce concrete results, it is important to put in place a system for reporting on, collecting and presenting data on the action specific indicators as soon as possible and revisiting the programmatic indicators to ensure that the key programme goals are covered. This could be built into the electronic reporting system for beneficiaries managed by Chafea.

94


Dissemination

10. As with previous HP, there is a need for DG SANTE and Chafea to continue to step up efforts to communicate about the HP with core stakeholders and wider audiences: while the HP has in place a formal communication strategy and dedicated Dissemination Officer, the strategy has a long horizon and is only in its first year. Given that only half of the programming period remains, it is crucial that DG SANTE and Chafea assemble the political will and resources to roll out the strategy in the near term. This would address persistent criticism about poor dissemination and thereby make it easier for the HP to inform potential beneficiaries about the possibilities of the programme (thereby improving accessibility), to publicise its substantial achievements and increase the take-up of programme results.

95

doi: 10.2875/359384

EW-02-17-440-EN

-N

mid-term evaluation of the third health programme (2014 ... · the third health programme (3hp) is...

Documents