kastori.net · web viewthe preparatory action for preserving and restoring cultural heritage in...
TRANSCRIPT
Annex VII-b
SPECIFIC TERMS OF REFERENCEEx-post Evaluation of the Pilot Schemes and Preparatory Actions for preserving and
restoring cultural heritage in conflict areas
FWC SIEA 2018 - LOT 4: Human Development and safety net - Sector 4: CultureEuropeAid/138778/DH/SER/multi
Contracting Authority:Directorate-General Neighbourhood and Enlargement Negotiations – DG NEAR, Unit D.5
1. BACKGROUND.................................................................................................................................................1.1 RELEVANT REGION................................................................................................................................................1.2 THE ACTIONS TO BE EVALUATED..............................................................................................................................1.3 STAKEHOLDERS OF THE ACTIONS............................................................................................................................
2 DESCRIPTION OF THE EVALUATION ASSIGNMENT...........................................................................................2.1 PURPOSE OF THE EVALUATION...............................................................................................................................2.2 REQUESTED SERVICES..........................................................................................................................................2.3 PHASES OF THE EVALUATION AND REQUIRED DELIVERABLES.........................................................................................2.4 MANAGEMENT AND STEERING OF THE EVALUATION.......................................................ERROR! BOOKMARK NOT DEFINED.2.5 LANGUAGE OF THE SPECIFIC CONTRACT...................................................................................................................
3 EXPERTISE REQUIRED AND ORGANISATION AND METHODOLOGY..................................................................3.1 NUMBER OF REQUESTED EXPERTS PER CATEGORY AND NUMBER OF WORKING DAYS PER EXPERT OR PER CATEGORY...............3.2 EXPERTISE REQUIRED...........................................................................................................................................3.3 PRESENCE OF MANAGEMENT TEAM FOR BRIEFING AND/OR DEBRIEFING.........................................................................3.4 SPECIFIC ORGANISATION AND METHODOLOGY (TECHNICAL OFFER)..............................................................................
4 LOCATION AND DURATION............................................................................................................................4.1 STARTING PERIOD...............................................................................................................................................4.2 FORESEEN DURATION...........................................................................................................................................4.3 PLANNING.........................................................................................................................................................4.4 LOCATION(S) OF ASSIGNMENT...............................................................................................................................
5 REPORTING....................................................................................................................................................5.1 CONTENT, TIMING AND SUBMISSION.......................................................................................................................5.2 COMMENTS.......................................................................................................................................................5.3 LANGUAGE........................................................................................................................................................5.4 NUMBER OF COPIES............................................................................................................................................5.5 FORMATTING OF REPORTS....................................................................................................................................
ANNEX I: SPECIFIC TECHNICAL EVALUATION CRITERIA............................................................................................
ANNEX II: INFORMATION THAT WILL BE PROVIDED TO THE EVALUATION TEAM.....................................................
ANNEX III: STRUCTURE OF THE FINAL REPORT AND OF THE EXECUTIVE SUMMARY.................................................
ANNEX IV: PLANNING SCHEDULE............................................................................................................................
ANNEX V: QUALITY ASSESSMENT GRID...................................................................................................................
1 BACKGROUND
1.1 Relevant regionAlbania, Croatia1, Bosnia and Herzegovina, Kosovo*, the Republic of North Macedonia, Montenegro and Serbia
1.2 The Actions to be evaluated2
General InformationBetween 2008 and 2012, the European Commission ran two Pilot Schemes and three Preparatory Actions (hereinafter also referred to as “the Actions”) for preserving and restoring cultural heritage in conflict areas in the Western Balkans. A total amount of EUR 11.75 million were allocated by the European Parliament to i) rehabilitate cultural heritage sites affected by war-related actions, involving several ethnic and religious communities; ii) foster inter-cultural dialogue, reconciliation and return; iii) raise the awareness of cultural diversity in the Western Balkans among the youth; and iv) eventually support human or local economic development. Specifically, between 2008 and 2011, the European Parliament granted EUR 8.75 million under Articles 22. 0208 and 22.0209 of the EU Budget for the preservation and restoration of cultural heritage in conflict areas in the Western Balkans. For 2012, the European Parliament allocated an additional EUR 3 million, under Article 22.0209 of the EU Budget, for the development of a preparatory action for the same purpose.
The objective was to launch a number of actions aiming to rehabilitate cultural heritage sites affected by war-related actions in conflict areas of Kosovo, Bosnia and Herzegovina, the Republic of Macedonia, Montenegro, Albania, Croatia and Serbia. The actions also aimed to raise the awareness of cultural diversity in the Western Balkans, to build confidence and support to local education, and to develop knowledge and skills of high cultural quality. The European Parliament also envisaged that experience gained under these pilot schemes and preparatory actions could help develop a wider approach for preserving and restoring cultural heritage in conflict areas in other geographical regions.
The European Commission has managed these pilot and preparatory actions; Annex VI offers more details regarding the relevant financing decisions adopted.
Different modes of implementation were used for the Pilot Schemes and Preparatory Actions, the benefits of which will be the subject of this evaluation:
• 2008 and 2009 pilot schemes: Works contracts managed by the EU Delegations in the Western Balkans;• 2010 preparatory action: Call for Proposals managed by the European Commission centrally;• 2011 preparatory action: Direct grant to the Regional Cooperation Council (RCC) to be implemented by the RCC Task Force for Culture and Society (TFCS);• 2012 preparatory action: Works contracts (and service contracts for the supervision of the works) managed by the EU Delegations in the Western Balkans.
Specific information about the Pilot Schemes and Preparatory Actions
1* This designation is without prejudice to positions on status, and is in line with UNSCR 1244 and the ICJ Opinion on the Kosovo Declaration of Independence. Until 2013.2 The term ‘Action’ is used throughout the report as a synonym of ‘project and programme’.
Through the Pilot Schemes and Preparatory Actions, different methods of implementation were used, the benefits of which will be the subject of this evaluation. The different methods and more details are described below:
The Pilot Scheme for preserving and restoring cultural heritage in conflict areas in the Western Balkans for the year 2008 (DN020467) was implemented through works contracts for the preservation of cultural and religious monuments managed by the EU Delegations in the Western Balkans. The pilot scheme was developed through two complementary projects, each of which implemented through different contracts, as described below:
The first project, "Development of a Regional Cultural Heritage Facility in Kosovo" aimed to set up a mechanism for vocational training and scholarships in the field of cultural heritage for the Western Balkans. This project focused on building human capacity for the preservation and restoration of cultural heritage and was developed in Prizren, Kosovo.
Contracts:PHARE/2009/217-113 Design & Supervision - Restoration of the 'Beledije' building in PrizrenPHARE/2009/217-124 Drafting of the TORs for a Service contract under the 'Pilot scheme for preserving & restoring cultural heritage in conflict areas in the Western Balkans'PHARE/2010/244-922 Development of a Regional Cultural Heritage Facility in KosovoPHARE/2010/247-951 Restoration of the Beledije Building in Prizren
The second project, "ARCH-Stolac: Reconstruction of Architectural Heritage in Stolac" aimed to rehabilitate the region through the restoration of identified historical sites with a view to supporting local economic development and making Stolac an attractive place to live, work and visit. This project sought to contribute to reconciliation between the different communities in Bosnia and Herzegovina and promote tolerance and cultural-diversity within the communities.
Contracts:PHARE/2011/268-975 ARCH-Stolac: Reconstruction of Architectural Heritage in StolacBosnia and HerzegovinaPHARE/2012/301-149 Restoration, conservation, reconstruction and revitalisation of selected objects in StolacPHARE/2012/302-455 Bridges to the Future - Conservation, Science, Technology & Education in Stolac
The Pilot scheme for preserving and restoring cultural heritage in conflict areas in the Western Balkans for the year 2009 ( DN021771) was implemented through works contracts for the preservation of cultural and religious monuments managed by the EU Delegations in the Western Balkans. The pilot scheme was developed through three complementary projects covering different aspects of rehabilitation of cultural heritage i.e.: cross-border component, local development of a border area, industrial heritage and re-conciliation of communities. Each of the three complementary projects under the pilot scheme was implemented through different contracts, as described below:
The first project, was “ARCH-Vukovar: Heritage as a means of development. Reconstruction of Architectural Heritage in Vukovar Town Centre - Sustainable Revival and Development of a war-torn community”. It aimed to rehabilitate the external appearance of the most representative national monument of the war-torn city of Vukovar in Croatia, the Worker’s Hall, contributing to the conservation of the urban centre, as well as to enhance intercultural and inter-ethnic dialogue within the community and the process of social and economic revival in the region.
Contract:CARDS/2011/265-589 ARCH - Vukovar: ''Heritage as a menas of development. reconstruction of Architectural Heritage in Vukovar Town Centre - Sustainable Revival and Development of a war-torn community''
The second project, "Revitalization of cultural heritage in the frame of Sustainable Development in Besac – Montenegro ", intended to restore the Ottoman fortress of Besac set in the Albano-Montenegrin frontier region of the lake Skadar. The monument had the purpose of serving as a catalyst for the socio-economic development of the border region.
Contract:CARDS/2013/310-028 Restoration works in the fortress ''Besac'' in Virpazar, Montenegro
Lastly, the third project, "Cultural Heritage Facility in Senjski Rudnik - Serbia " aimed to rehabilitate an old industrial centre and a coal mine in Senjski Rudnik, Serbia, and convert them into a regional industrial heritage centre, contributing in this way to sustainable tourism and development.
Contracts:CARDS/2012/290-927 Technical Assistance for the establishment of the Regional Heritage Center in Senjski rudnik old mineCARDS/2012/290-217 Supervision of reconstruction and restoration of Aleksandar's shaft buildingsCARDS/2012/290-916 Reconstruction and restoration of the buildings of Aleksandar's shaft
These Pilot Schemes were followed by three Preparatory Actions for the following consecutive years 2010, 2011 and 2012. The Preparatory Actions also aimed to rehabilitate cultural heritage sites affected by war-related actions, involving several ethnic and religious communities; they were also intended to raise the awareness of cultural diversity in the Western Balkans, to build confidence and support to local education and to develop restoration knowledge and skills of high cultural quality.
The Preparatory Action for preserving and restoring cultural heritage in conflict areas in the Western Balkans for 2010 ( DN022659) was implemented through a call for proposals managed by the European Commission centrally, that aimed to preserve and restore valuable cultural and religious objects damaged or destroyed by war or other political conflicts. The 8 actions selected through the call for proposals also had the objective of fostering knowledge in the affected areas aiming at broadening the skill base at regional and local level. It was also intended that those actions would help the development of local socio-economic conditions both in terms of education possibilities and revenue generating activities. For each action selected after the Call for Proposals, a grant contract with a Civil Society Organisation was signed, as described below:
Contract 2011/277-768, “Western Balkans - From historical integration to contemporary active participation” aimed to use cultural heritage as a platform to meet and create conditions for reconciliation as a prerequisite for peace and democracy with respect to human rights.
Contract 2011/277-778, “Jankovic Castle: historic site, generating sustainable development of the Ravni Kotari region” aimed at the revalorisation of the Jankovic Castle (Croatia) cultural heritage expertise for knowledge transfer purposes.
Contract 2011/277-801, “HER.CUL - Valorisation of Medieval Tombstone CULtural HERitage in the Western Balkans” had the purpose of preserving and restoring valuable cultural objects damaged or destroyed primarily by various conflicts but also due to negligence caused by changing political environment and conflicts.
Contract 2011/277-811, “Restore our past to build the future. Preparatory actions of participatory and intercultural requalification of civil war-damaged mosques and churches in the former Yugoslav Republic of Macedonia,” aimed to contribute to the restoration process of the Macedonian cultural heritage damaged by the armed conflict in a reconciliation and intercultural perspective.
Likewise, contract 2011/277-838, “MANUBALK” had the objective of preserving and restoring valuable cultural written heritage affected by war-related actions in the Western Balkans, and encouraging the access to collections of great historic and symbolic value promoting the stabilisation of the region.
Contract 2011/277-842, “Past for the future” aimed to contribute to the strategies that support long term development and to improve the quality of life of the local inhabitants by using tourism based on recourses of cultural heritage.
Contract 2011/277-929, “Balkans' memory” had the purpose of contributing to the construction of a common audio-visual memory in the Western Balkans in order to stimulate intercultural dialogue and to encourage the understanding of a shared history.
Contract 2011/277-950, “Revitalization of Albanian Historic Towns of Gjirokastra and Berat by resolving political conflicts (REVATO)”, aimed to foster the revitalization process of the two UNESCO World Heritage towns in Albania
The Preparatory action for preserving and restoring cultural heritage in conflict areas in the Western Balkans for 2011 (DN023372) was a direct grant to the Regional Cooperation Council (RCC) for “Sustaining the Rehabilitation of Cultural Heritage in the Western Balkans”, (contract 2012/295-647). The grant was implemented by the RCC Task Force on Culture and Society (TFCS). Through the grant, as a further step in the process, the Commission intended to assist the participating public administrations in launching a number of actions to contribute to strengthening regional and national capacities in the field of preservation and restoration of cultural and religious objects damaged or destroyed by war or other political conflicts. Specific actions were to be proposed by the national administrations with the assistance of international experts and implemented by the RCC TFCS through the award of procurement contracts for urgent intervention or consolidation works; preliminary expertise investigations; project designs for tender and building permits; management plans and related training workshops.
The Preparatory Action for the year 2012 ( DN023867) was implemented through works contracts (and service contracts for the supervision of the works) resulting from calls for tenders managed by the EU Delegations in the Western Balkans. The main purpose was the identification, selection and rehabilitation of indicatively 3 cultural heritage sites. The Preparatory Action intended to rehabilitate cultural heritage sites affected by war-related actions, to contribute to a sustainable development of sites concerning valuable cultural objects, to foster inter-cultural dialogue and reconciliation, to raise the awareness on cultural diversity in the Western Balkans among the population, and eventually to support economic development. The preparatory action was developed through three complementary projects, each of which implemented through different contracts, as described below:
The first project, “Rehabilitation of the monastery of St. Mary for museum use and landscaping of the archaeological site of Apollonia, Fier Albania” encompassed works and their supervision services for the rehabilitation and reconstruction of the Monastery of St. Mary in Apollonia, Fier, including the respective Defects Liability Period.
Contracts:CARDS/2015/365-175 Supervision of Works for Rehabilitation and Restoration of the Monastery of St. Mary in ApolloniaCARDS/2015/369-281 Rehabilitation of the Monastery of St. Mary and landscaping in Apollonia
The second project, “Reconstruction of the Kastel Fortress phase II - Unit 2- Cultural Centre with Auxiliary services” aimed to constructive stabilization of the bastions, walls and underground arsenal in the north and west part of the fortress complex, as well as to the reconstruction of the towers 3 and 4, and the reconstruction of the foreign lines of general infrastructure and auxiliary facilities needed for the functioning of the summer stage (sanitary and auxiliary facilities).
Contracts:CARDS/2014/349-104 Supervision of Reconstruction of Kastel Fortress Spatial Unit 2- Cultural Centre with Auxiliary Facilities
CARDS/2015/357-106 The reconstruction of the Kastel Fortress phase II - unit 2 ''Cultural Centre with auxiliary services''
The third project, “Rehabilitation of the Franciscan Monastery in Bac, Serbia”, had the objective of rehabilitating the outstanding monument in Bac, and the adaptation of the grant floor for educational and museum purposes.
Contracts:CARDS/2016/373-414 “Supervision of works for rehabilitation of the Franciscan Monastery in Bac”CARDS/2016/373-639 “Rehabilitation of the Franciscan Monastery in Bac”
Wider policy background regarding EU support to Cultural Heritage in the Western Balkans
The support described above should be seen principally in the context of the so-called "Ljubljana Process", an initiative launched during a Ministerial Conference, under Slovenia's EU Presidency, in May 2008, which targeted the preservation and restoration of cultural heritage in the Western Balkans. The two major objectives were: i) that historic places should be conserved to the highest standards and ii) that they are promoted in such a way as to maximise their contribution to the local economy without diminishing their cultural significance.
This type of commitment to cultural heritage preservation and restoration had started already before, with a joint action entitled “Integrated Rehabilitation Project Plan/Survey of the Architectural and Archaeological Heritage” (IRPP/SAAH) implemented by the Council of Europe and the European Commission between 2003 and 2010. The purpose of this action was to implement a heritage management tool based on the identification of priority interventions and the drawing up of integrated rehabilitation projects targeting the social and economic potential of monuments and sites. Its success was acknowledged at the European Ministerial Conference on “Rehabilitating our Common Heritage” held by the Ministers of Culture of the Western Balkans in November 2009 in Ljubljana, Slovenia.
The "Ljubljana Process II", which started on July 2011, continues to implement the IRPP/SAAH methodology in order to ensure its institutionalisation in national policies and strategies. The objective is to maintain the trans-national dimension of the activities and to allow permanent management structures to be set up and managed by the Beneficiaries themselves. It encourages the development of rehabilitation projects, while the implementation and supervision of the work on monuments and sites will remain the responsibility of the relevant institutions in accordance with the legal and financial procedures in each country. Furthermore, the Commission Communication "Enlargement Strategy and Main Challenges 2010-2011" states: "The Commission has continued to support the rehabilitation of cultural heritage in the context of the Ljubljana Process. The RCC has established a Task Force on Culture and Society, supported by a permanent secretariat benefiting from EU pre-accession financial assistance. Its task is to manage the Ljubljana Process, in cooperation with the Council of Europe and the Commission" (CRIS 2016/380-471 (Actions: A.1.3.2 and A 3.3.4)).
A recent broader but yet related initiative is the Joint Communication "Towards and EU strategy for international cultural relations", adopted on 8 June 2016 by the European External Action Service and the Commission. This proposed an EU Strategy for International Cultural Relations aiming at encouraging cultural cooperation between the EU and its partner countries and promoting a global order based on peace, rule of law, freedom of expression, mutual understanding and respect for fundamental rights. The Strategy was not targeting the Western Balkan region specifically, but highlighted the importance of working closely with the WB and enlargement countries to help protect their cultural heritage, promote the development of their cultural and creative industries, and foster their participation in existing EU cultural programmes
The celebration of the first European Year of Cultural Heritage 2018 (EYCH) aims to raise the awareness of the social and economic importance of cultural heritage by encouraging the sharing and appreciation of cultural heritage as a joint resource: enhancing awareness of common history and values and promoting a common sense of ownership.
Most recently, the Commission Communication "A credible enlargement perspective for and enhanced EU engagement with the Western Balkans" from 6 February 2018, also draws attention to the initiatives for preserving and restoring cultural heritage in the region. It states that "The people of the EU and the region have a common heritage and history and a future defined by shared opportunities and challenges" and it highlights that the Commission will continue to foster the cultural links with the region, what will include work to protect the Western Balkans cultural heritage.
Other relevant support channelled to the region in the area of cultural heritage, reconciliation and cultural dialogue
In parallel to the Pilot Schemes and Preparatory Actions, other initiatives were taken by the European Commission in the field of reconciliation and cultural dialogue (i.e. beyond the field of cultural heritage). These are listed below. To ensure that this evaluation provides a comprehensive analysis, including possible synergies and overlaps, it is essential to include a wider range of projects in the universe of this evaluation. The projects listed below should therefore as a minimum be covered by the desk research; some key projects should be visited too.
Contract Number Total Eligible Cost (EUR) Title2012/306-585 944,118.00 Partnership Reconciliation through Early Childhood Education and
Development in Europe (PRECEDE)2012/307-518 126,268.16 FOr CUlture in Serbia – FOCUS
2013/333-244 99,760.14 Portraits and memories of the Jewish Community in Serbia before the Holocaust
2013/333-296 74,385.37 We Must Learn to Live in a World of Differences
2013/333-330 91,618.32 Strength of dialogue and respect for diversity among youth from different ethnic communities
2013 /333-331 110,366.21 Interconnecting Stakeholders in Learning and Drama in Serbia2013/333-335 63,913.48 Enhancing cultural and personal communication between two communities2013/333-940 269,432.00 The Forum for Cultural Development
2014 /351-609 899,932.06 Partnership for Reconciliation through Early Childhood Education and Development in Europe (PRECEDE)
2014 /351-845 199,300.00 Regional Youth Exchange Association2014 /353-444 143,732.66 R&J - Theatre play Romeo and Juliet
2014 /353-516 133,072.72 Encyclopaedia of the Alive – An Artistic Intervention in Serbia and Kosovo Realities
2014 /353-535 67,909.42 Lajk it – promoting intercultural dialogue between communities in Serbia and Kosovo
2014 /353-556 141,950.07 Serbia and Kosovo: Intercultural dialogue 2014 /353-589 128,416.93 Promoting youth employment through Social Partnership and Cooperation2014 /353-704 103,953.75 Kosovo Serbia: LIVE2014 /344-401 1,000,000.00 Culture for all 3 2015 /364-189 1,072,747.00 Strengthening the RECOM Process (Phase II)2015 /369-774 281,064.00 Decade of Belonging - Celebrating Diversity 2015 /369-849 277,390.64 Diversity is trendy - promoting local multiculturalism!2015 /370-393 1,005,249.04 Divided Past - Joint Future
2015/370-755 323,460.47 Strengthening cooperation and inter-community dialogue through the Ohrid Framework Agreement (OFA)
2015/370-761 329,584.78 Joint forces for common interests!2015 /371-378 187,400.00 Regional Youth Exchange Association2015 /372-086 255,309.00 Support of the Process of Integrated Education in Schools
2016 /373-876 58,061.26 Work Programme - Reconciliation between communities2016 /383-183 376,474.15 Changing Minds: Trust through Innovation2016 /383-188 415,989.00 prEUgovor Policy Watch: Monitoring Reforms in Chapters 23 and 242016 /383-189 333,356.80 Pertej/Beyond/Preko 20 years
WBIF project not encoded in CRIS TBC Mostar: Integra Cultural and Sports Centre WBIF project not encoded in CRIS TBC Alps Area: Communal Infrastructure Works WBIF project not encoded in CRIS TBC Municipal and Regional Infrastructure Programme
Indirect management action not encoded in CRIS TBC Conservation and revitalization of the cultural - tourist site Sv. Gjorgjija – Staro Nagorichane (near to the city of Kumanovo)
Indirect management action not encoded in CRIS TBC Reconstruction of the Shengjul Hamam in Skopje
1.3 Stakeholders of the ActionsThe beneficiaries of this thematic evaluation will be the European Commission and the beneficiaries of the assistance covered by it. The stakeholders for this evaluation include:
IPAII beneficiary stakeholders include (non-exhaustive list):- Public authorities of the Western Balkans, including Ministries of Culture or
equivalent; Ministries of Education or equivalent; Ministries of Justice; Ministries of Interior; Ministries of European Integration or equivalent; where appropriate, equivalent local authorities; where applicable, public authorities responsible for transitional justice processes, such as BiH Ministry of Human Rights and Refugees,
- National IPA Coordinators offices- Regional Cooperation Council (RCC)- Other relevant bodies directly or indirectly involved in restoration and rehabilitation of
cultural heritage, including national and, where relevant, local museums.
EU stakeholders (non-exhaustive list):- European Commission DG NEAR (Directorate D Western Balkans; Unit A.1 Strategy,
Policy; EEA/EFTA, Unit A.3 Thematic support, Economic governance & IFIs, Public Administration Reform, and Unit A.4: MFF, Programming & Evaluation);
- European Commission DG EAC- EU Delegations/Office in the Western Balkans;- EEAS, EURCA (Europe and Central Asia): WEST - Western Europe, Western Balkans and
Turkey
Other stakeholders (non-exhaustive list): - Civil Society Organisations, networks, platforms, etc., including the International
Council of Museums, RECOM;- Cultural Institutions; - International Organizations, including UNESCO.
2DESCRIPTION OF THE EVALUATION ASSIGNMENTType of evaluation Ex-post Coverage Two Pilot Schemes and three Preparatory Actions for
preserving and restoring cultural heritage in conflict areas in the Western Balkans + a selection of other relevant projects based on a desk review of the projects listed above
Geographic scope Albania, Bosnia and Herzegovina, Croatia, the Republic of North Macedonia, Kosovo, Montenegro and Serbia
Period to be evaluated 2008-2012;
Page 13 of 48
2.1 Purpose of the evaluationSystematic and timely evaluation of its programmes and activities is an established priority 0 of the European Commission0. The focus of evaluations is on the assessment of achievements, the quality and the results0 of Actions in the context of an evolving cooperation policy with an increasing emphasis on result-oriented approaches0. From this perspective, evaluations should look for evidence of why, whether or how these results are linked to the EU intervention and seek to identify the factors driving or hindering progress. Evaluations should provide an understanding of the cause and effects links between inputs and activities, and outputs, outcomes and impacts. Evaluations should serve accountability, decision-making, learning and management purposes.
The main objectives of this evaluation are to provide the relevant services of the European Union and the interested stakeholders with:
an overall independent assessment of the past performance of the two Pilot Schemes and three Preparatory Actions as well as of some of the other relevant support channelled to the region by the European Commission, paying particular attention to their results measured against their objectives; that were:
i. rehabilitation of cultural heritage sites affected by war-related actions, involving several ethnic and religious communities;
ii. foster inter-cultural dialogue, reconciliation and return; iii. raise the awareness of cultural diversity in the Western Balkans among the
youth; iv. support human or local economic development.
key lessons and recommendations in order to improve future interventions in the areas targeted by the Actions.
The main users of this evaluation will be:
- relevant services of the European Union, i.e. DG NEAR Headquarters and EU Delegations in the Western Balkans;
- interested stakeholders as presented in section 1.3
Findings and recommendations of this evaluation should assist DG NEAR in assessing how to follow up on the 5 pilot and preparatory actions in its planning, programming and implementation of further EU pre-accession assistance for potential candidates and candidate countries.
0 COM(2013) 686 final “Strengthening the foundations of Smart Regulation – improving evaluation” - http://ec.europa.eu/smart-regulation/docs/com_2013_686_en.pdf; EU Financial regulation (art 27); Regulation (EC) No 1905/200; Regulation (EC) No 1889/2006; Regulation (EC) No 1638/2006; Regulation (EC) No 1717/2006; Council Regulation (EC) No 215/20080 SEC (2007)213 "Responding to Strategic Needs: Reinforcing the use of evaluation", http://ec.europa.eu/smart-regulation/evaluation/docs/eval_comm_sec_2007_213_en.pdf ; SWD (2015)111 “Better Regulation Guidelines”, http://ec.europa.eu/smart-
regulation/guidelines/docs/swd_br_guidelines_en.pdf 0 Reference is made to the entire results chain, covering outputs, outcomes and impacts. Cfr. Regulation (EU) No 236/2014 “Laying down common rules and procedures for the implementation of the Union's instruments for financing external action” - https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/pdf/financial_assistance/ipa/2014/236-2014_cir.pdf.0 COM (2011) 637 final "Increasing the impact of EU Development Policy: an Agenda for Change" - http://www.europarl.europa.eu/meetdocs/2009_2014/documents/acp/dv/communication_/communication_en.pdf
Page 14 of 48
Based on relevant findings, conclusions and lessons learned, the evaluators will provide operational recommendations aiming to improve the programming and performance of future assistance in this field, as well as corrective measures, where applicable, to improve the implementation and monitoring of ongoing actionsIn particular, findings, conclusions and lessons learned identified by the evaluators will include recommendations and concrete proposals on definition of objectives and related SMART indicators to facilitate the measurement of performance and design of future support in the field of cultural heritage.
2.2 Requested services
2.2.1 Scope of the evaluationThe evaluation will assess the Actions using the five standard DAC evaluation criteria, namely: relevance, effectiveness, efficiency, sustainability and impact. In addition, the evaluation will include two EU specific evaluation criteria:
- the EU added value to the preservation and restoration of cultural heritage in conflict areas in the Western Balkans (the extent to which the Actions add benefits to what would have resulted from Member States' interventions only);
- the coherence of the Actions with the EU policies and strategies in the Western Balkans, including the EU accession strategy and the Multi-country Indicative Strategy Paper 2014-20200, and with other EU policies and Member State Actions.
To that goal, in the different phase of the evaluation the evaluators will:
Inception phase:a) Attend the kick-off and the briefing meetings (some experts’ presence might be
required only in remote) b) Collect relevant documents and data and define the analytical methods. Review
the five actions as well as the projects listed above in the section “Other relevant support channelled to the region in the area of reconciliation and cultural dialogue”
c) Conduct a background analysis, define the consultation strategy and conduct inception interviews
d) Reconstruct the Intervention Logice) Propose a complete and finalised set of Evaluation Questions with indication of
specific Judgement Criteria and Indicators, as well as of the relevant data collection sources and tools
f) Develop a sound methodology for the required task and sector, to answer to EQs, including the identification of data and information to be collected, and of the methods to analyse the data
g) Draft the inception report and prepare the slide presentation for the Reference Group
Desk phase:
0 C(2014) 4293, 30.06.2014
Page 15 of 48
a) Conduct interviews, conduct an in-depth analysis of documents and of the different approaches that have been taken in the actions. Assess the effectiveness, efficiency and synergies between approaches and propose possible improvements for the future.
b) Provide preliminary answer to the EQs and identify hypotheses to be tested in the field phase, as well as of information gaps
c) Design the methodology of the Field Phase d) Draft the Desk phase report and prepare the slide presentation for the Reference
Groupe) Attend the briefing meeting (some experts’ presence might be required only in
remote)
Field phase:a) Organise and hold meetings with relevant stakeholders at IPA beneficiary level (EU
Delegations/Office, the NIPAC offices, other relevant central and local authorities, selected final beneficiaries, etc.)
b) Gather primary evidence with the use of the most appropriate techniques, collect data and analyse it
c) Prepare a slide presentation to present the field phase findings (and include them in the evaluation matrix)
d) Attend the briefing meeting (some experts’ presence might be required only in remote)
Synthesis phase:a) Carry out a final analysis of findings (with focus on the Evaluation Questions)b) Formulate the overall assessment, conclusions and recommendations on the actions
evaluatedc) Draft the Final Report and the Executive Summary, prepare the slide presentation for
the Reference group d) Organise the final presentation meeting (some experts’ presence might be required
only in remote), gather the Reference group commentse) Finalise the Report and the Executive Summary.
Additional information on each phase of the evaluation and the related deliverables is provided in sections 2.3 and 5.
Throughout all the assignment, the contractor will ensure:- A participatory approach and continuous consultations with the main beneficiaries
and stakeholders listed in section 1.3.;- The consistency and coordination of this evaluation with other evaluations as well as
with ROM exercises planned or on-going in the same period, and in the same field and geographical area under DG NEAR responsibility.
Page 16 of 48
The evaluation team shall furthermore consider whether the following cross-cutting issues (promotion of human rights, gender equality, democracy, good governance, children's rights and indigenous peoples, environmental sustainability as relevant) were taken into account in the identification/formulation documents and the extent to which they have been reflected in the implementation of the Actions and their monitoring.
2.2.2 Indicative Evaluation QuestionsThe Evaluation Questions as formulated below are indicative. Based on them and following initial consultations and documental analysis, the evaluation team will propose in their Inception Report a complete and finalised set of Evaluation Questions with indication of specific Judgement Criteria and Indicators, as well as the relevant data collection sources and tools. The final Evaluation Questions will be preferably structured following the 4 main objectives of the Actions being evaluated, namely:
I. rehabilitation of cultural heritage sites affected by war-related actions, involving several ethnic and religious communities;
II. foster inter-cultural dialogue, reconciliation and return; III. raise the awareness of cultural diversity in the Western Balkans among the youth; IV. support human or local economic development.
An indicative list of Evaluation Questions is presented hereafter. The contractor is expected to critically reflect on them during the Inception phase, discuss them with the Project Manager0, and propose a final version for approval in their Inception Report. Once agreed with the approval of the Inception Report, the Evaluation Questions will become contractually binding.
RELEVANCE 1. To what extent are the actions' objectives relevant to the pre-accession
objectives and current EU priorities in the Western Balkans?2. To what extent are the objectives at different levels clear, measurable and
realistic? 3. To what extent do programming and monitoring mechanisms include SMART
(Specific, Measurable, Available, Relevant and Time-bound) indicators to measure progress towards achievement of objectives?
4. Which are the main gaps/weaknesses of the current programming framework?5. Should the European Commission maintain the same approach for future
programmes? How can the impact of future programmes be maximized?6. To what extent has the allocation and use of the financial assistance in the
Western Balkans been consistent with the objective of the EP and the European Commission of preserving and restoring cultural heritage in conflict areas?
7. How well did the financial assistance respond to the specific needs of each IPA beneficiary in the targeted sector?
EFFECTIVENESS and EFFICIENCY
0 The Project Manager is the staff of the Contracting Authority managing the evaluation contract.
Page 17 of 48
8. Did the assistance reach the four objectives set out (i.e. rehabilitation of cultural heritage; fostering inter-cultural dialogue and reconciliation; awareness-raising on cultural diversity among the youth; support to human or local economic development) in an effective and efficient way?
9. Was the mix of implementation methods effective and efficient, or are changes needed for the future? If so, which ones?
IMPACT and SUSTAINABILITY
10. What have been the impacts of the pilot schemes and preparatory actions and other key projects on each objective of the actions (i.e. rehabilitation of cultural heritage; fostering inter-cultural dialogue and reconciliation; awareness-raising on cultural diversity among the youth; support to human or local economic development)?
11. To what extent are these impacts sustainable and what further improvements are needed? Were these actions followed by an increased commitment of national and local authorities in terms of policy creation, budget allocations, design of culture and reconciliation programmes, presence of cultural heritage in education curricula? Specifically regarding objective 4 (human and local economic development), did the assistance result in governments'/municipalities' additional investments (roads, sports facilities, hotels, etc)?
12. Are there any elements, which have hampered/could hamper the impact and/or sustainability of this assistance?
ADDED VALUE13. What is the added value of EU interventions, compared to what could be
achieved without EU assistance?14. Which areas, within the field of cultural heritage (for example rehabilitation of
cultural heritage; fostering inter-cultural dialogue and reconciliation; awareness-raising on cultural diversity among the youth; support to human or local economic development), do not require the involvement of further EU support because they are adequately covered by other donors?
COHERENCE, COORDINATION AND CONSISTENCY15. To what extent and how were the pilot schemes and preparatory actions and
other programmes coherent/complementary/overlapping? 16. To what extent and how did the programmes take into consideration and
interconnect with existing national authorities' policies and strategies in support to the preservation and restoration of cultural heritage?
17. To what extent and how did the programmes complement/overlap with interventions supported by other donors?
LESSONS LEARNT AND RECOMMENDATIONS REGARDING FUTURE PROGRAMMING AND SUPPORT
18. What arguments speak in favour of a continued EU support to actions in the area of cultural heritage?
Page 18 of 48
19. How can future programming/support be adjusted to efficiently and effectively reach the objectives set in the relevant strategic documents for the Western Balkans?
20. How can future programming/support be enhanced to improve the impact and sustainability of financial assistance in this specific field?
21. How can programming/support be improved so to make a stronger link between project activities, the needs and priorities of the cultural heritage sector, and the enlargement objectives?
22. Are there any potential actions, which would improve the efficiency and effectiveness of ongoing or future assistance, including actions on the administrative and organisational setting?
23. Are there any actions, which would improve prospects for impact and sustainability of ongoing or future assistance?
2.3 Phases of the evaluation and required deliverablesThe evaluation process will be carried out in four phases: an Inception Phase, a Desk Phase, a Field Phase, and a Synthesis Phase. Deliverables in the form of reports and slide presentations, should be submitted at the end of the corresponding phases as specified in the synoptic table below.
The submission of deliverables by the selected contractor will be performed through their uploading in the EVAL Module, an evaluation process management tool of the European Commission; the selected consultant will have access to online guidance in order to operate with the module.
2.3.1 Synoptic tableThe following table presents an overview of the key activities to be conducted during each phase (not necessarily in chronological order) and lists the deliverables to be produced by the team, including the key meetings with the Contracting Authority and the Reference Group. The main content of each deliverable is described in Chapter 5.
Phases of the evaluation
Key activities Deliverables and meetings
Inception Phase
Initial document/data collection and definition of methods of analysis
Background analysis and definition of the consultation strategy
Initial interviews Reconstruction of Intervention
Logic, incl. objectives, specific features and target beneficiaries
Finalisation of the EQs
Kick-off meeting with the Reference Group (Team Leader face-to-face + 2 experts connected in remote via WebEx or other suitable system)
Inception report incl.: Final intended / planned Intervention
Logic Evaluation Questions (EQs), with
judgment criteria & indicators Data analysis and collection
methods, incl. case studies proposal EU external action spending
interventions inventory
Page 19 of 48
Phases of the evaluation
Key activities Deliverables and meetings
Methodological design to answer to the EQs, incl. data collection & definition of analysis methods
Report writing (& quality control)
Work plan
Consultation strategy0
Slide presentation and meeting with Reference Group to present the Inception Report (all experts face-to-face)
0 It is expected that the evaluation team presents its strategy for stakeholders' consultation during the evaluation exercise.
Page 20 of 48
Phases of the evaluation
Key activities Deliverables and meetings
Desk Phase
In-depth document analysis (focused on the Evaluation Questions)
Interviews Preliminary answer to the EQs Identification of information
gaps and of hypotheses to be tested in the field phase
Methodological design of the Field Phase
Report writing (& quality control)
Desk report incl.: Background and key methodological
elements Preliminary answers to the
evaluation questions Field visit methodology Remaining work for the synthesis
phase Update work plan, if needed Main annexes:
Evaluation matrix with information gathered and analysed by indicator
Case study notes0
Slide presentation Debriefing with the Reference
Group (Team Leader face-to-face + 2 experts connected in remote via WebEx or other suitable system)
Field Phase
Meetings at IPA beneficiary level with the EU Delegations/Office, the NIPAC offices and selected final beneficiaries
Gathering of primary evidence with the use of the most appropriate techniques
Data collection and analysis Writing of field phase findings
(inclusion in the evaluation matrix)
Slide presentation Debriefing with the Reference
Group (all experts face-to-face)
Synthesis phase
Final analysis of findings (with focus on the Evaluation Questions)
Formulation of the overall assessment, conclusions and recommendations
Organisation of the final presentation meeting
Synthesis report writing (& quality control)
Draft Final Report incl.: Synthesis of methodological steps
undertaken during the evaluation exercise, including limitations, if any
Background analysis Findings by evaluation question Overall assessment, lessons learnt,
conclusions and recommendations Main annexes: incl. Evaluation matrix
with information gathered and analysis by indicator
Slide presentation and Draft Executive Summary
0 The case studies notes should not exceed 10 pages, but if required this number can be reasonably increased following consultation with the EC project manager. The EC project manager will provide the template.
Page 21 of 48
Phases of the evaluation
Key activities Deliverables and meetings
Debriefing with the Reference Group ((all experts face-to-face))
Final Report Executive Summary
Page 22 of 48
2.3.2 Inception PhaseThis phase aims at structuring the evaluation and clarifying its key issues.
The phase will start with a kick-off session in Brussels between the Reference Group and the evaluators. Half-day presence of the Team Leader is required, the two additional experts will be connected via WebEx or other suitable system. The meeting has the purpose to arrive at a clear and shared understanding of the scope of the evaluation, its limitations and feasibility.
In the Inception phase, the relevant documents will be reviewed (see annex II).
Further to a first desk review of the political, institutional and/or technical/cooperation framework of the EU support to the two Pilot Schemes and three Preparatory Actions for preserving and restoring cultural heritage in conflict areas in the Western Balkans, the evaluation team, in consultation with the Project Manager, will reconstruct the Intervention Logic of the Actions to be evaluated.
Based on the reconstructed Intervention Logic, the evaluators will finalise the evaluation methodology, the Evaluation Questions, the definition of judgement criteria and indicators, the selection of data collection tools and sources, the consultation strategy, and the planning of the following phases. They will also summarise their approach in an Evaluation Design Matrix, which will be included in the Inception Report.
The limitations faced, or to be faced, during the evaluation exercise will be discussed and mitigation measures defined. Finally, the work plan for the overall evaluation process will be presented and agreed in this phase; this work plan shall be in line with that proposed in the present terms of reference. Any modifications shall be justified and agreed with the Project Manager. On the basis of the information collected, the evaluation team should prepare an Inception Report; its content is described in Chapter 5.
The evaluation team will then present the Inception Report to the Reference Group. The presence of the full team of experts is required.
2.3.3 Desk Phase This phase aims at conducting most of the documental analysis needed for carrying out the evaluation; the analysis should include references to the existing literature relevant to the actions, especially evaluations, academic work and the latest research studies carried out by civil society, Governments, other donors (especially EU Member States) and/or the private sector. This is to ensure a more robust approach to identifying information gaps and to ensure complementarity with evaluations that have already been done. Experts will contribute and add value to the documental analysis through their general academic knowledge of cultural heritage related issues.
The analysis of the relevant documents shall be systematic and reflect the methodology developed and approved during the Inception Phase.
Page 23 of 48
Selected interviews with the programme management, the relevant EU services (DG NEAR Headquarters, EEAS, European Parliament), and key partners can be conducted during this phase as to support the analysis of secondary sources. Considering the various locations of the mentioned stakeholders, these interviews should mostly take place via remote conference. More in-depth interviews could take place during the field phase.
The activities to be conducted during this phase should allow for the provision of preliminary responses to each evaluation question, stating the information already gathered and its limitations. They should also identify the issues still to be covered and the preliminary hypotheses to be tested.
Page 24 of 48
During this phase the evaluation team shall furthermore confirm the evaluation tools and methodology already developed during the Inception Phase. In addition to the tools to be used during the Field Phase, the evaluation team will describe the preparatory steps already taken and those to be taken for the next phase organisation, including the list of people to be interviewed, dates and itinerary of visits, and attribution of tasks within the team.
At the end of the desk phase, a Desk Report will be prepared; its content is described in Chapter 5. A presentation by the evaluation team to the Reference Group will take place during a de-briefing meeting (TL presence required, the additional two experts will be connected via WebEx or other suitable system).
2.3.4 Field PhaseThe Field Phase starts after approval of the Desk Report by the Reference Group. The Field Phase aims at validating/changing the preliminary answers formulated during the Desk phase and bringing further information through primary research.If any significant deviation from the agreed work plan or schedule is perceived as creating a risk for the quality of the evaluation, these elements are to be immediately discussed with the Project Manager.
In the first days of the field phase, the evaluation team shall hold a briefing meeting with the programme management, EU Delegations/Office, local authorities and other relevant stakeholders.
During the entire field phase, the evaluation team shall ensure adequate contact and consultation with, and involvement of, the different stakeholders, including the relevant government and local authorities. Throughout the mission, the evaluation team shall use the most reliable and appropriate sources of information, respect the rights of individuals to provide information in confidence, and be sensitive to the beliefs and customs of local social and cultural environments.
At the end of the field phase, the evaluation team shall summarise its work, analyse the reliability and coverage of data collection, and present preliminary findings in a meeting with the Reference Group (presence of all experts required). At the end of the Field Phase a Slide presentation will be prepared; its content is described in Chapter 5.
2.3.5 Synthesis PhaseThis phase is devoted to the preparation of the Final Report and entails the analysis of the data collected during the desk and field phases to finalise the answers to the Evaluation Questions and prepare the overall assessment, conclusions and recommendations of the evaluation.The evaluation team will present in a single Report plus Annexes their findings, conclusions and recommendations in accordance with the agreed structure (see Annex III); a separate Executive Summary will be produced as well.
The evaluation team will make sure that: Their assessments are objective and balanced, statements are accurate and evidence-
based, and recommendations realistic.
Page 25 of 48
When drafting the report, they will acknowledge clearly where changes in the desired direction are known to be already taking place.
The evaluation team will deliver, and then present, in Brussels the Draft Final Report and the Draft Executive Summary to the Reference Group to discuss the draft findings, conclusions and recommendations. The presence of all experts is required.
The Project Manager consolidates the comments expressed by the Reference Group members and sends them to the evaluation team for revision, together with a first version of the Quality Assessment Grid assessing the quality of the Draft Final Report. The content of the Quality Assessment Grid will be discussed with the evaluation team to verify if further improvements are required.
The evaluation team will then finalise the Final Report and the Executive Summary by addressing the relevant comments. While potential quality issues, factual errors or methodological problems should be corrected, comments linked to diverging judgements may be either accepted or rejected. In the latter instance, the evaluation team should explain the reasons in writing.2.4 Specific Contract Organisation and Methodology (Technical offer)The invited Framework Contractors will submit their specific Contract Organisation and Methodology by using the standard SIEA template B-VII-d-i and its annexes 1 and 2 (B-VII-d-ii). The evaluation methodology proposed to undertake the assignment will be described in the Chapter 3 (Strategy and timetable of work) of the template B-VII-d-i. Contractors will describe how their proposed methodology will address the cross-cutting issues mentioned in these Terms of Reference and notably gender equality and the empowerment of women. This will include (if applicable) the communication action messages, materials and management structures.
By derogation of what is specified in the standard SIEA template B-VII-d-i, the maximum length of the specific Contract Organisation and Methodology is 7 pages, written in Times New Roman 12 or Arial size 11, single interline, excluding the framework contractor’s own annexes (maximum length of such annexes: 3 pages), additional to the Annexes foreseen as part of the present Specific ToRs. The timetable is not accounted and may be presented on an A3 page]
Page 26 of 48
2.5 Management and Steering of the evaluation
2.5.1 At the EU levelThe evaluation is managed by the Project Manager responsible for the evaluation of the two pilot schemes and three preparatory actions within Unit D.5 of DG NEAR.. The progress of the evaluation will be followed closely with the assistance of a Reference Group consisting of members of relevant EU Services (DG NEAR headquarters, EUDs/EUO, other relevant DGs of the European Commission, EEAS, the European Parliament).
The Reference Group members' main functions are: To facilitate contacts between the evaluation team and the EU services and external
stakeholders; To ensure that the evaluation team has access to and has consulted all relevant
information sources and documents related to the Actions; To define and validate the Evaluation Questions; To discuss and comment on reports delivered by the evaluation team. Comments by
individual group members are compiled into a single document by the Project Manager and subsequently transmitted to the evaluation team;
To assist in feedback on the findings, conclusions, lessons and recommendations from the evaluation;
To support the development of a proper follow-up action plan after completion of the evaluation.
2.5.2 At the Contractor level Further to the Requirements set in the art. 6 of the Global Terms of Reference and in
the Global Organisation and Methodology, respectively annexes II and III of the Framework contract SIEA 2018, the contractor is responsible for the quality of: the process; the evaluation design; the inputs and the outputs of the evaluation. In particular, it will:Support the Team Leader in its role, mainly from a team management perspective. In this regard, the contractor should make sure that, for each evaluation phase, specific tasks and outputs for each team member are clearly defined and understood. Provide backstopping and quality control of the evaluation team’s work throughout the assignment.
Ensure that the evaluators are adequately resourced to perform all required tasks within the time framework of the contract.
2.6 Language of the specific contract
Page 27 of 48
The language of the specific contract is to be English.
3 EXPERTISE REQUIRED AND ORGANISATION AND METHODOLOGYThe contractor shall provide an appropriate team of experts to complete the requested evaluation.
3.1 Number of requested experts per category and number of working days Three experts Category I are required for this assignment. All experts are expected to be available for the whole duration of the assignment. The precise time inputs of the experts will be proposed by the Framework Contractor in the specific offer. However, 170 working days, of which 40 will have to be spent in the field, are to be considered an absolute minimum input in terms of working days for the team of experts as a whole.
3.2 Expertise requiredA team of three experts Category I is required. The experts will work in cooperation with each other, with one appointed as Team Leader (to be identified in the Organisation and Methodology and in the Financial Offer).The evaluation team will have to be able to satisfy the highest quality standards. In this regard, the expertise required from the team of experts for this assignment is as follows.
Team of Experts (Category I)
The minimum requirements of the team made of three experts Category I for this contract are as follows:
Minimum requirements of the teamQualifications
All experts shall have at least a University Degree – or, in its absence, an equivalent professional experience of at least 5 years – in cultural heritage and the related fields relevant to the tasks to be performed for this evaluation.
General professional experience All experts shall have at least 12 years' experience in the sector(s) related to the
evaluation. This minimum number of years of experience is to be added to 5 additional years of experience required in case of absence of a University Degree (see point above)
Specific professional experience Very good working knowledge of evaluation methods and techniques is required for
each team member, in particular on analytical methods which can evaluate change and contribution. This includes quantitative and qualitative data collection and analysis.
Page 28 of 48
The team of experts as a whole should have professional experience with the development of cultural heritage programmes, in particular the team as a whole must have knowledge and experience with the following aspects of cultural heritage programmes:- rehabilitation of cultural heritage sites affected by war-related actions, involving several ethnic and religious communities; - fostering inter-cultural dialogue, reconciliation and return; - raising awareness of cultural diversity in the Western Balkans among the youth; - support to human or local economic development.
The team of experts as a whole should have relevant expertise and/or professional experience in the Western Balkans region.
Professional experience in relation to the development of cultural heritage projects is required.
Relevant expertise and/or professional experience in Western Balkans region would be an asset.
Soft skills All experts will have excellent writing and editing skills.
Language skills All experts will have an excellent command of English – both spoken and written.
Additional requirements of the team Professional experience with the development of programmes in the Western
Balkans region would be an advantage. The knowledge of one or several of the following languages would be an asset:
Albanian, Serbo-Croat-Bosnian, Montenegrin and Macedonian.
Specific minimum requirements for the Team LeaderThe Team Leader is expected to possess a demonstrable senior evaluation expertise coherent with the requirements of this assignment. He/She should have excellent communication, team co-ordination, presentation and proven report writing and editing skills in English.
Languages levels are defined for understanding, speaking and writing skills by the Common European Framework of Reference for Languages available at https://europass.cedefop.europa.eu/en/resources/european-language-levels-cefr and shall be demonstrated by certificates or by past relevant experience.
The European Union pursues an equal opportunities policy. Gender balance in the proposed team, at all levels, is highly recommended.The offer will include the CVs of all the experts. References relevant to the assignment must be highlighted in bold. The technical proposal should include a table showing how the proposed experts meet the above requirements.
The team members must be independent from the EU external action support interventions which will be covered under this assignment. Should a conflict of interest be identified in the course of the evaluation, it will be notified without delay to the Contracting Authority.
Page 29 of 48
3.3 Presence of management team for briefing and/or debriefingThe presence of the three experts is required, either face-to-face or in remote via WebEx or other suitable system, for briefing or debriefing purposes.
3.4 Specific Organisation and Methodology (Technical offer)The tenderers are expected to provide in their technical offers the following:
An outline of the approach and methodology proposed to conduct the assignment; Comments on the terms of reference for the successful execution of the assignment as
well as regarding the indicative evaluation issues, thus demonstrating the degree of understanding of the assignment;
Comments on the timing, sequence and duration of the proposed tasks; Identification, if relevant, of risks and assumptions that may affect the execution of
the assignment. An outline of the quality control mechanisms to be put in place by the Contractor
4 LOCATION AND DURATION
4.1 Starting period Provisional start of the assignment: June 2019.
4.2 Foreseen duration The maximum expected duration of the assignment: six months (including time for finalising the final report).
Page 30 of 48
The assignment is expected to be completed in December 2019 (depending on the start date of activities). It is assumed that the consultants will work on the basis of a five-day week.
4.3 Planning0 As part of the technical offer, the framework contractor must fill-in the timetable in the Annex IV (to be finalised in the Inception Report). The ‘Indicative dates’ are not to be formulated as fixed dates but rather as days (or weeks, or months) from the beginning of the assignment (to be referenced as ‘0’).Attention is drawn to the fact that sufficient forward planning is needed in order to ensure active participation and consultation with government representatives and national stakeholders.
4.4 Location(s) of assignmentThe assignment will take place in Brussels, with field visits in the Western Balkans. The desk tasks will be performed at the experts’ office(s).
5 REPORTING
5.1 Content, timing and submissionThe reports must match quality standards. The text of the report should be illustrated, as appropriate, with maps, graphs and tables; a map of the area(s) of Action is required (to be attached as Annex). The evaluation team will submit the following reports/other outputs:
Number of Pages (excluding annexes)
Main Content Timing for submission
Inception Report + Slide presentation
25-30 pages(nr of slides necessary to present the report appropriately)
Intervention logic reconstructed Methodology for the evaluation Evaluation Questions, judgement
criteria and indicators Evaluation Matrix Data analysis and collection methods Stakeholder map Identification of a sample of projects
to be analysed in further details Consultation strategy Work plan Analysis of risks and of mitigating
measures
End of Inception Phase
Desk Report 25-30 pages Preliminary answer to each Evaluation Question, with indication of the limitations of the available information
Issues still to be covered and
End of the Desk Phase
0 including the period for notification for placement of the staff as per art 16.4 a)
Page 31 of 48
Number of Pages (excluding annexes)
Main Content Timing for submission
assumptions to be tested Field visit approach, including the
criteria to select the field visits Slide Presentation
15-20 slides Activities conducted during the field phase
Difficulties encountered during the phase and mitigation measures adopted
Key preliminary findings
End of the Field Phase
Draft Final Report + Draft Executive Summary + Slide presentation
50-60 pages+ 5 pages+ nr of slides necessary to present the report appropriately
Cf. detailed structure in Annex III End of Synthesis Phase
Final report and Executive Summary
50-60 pages+ 5 pages
Same specifications as for the Draft Final Report and Draft Executive Summary, incorporating any comments received from the concerned parties on the draft report and draft Executive Summary that have been accepted
2 weeks after having received comments to the Draft Final Report and Draft Executive Summary
5.2 Use of the EVAL module by the evaluatorsIt is strongly recommended that the submission of deliverables by the selected contractor be performed through their uploading in the EVAL Module, an evaluation process management tool and repository of the European Commission. The selected contractor will receive access to online and offline guidance in order to operate with the module during the related Specific contract validity.
Page 32 of 48
5.3 Comments on the outputsFor each report, the Project Manager will send to the Contractor consolidated comments received from the Reference Group or the approval of the report within 15 calendar days. The revised reports incorporating comments received from the Reference Group shall be submitted within 7 calendar days from the date of receipt of the comments. The evaluation team should provide a separate document explaining how and where comments have been integrated or the reason for not integrating certain comments, if this is the case..
5.4 Assessment of the quality of the Final Report and of the Executive SummaryThe quality of the draft versions of the Final Report and of the Executive Summary will be assessed by the Evaluation Manager using the online Quality Assessment Grid (QAG) in the EVAL Module (text provided in Annex V). The Contractor is given – through the EVAL module - the possibility to comment on the assessments formulated by the Evaluation Manager. The QAG will then be reviewed following the submission of the final version of the Final Report and of the Executive Summary.The compilation of the QAG will support/inform the compilation by the Evaluation Manager of the FWC SIEA’s Specific Contract Performance Evaluation.
5.5 Language All reports shall be submitted in English.
5.6 Number of copiesApart from their submission -preferably via the EVAL Module-, the approved version of the Final Report and of the Executive Summary will be also provided in 3 paper copies and in electronic version (PDF format sent by email) at no extra cost.
5.7 Formatting of reportsAll reports will be produced using Font Arial or Times New Roman minimum letter size 11 and 12 respectively, single spacing double sided. They will be sent in Word and PDF formats.
Page 33 of 48
ANNEXES
Annex I: Specific Technical Evaluation Criteria
SPECIFIC TECHNICAL EVALUATION CRITERIA Request for Services n. 2019/405979/1
FWC SIEA 2018 - LOT 4: Human Development and safety net - Sector 4: CultureEuropeAid/138778/DH/SER/multi
1. TECHNICAL EVALUATION CRITERIA The Contracting Authority selects the offer with the best value for money using an 80/20 weighing between technical quality and price0. Technical quality is evaluated on the basis of the following grid:
Criteria MaximumTotal score for Organisation and Methodology 100 Understanding of ToR and the aim of the
services to be provided10
Overall methodological approach, quality control approach, appropriate mix of tools and estimate of difficulties and challenges
30
Strategy, including team composition, organisation of tasks, and work plan
60
OVERALL TOTAL SCORE 100
2. TECHNICAL THRESHOLD Any offer falling short of the technical threshold of 75 out of 100 points, will be automatically rejected.
3. INTERVIEWS DURING THE EVALUATION OF THE OFFERSDuring the evaluation process of the offers received the Contracting Authority reserves the right to interview by phone one or several members of the proposed evaluation teams. Phone interviews will be tentatively carried out during the month of June 2019.
0 For more details about the 80/20 rule, please see the PRAG, chapter 3.3.10.5 - https://ec.europa.eu/europeaid/funding/about-funding-and-procedures/procedures-and-practical-guide-prag_en
Page 34 of 48
Annex II: Information that will be provided to the evaluation team
Legal texts and political commitments pertaining to the Actions to be evaluated
Country and Multi-Country Strategy Papers and Indicative Programmes (and equivalent) for the periods covered
Relevant national / sector policies and plans from National and Local partners and other donors
Identification studies of the Actions
Feasibility / formulation studies of the Actions
Financing agreement and addenda of the Actions
Actions ‘ quarterly and annual progress reports, and technical reports
European Commission’s Result Oriented Monitoring (ROM) Reports, and other external and internal monitoring reports of the Actions
Actions’ mid-term evaluation report and other relevant evaluations, audit, reports.
Relevant documentation from national/local partners and other donors
Guidance for Gender sensitive evaluations
Calendar and minutes of all the meeting of the Steering Committee of the Action(s)
Any other relevant document
A non-exhaustive list of specific reference documents is provided here below:
Instrument for Pre-accession Assistance (IPA I and II) 2011-2016. Available at: http://ec.europa.eu/enlargement/instruments/multi-beneficiary-programme/index_en.htm
Commission Communication COM(2012) 492 final : 'The roots of democracy and sustainable development: Europe's engagement with Civil Society in external relations', Brussels 12/09/2012: Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0492:FIN:EN:PDF
European Commission Enlargement Strategies (2011-2015). Available at: http://ec.europa.eu/enlargement/countries/package/index_en.htm
Commission Communication COM(2018) 65 final “A credible enlargement perspective for and enhanced EU engagement with the Western Balkans” of 06/02/2018 available at https://ec.europa.eu/commission/sites/beta-political/files/communication-credible-enlargement-perspective-western-balkans_en.pdf
Multi-Country Indicative Strategy Paper (2014-2020). Available at: http://ec.europa.eu/enlargement/pdf/key_documents/2014/20140919-multi-country-strategy-paper.pdf
Page 35 of 48
Financing Decisions of the Civil Society Facility and Media Programmes and Civil Society Facility Regional and National Action Documents for the years 2011-2013, 2014-2015, 2016-2017, and 2018-2019. Available at: http://ec.europa.eu/enlargement/instruments/multi-beneficiary-programme/index_en.htm (see bottom of the page under different years):
o Decision 2011/22-965 and 2012/23-324 (allocations for 2011 and 2012)
o Decision 2013/024-081 (allocations for 2013)
o Decisions 2014/031-605 and 2015/037-653 (allocations for 2014 and 2015)
o Decisions 2016/038-960 and 2017/038-961 (allocations for 2016 and 2017)
o Decisions 2018/040-646 and 2019/040-647 (allocations for 2018 and 2019)
Guidelines for EU support to Civil Society in enlargement countries 2014-2020: Available at:
http://ec.europa.eu/enlargement/pdf/civil_society/doc_guidelines_cs_support.pdf
Civil Society Facility projects Database (Western Balkans and Turkey) (2011-2015). Available upon request
TACSO Baseline and Monitoring reports for 2013, 2014 and 2015. Available at: http://www.tacso.org/documents/otherdoc/?id=9887&template_id=73&langTag=en-US
Thematic Evaluation of EU's Support to Civil Society in the Western Balkans (namely, Albania, Bosnia and Herzegovina, Croatia, former Yugoslav Republic of Macedonia, Kosovo under UNSCR 1244, Montenegro and Serbia) and Turkey 2012. http://ec.europa.eu/enlargement/pdf/financial_assistance/phare/evaluation/2012_eval_cs_final_report_2.pdf
Mid-term Review of Partnership Programmes for Civil Society Organisations (CSOs). August 2014. Available upon request at DG NEAR Unit D5
Technical Assistance for Civil Society Organisations (TACSO) needs assessment reports. Available at:
http://www.tacso.org/documents/reports/Archive.aspx?langTag=en-US&template_id=73&pageIndex=3
Results Oriented Monitoring (ROM) reports of projects financed by the CSF Programmes. Available upon request
DG NEAR Unit D5 mission reports. Available upon request
Relevant National Strategies for civil society development
Available publications, surveys and reviews.
Note: The evaluation team has to identify and obtain any other document worth analysing, through independent research and during interviews with relevant informed parties and stakeholders of the Actions.
Page 36 of 48
Annex III: Structure of the Final Report and of the Executive Summary
The contractor will deliver – preferably through their uploading in the EVAL Module two distinct documents: the Final Report and the Executive Summary. They must be consistent, concise and clear and free of linguistic errors both in the original version and in their translation – if foreseen.The Final Report should not be longer than the number of pages indicated in Chapter 5. Additional information on the overall context of the Actions, description of methodology and analysis of findings should be reported in an Annex to the main text. The presentation must be properly spaced and the use of clear graphs, tables and short paragraphs is strongly recommended.
The cover page of both deliverables shall carry the following text:
‘’This evaluation is supported and guided by the European Commission and presented by [name of consulting firm]. The report does not necessarily reflect the views and opinions of the European Commission’’.
Page 37 of 48
Executive Summary A tightly-drafted, to-the-point and free-standing Executive Summary. It should be short, no more than five pages. It should focus on the key purpose or issues of the evaluation, outline the main analytical points, and clearly indicate the main conclusions, lessons to be learned and specific recommendations. It is to be prepared by using the specific format foreseen in the EVAL Module.
Page 38 of 48
The main sections of the evaluation report shall be as follows:
1. Introduction A description of the Actions, of the relevant beneficiary/region/sector background and of the evaluation, providing the reader with sufficient methodological explanations to gauge the credibility of the conclusions and to acknowledge limitations or weaknesses, where relevant.
2. Answered questions / Findings A chapter presenting the answers to the Evaluation Questions, supported by evidence and reasoning.
3. Overall assessment A chapter synthesising all answers to Evaluation Questions into an overall assessment of the Actions. The detailed structure of the overall assessment should be refined during the evaluation process. The relevant chapter has to articulate all the findings, conclusions and lessons in a way that reflects their importance and facilitates the reading. The structure should not follow the Evaluation Questions, the logical framework or the evaluation criteria.
4. Conclusions and Recommendations
4.1 Conclusions This chapter contains the conclusions of the evaluation, organised per evaluation criterion.
In order to allow better communication of the evaluation messages that are addressed to the Commission, a table organising the conclusions by order of importance can be presented, or a paragraph or sub-chapter emphasizing the 3 or 4 major conclusions organised by order of importance, while avoiding being repetitive. If possible, the evaluation report identifies one or more transferable lessons, which are highlighted in the executive summary and can be presented in appropriate seminars or other dissemination activities
4.2 Recommendations They are intended to improve or reform the Actions in the framework of the cycle under way, or to prepare the design of a new Action for the next cycle.
Page 39 of 48
Recommendations must be clustered and prioritised, carefully targeted to the appropriate audiences at all levels, especially within the Commission structure.
5. Annexes to the report The report should include the following annexes:
The Terms of Reference of the evaluation The names of the evaluators (CVs can be
shown, but summarised and limited to one page per person)
Detailed evaluation methodology including: options taken, difficulties encountered and limitations. Detail of tools and analyses.
Evaluation Matrix Intervention logic / Logical Framework
matrices (planned/real and improved/updated)
Relevant geographic map(s) where the Actions took place
List of persons/organisations consulted Literature and documentation consulted Other technical annexes (e.g. statistical
analyses, tables of contents and figures, matrix of evidence, databases) as relevant
Detailed answers to the Evaluation Questions, judgement criteria and indicators
Page 40 of 48
Annex IV: Planning schedule
This annex must be included by Framework Contractors in their Specific Contract Organisation and Methodology and forms an integral part of it. Framework Contractors can add as many rows and columns as needed.The phases of the evaluation shall reflect those indicated in the present Terms of Reference.
Indicative Duration in working days0
Activity Location Team Leader Expert n°1 Expert n°2 Indicative Dates
Inception phase: total days days days days
Desk phase: total days days days days
Field phase: total days days days days
Synthesis phase: total days days days days
TOTAL working days (maximum)
0 Add one column per each expert
Page 41 of 48
Annex V: Quality Assessment Grid
The quality of the Final Report will be assessed by the Project Manager (since the submission of the draft Report and Executive Summary) using the following quality assessment grid, which is included in the EVAL Module; the grid will be shared with the evaluation team, which will have the possibility to include their comments.
Action (Project/Programme) evaluation – Quality Assessment Grid Final Report
Evaluation data
Evaluation title
Evaluation managed by Type of evaluation
CRIS ref. of the evaluation contract EVAL ref.
Evaluation budget
EUD/Unit in charge Evaluation Manager
Evaluation dates Start: End:
Date of draft final report Date of Response of the Services
Comments
Project dataMain project evaluated
CRIS # of evaluated project(s)
DAC Sector
Contractor's detailsEvaluation Team Leader Evaluation Contractor
Evaluation expert(s)
Page 42 of 48
Legend: scores and their meaning
Very satisfactory: criterion entirely fulfilled in a clear and appropriate waySatisfactory: criterion fulfilledUnsatisfactory: criterion partly fulfilled Very unsatisfactory: criterion mostly not fulfilled or absent
The evaluation report is assessed as follows
1. Clarity of the reportThis criterion analyses the extent to which both the Executive Summary and the Final Report:
Are easily readable, understandable and accessible to the relevant target readers Highlight the key messages The length of the various chapters and annexes of the Report are well balanced Contain relevant graphs, tables and charts facilitating understanding Contain a list of acronyms (only the Report) Avoid unnecessary duplications Have been language checked for unclear formulations, misspelling and grammar errors The Executive Summary is an appropriate summary of the full report and is a free-standing document
Strengths Weaknesses Score
Contractor's comments Contractor's comments
2. Reliability of data and robustness of evidence This criterion analyses the extent to which:
Data/evidence was gathered as defined in the methodology The report considers, when relevant, evidence from EU and/or other partners’ relevant studies, monitoring reports and/or evaluations The report contains a clear description of the limitations of the evidence, the risks of bias and the mitigating measures
Strengths Weaknesses Score
Contractor's comments Contractor's comments
3. Validity of Findings
Page 43 of 48
This criterion analyses the extent to which: Findings derive from the evidence gathered Findings address all selected evaluation criteria Findings result from an appropriate triangulation of different, clearly identified sources When assessing the effect of the EU intervention, the findings describe and explain the most relevant cause/effect links between outputs, outcomes and impacts The analysis of evidence is comprehensive and takes into consideration contextual and external factors
Strengths Weaknesses Score
Contractor's comments Contractor's comments
4. Validity of conclusionsThis criterion analyses the extent to which:
Conclusions are logically linked to the findings, and go beyond them to provide a comprehensive analysis Conclusions appropriately address the selected evaluation criteria and all the evaluation questions, including the relevant cross-cutting dimensions Conclusions take into consideration the various stakeholder groups of the evaluation Conclusions are coherent and balanced (i.e. they present a credible picture of both strengths and weaknesses), and are free of personal or partisan considerations (If relevant) whether the report indicates when there are not sufficient findings to conclude on specific issues
Strengths Weaknesses Score
Contractor's comments Contractor's comments
5. Usefulness of recommendationsThis criterion analyses the extent to which the recommendations:
Are clearly linked to and derive from the conclusions Are concrete, achievable and realistic Are targeted to specific addressees Are clustered (if relevant), prioritised, and possibly time-bound (If relevant) provide advice for the Action’s exit strategy, post-Action sustainability or for adjusting Action’s design or plans
Strengths Weaknesses Score
Contractor's comments Contractor's comments
6. Appropriateness of lessons learnt analysis (if requested by the ToR or included by the evaluators)
Page 44 of 48
This criterion is to be assessed only when requested by the ToR or included by evaluators and is not to be scored. It analyses the extent to which: Lessons are identified When relevant, they are generalised in terms of wider relevance for the institution(s)
Strengths Weaknesses
Contractor's comments Contractor's comments
Final comments on the overall quality of the report Overall score
Page 45 of 48
Annex VI: Summary table of the Financing decisions and the related projects concerned by the evaluation
Type Decision Number
Description Projects Contract
Number
Pilot Scheme(22 02 77 01)
DN020467 Pilot
Scheme 2008
Project 1: Development of a Regional Cultural Heritage Facility in Kosovo
2009/217-1132009/217-124 2010/247-9512010/244-922
Project 2: ARCH-Stolac: Reconstruction of Architectural Heritage in Stolac
2011/268-9752012/301-1492012/302-455
DN021771 Pilot
Scheme 2009
Project 1 - ARCH-Vukovar: “Heritage as a means of development. Reconstruction of Architectural Heritage in Vukovar Town Centre - Sustainable Revival and Development of a war-torn community”
2011/265-589
Project 2 – Revitalization of cultural heritage in the frame of Sustainable Development in Besac – Montenegro
2013/310-028
Project 3 – Cultural Heritage Facility in Senjski Rudnik - Serbia
2012/290-9272012/290-2172012/290-916
Preparatory
DN022659 Preparatory
Contract 1: Western Balkans - From historical integration to contemporary active participation 2011/277-768
46
Action(22 02 77 02)
Action Cultural Heritage
2010
Contract 2: Jankovic Castel - Historic Site, generating sustainable development of the Ravni Kotari region (Croatia)
2011/277-778
Contract 3: HER.CUL - Valorisation of Medieval Tombstone. CULtural HERitage in the Western Balkans
2011/277-801
Contract 4: Restore our past to build the future. Preparatory actions of participatory and intercultural requalification of civil war-damaged mosques and churches in the former Yugoslav Republic of Macedonia
2011/277-811
Contract 5: MANUBALK 2011/277-838
Contract 6: Past for the Future 2011/277-842
Contract 7: Balkans’ Memory 2011/277-929
Contract 8: Revitalization of Albanian Hiritoric Towns of Gjirokastra and Berat by resolving political conflicts (REVATO)
2011/277-950
DN023372
Preparatory
Action Cultural Heritage
2011
Contract 1: Sustaining the Rehabilitation of Cultural Heritage in the Western Balkans 2012/295-647
DN023867 Preparatory
Action Cultural Heritage
2012
Project 1: Rehabilitation of the monastery of St. Mary for museum use and landscaping of the archaeological site of Apollonia, Fier Albania–
2015/365-1752015/369-281
Project 2: Reconstruction of the Kastel Fortress phase II - Unit 2- Cultural Centre with Auxiliary services
2014/349-104
2015/357-106
47
Project 3: Rehabilitation of the Franciscan Monastery in Bac, Serbia
2016/373-414
2016/373-639
48